Welcome!

@CloudExpo Authors: Liz McMillan, Karthick Viswanathan, Jason Bloomberg, Pat Romanski, Yeshim Deniz

Related Topics: @CloudExpo

@CloudExpo: Article

BPM in the Cloud: Disruptive Technology

Why the middleware-in-the-Cloud approach to PaaS is doomed to fail

Sometimes, it feels like ZapThink is the lone voice of reason in the wilderness. We fought to cast SOA as a business-driven architectural approach in the face of massive vendor misinformation, hell-bent on turning SOA into an excuse to buy middleware. And while many organizations are successfully implementing SOA, perhaps with our help, in large part we lost that battle.

Today we’re fighting the corresponding battle over the Cloud. The general battle lines are similar to the SOA days—ZapThink espousing business-driven architecture, while the vendors do their best to spin Cloud as an excuse to buy more gear. But just as the players in the two World Wars were largely the same even though the tactics were quite different, so too with SOA and the Cloud.

The Original BPM Battle Lines
Today, a battle is brewing that promises to reopen some old wounds of the SOA days to be fought over the new territory of the Cloud. That battle is over Business Process Management (BPM). The fundamental idea behind BPM software is that you need some kind of engine to coordinate interactions between actors and disparate applications into business processes. Those actors may be human, or other applications themselves. To specify the process, you must create some kind of process representation the process engine can run.

Vendors loved BPM because process engines were a natural add-on to their middleware stacks. Coordinating multiple applications meant creating multiple integrations, and for that you need middleware. Lots of it. And in spite of paying lip service to cross-platform Service compositions that would implement vendor-independent processes, in large part each vendor rolled out a proprietary toolset.

ZapThink, of course, saw the world of SOA-enabled BPM quite differently. In our view, the Service-oriented way of looking at BPM was to free it from the engines, and focus on Services: composing them and consuming them. But there was a catch: Services are inherently stateless. The challenge with the Service-oriented approach to BPM was how to main state for each process instance in an inherently stateless environment.

We tackled the problem of state head on back in 2006, when we discussed how to maintain state in compositions of stateless Services that implement processes. Then in 2008, we discussed how you could free SOA-based BPM from middleware by maintaining state in message interactions between Services and their consumers. If you got this right, you wouldn’t need a heavyweight process engine to do it for you.

The ESB vendors largely ignored us (Fiorano Software being a notable exception). Sure, WS-Addressing might have helped, but the vendors chose not to implement this standard consistently, instead relying on threads or other object instances to maintain process instance state. After all, once you fall for the heavyweight, big engine approach to BPM, you’re essentially locked into the platform you have chosen. And vendors love nothing more than customer lock-in.

The New Battle Line in the Cloud
While we have to award victory to the vendors in the SOA-based BPM war, the move to the Cloud offers an entirely new battleground with completely new rules. Today, of course, the vendors (and it seems, everyone else) want to put their software in the Cloud. So it’s a natural consequence that the BPM vendors would seek to move their BPM engines into the Cloud as well, perhaps as part of a PaaS provider strategy. Clearly such a move would be a good bet from the business perspective, as it’s likely that many BPM customers would find value in a Cloud-based offering.

Here’s where the story gets interesting. In order to achieve the elasticity benefit of the Cloud for a distributed application, it’s essential for the application tier to be stateless. The Cloud may need to spawn additional instances to handle the load, and any particular instance may crash. But because the Cloud is highly available and partition tolerant, such a crash mustn’t hose the process that Cloud instance is supporting.

As a result, there is simply no way a traditional BPM engine can run properly in the Cloud. After all, BPM engines’ raison d’être is to maintain process state, but you can’t do that on a Cloud instance without sacrificing elasticity! In other words, all the work the big vendors put into building their SOA-platform-centric BPM engines must now be chucked out the door. The Cloud has changed the rules of BPM.

Hypermedia-Oriented Architecture, Anyone?
ZapThink has the answer. The same answer we had in 2006, only now we must recast the answer for the Cloud: maintain process state information in the messages.

The messages we’re talking about are the interactions between resources on the server and the clients that the process actors are using, what we call representations in the REST context. In other words, it’s essential to transfer application state in representations to the client. This is the reason why REST is “representational state transfer,” by the way.

The vendors, however, are far from shaking in their boots, since ZapThink once again has the lone voice in the wilderness when we talk about REST—and this time, the vendors aren’t the ones responsible for the misinformation. Instead, it’s the community of developers who have miscast REST as a way to build uniform APIs to resources. But that’s not what REST is about. It’s about building hypermedia applications. In fact, REST isn’t about Resource-Oriented Architecture at all. You could say it’s about Hypermedia Oriented Architecture.

At the risk of extending an already-tired cliché, let’s use the abbreviation HOA. In HOA, hypermedia are the engine of application state. (RESTafarians recognize this phrase as the dreaded HATEOAS REST constraint.) With HATEOAS, hyperlinks dynamically describe the contract between the client and the server in the form of a workflow at runtime. That’s right. No process engine needed! All you need are hyperlinks.

The power of this idea is obvious once you think through it, since the World Wide Web itself is the prototypical example of such a runtime workflow. You can think of any sequence of clicking links and loading Web pages as a workflow, after all—where those pages may be served up by different resources on different servers anywhere in the world. No heavyweight, centralized process engine in sight.

The New BPM Battle Lines
The big vendors aren’t worried now, because they don’t understand the power of hypermedia. If HOA-based BPM takes off, however, then their whole middleware-in-the-Cloud approach to PaaS is doomed to fail. If they’re smart, they’d better dig their trenches now. After all, they have too much invested in the old way of doing things. HOA-based BPM is potentially a disruptive technology, and they’re facing the innovator’s dilemma.

On the other hand, there is a substantial opportunity for the innovators—new entrants to the BPM marketplace who figure out how to build a Cloud-friendly BPM engine. Think you have what it takes? Here are some pointers to architecting a partition-tolerant, RESTful BPM application.

  • Separate the resources that build process applications from the representations of those resources. Servers are responsible for generating self-descriptive process maps that contain all the context necessary for any actor to work through a process. After all, orchestrations can be resources as well. In other words, the persistence tier doesn’t host a process engine, it hosts a model-driven resource that generates stateless process applications to run on the elastic application tier.
  • The application tier acts as the client for such process representations, and as a server that supports the clients of the process. Keep the application tier stateless by serving application state metadata in representations to the clients of the process. In other words, the application tier processes interactions with clients statelessly. As a result, any application tier instance is interchangeable with any other. If one crashes, you can bootstrap a replacement without affecting processes in progress. This interchangeability is the secret to maintaining elasticity and fault tolerance.
  • Separate UI representations from application state representations. If a client has the state representation, it should be able to fetch the appropriate UI representation from any application tier instance. As a result, the state representations are portable from one client to another. You could begin a process on a laptop and transfer it to a mobile phone, for example, and continue where you left off.
  • Use a lightweight, distributed queuing mechanism to address client uptime issues. If a client (typically a browser) crashes in the middle of a process, you want to be able to relaunch the client and pick up where you left off. But if the client has the only copy of the application state, you have a problem. Instead, allow the client to fetch a cached copy from a queue.
  • For processes that require heavy interactions among multiple actors, follow a peer-to-peer model. Most processes that involve multiple actors call for infrequent interactions between those actors, for example, processes with approval steps. For such processes, support those interactions via the resource state in the persistence tier. However, when you require heavy interactions among actors (imagine a chat window, for example), enable the actors to share a single application instance that initiates a peer-to-peer interaction.
  • Maintain integrity via checksums. Conventional wisdom states that you don’t want to let the client have too much control, or a bad actor could do real damage. To prevent such mischief, ensure that any invalid request from the client triggers an error response on the application tier. As a result, the worst a hacker can do is screw up their own session. Serves them right!
  • Not much to it, is there? On the one hand, it’s surprising no vendor has stepped up to the plate to build a fully partition tolerant, RESTful BPM app. But then again, it’s not the technical complexity that’s the problem—it’s the paradigm shift in thinking about the nature of stateful applications in today’s Cloud-ready world. That shift is what makes Cloud-friendly BPM a disruptive technology.

    The ZapThink Take

    Perhaps I should have named this ZapFlash “RESTful BPM,” because, of course, that’s what it’s about. But that would have introduced unfortunate confusion, since that phrase has already been co-opted by vendors who use it to mean “traditional BPM engines with RESTful APIs.” But that’s the old way of thinking. What this ZapFlash proposes is a different way of thinking about BPM applications altogether.

    This paradigm shift, as with all such shifts, is part of a larger trend from old ways of thinking to entirely different approaches to building and consuming applications. The table below illustrates some aspects of this shift:

    Old way

    New way

    Partition intolerant

    Partition tolerant

    Vertical scalability

    Horizontal scalability

    ACID

    BASE

    Process engines

    RESTful BPM

    State management with objects (threads, session beans, etc.)

    State management with HATEOAS

    Web Services-based SOA

    REST-based SOA

    Legacy enterprise apps poorly shoehorned into the Cloud

    Next generation enterprise apps built for the Cloud (enterprise SaaS)

    Middleware centricity

    Cloud centricity

    Imperative programming

    Declarative programming


    It may seem the new way is at the bleeding edge, but in reality, many of today’s Cloud-based companies are already living the new paradigm. Only enterprises and incumbent vendors find themselves struggling with the old way. But that is changing. Stay tuned, ZapThink will continue to lead the way!

    Image source: Leszek Leszczynski

More Stories By Jason Bloomberg

Jason Bloomberg is the leading expert on architecting agility for the enterprise. As president of Intellyx, Mr. Bloomberg brings his years of thought leadership in the areas of Cloud Computing, Enterprise Architecture, and Service-Oriented Architecture to a global clientele of business executives, architects, software vendors, and Cloud service providers looking to achieve technology-enabled business agility across their organizations and for their customers. His latest book, The Agile Architecture Revolution (John Wiley & Sons, 2013), sets the stage for Mr. Bloomberg’s groundbreaking Agile Architecture vision.

Mr. Bloomberg is perhaps best known for his twelve years at ZapThink, where he created and delivered the Licensed ZapThink Architect (LZA) SOA course and associated credential, certifying over 1,700 professionals worldwide. He is one of the original Managing Partners of ZapThink LLC, the leading SOA advisory and analysis firm, which was acquired by Dovel Technologies in 2011. He now runs the successor to the LZA program, the Bloomberg Agile Architecture Course, around the world.

Mr. Bloomberg is a frequent conference speaker and prolific writer. He has published over 500 articles, spoken at over 300 conferences, Webinars, and other events, and has been quoted in the press over 1,400 times as the leading expert on agile approaches to architecture in the enterprise.

Mr. Bloomberg’s previous book, Service Orient or Be Doomed! How Service Orientation Will Change Your Business (John Wiley & Sons, 2006, coauthored with Ron Schmelzer), is recognized as the leading business book on Service Orientation. He also co-authored the books XML and Web Services Unleashed (SAMS Publishing, 2002), and Web Page Scripting Techniques (Hayden Books, 1996).

Prior to ZapThink, Mr. Bloomberg built a diverse background in eBusiness technology management and industry analysis, including serving as a senior analyst in IDC’s eBusiness Advisory group, as well as holding eBusiness management positions at USWeb/CKS (later marchFIRST) and WaveBend Solutions (now Hitachi Consulting).

@CloudExpo Stories
yperConvergence came to market with the objective of being simple, flexible and to help drive down operating expenses. It reduced the footprint by bundling the compute/storage/network into one box. This brought a new set of challenges as the HyperConverged vendors are very focused on their own proprietary building blocks. If you want to scale in a certain way, let’s say you identified a need for more storage and want to add a device that is not sold by the HyperConverged vendor, forget about it....
SYS-CON Events announced today that Pulzze Systems will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Pulzze Systems Inc, provides the software product "The Interactor" that uniquely simplifies building IoT, Web and Smart Enterprise Solutions. It is a Silicon Valley startup funded by US government agencies, NSF and DHS to bring innovative solutions to market.
With Cloud Foundry you can easily deploy and use apps utilizing websocket technology, but not everybody realizes that scaling them out is not that trivial. In his session at 21st Cloud Expo, Roman Swoszowski, CTO and VP, Cloud Foundry Services, at Grape Up, will show you an example of how to deal with this issue. He will demonstrate a cloud-native Spring Boot app running in Cloud Foundry and communicating with clients over websocket protocol that can be easily scaled horizontally and coordinate...
SYS-CON Events announced today that Elastifile will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Elastifile Cloud File System (ECFS) is software-defined data infrastructure designed for seamless and efficient management of dynamic workloads across heterogeneous environments. Elastifile provides the architecture needed to optimize your hybrid cloud environment, by facilitating efficient...
For financial firms, the cloud is going to increasingly become a crucial part of dealing with customers over the next five years and beyond, particularly with the growing use and acceptance of virtual currencies. There are new data storage paradigms on the horizon that will deliver secure solutions for storing and moving sensitive financial data around the world without touching terrestrial networks. In his session at 20th Cloud Expo, Cliff Beek, President of Cloud Constellation Corporation, d...
Businesses and business units of all sizes can benefit from cloud computing, but many don't want the cost, performance and security concerns of public cloud nor the complexity of building their own private clouds. Today, some cloud vendors are using artificial intelligence (AI) to simplify cloud deployment and management. In his session at 20th Cloud Expo, Ajay Gulati, Co-founder and CEO of ZeroStack, discussed how AI can simplify cloud operations. He covered the following topics: why cloud mana...
As more and more companies are making the shift from on-premises to public cloud, the standard approach to DevOps is evolving. From encryption, compliance and regulations like GDPR, security in the cloud has become a hot topic. Many DevOps-focused companies have hired dedicated staff to fulfill these requirements, often creating further siloes, complexity and cost. This session aims to highlight existing DevOps cultural approaches, tooling and how security can be wrapped in every facet of the bu...
DevOps at Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to w...
SYS-CON Events announced today that CAST Software will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CAST was founded more than 25 years ago to make the invisible visible. Built around the idea that even the best analytics on the market still leave blind spots for technical teams looking to deliver better software and prevent outages, CAST provides the software intelligence that matter ...
SYS-CON Events announced today that Golden Gate University will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Since 1901, non-profit Golden Gate University (GGU) has been helping adults achieve their professional goals by providing high quality, practice-based undergraduate and graduate educational programs in law, taxation, business and related professions. Many of its courses are taug...
Recently, WebRTC has a lot of eyes from market. The use cases of WebRTC are expanding - video chat, online education, online health care etc. Not only for human-to-human communication, but also IoT use cases such as machine to human use cases can be seen recently. One of the typical use-case is remote camera monitoring. With WebRTC, people can have interoperability and flexibility for deploying monitoring service. However, the benefit of WebRTC for IoT is not only its convenience and interopera...
Translating agile methodology into real-world best practices within the modern software factory has driven widespread DevOps adoption, yet much work remains to expand workflows and tooling across the enterprise. As models evolve from pockets of experimentation into wholescale organizational reinvention, practitioners find themselves challenged to incorporate the culture and architecture necessary to support DevOps at scale.
Vulnerability management is vital for large companies that need to secure containers across thousands of hosts, but many struggle to understand how exposed they are when they discover a new high security vulnerability. In his session at 21st Cloud Expo, John Morello, CTO of Twistlock, will address this pressing concern by introducing the concept of the “Vulnerability Risk Tree API,” which brings all the data together in a simple REST endpoint, allowing companies to easily grasp the severity of t...
SYS-CON Events announced today that CA Technologies has been named “Platinum Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business – from apparel to energy – is being rewritten by software. From planning to development to management to security, CA creates software that fuels transformation for companies in the applic...
When shopping for a new data processing platform for IoT solutions, many development teams want to be able to test-drive options before making a choice. Yet when evaluating an IoT solution, it’s simply not feasible to do so at scale with physical devices. Building a sensor simulator is the next best choice; however, generating a realistic simulation at very high TPS with ease of configurability is a formidable challenge. When dealing with multiple application or transport protocols, you would be...
In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, discussed how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He also discussed how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
WebRTC is the future of browser-to-browser communications, and continues to make inroads into the traditional, difficult, plug-in web communications world. The 6th WebRTC Summit continues our tradition of delivering the latest and greatest presentations within the world of WebRTC. Topics include voice calling, video chat, P2P file sharing, and use cases that have already leveraged the power and convenience of WebRTC.
IT organizations are moving to the cloud in hopes to approve efficiency, increase agility and save money. Migrating workloads might seem like a simple task, but what many businesses don’t realize is that application migration criteria differs across organizations, making it difficult for architects to arrive at an accurate TCO number. In his session at 21st Cloud Expo, Joe Kinsella, CTO of CloudHealth Technologies, will offer a systematic approach to understanding the TCO of a cloud application...
An increasing number of companies are creating products that combine data with analytical capabilities. Running interactive queries on Big Data requires complex architectures to store and query data effectively, typically involving data streams, an choosing efficient file format/database and multiple independent systems that are tied together through custom-engineered pipelines. In his session at @BigDataExpo at @ThingsExpo, Tomer Levi, a senior software engineer at Intel’s Advanced Analytics ...
"With Digital Experience Monitoring what used to be a simple visit to a web page has exploded into app on phones, data from social media feeds, competitive benchmarking - these are all components that are only available because of some type of digital asset," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.