SOA in the Cloud on Ulitzer

Certainly no one would seriously argue that web applications are fast enough for everyone. SPDY is one suggested solution, but what if we combine MapReduce and SPDY? Could we develop an architectural solution that leverages the best of SPDY without requiring entire infrastructure changes to support a new protocol?

More than a couple of people have mentioned Map/Reduce as a means to achieve workload-level distribution of applications in a cloud computing environment. I hadn’t looked into Map/Reduce but finally decided that if that many very smart people were thinking it was a solution, I should look into it. After reading through a few tutorials and articles on the subject, including a much referenced lecture from a UW Madison (yeah! Badgers!) professor, I began to wonder how well we could combined MapReduce with SPDY as a means to improve application delivery. [The referenced ‘illustrated’ PDF from the lecture is hard to find. You can access it here. ]

From Google’s paper on Map/Reduce:

MapReduce is a programming model and an associated implementation for processing and generating large datasets that is amenable to a broad variety of real-world tasks. Users specify the computation in terms of a map and a reduce function, and the underlying runtime system automatically parallelizes the computation across large-scale clusters of machines, handles machine failures, and schedules inter-machine communication to make efficient use of the network and disks.

Programmers find the system easy to use: more than ten thousand distinct MapReduce programs have been implemented internally at Google over the past four years, and an average of one hundred thousand MapReduce jobs are executed on Google’s clusters every day, processing a total of more than twenty petabytes of data per day.

It isn’t just the protocol (SPDY) that’s apposite to application performance and more specifically, web application performance. After looking through Map/Reduce, it would certainly appear that the combination of the “programmatic model” and SPDY would definitely provide the kind of scale and processing speed necessary to achieve a “speedier web.”

When we want to scale a web application today we need to build out an architecture that load balances requests across a pool of servers. Clients are limited in the number of connections that can be opened to any given host, but that number is now in the 6-8 range for modern browsers. The connections are synchronous, meaning that once a request is sent a reply must be received before the next request can be sent.

Each object in a page can be mapped to a request and thus the browser’s task is to distribute object requests across its available connections and then to aggregate the responses into a document that can be rendered for the user’s viewing pleasure.

In much the same way, the load balancer also distributes the requests across its pool of available resources: the application instances. The Load balancer is capable of handling much high volumes of connections, of course, and it can intelligently distribute requests based on a variety of parameters. An advanced load balancer (application delivery controller) can distribute requests based on the URI, values in HTTP headers, and on data in the actual request (payload). But it is still bound to the same synchronous request/reply pattern as the browser. In order to achieve high scalability and fast performance, the load balancer optimizes connections and uses as much information as possible when distributing requests. The latter is often a matter of configuration: even though the load balancer can use a wide variety of environmental factors upon which to base its load balancing decision it must be configured to do so and many an administrator/architect ignores these capabilities.

The result is still synchronous, with potentially multiple connections per client being utilized to return as many objects in parallel as possible. Both the browser and the load balancer are essentially parallelizing requests and responses.

The biggest difference to note immediately is the lack of synchronous communication. SPDY is asynchronous, and thus the browser need not parallelize the requests. Using SPDY the browser could, as it was parsing the main page, simply send a request for each object it encounters back to the origin server.

Remember that SPDY allows for only one connection per browser, so all requests for component objects in a web page would need to be sent over that single connection. Aside from the synchronicity, this is not much different than would be the case is browsers were programmatically limited to a single connection per host.

Right now it appears the usage of SPDY is simulating traditional behavior; that is, the browser is still responsible for parsing out the “main page” and initiating individual requests for each component, albeit in the case of SPDY over the same connection.

If you have the capabilities afforded by Map/Reduce on the web/application server (or intermediary of some kind), could we not take advantage of that? Using Map/Reduce it certainly appears (and I may be completely off-base, but someone, I’m sure, will correct me if that’s the case) you could push the parsing (disaggregation) of the “main page” to the server/intermediary and let it “map” and “reduce” (aggregate) its component objects into a single, completed page that can then be returned to the client over that single connection. The “map” function is used to apply the same function to a large set of inputs, and all we’re doing is saying the function is “load/generate this page”, after all. The application of compression and security policies can be applied either at the component or complete page comprising all HTML required. The rest of the infrastructure need only act on a single, completed page in which all pertinent data exists, greatly simplifying processing.

It would have to be selective in that only some included content needs to be “reduced” into the main page. Some objects – navigational links, for example – can’t be included because, well, it would break the entire web. But there is a subset of objects that could be included that might result in improved performance overall. This is where SPDY (or at least its core functionality as applied to HTTP) comes into the picture, as its asynchronous nature would improve the delivery of objects that can’t be included in the core HTML for whatever reason. Distinguishing between the two could be as simple as an attribute on an anchor element such as “aggregate=true” with a default of false, just to try to maintain backward compatibility.

This would remove the need for the browser to parse the original page and subsequently issue requests, eliminating the round trip time for each object from the overall response time. While the resulting page is larger because it contains the complete HTML necessary, the browser can more effectively employ  progressive rendering techniques on the complete page as soon as data begins returning.

The draft SPDY protocol, by allowing asynchronous requests, eliminates approximately half the round trip times by not requiring immediate responses, but by leveraging Map/Reduce capable systems on the server/intermediary side we can eliminate more ( #objects * RTT to be exact). We also completely eliminate the negative impact on the network (and thus application performance) from dealing with many small packets generated from many small objects.

The RTT between the server/intermediary and internal application servers is still applicable, but because this is almost always over high-speed, low-latency LAN connections (and we’re paying that price regardless) the impact on overall performance remains minimal.

If you think of application delivery controllers as nothing more than load balancers then it certainly might appear that way, wouldn’t it? But load balancing, while an integral component to an application delivery controller, is not the be-all and end-all of its capabilities or its only role in high-availability architectures. Optimization and acceleration still applies, as does security and its myriad related functions. So, too, does ability to transform requests on-demand, both ingress and egress. Context is still as important, if not more so in an architecture such as the one described, and given an application delivery controller generally sits in what is a strategic point of control in a data center architecture- traditional, virtualized, or cloud computing – it is still the best place to provide most application delivery functionality.

So no, I’m not shooting myself in the foot by postulating on a web-application architecture using SPDY and Map/Reduce (or some similar mechanism that has yet to be designed) as a core means to achieve fast and highly-scalable web applications. The use of SPDY and Map/Reduce would only speed up the internal processing and reduce the latency associated with the traditional request/reply paradigm. It does not address high-latency links, congestion, conditional network problems, or security-related issues. It doesn’t solve the problem of regulating request rates nor prioritization nor business-layer load balancing. And there are many BHQ (Big Hairy Questions) involving such a solution that would need answers before it could be useful, such as the handling of off-domain requests and credential mapping for integrated widgets/gadgets/sites.

Besides, it is somewhat interesting to note that much of the functionality described by Map/Reduce, when applied strictly to URI-based workloads (think REST and even SOA) already exists in application delivery controllers. It isn’t, after all, just about load-balancing, it’s about intelligent routing of requests based on context, like the URI. The single-session concept is something already demanded by service-providers (RADIUS, DIAMETER, SIP) and some application delivery controllers can handle this type of message-based load balancing [PDF] scenario, so all that’s left is the aggregation of the disparate components into a single page for delivery. So it’s possible that the definition of such an architecture combined with the protocol could be natively supported by application delivery controllers with relative ease. What’s necessary is to break out of the connection-oriented processing paradigm inherent in load balancing and proxies and HTTP, and in some cases we’re half-way there already.

It is definitely interesting to contemplate a new architectural solution to the problems associated with HTTP and performance. Map/Reduce is also certainly one answer to moving cloud computing out of its current instantiation toward truly on-demand resource utilization on a per-workload basis. It’s an interesting concept and one that obviously works well for Google, given the number of applications in its repertoire that apparently take advantage of the model. Thus it (or similar concepts) is certainly something to consider for potentially broader usage outside of Google’s infrastructure.

I don’t think anyone would argue that the web is “speedy” enough as it is, so exploring new concepts is something we need to do. We may find a thousand ways not to do it – and this may be one of those ‘not’ ways – but eventually someone will find a way.