Click here to close now.

Welcome!

@CloudExpo Authors: Pat Romanski, AppDynamics Blog, Liz McMillan, Carmen Gonzalez, Jason Bloomberg

Related Topics: @CloudExpo, Microservices Expo

@CloudExpo: Blog Post

Load Balancing in a Cloud

How should auto-scaling work, and why doesn’t it?

Although “rapid elasticity” is part of NIST’s definition of cloud computing, it may be interesting to note that many cloud computing environments don’t include this capability at all – or charge you extra for it. Many providers offer the means by which you can configure a load balancing service and manually add or remove instances, but there may not be a way to automate that process. If it’s manual, it’s certain “rapid” in the sense that’s it’s probably faster than you can do it (because you’d have to acquire hardware and deploy the application, as opposed to simply hitting a button and “cloning” the environment ‘out there’) but it’s not necessarily as fast as it could be (because it’s manual) nor is it automated. There’s a number of reasons for that – but we’ll get to that later. First, let’s look at how auto-scaling is supposed to work, at least theoretically.

image

1. Some external entity that is monitoring capacity for “Application A” triggers an event that indicates a new instance is required to maintain availability. This external entity could be an APM solution, the cloud management console, or a custom developed application. How it determines capacity limitations is variable and might be based solely on VM status (via VMware APIs), data received from the load balancer, or a combination thereof.

2. A new instance is launched. This is accomplished via the cloud management console or an external API as part of a larger workflow/orchestration.

3. The external entity grabs the IP address of the newly launched instance and instructs the load balancer to add it to the pool for resources for “Application A.” This is accomplished via the standards-based API which presents the configuration and management control plane of the load balancer to external consumers as services.

4. The load balancer adds the new application instance to the appropriate pool and as soon as it has confirmation that the instance is available and responding to requests, begins to direct traffic to the instance.

This process is easily reversed upon termination of an instance. Note: there are other infrastructure components that are involved in this process that must also be notified on launch and decommission, but for this discussion we’re just looking at the load balancing piece as it’s critical to the concept of auto-scaling.

The important thing to note here is that the process for adding – and subsequently using – the new application instance to the load balancer should be automatic. Once the instance is  launched, all that remains is to inform the load balancer that a new instance is available. The load balancer should take care of the rest, and it should do so transparently. That means no interruption of service, no reboot, no reload the configuration, nothing. Similarly the termination and removal of the application instance from the pool of resources should be as seamless and non-disruptive. It does not matter whether the load balancing solution is hardware or a VNA (Virtual Network Appliance) as long as it has an API through which available resources can be added and removed on-demand.

This process is relatively straightforward and can even be accomplished without the “external entity” by directly integrating the application with the load balancing service. By hooking the appropriate setup and tear-down routines in imagethe application, the load balancer can be automatically instructed to perform the appropriate actions whenever an instance of that application is launched or terminated.

1. Application is initialized and instructs load balancer to add it to the appropriate resource pool.

2. Load balancer adds the application instance to the right pool of resources and begins directing traffic to it as configured.

3. Application is terminating, instructs load balancer to remove it from the available resource pool.

4. Load balancer removes the application instance from pool of resources.

This requires tighter coupling of the load balancer with the application, however, which may be less than ideal unless you are very tied to your load balancing solution. The advantage of this solution is that regardless of whether the application is “physical” or “virtual”, the same process occurs, which lets you mix and match and be a lot more flexible in your architecture.


WHY is THIS so HARD THEN?

It’s not, if the load balancer has an API. Herein lies the problem: building out an automated, dynamic infrastructure requires the ability to communicate, to collaborate, to integrate network and application network infrastructure with the data center management system. That means some sort of API is required. Oh, you could do it with a little SSH and a local script, but that’s not nearly as efficient or flexible as using an API that’s designed to be leveraged for integration and automation in the first place.

While just about every hardware load balancer (and their virtualized equivalents) is enabled with a standards-based control plane, most web-server-turned-rudimentary-load-balancers  are not enabled with an API that allows this level of integration. That means hacking the system with scripts that inject or remove configurations, forcing reloading of the configuration (or worse, stopping and starting the system) and most likely interrupting service. Not only does it mean hacking the system but the scripts must somehow pull from customer-specific meta-data information in order to properly configure the load balancing solution.

 

 

So in order to achieve this in a way that maintains availability and is easy to use and essentially a multi-tenant solution needs the services of an API. When it comes to integration there are few options and the most flexible method is to use a standards-based API to manage a solution that supports the notion of multi-tenant configuration natively. The problem is, of course, that most of the API-enabled (and therefore easy to integrate and automate) load balancers cost money. It requires an investment, up front, and many folks just aren’t willing to do that. Except they are doing that; they’re just trading capital expenses for longer-term development of scripts and research that approximate (but never quite reach the reliability) of an API that’s been specifically developed, tested, and deployed for the purpose of integrating with external management systems and scripts. What’s worse is that eventually most of these solutions developed on sweat and scripts will end up turning to a more sophisticated, proven API-based solution anyway because of the inherent challenges associated with scalability of the solution itself. That results in a financially inefficient way to go about building a dynamic infrastructure that supports rapid, elastic scalability.

So many organizations faced with the choice of investing in a more robust, flexible solution when they’ve already sunk a lot of time, effort, and money into their existing “free” solution, simply continue to hack and paste and shove a square peg into a round hole, which means auto-scaling never works quite as expected and advanced features come at a snail’s pace because of the difficulty in scripting and integration.

It doesn’t have to be so hard, but you do have to view load balancing for what it is: a key component of the cloud computing strategy, and give it the focus it deserves, even if that means it’ll cost you more up front because in the end it’ll cost you far less than the alternative.

Read the original blog entry...

More Stories By Lori MacVittie

Lori MacVittie is responsible for education and evangelism of application services available across F5’s entire product suite. Her role includes authorship of technical materials and participation in a number of community-based forums and industry standards organizations, among other efforts. MacVittie has extensive programming experience as an application architect, as well as network and systems development and administration expertise. Prior to joining F5, MacVittie was an award-winning Senior Technology Editor at Network Computing Magazine, where she conducted product research and evaluation focused on integration with application and network architectures, and authored articles on a variety of topics aimed at IT professionals. Her most recent area of focus included SOA-related products and architectures. She holds a B.S. in Information and Computing Science from the University of Wisconsin at Green Bay, and an M.S. in Computer Science from Nova Southeastern University.

@CloudExpo Stories
"ciqada is a combined platform of hardware modules and server products that lets people take their existing devices or new devices and lets them be accessible over the Internet for their users," noted Geoff Engelstein of ciqada, a division of Mars International, in this SYS-CON.tv interview at @ThingsExpo, held June 9-11, 2015, at the Javits Center in New York City.
The time is ripe for high speed resilient software defined storage solutions with unlimited scalability. ISS has been working with the leading open source projects and developed a commercial high performance solution that is able to grow forever without performance limitations. In his session at Cloud Expo, Alex Gorbachev, President of Intelligent Systems Services Inc., shared foundation principles of Ceph architecture, as well as the design to deliver this storage to traditional SAN storage co...
Countless business models have spawned from the IaaS industry. Resell Web hosting, blogs, public cloud, and on and on. With the overwhelming amount of tools available to us, it's sometimes easy to overlook that many of them are just new skins of resources we've had for a long time. In his General Session at 16th Cloud Expo, Phil Jackson, Lead Technology Evangelist at SoftLayer, broke down what we've got to work with and discuss the benefits and pitfalls to discover how we can best use them to d...
"We have seen the evolution of WebRTC right from the starting point to what it has become today, that people are using in real applications," noted Dr. Natasha Tamaskar, Vice President and Head of Cloud and Mobile Strategy and Ecosystem at GENBAND, in this SYS-CON.tv interview at WebRTC Summit, held June 9-11, 2015, at the Javits Center in New York City.
Enterprises are turning to the hybrid cloud to drive greater scalability and cost-effectiveness. But enterprises should beware as the definition of “policy” varies wildly. Some say it’s the ability to control the resources apps’ use or where the apps run. Others view policy as governing the permissions and delivering security. Policy is all of that and more. In his session at 16th Cloud Expo, Derek Collison, founder and CEO of Apcera, explained what policy is, he showed how policy should be arch...
The 3rd International WebRTC Summit, to be held Nov. 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA, announces that its Call for Papers is now open. Topics include all aspects of improving IT delivery by eliminating waste through automated business models leveraging cloud technologies. WebRTC Summit is co-located with 15th International Cloud Expo, 6th International Big Data Expo, 3rd International DevOps Summit and 2nd Internet of @ThingsExpo. WebRTC (Web-based Real-Time Com...
Internet of Things (IoT) will be a hybrid ecosystem of diverse devices and sensors collaborating with operational and enterprise systems to create the next big application. In their session at @ThingsExpo, Bramh Gupta, founder and CEO of robomq.io, and Fred Yatzeck, principal architect leading product development at robomq.io, discussed how choosing the right middleware and integration strategy from the get-go will enable IoT solution developers to adapt and grow with the industry, while at th...
It is one thing to build single industrial IoT applications, but what will it take to build the Smart Cities and truly society-changing applications of the future? The technology won’t be the problem, it will be the number of parties that need to work together and be aligned in their motivation to succeed. In his session at @ThingsExpo, Jason Mondanaro, Director, Product Management at Metanga, discussed how you can plan to cooperate, partner, and form lasting all-star teams to change the world...
SYS-CON Events announced today that Harbinger Systems will exhibit at SYS-CON's 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Harbinger Systems is a global company providing software technology services. Since 1990, Harbinger has developed a strong customer base worldwide. Its customers include software product companies ranging from hi-tech start-ups in Silicon Valley to leading product companies in the US a...
The 4th International Internet of @ThingsExpo, co-located with the 17th International Cloud Expo - to be held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA - announces that its Call for Papers is open. The Internet of Things (IoT) is the biggest idea since the creation of the Worldwide Web more than
SYS-CON Events announced today that ProfitBricks, the provider of painless cloud infrastructure, will exhibit at SYS-CON's 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. ProfitBricks is the IaaS provider that offers a painless cloud experience for all IT users, with no learning curve. ProfitBricks boasts flexible cloud servers and networking, an integrated Data Center Designer tool for visual control over the...
"We got started as search consultants. On the services side of the business we have help organizations save time and save money when they hit issues that everyone more or less hits when their data grows," noted Otis Gospodnetić, Founder of Sematext, in this SYS-CON.tv interview at @DevOpsSummit, held June 9-11, 2015, at the Javits Center in New York City.
"CenturyLink brings a full suite of services to the table and that enables us to be an IT service provider," explained Jeff Katzen, Director of the Cloud Practice at CenturyLink, in this SYS-CON.tv interview at 16th Cloud Expo, held June 9-11, 2015, at the Javits Center in New York City.
17th Cloud Expo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterprises ar...
One of the hottest areas in cloud right now is DRaaS and related offerings. In his session at 16th Cloud Expo, Dale Levesque, Disaster Recovery Product Manager with Windstream's Cloud and Data Center Marketing team, will discuss the benefits of the cloud model, which far outweigh the traditional approach, and how enterprises need to ensure that their needs are properly being met.
In the midst of the widespread popularity and adoption of cloud computing, it seems like everything is being offered “as a Service” these days: Infrastructure? Check. Platform? You bet. Software? Absolutely. Toaster? It’s only a matter of time. With service providers positioning vastly differing offerings under a generic “cloud” umbrella, it’s all too easy to get confused about what’s actually being offered. In his session at 16th Cloud Expo, Kevin Hazard, Director of Digital Content for SoftL...
DevOps is about increasing efficiency, but nothing is more inefficient than building the same application twice. However, this is a routine occurrence with enterprise applications that need both a rich desktop web interface and strong mobile support. With recent technological advances from Isomorphic Software and others, it is now feasible to create a rich desktop and tuned mobile experience with a single codebase, without compromising performance or usability.
WebRTC converts the entire network into a ubiquitous communications cloud thereby connecting anytime, anywhere through any point. In his session at WebRTC Summit,, Mark Castleman, EIR at Bell Labs and Head of Future X Labs, will discuss how the transformational nature of communications is achieved through the democratizing force of WebRTC. WebRTC is doing for voice what HTML did for web content.
Discussions about cloud computing are evolving into discussions about enterprise IT in general. As enterprises increasingly migrate toward their own unique clouds, new issues such as the use of containers and microservices emerge to keep things interesting. In this Power Panel at 16th Cloud Expo, moderated by Conference Chair Roger Strukhoff, panelists addressed the state of cloud computing today, and what enterprise IT professionals need to know about how the latest topics and trends affect t...
Malicious agents are moving faster than the speed of business. Even more worrisome, most companies are relying on legacy approaches to security that are no longer capable of meeting current threats. In the modern cloud, threat diversity is rapidly expanding, necessitating more sophisticated security protocols than those used in the past or in desktop environments. Yet companies are falling for cloud security myths that were truths at one time but have evolved out of existence.