@CloudExpo Authors: Yeshim Deniz, Elizabeth White, Jason Bloomberg, Carmen Gonzalez, Pat Romanski

Related Topics: @CloudExpo, Microservices Expo

@CloudExpo: Blog Post

Load Balancing in a Cloud

How should auto-scaling work, and why doesn’t it?

Although “rapid elasticity” is part of NIST’s definition of cloud computing, it may be interesting to note that many cloud computing environments don’t include this capability at all – or charge you extra for it. Many providers offer the means by which you can configure a load balancing service and manually add or remove instances, but there may not be a way to automate that process. If it’s manual, it’s certain “rapid” in the sense that’s it’s probably faster than you can do it (because you’d have to acquire hardware and deploy the application, as opposed to simply hitting a button and “cloning” the environment ‘out there’) but it’s not necessarily as fast as it could be (because it’s manual) nor is it automated. There’s a number of reasons for that – but we’ll get to that later. First, let’s look at how auto-scaling is supposed to work, at least theoretically.


1. Some external entity that is monitoring capacity for “Application A” triggers an event that indicates a new instance is required to maintain availability. This external entity could be an APM solution, the cloud management console, or a custom developed application. How it determines capacity limitations is variable and might be based solely on VM status (via VMware APIs), data received from the load balancer, or a combination thereof.

2. A new instance is launched. This is accomplished via the cloud management console or an external API as part of a larger workflow/orchestration.

3. The external entity grabs the IP address of the newly launched instance and instructs the load balancer to add it to the pool for resources for “Application A.” This is accomplished via the standards-based API which presents the configuration and management control plane of the load balancer to external consumers as services.

4. The load balancer adds the new application instance to the appropriate pool and as soon as it has confirmation that the instance is available and responding to requests, begins to direct traffic to the instance.

This process is easily reversed upon termination of an instance. Note: there are other infrastructure components that are involved in this process that must also be notified on launch and decommission, but for this discussion we’re just looking at the load balancing piece as it’s critical to the concept of auto-scaling.

The important thing to note here is that the process for adding – and subsequently using – the new application instance to the load balancer should be automatic. Once the instance is  launched, all that remains is to inform the load balancer that a new instance is available. The load balancer should take care of the rest, and it should do so transparently. That means no interruption of service, no reboot, no reload the configuration, nothing. Similarly the termination and removal of the application instance from the pool of resources should be as seamless and non-disruptive. It does not matter whether the load balancing solution is hardware or a VNA (Virtual Network Appliance) as long as it has an API through which available resources can be added and removed on-demand.

This process is relatively straightforward and can even be accomplished without the “external entity” by directly integrating the application with the load balancing service. By hooking the appropriate setup and tear-down routines in imagethe application, the load balancer can be automatically instructed to perform the appropriate actions whenever an instance of that application is launched or terminated.

1. Application is initialized and instructs load balancer to add it to the appropriate resource pool.

2. Load balancer adds the application instance to the right pool of resources and begins directing traffic to it as configured.

3. Application is terminating, instructs load balancer to remove it from the available resource pool.

4. Load balancer removes the application instance from pool of resources.

This requires tighter coupling of the load balancer with the application, however, which may be less than ideal unless you are very tied to your load balancing solution. The advantage of this solution is that regardless of whether the application is “physical” or “virtual”, the same process occurs, which lets you mix and match and be a lot more flexible in your architecture.


It’s not, if the load balancer has an API. Herein lies the problem: building out an automated, dynamic infrastructure requires the ability to communicate, to collaborate, to integrate network and application network infrastructure with the data center management system. That means some sort of API is required. Oh, you could do it with a little SSH and a local script, but that’s not nearly as efficient or flexible as using an API that’s designed to be leveraged for integration and automation in the first place.

While just about every hardware load balancer (and their virtualized equivalents) is enabled with a standards-based control plane, most web-server-turned-rudimentary-load-balancers  are not enabled with an API that allows this level of integration. That means hacking the system with scripts that inject or remove configurations, forcing reloading of the configuration (or worse, stopping and starting the system) and most likely interrupting service. Not only does it mean hacking the system but the scripts must somehow pull from customer-specific meta-data information in order to properly configure the load balancing solution.



So in order to achieve this in a way that maintains availability and is easy to use and essentially a multi-tenant solution needs the services of an API. When it comes to integration there are few options and the most flexible method is to use a standards-based API to manage a solution that supports the notion of multi-tenant configuration natively. The problem is, of course, that most of the API-enabled (and therefore easy to integrate and automate) load balancers cost money. It requires an investment, up front, and many folks just aren’t willing to do that. Except they are doing that; they’re just trading capital expenses for longer-term development of scripts and research that approximate (but never quite reach the reliability) of an API that’s been specifically developed, tested, and deployed for the purpose of integrating with external management systems and scripts. What’s worse is that eventually most of these solutions developed on sweat and scripts will end up turning to a more sophisticated, proven API-based solution anyway because of the inherent challenges associated with scalability of the solution itself. That results in a financially inefficient way to go about building a dynamic infrastructure that supports rapid, elastic scalability.

So many organizations faced with the choice of investing in a more robust, flexible solution when they’ve already sunk a lot of time, effort, and money into their existing “free” solution, simply continue to hack and paste and shove a square peg into a round hole, which means auto-scaling never works quite as expected and advanced features come at a snail’s pace because of the difficulty in scripting and integration.

It doesn’t have to be so hard, but you do have to view load balancing for what it is: a key component of the cloud computing strategy, and give it the focus it deserves, even if that means it’ll cost you more up front because in the end it’ll cost you far less than the alternative.

Read the original blog entry...

More Stories By Lori MacVittie

Lori MacVittie is responsible for education and evangelism of application services available across F5’s entire product suite. Her role includes authorship of technical materials and participation in a number of community-based forums and industry standards organizations, among other efforts. MacVittie has extensive programming experience as an application architect, as well as network and systems development and administration expertise. Prior to joining F5, MacVittie was an award-winning Senior Technology Editor at Network Computing Magazine, where she conducted product research and evaluation focused on integration with application and network architectures, and authored articles on a variety of topics aimed at IT professionals. Her most recent area of focus included SOA-related products and architectures. She holds a B.S. in Information and Computing Science from the University of Wisconsin at Green Bay, and an M.S. in Computer Science from Nova Southeastern University.

@CloudExpo Stories
SYS-CON Events announced today that Sheng Liang to Keynote at SYS-CON's 19th Cloud Expo, which will take place on November 1-3, 2016 at the Santa Clara Convention Center in Santa Clara, California.
"Matrix is an ambitious open standard and implementation that's set up to break down the fragmentation problems that exist in IP messaging and VoIP communication," explained John Woolf, Technical Evangelist at Matrix, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Effectively SMBs and government programs must address compounded regulatory compliance requirements. The most recent are Controlled Unclassified Information and the EU’s GDPR have Board Level implications. Managing sensitive data protection will likely result in acquisition criteria, demonstration requests and new requirements. Developers, as part of the pre-planning process and the associated supply chain, could benefit from updating their code libraries and design by incorporating changes.
@ThingsExpo has been named the Top 5 Most Influential Internet of Things Brand by Onalytica in the ‘The Internet of Things Landscape 2015: Top 100 Individuals and Brands.' Onalytica analyzed Twitter conversations around the #IoT debate to uncover the most influential brands and individuals driving the conversation. Onalytica captured data from 56,224 users. The PageRank based methodology they use to extract influencers on a particular topic (tweets mentioning #InternetofThings or #IoT in this ...
What are the new priorities for the connected business? First: businesses need to think differently about the types of connections they will need to make – these span well beyond the traditional app to app into more modern forms of integration including SaaS integrations, mobile integrations, APIs, device integration and Big Data integration. It’s important these are unified together vs. doing them all piecemeal. Second, these types of connections need to be simple to design, adapt and configure...
DevOps is being widely accepted (if not fully adopted) as essential in enterprise IT. But as Enterprise DevOps gains maturity, expands scope, and increases velocity, the need for data-driven decisions across teams becomes more acute. DevOps teams in any modern business must wrangle the ‘digital exhaust’ from the delivery toolchain, "pervasive" and "cognitive" computing, APIs and services, mobile devices and applications, the Internet of Things, and now even blockchain. In this power panel at @...
November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Penta Security is a leading vendor for data security solutions, including its encryption solution, D’Amo. By using FPE technology, D’Amo allows for the implementation of encryption technology to sensitive data fields without modification to schema in the database environment. With businesses having their data become increasingly more complicated in their mission-critical applications (such as ERP, CRM, HRM), continued ...
The IoT has the potential to create a renaissance of manufacturing in the US and elsewhere. In his session at 18th Cloud Expo, Florent Solt, CTO and chief architect of Netvibes, discussed how the expected exponential increase in the amount of data that will be processed, transported, stored, and accessed means there will be a huge demand for smart technologies to deliver it. Florent Solt is the CTO and chief architect of Netvibes. Prior to joining Netvibes in 2007, he co-founded Rift Technologi...
SYS-CON Events announced today that Streamlyzer will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Streamlyzer is a powerful analytics for video streaming service that enables video streaming providers to monitor and analyze QoE (Quality-of-Experience) from end-user devices in real time.
@DevOpsSummit has been named the ‘Top DevOps Influencer' by iTrend. iTrend processes millions of conversations, tweets, interactions, news articles, press releases, blog posts - and extract meaning form them and analyzes mobile and desktop software platforms used to communicate, various metadata (such as geo location), and automation tools. In overall placement, @DevOpsSummit ranked as the number one ‘DevOps Influencer' followed by @CloudExpo at third, and @MicroservicesE at 24th.
@ThingsExpo has been named the Top 5 Most Influential M2M Brand by Onalytica in the ‘Machine to Machine: Top 100 Influencers and Brands.' Onalytica analyzed the online debate on M2M by looking at over 85,000 tweets to provide the most influential individuals and brands that drive the discussion. According to Onalytica the "analysis showed a very engaged community with a lot of interactive tweets. The M2M discussion seems to be more fragmented and driven by some of the major brands present in the...
Established in 1998, Calsoft is a leading software product engineering Services Company specializing in Storage, Networking, Virtualization and Cloud business verticals. Calsoft provides End-to-End Product Development, Quality Assurance Sustenance, Solution Engineering and Professional Services expertise to assist customers in achieving their product development and business goals. The company's deep domain knowledge of Storage, Virtualization, Networking and Cloud verticals helps in delivering ...
Most of us already know that adopting new cloud applications can boost a business’s productivity by enabling organizations to be more agile and ready to change course in our fast-moving and connected digital world. But the rapid adoption of cloud apps and services also brings with it profound security threats, including visibility and control challenges that aren’t present in traditional on-premises environments. At the same time, the cloud – because of its interconnected, flexible and adaptable...
Explosive growth in connected devices. Enormous amounts of data for collection and analysis. Critical use of data for split-second decision making and actionable information. All three are factors in making the Internet of Things a reality. Yet, any one factor would have an IT organization pondering its infrastructure strategy. How should your organization enhance its IT framework to enable an Internet of Things implementation? In his session at @ThingsExpo, James Kirkland, Red Hat's Chief Arch...
Extreme Computing is the ability to leverage highly performant infrastructure and software to accelerate Big Data, machine learning, HPC, and Enterprise applications. High IOPS Storage, low-latency networks, in-memory databases, GPUs and other parallel accelerators are being used to achieve faster results and help businesses make better decisions. In his session at 18th Cloud Expo, Michael O'Neill, Strategic Business Development at NVIDIA, focused on some of the unique ways extreme computing is...
Virgil consists of an open-source encryption library, which implements Cryptographic Message Syntax (CMS) and Elliptic Curve Integrated Encryption Scheme (ECIES) (including RSA schema), a Key Management API, and a cloud-based Key Management Service (Virgil Keys). The Virgil Keys Service consists of a public key service and a private key escrow service. 

SYS-CON Media announced today that @WebRTCSummit Blog, the largest WebRTC resource in the world, has been launched. @WebRTCSummit Blog offers top articles, news stories, and blog posts from the world's well-known experts and guarantees better exposure for its authors than any other publication. @WebRTCSummit Blog can be bookmarked ▸ Here @WebRTCSummit conference site can be bookmarked ▸ Here
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, will discuss the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docke...
You have great SaaS business app ideas. You want to turn your idea quickly into a functional and engaging proof of concept. You need to be able to modify it to meet customers' needs, and you need to deliver a complete and secure SaaS application. How could you achieve all the above and yet avoid unforeseen IT requirements that add unnecessary cost and complexity? You also want your app to be responsive in any device at any time. In his session at 19th Cloud Expo, Mark Allen, General Manager of...
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, provided an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life sett...