Welcome!

@CloudExpo Authors: Pat Romanski, Yeshim Deniz, Charles Araujo, Kevin Benedict, Liz McMillan

Related Topics: @CloudExpo, Microservices Expo

@CloudExpo: Blog Post

Load Balancing in a Cloud

How should auto-scaling work, and why doesn’t it?

Although “rapid elasticity” is part of NIST’s definition of cloud computing, it may be interesting to note that many cloud computing environments don’t include this capability at all – or charge you extra for it. Many providers offer the means by which you can configure a load balancing service and manually add or remove instances, but there may not be a way to automate that process. If it’s manual, it’s certain “rapid” in the sense that’s it’s probably faster than you can do it (because you’d have to acquire hardware and deploy the application, as opposed to simply hitting a button and “cloning” the environment ‘out there’) but it’s not necessarily as fast as it could be (because it’s manual) nor is it automated. There’s a number of reasons for that – but we’ll get to that later. First, let’s look at how auto-scaling is supposed to work, at least theoretically.

image

1. Some external entity that is monitoring capacity for “Application A” triggers an event that indicates a new instance is required to maintain availability. This external entity could be an APM solution, the cloud management console, or a custom developed application. How it determines capacity limitations is variable and might be based solely on VM status (via VMware APIs), data received from the load balancer, or a combination thereof.

2. A new instance is launched. This is accomplished via the cloud management console or an external API as part of a larger workflow/orchestration.

3. The external entity grabs the IP address of the newly launched instance and instructs the load balancer to add it to the pool for resources for “Application A.” This is accomplished via the standards-based API which presents the configuration and management control plane of the load balancer to external consumers as services.

4. The load balancer adds the new application instance to the appropriate pool and as soon as it has confirmation that the instance is available and responding to requests, begins to direct traffic to the instance.

This process is easily reversed upon termination of an instance. Note: there are other infrastructure components that are involved in this process that must also be notified on launch and decommission, but for this discussion we’re just looking at the load balancing piece as it’s critical to the concept of auto-scaling.

The important thing to note here is that the process for adding – and subsequently using – the new application instance to the load balancer should be automatic. Once the instance is  launched, all that remains is to inform the load balancer that a new instance is available. The load balancer should take care of the rest, and it should do so transparently. That means no interruption of service, no reboot, no reload the configuration, nothing. Similarly the termination and removal of the application instance from the pool of resources should be as seamless and non-disruptive. It does not matter whether the load balancing solution is hardware or a VNA (Virtual Network Appliance) as long as it has an API through which available resources can be added and removed on-demand.

This process is relatively straightforward and can even be accomplished without the “external entity” by directly integrating the application with the load balancing service. By hooking the appropriate setup and tear-down routines in imagethe application, the load balancer can be automatically instructed to perform the appropriate actions whenever an instance of that application is launched or terminated.

1. Application is initialized and instructs load balancer to add it to the appropriate resource pool.

2. Load balancer adds the application instance to the right pool of resources and begins directing traffic to it as configured.

3. Application is terminating, instructs load balancer to remove it from the available resource pool.

4. Load balancer removes the application instance from pool of resources.

This requires tighter coupling of the load balancer with the application, however, which may be less than ideal unless you are very tied to your load balancing solution. The advantage of this solution is that regardless of whether the application is “physical” or “virtual”, the same process occurs, which lets you mix and match and be a lot more flexible in your architecture.


WHY is THIS so HARD THEN?

It’s not, if the load balancer has an API. Herein lies the problem: building out an automated, dynamic infrastructure requires the ability to communicate, to collaborate, to integrate network and application network infrastructure with the data center management system. That means some sort of API is required. Oh, you could do it with a little SSH and a local script, but that’s not nearly as efficient or flexible as using an API that’s designed to be leveraged for integration and automation in the first place.

While just about every hardware load balancer (and their virtualized equivalents) is enabled with a standards-based control plane, most web-server-turned-rudimentary-load-balancers  are not enabled with an API that allows this level of integration. That means hacking the system with scripts that inject or remove configurations, forcing reloading of the configuration (or worse, stopping and starting the system) and most likely interrupting service. Not only does it mean hacking the system but the scripts must somehow pull from customer-specific meta-data information in order to properly configure the load balancing solution.

 

 

So in order to achieve this in a way that maintains availability and is easy to use and essentially a multi-tenant solution needs the services of an API. When it comes to integration there are few options and the most flexible method is to use a standards-based API to manage a solution that supports the notion of multi-tenant configuration natively. The problem is, of course, that most of the API-enabled (and therefore easy to integrate and automate) load balancers cost money. It requires an investment, up front, and many folks just aren’t willing to do that. Except they are doing that; they’re just trading capital expenses for longer-term development of scripts and research that approximate (but never quite reach the reliability) of an API that’s been specifically developed, tested, and deployed for the purpose of integrating with external management systems and scripts. What’s worse is that eventually most of these solutions developed on sweat and scripts will end up turning to a more sophisticated, proven API-based solution anyway because of the inherent challenges associated with scalability of the solution itself. That results in a financially inefficient way to go about building a dynamic infrastructure that supports rapid, elastic scalability.

So many organizations faced with the choice of investing in a more robust, flexible solution when they’ve already sunk a lot of time, effort, and money into their existing “free” solution, simply continue to hack and paste and shove a square peg into a round hole, which means auto-scaling never works quite as expected and advanced features come at a snail’s pace because of the difficulty in scripting and integration.

It doesn’t have to be so hard, but you do have to view load balancing for what it is: a key component of the cloud computing strategy, and give it the focus it deserves, even if that means it’ll cost you more up front because in the end it’ll cost you far less than the alternative.

Read the original blog entry...

More Stories By Lori MacVittie

Lori MacVittie is responsible for education and evangelism of application services available across F5’s entire product suite. Her role includes authorship of technical materials and participation in a number of community-based forums and industry standards organizations, among other efforts. MacVittie has extensive programming experience as an application architect, as well as network and systems development and administration expertise. Prior to joining F5, MacVittie was an award-winning Senior Technology Editor at Network Computing Magazine, where she conducted product research and evaluation focused on integration with application and network architectures, and authored articles on a variety of topics aimed at IT professionals. Her most recent area of focus included SOA-related products and architectures. She holds a B.S. in Information and Computing Science from the University of Wisconsin at Green Bay, and an M.S. in Computer Science from Nova Southeastern University.

@CloudExpo Stories
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...
As Cybric's Chief Technology Officer, Mike D. Kail is responsible for the strategic vision and technical direction of the platform. Prior to founding Cybric, Mike was Yahoo's CIO and SVP of Infrastructure, where he led the IT and Data Center functions for the company. He has more than 24 years of IT Operations experience with a focus on highly-scalable architectures.
Headquartered in Plainsboro, NJ, Synametrics Technologies has provided IT professionals and computer systems developers since 1997. Based on the success of their initial product offerings (WinSQL and DeltaCopy), the company continues to create and hone innovative products that help its customers get more from their computer applications, databases and infrastructure. To date, over one million users around the world have chosen Synametrics solutions to help power their accelerated business or per...
We are seeing a major migration of enterprises applications to the cloud. As cloud and business use of real time applications accelerate, legacy networks are no longer able to architecturally support cloud adoption and deliver the performance and security required by highly distributed enterprises. These outdated solutions have become more costly and complicated to implement, install, manage, and maintain.SD-WAN offers unlimited capabilities for accessing the benefits of the cloud and Internet. ...
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
In an era of historic innovation fueled by unprecedented access to data and technology, the low cost and risk of entering new markets has leveled the playing field for business. Today, any ambitious innovator can easily introduce a new application or product that can reinvent business models and transform the client experience. In their Day 2 Keynote at 19th Cloud Expo, Mercer Rowe, IBM Vice President of Strategic Alliances, and Raejeanne Skillern, Intel Vice President of Data Center Group and ...
Founded in 2000, Chetu Inc. is a global provider of customized software development solutions and IT staff augmentation services for software technology providers. By providing clients with unparalleled niche technology expertise and industry experience, Chetu has become the premiere long-term, back-end software development partner for start-ups, SMBs, and Fortune 500 companies. Chetu is headquartered in Plantation, Florida, with thirteen offices throughout the U.S. and abroad.
Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: Driving Business Strategies with Data Science," is responsible for setting the strategy and defining the Big Data service offerings and capabilities for EMC Global Services Big Data Practice. As the CTO for the Big Data Practice, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He's written several white papers, is an avid blogge...
DXWorldEXPO LLC announced today that Dez Blanchfield joined the faculty of CloudEXPO's "10-Year Anniversary Event" which will take place on November 11-13, 2018 in New York City. Dez is a strategic leader in business and digital transformation with 25 years of experience in the IT and telecommunications industries developing strategies and implementing business initiatives. He has a breadth of expertise spanning technologies such as cloud computing, big data and analytics, cognitive computing, m...
Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: Driving Business Strategies with Data Science," is responsible for setting the strategy and defining the Big Data service offerings and capabilities for EMC Global Services Big Data Practice. As the CTO for the Big Data Practice, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He's written several white papers, is an avid blogge...
HyperConvergence came to market with the objective of being simple, flexible and to help drive down operating expenses. It reduced the footprint by bundling the compute/storage/network into one box. This brought a new set of challenges as the HyperConverged vendors are very focused on their own proprietary building blocks. If you want to scale in a certain way, let's say you identified a need for more storage and want to add a device that is not sold by the HyperConverged vendor, forget about it...
Historically, some banking activities such as trading have been relying heavily on analytics and cutting edge algorithmic tools. The coming of age of powerful data analytics solutions combined with the development of intelligent algorithms have created new opportunities for financial institutions. In his session at 20th Cloud Expo, Sebastien Meunier, Head of Digital for North America at Chappuis Halder & Co., discussed how these tools can be leveraged to develop a lasting competitive advantage ...
In his session at 21st Cloud Expo, James Henry, Co-CEO/CTO of Calgary Scientific Inc., introduced you to the challenges, solutions and benefits of training AI systems to solve visual problems with an emphasis on improving AIs with continuous training in the field. He explored applications in several industries and discussed technologies that allow the deployment of advanced visualization solutions to the cloud.
As you move to the cloud, your network should be efficient, secure, and easy to manage. An enterprise adopting a hybrid or public cloud needs systems and tools that provide: Agility: ability to deliver applications and services faster, even in complex hybrid environments Easier manageability: enable reliable connectivity with complete oversight as the data center network evolves Greater efficiency: eliminate wasted effort while reducing errors and optimize asset utilization Security: implemen...
From 2013, NTT Communications has been providing cPaaS service, SkyWay. Its customer’s expectations for leveraging WebRTC technology are not only typical real-time communication use cases such as Web conference, remote education, but also IoT use cases such as remote camera monitoring, smart-glass, and robotic. Because of this, NTT Communications has numerous IoT business use-cases that its customers are developing on top of PaaS. WebRTC will lead IoT businesses to be more innovative and address...
Charles Araujo is an industry analyst, internationally recognized authority on the Digital Enterprise and author of The Quantum Age of IT: Why Everything You Know About IT is About to Change. As Principal Analyst with Intellyx, he writes, speaks and advises organizations on how to navigate through this time of disruption. He is also the founder of The Institute for Digital Transformation and a sought after keynote speaker. He has been a regular contributor to both InformationWeek and CIO Insight...
In this presentation, you will learn first hand what works and what doesn't while architecting and deploying OpenStack. Some of the topics will include:- best practices for creating repeatable deployments of OpenStack- multi-site considerations- how to customize OpenStack to integrate with your existing systems and security best practices.
Your homes and cars can be automated and self-serviced. Why can't your storage? From simply asking questions to analyze and troubleshoot your infrastructure, to provisioning storage with snapshots, recovery and replication, your wildest sci-fi dream has come true. In his session at @DevOpsSummit at 20th Cloud Expo, Dan Florea, Director of Product Management at Tintri, provided a ChatOps demo where you can talk to your storage and manage it from anywhere, through Slack and similar services with...
Evan Kirstel is an internationally recognized thought leader and social media influencer in IoT (#1 in 2017), Cloud, Data Security (2016), Health Tech (#9 in 2017), Digital Health (#6 in 2016), B2B Marketing (#5 in 2015), AI, Smart Home, Digital (2017), IIoT (#1 in 2017) and Telecom/Wireless/5G. His connections are a "Who's Who" in these technologies, He is in the top 10 most mentioned/re-tweeted by CMOs and CIOs (2016) and have been recently named 5th most influential B2B marketeer in the US. H...
Gemini is Yahoo’s native and search advertising platform. To ensure the quality of a complex distributed system that spans multiple products and components and across various desktop websites and mobile app and web experiences – both Yahoo owned and operated and third-party syndication (supply), with complex interaction with more than a billion users and numerous advertisers globally (demand) – it becomes imperative to automate a set of end-to-end tests 24x7 to detect bugs and regression. In th...
"With Digital Experience Monitoring what used to be a simple visit to a web page has exploded into app on phones, data from social media feeds, competitive benchmarking - these are all components that are only available because of some type of digital asset," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.