Welcome!

@CloudExpo Authors: Yeshim Deniz, Pat Romanski, Liz McMillan, Elizabeth White, Ed Featherston

Related Topics: @CloudExpo, Java IoT, Microservices Expo, Linux Containers, SDN Journal, @DevOpsSummit

@CloudExpo: Article

How to Choose Where Your Load Should Come From

The goal of a load test is to replicate the traffic & conditions your app experiences in production as realistically as possible

The goal of a load test is to replicate the traffic & conditions your app experiences in production as realistically as possible

As a tester, you understand how important it is to create the most realistic load test possible to provide confidence that your web application won't fail in the field. But how do you know where your load should come from to produce realistic results?

In this article, we have outlined three options most organizations can choose from when determining where your load should come from.

  1. Generate load on dedicated infrastructure in the datacenter
  2. Generate load from a private cloud
  3. Generate load from distributed public cloud locations

There is a time and a place for all three of these options. In this post we will go over when you should do each test and why.

1. Generate load on dedicated infrastructure in the datacenter

When: Perform internal load generation for early stage performance tests and tests on internal IT apps that will be accessed by users behind your firewall.

Why: Load testing from dedicated infrastructure inside your own datacenter is the most common and typically the most accessible way to wring out performance issues in your applications. This type of testing should be performed as part of your regular testing process.

Essentially what happens is you define specific load and performance tests that push on the individual modules and user paths within the system, and make sure that no unintended bottlenecks were introduced during development. The kinds of issues you'll work out with this localized, constrained testing include:

  • Inadequate transaction design
  • Broken links
  • Poor configuration
  • Memory leaks
  • Unoptimized code
  • Sub-optimal session model
  • Ineffective caching
  • Database indexing, connection, and concurrency issues

While performing this type of load test on dedicated equipment is easy, there are a few issues with using this approach as your sole means of load testing - the first of which is that many companies simply do not have adequate dedicated infrastructure resources to realistically test application performance at scale. The capital expense involved with acquiring and managing network equipment, servers, and applications for the sole purpose of performance testing can be hard to justify - especially when you consider that this infrastructure may sit idle for a significant portion of its life.

Second, load testing from anywhere inside your own datacenter environment will test only application performance. It will not test the full capabilities of the datacenter (including network configurations, load balancing, or security systems) and therefore will not produce a realistic picture of the user experience.

Running load tests from your internal environment is an important part of improving application performance, but it is not sufficient. You should integrate tests like those outlined above as a frequent part of your build & integration process. As you near the end of a release cycle, you'll have confidence that the application's functionality performs well under periods of high traffic.

2. Generate load from within a private cloud

When: Perform tests from a private cloud when you need to ramp up your load generation infrastructure quickly for larger tests.

Why: To address some of the issues above, many organizations are running load tests from within their private clouds. Operating on this type of infrastructure has several benefits when compared with the first approach that has been outlined.

Functionally, this method of testing is best suited to the same types of performance tests that you would execute using dedicated hardware: database optimization, transaction design, and the rest. However, private clouds address some of the problems with running on dedicated hardware.

First, a private cloud can be significantly less expensive to operate, specifically because the cloud infrastructure can be leveraged across many different applications. When you are done with your load test, virtual machines can be reassigned to complete other tasks such as analytics, development, or other cloud-friendly business processes. This allows an IT organization to distribute infrastructure costs, thus lowering the overall expense.

Second, this approach lends itself very easily to more realistic scaling. You can grab more cycles and more disk space as needed, ultimately driving a more aggressive stress test of the application.

Chances are, however, since you are testing from within your own network and environment, you are still not stressing the complete datacenter. As a result, this type of load generation approach won't necessarily give you an accurate picture of what your users will experience in periods of high traffic.

3. Generate load from distributed public cloud locations

When: Perform this test for when your app will be accessed primarily from users outside your firewall from distributed locations.

Why: By running your load tests from outside the datacenter in a public cloud or even a remote private cloud environment, you will get a good picture of how your datacenter infrastructure handles high traffic. This third type of load will test a part of the application's delivery that no amount of testing from your own datacenter can duplicate. This includes:

  • Load balancers
  • Networking equipment configuration
  • Data center connectivity and bandwidth
  • Web firewalls and security systems
  • In-network caching and content distribution
  • Network architecture
  • DNS, web latency, network congestion

It's important to focus your testing so it puts the load in the right place. For example, load testing firewall or load-balancing infrastructure. You may even choose to load test multiple applications that exist within the same data center at the same time to see if it chokes up for any reason when the load is applied.

Another major point that many testers don't often consider is the fact users are accessing your app from many different regions across different service providers. To get a realistic picture of the actual user experience, it is important to see what their experience looks like from their access points anywhere in the world. This will give you a better understanding of what happens to your web application when users from different parts of the world are flooding your website with traffic.

Prevent a Disaster, Don't Cause One
The ultimate goal of a load test is to replicate the traffic and conditions your app experiences in production as realistically as possible to give you confidence that your site will not fail at peak times with actual live users. By looking at the three major options of where your load should come from, it will give you a better idea of how your app will perform under stress.

No matter what, each of these tests will catch something different in your application. And we all should know by now, if you catch a bug earlier in the process you won't have to pull out the big bucks to fix a major disaster down the road. Prevent a disaster, don't cause one.

More Stories By Tim Hinds

Tim Hinds is the Product Marketing Manager for NeoLoad at Neotys. He has a background in Agile software development, Scrum, Kanban, Continuous Integration, Continuous Delivery, and Continuous Testing practices.

Previously, Tim was Product Marketing Manager at AccuRev, a company acquired by Micro Focus, where he worked with software configuration management, issue tracking, Agile project management, continuous integration, workflow automation, and distributed version control systems.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@CloudExpo Stories
JETRO showcased Japan Digital Transformation Pavilion at SYS-CON's 21st International Cloud Expo® at the Santa Clara Convention Center in Santa Clara, CA. The Japan External Trade Organization (JETRO) is a non-profit organization that provides business support services to companies expanding to Japan. With the support of JETRO's dedicated staff, clients can incorporate their business; receive visa, immigration, and HR support; find dedicated office space; identify local government subsidies; get...
Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more business becomes digital the more stakeholders are interested in this data including how it relates to business. Some of these people have never used a monitoring tool before. They have a question on their mind like “How is my application doing” but no id...
22nd International Cloud Expo, taking place June 5-7, 2018, at the Javits Center in New York City, NY, and co-located with the 1st DXWorld Expo will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud ...
"We're focused on how to get some of the attributes that you would expect from an Amazon, Azure, Google, and doing that on-prem. We believe today that you can actually get those types of things done with certain architectures available in the market today," explained Steve Conner, VP of Sales at Cloudistics, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
As organizations shift towards IT-as-a-service models, the need for managing and protecting data residing across physical, virtual, and now cloud environments grows with it. Commvault can ensure protection, access and E-Discovery of your data – whether in a private cloud, a Service Provider delivered public cloud, or a hybrid cloud environment – across the heterogeneous enterprise. In his general session at 18th Cloud Expo, Randy De Meno, Chief Technologist - Windows Products and Microsoft Part...
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
DXWorldEXPO LLC announced today that ICC-USA, a computer systems integrator and server manufacturing company focused on developing products and product appliances, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City. ICC is a computer systems integrator and server manufacturing company focused on developing products and product appliances to meet a wide range of ...
Business professionals no longer wonder if they'll migrate to the cloud; it's now a matter of when. The cloud environment has proved to be a major force in transitioning to an agile business model that enables quick decisions and fast implementation that solidify customer relationships. And when the cloud is combined with the power of cognitive computing, it drives innovation and transformation that achieves astounding competitive advantage.
Vulnerability management is vital for large companies that need to secure containers across thousands of hosts, but many struggle to understand how exposed they are when they discover a new high security vulnerability. In his session at 21st Cloud Expo, John Morello, CTO of Twistlock, addressed this pressing concern by introducing the concept of the “Vulnerability Risk Tree API,” which brings all the data together in a simple REST endpoint, allowing companies to easily grasp the severity of the ...
Cloud-enabled transformation has evolved from cost saving measure to business innovation strategy -- one that combines the cloud with cognitive capabilities to drive market disruption. Learn how you can achieve the insight and agility you need to gain a competitive advantage. Industry-acclaimed CTO and cloud expert, Shankar Kalyana presents. Only the most exceptional IBMers are appointed with the rare distinction of IBM Fellow, the highest technical honor in the company. Shankar has also receive...
Michael Maximilien, better known as max or Dr. Max, is a computer scientist with IBM. At IBM Research Triangle Park, he was a principal engineer for the worldwide industry point-of-sale standard: JavaPOS. At IBM Research, some highlights include pioneering research on semantic Web services, mashups, and cloud computing, and platform-as-a-service. He joined the IBM Cloud Labs in 2014 and works closely with Pivotal Inc., to help make the Cloud Found the best PaaS.
We all know that end users experience the Internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices – not doing so will be a path to eventual b...
We all know that end users experience the internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices - not doing so will be a path to eventual ...
We all know that end users experience the internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices - not doing so will be a path to eventual ...
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, led attendees through the exciting evolution of the cloud. He looked at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering m...
DXWorldEXPO LLC announced today that All in Mobile, a mobile app development company from Poland, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. All In Mobile is a mobile app development company from Poland. Since 2014, they maintain passion for developing mobile applications for enterprises and startups worldwide.
Sanjeev Sharma Joins November 11-13, 2018 @DevOpsSummit at @CloudEXPO New York Faculty. Sanjeev Sharma is an internationally known DevOps and Cloud Transformation thought leader, technology executive, and author. Sanjeev's industry experience includes tenures as CTO, Technical Sales leader, and Cloud Architect leader. As an IBM Distinguished Engineer, Sanjeev is recognized at the highest levels of IBM's core of technical leaders.
We are seeing a major migration of enterprises applications to the cloud. As cloud and business use of real time applications accelerate, legacy networks are no longer able to architecturally support cloud adoption and deliver the performance and security required by highly distributed enterprises. These outdated solutions have become more costly and complicated to implement, install, manage, and maintain.SD-WAN offers unlimited capabilities for accessing the benefits of the cloud and Internet. ...
As Cybric's Chief Technology Officer, Mike D. Kail is responsible for the strategic vision and technical direction of the platform. Prior to founding Cybric, Mike was Yahoo's CIO and SVP of Infrastructure, where he led the IT and Data Center functions for the company. He has more than 24 years of IT Operations experience with a focus on highly-scalable architectures.
Headquartered in Plainsboro, NJ, Synametrics Technologies has provided IT professionals and computer systems developers since 1997. Based on the success of their initial product offerings (WinSQL and DeltaCopy), the company continues to create and hone innovative products that help its customers get more from their computer applications, databases and infrastructure. To date, over one million users around the world have chosen Synametrics solutions to help power their accelerated business or per...
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.