Welcome!

@CloudExpo Authors: Elizabeth White, Pat Romanski, Yeshim Deniz, Liz McMillan, Aruna Ravichandran

Related Topics: @CloudExpo, Java IoT, Microservices Expo

@CloudExpo: Article

Dos & Don’ts of Load Testing in the Cloud By @Neotys | @CloudExpo [#Cloud]

The cloud allows you to achieve a degree of realism with your performance and load testing

As the cloud continues to grow as a topic of conversation, so too does load and performance testing within the cloud. You may be asking yourself questions like: When is it most appropriate to involve the cloud when load testing? When is the right time to conduct performance testing using the cloud? How can cloud load testing benefit my company?

What should I do and what shouldn't I do?

This week at Neotys, we've decided to briefly cover a few benefits of load testing in the cloud, but also want to unearth what you should consider doing (or not doing) in your future cloud testing projects.

Use The Cloud for the Most Realistic Load Tests
The cloud allows you to achieve a degree of realism with your performance and load testing that is virtually impossible to achieve any other way. That gives any web or mobile application that has gone through a cycle of cloud-based performance testing a distinct advantage. The cloud provides for a geographical distribution of simulated users that mimics your real-world users. It also provides for almost limitless scaling of load generators. Cloud-based performance testing also lets you easily test the entire delivery chain, including your outermost firewalls and load balancers. For on-site testing, these types of capabilities may be difficult to acquire.

The cloud is also more cost-effective and time-friendly. On-premise testing often requires an administrative team to set up, maintain and manage the load testing infrastructure. Of course, you'll need to buy and maintain actual machines, whether or not you use them on a regular basis. Cloud testing reduces a lot of red tape. You can set up the testing infrastructure as you need it and pay as you go.

On-Premise Load Testing Still Has a Purpose
But don't throw away your internal testing apparatus - by any means. You definitely benefit from both types of testing. Your on-premise performance testing systems are much better suited for determining baseline performance. Plus, they should be your go-to platform for troubleshooting an easily-reproduced bottleneck, or testing internal componentry. The trick is knowing when to use your own systems, and when to use the cloud.

Cloud load testing is an essential part of your load testing strategy, and we at Neotys feel your pain as it can be difficult to know exactly how to best deploy it. To help make things clearer, we've decided to compile a brief list of do's and don'ts on cloud performance testing. Hopefully, it un-clouds your opinion on load testing procedure!

Indispensable Do's and Don'ts for Cloud Load Testing


DO DON'T
DO distribute load generators geographically, emulating where your site users are actually from. This creates more realistic tests, which give you better insight into any performance limitations of your web applications. DON'T simply replicate your local load generation setup in the cloud somewhere. You'll need the enhanced capabilities that come with a integrated cloud platform. NeoLoad offers an on-demand, easy setup to save you some headaches.
DO separate your load generators from your application. Serve them up from completely different environments. DON'T run your load generators in the same cloud as that behavior is almost guaranteed to contaminate test results.
DO maximize the benefits of using the cloud where it works best, and internal systems where they work best. The cloud is ideal for simulating real usage scenarios and running complex tests with many transactions. Once you have isolated the problem and can reproduce it effectively, switching to a local system may be a more efficient way of squashing the bug. DON'T eliminate your local testing infrastructure. As noted earlier, it still has an important purpose. You will need it to isolate effects from infrastructures beyond your firewall and identify solitary sources of performance slow-downs.
DO maintain a central dashboard for all the testing activities. When you move to the cloud you can have tests running all over the place and you definitely don't want to lose track of them. Make sure all your testing is coordinated and keep track of it through a good, centralized platform. DON'T ignore the meter! If you are using the cloud, you are probably paying for usage (that includes CPU cycles and network bandwidth). A long-running, high traffic load test could end up costing more than you expected, if you aren't careful.
DO use multiple cloud vendors. Different services have different characteristics. By using a wide variety, you'll end up with more realistic usage scenarios. NeoLoad's cloud testing platform allows you to access a range of cloud vendors, while avoiding the hassle of dealing with multiple providers. DON'T procrastinate on keeping your tests up-to-date. When using multiple cloud vendors, it's possible to lose track of testing scenarios and application versions. Keep things synchronized between all the environments used and your latest versions in development, QA, and production.
DO integrate mobile testing. Make sure your cloud generators are using mobile emulators and exercising the parts of your app that are suited for the small screen. Make sure the right information gets translated from the correct device while testing, and analyze the effects of network variances and bandwidths. DON'T ignore your perimeter. When you test from the cloud, you are setting up a geographically realistic scenario that includes important perimeter elements like content distribution networks, firewalls, perimeter security, and load balancers. You'll want to monitor the performance of all these levels of technology, in addition to your court application. Remember, you never know exactly where bottlenecks may occur.

What's the Best Way to Maximize the Capabilities of Cloud Testing?
These do's and don'ts should make getting started with cloud load testing much easier, if you are doing it on your own. However, if you are looking for a great out-of-the-box solution, NeoLoad offers maximum flexibility, scalability and cost effectiveness. You'll be able to monitor performance in real time as well as ensure that your applications are meeting minimum performance levels. Quickly identify the root cause of any issue with automated reports and analytics tools. This range of capabilities provides you with the opportunity to utilize the cloud for load and performance testing in a way that truly suits and benefits your company.

More Stories By Tim Hinds

Tim Hinds is the Product Marketing Manager for NeoLoad at Neotys. He has a background in Agile software development, Scrum, Kanban, Continuous Integration, Continuous Delivery, and Continuous Testing practices.

Previously, Tim was Product Marketing Manager at AccuRev, a company acquired by Micro Focus, where he worked with software configuration management, issue tracking, Agile project management, continuous integration, workflow automation, and distributed version control systems.

@CloudExpo Stories
As you move to the cloud, your network should be efficient, secure, and easy to manage. An enterprise adopting a hybrid or public cloud needs systems and tools that provide: Agility: ability to deliver applications and services faster, even in complex hybrid environments Easier manageability: enable reliable connectivity with complete oversight as the data center network evolves Greater efficiency: eliminate wasted effort while reducing errors and optimize asset utilization Security: imple...
High-velocity engineering teams are applying not only continuous delivery processes, but also lessons in experimentation from established leaders like Amazon, Netflix, and Facebook. These companies have made experimentation a foundation for their release processes, allowing them to try out major feature releases and redesigns within smaller groups before making them broadly available. In his session at 21st Cloud Expo, Brian Lucas, Senior Staff Engineer at Optimizely, will discuss how by using...
In this strange new world where more and more power is drawn from business technology, companies are effectively straddling two paths on the road to innovation and transformation into digital enterprises. The first path is the heritage trail – with “legacy” technology forming the background. Here, extant technologies are transformed by core IT teams to provide more API-driven approaches. Legacy systems can restrict companies that are transitioning into digital enterprises. To truly become a lead...
The session is centered around the tracing of systems on cloud using technologies like ebpf. The goal is to talk about what this technology is all about and what purpose it serves. In his session at 21st Cloud Expo, Shashank Jain, Development Architect at SAP, will touch upon concepts of observability in the cloud and also some of the challenges we have. Generally most cloud-based monitoring tools capture details at a very granular level. To troubleshoot problems this might not be good enough.
Companies are harnessing data in ways we once associated with science fiction. Analysts have access to a plethora of visualization and reporting tools, but considering the vast amount of data businesses collect and limitations of CPUs, end users are forced to design their structures and systems with limitations. Until now. As the cloud toolkit to analyze data has evolved, GPUs have stepped in to massively parallel SQL, visualization and machine learning.
SYS-CON Events announced today that CAST Software will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CAST was founded more than 25 years ago to make the invisible visible. Built around the idea that even the best analytics on the market still leave blind spots for technical teams looking to deliver better software and prevent outages, CAST provides the software intelligence that matter ...
SYS-CON Events announced today that Daiya Industry will exhibit at the Japanese Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Ruby Development Inc. builds new services in short period of time and provides a continuous support of those services based on Ruby on Rails. For more information, please visit https://github.com/RubyDevInc.
The next XaaS is CICDaaS. Why? Because CICD saves developers a huge amount of time. CD is an especially great option for projects that require multiple and frequent contributions to be integrated. But… securing CICD best practices is an emerging, essential, yet little understood practice for DevOps teams and their Cloud Service Providers. The only way to get CICD to work in a highly secure environment takes collaboration, patience and persistence. Building CICD in the cloud requires rigorous ar...
When it comes to cloud computing, the ability to turn massive amounts of compute cores on and off on demand sounds attractive to IT staff, who need to manage peaks and valleys in user activity. With cloud bursting, the majority of the data can stay on premises while tapping into compute from public cloud providers, reducing risk and minimizing need to move large files. In his session at 18th Cloud Expo, Scott Jeschonek, Director of Product Management at Avere Systems, discussed the IT and busine...
As businesses evolve, they need technology that is simple to help them succeed today and flexible enough to help them build for tomorrow. Chrome is fit for the workplace of the future — providing a secure, consistent user experience across a range of devices that can be used anywhere. In her session at 21st Cloud Expo, Vidya Nagarajan, a Senior Product Manager at Google, will take a look at various options as to how ChromeOS can be leveraged to interact with people on the devices, and formats th...
First generation hyperconverged solutions have taken the data center by storm, rapidly proliferating in pockets everywhere to provide further consolidation of floor space and workloads. These first generation solutions are not without challenges, however. In his session at 21st Cloud Expo, Wes Talbert, a Principal Architect and results-driven enterprise sales leader at NetApp, will discuss how the HCI solution of tomorrow will integrate with the public cloud to deliver a quality hybrid cloud e...
SYS-CON Events announced today that Yuasa System will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Yuasa System is introducing a multi-purpose endurance testing system for flexible displays, OLED devices, flexible substrates, flat cables, and films in smartphones, wearables, automobiles, and healthcare.
Is advanced scheduling in Kubernetes achievable? Yes, however, how do you properly accommodate every real-life scenario that a Kubernetes user might encounter? How do you leverage advanced scheduling techniques to shape and describe each scenario in easy-to-use rules and configurations? In his session at @DevOpsSummit at 21st Cloud Expo, Oleg Chunikhin, CTO at Kublr, will answer these questions and demonstrate techniques for implementing advanced scheduling. For example, using spot instances ...
DevOps is under attack because developers don’t want to mess with infrastructure. They will happily own their code into production, but want to use platforms instead of raw automation. That’s changing the landscape that we understand as DevOps with both architecture concepts (CloudNative) and process redefinition (SRE). Rob Hirschfeld’s recent work in Kubernetes operations has led to the conclusion that containers and related platforms have changed the way we should be thinking about DevOps and...
SYS-CON Events announced today that Taica will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Taica manufacturers Alpha-GEL brand silicone components and materials, which maintain outstanding performance over a wide temperature range -40C to +200C. For more information, visit http://www.taica.co.jp/english/.
When it comes to cloud computing, the ability to turn massive amounts of compute cores on and off on demand sounds attractive to IT staff, who need to manage peaks and valleys in user activity. With cloud bursting, the majority of the data can stay on premises while tapping into compute from public cloud providers, reducing risk and minimizing need to move large files. In his session at 18th Cloud Expo, Scott Jeschonek, Director of Product Management at Avere Systems, discussed the IT and busine...
We all know that end users experience the Internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices – not doing so will be a path to eventual b...
Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities – ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups. As a result, many firms employ new business models that place enormous impor...
SYS-CON Events announced today that SourceForge has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. SourceForge is the largest, most trusted destination for Open Source Software development, collaboration, discovery and download on the web serving over 32 million viewers, 150 million downloads and over 460,000 active development projects each and every month.
SYS-CON Events announced today that Dasher Technologies will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Dasher Technologies, Inc. ® is a premier IT solution provider that delivers expert technical resources along with trusted account executives to architect and deliver complete IT solutions and services to help our clients execute their goals, plans and objectives. Since 1999, we'v...