Welcome!

@CloudExpo Authors: Yeshim Deniz, Pat Romanski, Liz McMillan, Elizabeth White, Gopala Krishna Behara

Related Topics: @CloudExpo, Java IoT, @DevOpsSummit

@CloudExpo: Blog Feed Post

New to Cloud-Based Performance Testing? By @DanBoutinSOASTA | @CloudExpo #Cloud

You want to get with the times, but you can’t. Why? Because you don’t speak the new language

New to Cloud-Based Performance Testing?  Here's a Quick Language Lesson
by Dan Boutin

OK, go ahead… admit it. You listen to Sirius/XM Radio channels 8 and 9 only. You haven’t noticed who the president is since Bill Clinton left office. If you haven’t had your mid-life crisis, you’re thinking about having it soon. And you may still call your test tool Mercury LoadRunner, refusing to acknowledge HP’s purchase of Mercury since it happened eight years ago.

You want to get with the times, but you can’t. Why? Because you don’t speak the new language. And that language barrier is stopping you from entering the new world order in performance testing: cloud-based performance testing.

So, I’m here to help you feel young again with a language lesson. Today, we’re going to translate your circa-1989 client/server tool terminology to today’s web and mobile cloud-based performance testing language.

(To paraphrase a former University of Florida coach, Urban Meyer, who refused to ever mention Florida State University (FSU) during his entire tenure at the University of Florida — instead calling them “that school out west” — I’ll do the same in this blog post. I will henceforth refer to that tool that had the lion’s share of the marketplace and was named after a planet as “that tool out west”.)

This blog post is a translator for anyone using “that tool out west”. The goal of this post is to map familiar concepts within, say, a client/server test tool developed in 1989 to corresponding similar concepts in SOASTA CloudTest.

Cloud testing terminology and concepts
Both CloudTest and “that tool out west” require planning before creating performance test cases. The performance engineer needs to understand the system, user experience, and business drivers for load testing the system under test. The load test plan needs to identify test cases, Entry and Exit criteria, metrics to be collected, timelines, and include a final report.

Terminology Differences

  1. An environment in CloudTest is very similar to an environment in “that tool out west”: they are both web applications. Both of them perform the job of controlling virtual users for a test scenario, and both require a username and password to access the Performance Center manages and maintains scenarios. “That tool out west” controller, the stand-alone solution, also does this. It is done by SOASTA’s repository in CloudTest. One important difference is that the SOASTA CloudTest environment incorporates load test creation, browser-based functional test creation, and can serve as a Load Generator, all in one platform.
  1. A maestro in CloudTest is the same as a load generator in “that tool out west”. In the case of SOASTA CloudTest, a load server can be local machine or it can be in thecloud. Deploying load generators in the cloud is cost effective and can simulate traffic from different geographic locations on the Internet. With “that tool out west”, load generators are usually deployed locally. A recently announced version of “that tool out west” provides the ability to deploy both controller and load generator in the cloud, but this storm is still evolving and does not yet qualify for a name from the National Hurricane Center.
  1. Analytics dashboards in CloudTest are similar to performance analysis in “that tool out west”.
  1. Virtual users in CloudTest are similar to Vusers in “that tool out west”.
  1. Clips in CloudTest are similar to Vuser scripts in “that tool out west”.
  1. A composition in CloudTest is similar to a scenario in “that tool out west”. Both composition and scenario determine what should happen in a testing session with details about what test cases should be

Building scripts
“That tool out west” uses VuGen. VuGen captures all the recorded traffic based upon the type of protocol selected for the test.

With SOASTA CloudTest, recording is integrated in the product and uses an agent called the SOASTA Conductor on the machine from which recording is being done. The Conductor acts as a web proxy for capturing all the HTTP/HTTPS traffic when recording.

SOASTA CloudTest has the option of viewing recording in list and icon views.

Correlation
SOASTA CloudTest has a “Session Template Package Wizard” which is somewhat similar to “that tool out west’s” “Scan Script for Correlations”. However, SOASTA CloudTest scans for name/value pairs and not for differences between recording and playback. The Session Template Package Wizard identifies all dynamic values in the clip and required values can be selected from the UI.

LR-1

Seed data
SOASTA CloudTest has a seed data repository, which is similar to “that tool out west’s” parameterization functionality, but covering only the parameter type File. Test data can be stored within CloudTest or it can be hosted anywhere accessible by the load generators and used within a clip. Once test data is available, it can be used in a random fashion, as single shot data or unique data for each virtual user. This functionality can be achieved either from the UI or by using JavaScript.

LR-2LR-3

Validation
CloudTest has validation functionality similar to “that tool out west’s” “Content Check”. The validations can be done within a clip/script and added for text, HTML, JSON, XML and SOAP. Validations can be done for data within the header or body. The validations can be added for each and every message within a clip. Similar to the content check functionality in “that tool out west”, success and failure messages can be added as part of validations.

LR-3

Collections
CloudTest can group multiple messages or pages as transactions (also known as collections) and metrics can be tracked in the analytics dashboard for a particular transaction. This is similar to the concept of transaction in “that tool out west”.

LR-4

Test compositions
For creating a test in CloudTest, one or more clips are added to a composition. This is similar to adding vu scripts in a scenario. The composition specifies load servers, number of virtual users, number of iterations, ramp time and duration of the test. This is like adding load generators, building a schedule, specifying duration and ramp up and ramp down for the test in “that tool out west”.

LR-5

Analytics
SOASTA CloudTest has extensive metrics in the analytic dashboards. These dashboards show real-time data, independent of the scale of the test. They are similar to “that tool out west” performance analysis. SOASTA CloudTest results can be viewed during and after the test execution and results can be exported in .csv, .xml and .doc formats.

LR-6

Repository
The SOASTA repository stores results from the test. This is similar to the result directory in “that tool out west”. The SOASTA repository can be maintained on premise or in the cloud and can accessed over the internet. In both cases, the client is an Ajax- based browser.

Monitoring
Just like “that tool out west”, SOASTA CloudTest has integrated monitors for monitoring system utilization of systems under load. These monitors can be created as web monitors, database monitors, application server monitors, etc.

CloudTest can also integrate monitored data from third-party monitoring tools, such as those from New Relic, CA, Amazon, AppDynamics, Correlsense and more, displaying the results in the same real-time dashboards.

LR-7

LR-8

Takeaways
Now that you’re up to speed on the terminology, let’s keep the momentum going. Go ahead and download our free version of CloudTest, the performance testing solution used by every one of the top twenty online retailers.

And now that you speak the language, take the extra mile by getting trained and certified. Find out how here.

Once you are done with that, feel free to check out the other 200+ channels on Sirius/XM radio. And kiss 1989 goodbye. You deserve it.

performance testing

Read the original blog entry...

More Stories By SOASTA Blog

The SOASTA platform enables digital business owners to gain unprecedented and continuous performance insights into their real user experience on mobile and web devices in real time and at scale.

@CloudExpo Stories
We all know that end users experience the internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices - not doing so will be a path to eventual ...
We all know that end users experience the internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices - not doing so will be a path to eventual ...
"We were founded in 2003 and the way we were founded was about good backup and good disaster recovery for our clients, and for the last 20 years we've been pretty consistent with that," noted Marc Malafronte, Territory Manager at StorageCraft, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
DXWorldEXPO LLC announced today that ICC-USA, a computer systems integrator and server manufacturing company focused on developing products and product appliances, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City. ICC is a computer systems integrator and server manufacturing company focused on developing products and product appliances to meet a wide range of ...
We all know that end users experience the Internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices – not doing so will be a path to eventual b...
More and more brands have jumped on the IoT bandwagon. We have an excess of wearables – activity trackers, smartwatches, smart glasses and sneakers, and more that track seemingly endless datapoints. However, most consumers have no idea what “IoT” means. Creating more wearables that track data shouldn't be the aim of brands; delivering meaningful, tangible relevance to their users should be. We're in a period in which the IoT pendulum is still swinging. Initially, it swung toward "smart for smart...
Vulnerability management is vital for large companies that need to secure containers across thousands of hosts, but many struggle to understand how exposed they are when they discover a new high security vulnerability. In his session at 21st Cloud Expo, John Morello, CTO of Twistlock, addressed this pressing concern by introducing the concept of the “Vulnerability Risk Tree API,” which brings all the data together in a simple REST endpoint, allowing companies to easily grasp the severity of the ...
Sanjeev Sharma Joins November 11-13, 2018 @DevOpsSummit at @CloudEXPO New York Faculty. Sanjeev Sharma is an internationally known DevOps and Cloud Transformation thought leader, technology executive, and author. Sanjeev's industry experience includes tenures as CTO, Technical Sales leader, and Cloud Architect leader. As an IBM Distinguished Engineer, Sanjeev is recognized at the highest levels of IBM's core of technical leaders.
"DivvyCloud as a company set out to help customers automate solutions to the most common cloud problems," noted Jeremy Snyder, VP of Business Development at DivvyCloud, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
As Cybric's Chief Technology Officer, Mike D. Kail is responsible for the strategic vision and technical direction of the platform. Prior to founding Cybric, Mike was Yahoo's CIO and SVP of Infrastructure, where he led the IT and Data Center functions for the company. He has more than 24 years of IT Operations experience with a focus on highly-scalable architectures.
Headquartered in Plainsboro, NJ, Synametrics Technologies has provided IT professionals and computer systems developers since 1997. Based on the success of their initial product offerings (WinSQL and DeltaCopy), the company continues to create and hone innovative products that help its customers get more from their computer applications, databases and infrastructure. To date, over one million users around the world have chosen Synametrics solutions to help power their accelerated business or per...
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
In an era of historic innovation fueled by unprecedented access to data and technology, the low cost and risk of entering new markets has leveled the playing field for business. Today, any ambitious innovator can easily introduce a new application or product that can reinvent business models and transform the client experience. In their Day 2 Keynote at 19th Cloud Expo, Mercer Rowe, IBM Vice President of Strategic Alliances, and Raejeanne Skillern, Intel Vice President of Data Center Group and ...
Founded in 2000, Chetu Inc. is a global provider of customized software development solutions and IT staff augmentation services for software technology providers. By providing clients with unparalleled niche technology expertise and industry experience, Chetu has become the premiere long-term, back-end software development partner for start-ups, SMBs, and Fortune 500 companies. Chetu is headquartered in Plantation, Florida, with thirteen offices throughout the U.S. and abroad.
Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: Driving Business Strategies with Data Science," is responsible for setting the strategy and defining the Big Data service offerings and capabilities for EMC Global Services Big Data Practice. As the CTO for the Big Data Practice, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He's written several white papers, is an avid blogge...
"We are a well-established player in the application life cycle management market and we also have a very strong version control product," stated Flint Brenton, CEO of CollabNet,, in this SYS-CON.tv interview at 18th Cloud Expo at the Javits Center in New York City, NY.
DXWorldEXPO LLC announced today that Dez Blanchfield joined the faculty of CloudEXPO's "10-Year Anniversary Event" which will take place on November 11-13, 2018 in New York City. Dez is a strategic leader in business and digital transformation with 25 years of experience in the IT and telecommunications industries developing strategies and implementing business initiatives. He has a breadth of expertise spanning technologies such as cloud computing, big data and analytics, cognitive computing, m...
HyperConvergence came to market with the objective of being simple, flexible and to help drive down operating expenses. It reduced the footprint by bundling the compute/storage/network into one box. This brought a new set of challenges as the HyperConverged vendors are very focused on their own proprietary building blocks. If you want to scale in a certain way, let's say you identified a need for more storage and want to add a device that is not sold by the HyperConverged vendor, forget about it...
Enterprises are moving to the cloud faster than most of us in security expected. CIOs are going from 0 to 100 in cloud adoption and leaving security teams in the dust. Once cloud is part of an enterprise stack, it’s unclear who has responsibility for the protection of applications, services, and data. When cloud breaches occur, whether active compromise or a publicly accessible database, the blame must fall on both service providers and users. In his session at 21st Cloud Expo, Ben Johnson, C...
It is of utmost importance for the future success of WebRTC to ensure that interoperability is operational between web browsers and any WebRTC-compliant client. To be guaranteed as operational and effective, interoperability must be tested extensively by establishing WebRTC data and media connections between different web browsers running on different devices and operating systems. In his session at WebRTC Summit at @ThingsExpo, Dr. Alex Gouaillard, CEO and Founder of CoSMo Software, presented ...
In this presentation, you will learn first hand what works and what doesn't while architecting and deploying OpenStack. Some of the topics will include:- best practices for creating repeatable deployments of OpenStack- multi-site considerations- how to customize OpenStack to integrate with your existing systems and security best practices.