Welcome!

@CloudExpo Authors: Elizabeth White, Pat Romanski, Yeshim Deniz, Liz McMillan, Aruna Ravichandran

Related Topics: @CloudExpo, Java IoT, @DevOpsSummit

@CloudExpo: Blog Feed Post

New to Cloud-Based Performance Testing? By @DanBoutinSOASTA | @CloudExpo #Cloud

You want to get with the times, but you can’t. Why? Because you don’t speak the new language

New to Cloud-Based Performance Testing?  Here's a Quick Language Lesson
by Dan Boutin

OK, go ahead… admit it. You listen to Sirius/XM Radio channels 8 and 9 only. You haven’t noticed who the president is since Bill Clinton left office. If you haven’t had your mid-life crisis, you’re thinking about having it soon. And you may still call your test tool Mercury LoadRunner, refusing to acknowledge HP’s purchase of Mercury since it happened eight years ago.

You want to get with the times, but you can’t. Why? Because you don’t speak the new language. And that language barrier is stopping you from entering the new world order in performance testing: cloud-based performance testing.

So, I’m here to help you feel young again with a language lesson. Today, we’re going to translate your circa-1989 client/server tool terminology to today’s web and mobile cloud-based performance testing language.

(To paraphrase a former University of Florida coach, Urban Meyer, who refused to ever mention Florida State University (FSU) during his entire tenure at the University of Florida — instead calling them “that school out west” — I’ll do the same in this blog post. I will henceforth refer to that tool that had the lion’s share of the marketplace and was named after a planet as “that tool out west”.)

This blog post is a translator for anyone using “that tool out west”. The goal of this post is to map familiar concepts within, say, a client/server test tool developed in 1989 to corresponding similar concepts in SOASTA CloudTest.

Cloud testing terminology and concepts
Both CloudTest and “that tool out west” require planning before creating performance test cases. The performance engineer needs to understand the system, user experience, and business drivers for load testing the system under test. The load test plan needs to identify test cases, Entry and Exit criteria, metrics to be collected, timelines, and include a final report.

Terminology Differences

  1. An environment in CloudTest is very similar to an environment in “that tool out west”: they are both web applications. Both of them perform the job of controlling virtual users for a test scenario, and both require a username and password to access the Performance Center manages and maintains scenarios. “That tool out west” controller, the stand-alone solution, also does this. It is done by SOASTA’s repository in CloudTest. One important difference is that the SOASTA CloudTest environment incorporates load test creation, browser-based functional test creation, and can serve as a Load Generator, all in one platform.
  1. A maestro in CloudTest is the same as a load generator in “that tool out west”. In the case of SOASTA CloudTest, a load server can be local machine or it can be in thecloud. Deploying load generators in the cloud is cost effective and can simulate traffic from different geographic locations on the Internet. With “that tool out west”, load generators are usually deployed locally. A recently announced version of “that tool out west” provides the ability to deploy both controller and load generator in the cloud, but this storm is still evolving and does not yet qualify for a name from the National Hurricane Center.
  1. Analytics dashboards in CloudTest are similar to performance analysis in “that tool out west”.
  1. Virtual users in CloudTest are similar to Vusers in “that tool out west”.
  1. Clips in CloudTest are similar to Vuser scripts in “that tool out west”.
  1. A composition in CloudTest is similar to a scenario in “that tool out west”. Both composition and scenario determine what should happen in a testing session with details about what test cases should be

Building scripts
“That tool out west” uses VuGen. VuGen captures all the recorded traffic based upon the type of protocol selected for the test.

With SOASTA CloudTest, recording is integrated in the product and uses an agent called the SOASTA Conductor on the machine from which recording is being done. The Conductor acts as a web proxy for capturing all the HTTP/HTTPS traffic when recording.

SOASTA CloudTest has the option of viewing recording in list and icon views.

Correlation
SOASTA CloudTest has a “Session Template Package Wizard” which is somewhat similar to “that tool out west’s” “Scan Script for Correlations”. However, SOASTA CloudTest scans for name/value pairs and not for differences between recording and playback. The Session Template Package Wizard identifies all dynamic values in the clip and required values can be selected from the UI.

LR-1

Seed data
SOASTA CloudTest has a seed data repository, which is similar to “that tool out west’s” parameterization functionality, but covering only the parameter type File. Test data can be stored within CloudTest or it can be hosted anywhere accessible by the load generators and used within a clip. Once test data is available, it can be used in a random fashion, as single shot data or unique data for each virtual user. This functionality can be achieved either from the UI or by using JavaScript.

LR-2LR-3

Validation
CloudTest has validation functionality similar to “that tool out west’s” “Content Check”. The validations can be done within a clip/script and added for text, HTML, JSON, XML and SOAP. Validations can be done for data within the header or body. The validations can be added for each and every message within a clip. Similar to the content check functionality in “that tool out west”, success and failure messages can be added as part of validations.

LR-3

Collections
CloudTest can group multiple messages or pages as transactions (also known as collections) and metrics can be tracked in the analytics dashboard for a particular transaction. This is similar to the concept of transaction in “that tool out west”.

LR-4

Test compositions
For creating a test in CloudTest, one or more clips are added to a composition. This is similar to adding vu scripts in a scenario. The composition specifies load servers, number of virtual users, number of iterations, ramp time and duration of the test. This is like adding load generators, building a schedule, specifying duration and ramp up and ramp down for the test in “that tool out west”.

LR-5

Analytics
SOASTA CloudTest has extensive metrics in the analytic dashboards. These dashboards show real-time data, independent of the scale of the test. They are similar to “that tool out west” performance analysis. SOASTA CloudTest results can be viewed during and after the test execution and results can be exported in .csv, .xml and .doc formats.

LR-6

Repository
The SOASTA repository stores results from the test. This is similar to the result directory in “that tool out west”. The SOASTA repository can be maintained on premise or in the cloud and can accessed over the internet. In both cases, the client is an Ajax- based browser.

Monitoring
Just like “that tool out west”, SOASTA CloudTest has integrated monitors for monitoring system utilization of systems under load. These monitors can be created as web monitors, database monitors, application server monitors, etc.

CloudTest can also integrate monitored data from third-party monitoring tools, such as those from New Relic, CA, Amazon, AppDynamics, Correlsense and more, displaying the results in the same real-time dashboards.

LR-7

LR-8

Takeaways
Now that you’re up to speed on the terminology, let’s keep the momentum going. Go ahead and download our free version of CloudTest, the performance testing solution used by every one of the top twenty online retailers.

And now that you speak the language, take the extra mile by getting trained and certified. Find out how here.

Once you are done with that, feel free to check out the other 200+ channels on Sirius/XM radio. And kiss 1989 goodbye. You deserve it.

performance testing

Read the original blog entry...

More Stories By SOASTA Blog

The SOASTA platform enables digital business owners to gain unprecedented and continuous performance insights into their real user experience on mobile and web devices in real time and at scale.

@CloudExpo Stories
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, will discuss how from store operations...
Nordstrom is transforming the way that they do business and the cloud is the key to enabling speed and hyper personalized customer experiences. In his session at 21st Cloud Expo, Ken Schow, VP of Engineering at Nordstrom, will discuss some of the key learnings and common pitfalls of large enterprises moving to the cloud. This includes strategies around choosing a cloud provider(s), architecture, and lessons learned. In addition, he’ll go over some of the best practices for structured team migrat...
SYS-CON Events announced today that Datera will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Datera offers a radically new approach to data management, where innovative software makes data infrastructure invisible, elastic and able to perform at the highest level. It eliminates hardware lock-in and gives IT organizations the choice to source x86 server nodes, with business model option...
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Infoblox delivers Actionable Network Intelligence to enterprise, government, and service provider customers around the world. They are the industry leader in DNS, DHCP, and IP address management, the category known as DDI. We empower thousands of organizations to control and secure their networks from the core-enabling them to increase efficiency and visibility, improve customer service, and meet compliance requirements.
Recently, REAN Cloud built a digital concierge for a North Carolina hospital that had observed that most patient call button questions were repetitive. In addition, the paper-based process used to measure patient health metrics was laborious, not in real-time and sometimes error-prone. In their session at 21st Cloud Expo, Sean Finnerty, Executive Director, Practice Lead, Health Care & Life Science at REAN Cloud, and Dr. S.P.T. Krishnan, Principal Architect at REAN Cloud, will discuss how they bu...
Digital transformation is changing the face of business. The IDC predicts that enterprises will commit to a massive new scale of digital transformation, to stake out leadership positions in the "digital transformation economy." Accordingly, attendees at the upcoming Cloud Expo | @ThingsExpo at the Santa Clara Convention Center in Santa Clara, CA, Oct 31-Nov 2, will find fresh new content in a new track called Enterprise Cloud & Digital Transformation.
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp emp...
SYS-CON Events announced today that N3N will exhibit at SYS-CON's @ThingsExpo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. N3N’s solutions increase the effectiveness of operations and control centers, increase the value of IoT investments, and facilitate real-time operational decision making. N3N enables operations teams with a four dimensional digital “big board” that consolidates real-time live video feeds alongside IoT sensor data a...
Smart cities have the potential to change our lives at so many levels for citizens: less pollution, reduced parking obstacles, better health, education and more energy savings. Real-time data streaming and the Internet of Things (IoT) possess the power to turn this vision into a reality. However, most organizations today are building their data infrastructure to focus solely on addressing immediate business needs vs. a platform capable of quickly adapting emerging technologies to address future ...
The dynamic nature of the cloud means that change is a constant when it comes to modern cloud-based infrastructure. Delivering modern applications to end users, therefore, is a constantly shifting challenge. Delivery automation helps IT Ops teams ensure that apps are providing an optimal end user experience over hybrid-cloud and multi-cloud environments, no matter what the current state of the infrastructure is. To employ a delivery automation strategy that reflects your business rules, making r...
As people view cloud as a preferred option to build IT systems, the size of the cloud-based system is getting bigger and more complex. As the system gets bigger, more people need to collaborate from design to management. As more people collaborate to create a bigger system, the need for a systematic approach to automate the process is required. Just as in software, cloud now needs DevOps. In this session, the audience can see how people can solve this issue with a visual model. Visual models ha...
Enterprises are adopting Kubernetes to accelerate the development and the delivery of cloud-native applications. However, sharing a Kubernetes cluster between members of the same team can be challenging. And, sharing clusters across multiple teams is even harder. Kubernetes offers several constructs to help implement segmentation and isolation. However, these primitives can be complex to understand and apply. As a result, it’s becoming common for enterprises to end up with several clusters. Thi...
SYS-CON Events announced today that Avere Systems, a leading provider of hybrid cloud enablement solutions, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Avere Systems was created by file systems experts determined to reinvent storage by changing the way enterprises thought about and bought storage resources. With decades of experience behind the company’s founders, Avere got its ...
In the fast-paced advances and popularity in cloud technology, one of the most critical factors revolves around concerns for security of your critical data. How to assure both your company and your customers they can confidently trust and utilize your cloud environment is most often top on the list. There is a method to evaluating and providing security that exceeds conventional modes of protecting data both within the cloud as well externally on mobile and other devices. With the public failure...
Containers are rapidly finding their way into enterprise data centers, but change is difficult. How do enterprises transform their architecture with technologies like containers without losing the reliable components of their current solutions? In his session at @DevOpsSummit at 21st Cloud Expo, Tony Campbell, Director, Educational Services at CoreOS, will explore the challenges organizations are facing today as they move to containers and go over how Kubernetes applications can deploy with lega...
SYS-CON Events announced today that Avere Systems, a leading provider of enterprise storage for the hybrid cloud, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Avere delivers a more modern architectural approach to storage that doesn't require the overprovisioning of storage capacity to achieve performance, overspending on expensive storage media for inactive data or the overbui...
Gemini is Yahoo’s native and search advertising platform. To ensure the quality of a complex distributed system that spans multiple products and components and across various desktop websites and mobile app and web experiences – both Yahoo owned and operated and third-party syndication (supply), with complex interaction with more than a billion users and numerous advertisers globally (demand) – it becomes imperative to automate a set of end-to-end tests 24x7 to detect bugs and regression. In th...
Today most companies are adopting or evaluating container technology - Docker in particular - to speed up application deployment, drive down cost, ease management and make application delivery more flexible overall. As with most new architectures, this dream takes significant work to become a reality. Even when you do get your application componentized enough and packaged properly, there are still challenges for DevOps teams to making the shift to continuous delivery and achieving that reducti...
Microsoft Azure Container Services can be used for container deployment in a variety of ways including support for Orchestrators like Kubernetes, Docker Swarm and Mesos. However, the abstraction for app development that support application self-healing, scaling and so on may not be at the right level. Helm and Draft makes this a lot easier. In this primarily demo-driven session at @DevOpsSummit at 21st Cloud Expo, Raghavan "Rags" Srinivas, a Cloud Solutions Architect/Evangelist at Microsoft, wi...