Welcome!

@CloudExpo Authors: Pat Romanski, Elizabeth White, Ed Featherston, Liz McMillan, David Paquette

Related Topics: Microservices Expo, Java IoT, Mobile IoT, Agile Computing, Release Management

Microservices Expo: Article

Best Practices for Load Testing Mobile Applications | Part I

The differences between testing traditional web and mobile applications

Mobile applications and mobile websites have become a major channel for conducting business, improving employee efficiency, communicating, and reaching consumers. In the past, mobile played a smaller role in business applications, so performance issues and outages were less of a concern. This is no longer the case. Today, performance problems with mobile applications lead directly to revenue loss, brand damage, and diminished employee productivity. Part I of this article discusses the differences between testing traditional web and mobile applications, specific challenges associated with mobile load testing, mobile testing basics and best practices for recording mobile load test scenarios. In Part II, we will look at how to conduct realistic tests and how to best analyze the results.

Application developers have long understood the need for load testing conventional desktop web applications to ensure that they will behave properly under load with the expected number of users. With the advent of mobile apps and mobile websites the principles of load testing have not changed. There are, however, challenges specific to mobile load testing that must be addressed by your load testing solution.

Since mobile apps and applications for desktop web browsers use the same underlying technologies, the good news is that most load testing tasks and challenges are the same. This means that you don't necessarily need a brand new, mobile-specific load testing tool, but you do need a quality web load testing tool capable of handling the nuances of load testing mobile apps. Using a tool that enables testing of traditional and mobile web applications enables you to leverage existing in-house skills for designing and parameterizing your scripts, running your tests, and analyzing the results.

Aside from the similarities between traditional and mobile load testing, there are three key differences:

  • Simulating network and bandwidth for wireless protocols: With 3G wireless protocols, mobile devices typically connect to the Internet using a slower, lower quality connection than desktops and laptops. This has an effect on response times on the client side and on the server, which you'll need to account for as you define your tests and analyze your results. Additionally, latency and packet loss becomes more of a factor with mobile applications and needs to be considered.
  • Recording on mobile devices: Obviously, mobile apps run on mobile devices, and this can make it difficult to record test scenarios, particularly for secured applications that use HTTPS.
  • Supporting a wide range of devices: The many different kinds of mobile devices on the market have led web application designers to tailor content based on the capabilities of the client's platform. This presents challenges for recording and playing back test scenarios.

Mobile Load Testing Basics
As you may know, a typical automated functional test for a mobile application emulates user actions (including tap, swipe, zoom, and entering text) on a real device or an emulator. The objective of load testing, however, is not to test the functionality of the application for just a single user. Rather, the goal is to see how the server infrastructure performs when handling requests from a large number of users, and to understand how response times are affected by other users interacting with the application.

An effective load test simulates a high volume of simultaneous users accessing your server via your application. Using real devices or emulators for this task is impractical because it demands acquiring, configuring, and synchronizing hundreds or thousands of real devices or machines running emulators.

The solution, of course, is to use a load testing approach that is designed to scale as needed. With a client-based approach, user actions in the browser or the native application are recorded and played back. In contrast, a protocol-based approach involves recording and reproducing the network traffic between the device and the server. To verify performance under large loads, tools that enable protocol-based testing are superior to those that support only client-based testing because they can scale up to millions of users while checking errors and response times for each user.

The basic process for protocol-based mobile load testing is:

  1. Record the network traffic between the device and the server
  2. Replay the network requests for a large number of virtual users
  3. Analyze the results

It appears straightforward, but there are challenges at every step. The good news is that these challenges can be addressed with an effective load testing approach.

Recording Mobile Load Testing Scenarios
To generate a mobile test scenario, you first need to identify the type of mobile application under test. Challenges associated with capturing the data exchanges between a mobile application and the server depend on the design of the application:

  • Native apps - These apps are coded using a programming language (Objective-C for iOS, Java for Android) and API that is specific to the device. As such, they are tied to a mobile platform and are installed from an online store or market.
  • Web apps - Built with web technologies (such as HTML and JavaScript), these applications can be accessed from any mobile browser. More sophisticated web apps may use advanced features like geolocation or web storage for data or include customizations to better match the browser used. Two popular web apps are http://touch.linkedin.com and http://m.gmail.com.
  • Hybrid Apps - A web app embedded in a native app is known as a hybrid app. The native part of the application is limited to a few user interface elements like the menu or navigation buttons, and functions such as automatic login. The main content is displayed in an embedded web browser component. The Facebook application, installed from an online store or a market is a typical sample.

Recording Tests for Native Apps
Because native apps run on your device or within an emulator, to record a test you need to intercept the network traffic coming from the real device or the emulator.

To intercept this traffic, the equipment that records the traffic must be connected to the same network as the device. When the recording computer is on the intranet behind a firewall, it is not possible to record a mobile device connected via a 3G or 4G wireless network. The device and the computer running the recorder must be connected to the same Wi-Fi network.

Most load testing tools provide a proxy based recorder, which is the easiest way to record an application's network traffic. To use this approach, you need to configure the mobile device's Wi-Fi settings so that the traffic goes through the recording proxy. Some mobile operating systems, such as iOS and Android 4, support making this change, but older versions of Android may not. Moreover, some applications connect directly to the server regardless of the proxy settings of the operating system. In either of these cases, you need a tool that provides an alternative to proxy-based recording methods based on network capture or tunneling.

Note: You can use the following simple test to check if the application can be recorded using a proxy. First, configure the proxy settings on the device and record your interactions with any website in a mobile browser. Then, try to record interactions in the native application. If your testing tool successfully records the browser generated traffic, but does not record traffic generated by the native application then you can conclude that the native application is bypassing the proxy settings and that an alternative recording method is required.

Recording Tests for Web Apps and Mobile Version of Websites
Web apps use the same web technologies as modern desktop browsers. As a result, you can record the application or the mobile version of a website from a modern browser on your regular desktop computer, which is an easier and faster alternative to recording from the device.

Many web applications check the browser and platform used to access them. This enables the application, when accessed from a mobile device, to redirect to a mobile version of the content that may contain less text or fewer images. To test such an app from the desktop, you need to modify the requests to make them appear to the server to be coming from a mobile device. Otherwise, you will not be testing the mobile version of the application as the server may redirect to a desktop version. Some browser plugins provide the ability to alter the identity of the browser (by modifying the User-Agent header of requests). Support for this feature is also directly integrated in the recorder of advanced load testing tools.

Modifying the browser's identity is not always enough. You obviously cannot use this approach to transform Internet Explorer 6 into an HTML5 compatible browser. The browser you use on the desktop must be able to parse and render content created for mobile browsers, so it's best to record with a modern browser like Internet Explorer 9, Firefox 5, Chrome 15, or Safari 5 (or a more recent version of any of these if available). If the application includes WebKit specific features, you should use a WebKit based desktop browser, preferably either Chrome or Safari.

Recording Tests for Hybrid Apps
Obviously, tests for native apps cannot be recorded using a desktop browser. However, tests for many hybrid apps can. You may be able to directly access the URL used for the application, for example http://m.facebook.com for the Facebook application, and record your tests as you would for a classic web app.

Recording Tests for Secured Native Applications
There are additional challenges to consider when recording tests for a secured native application, that is, an application that uses HTTPS for the login procedure or any other processing.

By default, all HTTPS recording methods, whether proxy or tunnel based, are seen as man-in-the-middle attacks by the device. This raises a non-blocking alert in a desktop or mobile browser but it leads to an outright connection refusal in native applications, making it impossible to record the secured traffic.

The only way to record tests for secured native applications is to provide a root certificate that authorizes the connection with the proxy or tunnel. While this feature is currently supported by relatively few load testing solutions, it is essential for load testing any native application that relies on HTTPS.

Note: The root certificate must be installed on the device. This operation is simple for iOS devices; you can simply send the certificate via email and open the attachment on the device. For other platforms, including Android, the procedure is not as straightforward and may depend on the version of the operating system and the manufacturer of the device.

Running Realistic Tests
Once you've recorded a test scenario, you need to be parameterize it so that it can emulate users with different identities and behaviors as it is played back to produce a realistic load on the server. This step is required for traditional and mobile web applications, and the tools used to complete it are the same. When playing back the test scenarios, however, there are several challenges specific to mobile load testing and we will discuss this more in Part II of this article on "Best Practices for Load Testing Mobile Applications."

More Stories By Steve Weisfeldt

Steve Weisfeldt is a Senior Performance Engineer at Neotys, a provider of load testing software for Web applications. Previously, he has worked as the President of Engine 1 Consulting, a services firm specializing in all facets of test automation. Prior to his involvement at Engine 1 Consulting, he was a Senior Systems Engineer at Aternity. Prior to that, Steve spent seven years at automated testing vendor Segue Software (acquired by Borland). While spending most of his time at Segue delivering professional services and training, he was also involved in pre-sales and product marketing efforts.

Being in the load and performance testing space since 1999, Steve has been involved in load and performance testing projects of all sizes, in industries that span the retail, financial services, insurance and manufacturing sectors. His expertise lies in enabling organizations to optimize their ability to develop, test and launch high-quality applications efficiently, on-time and on-budget. Steve graduated from the University of Massachusetts-Lowell with a BS in Electrical Engineering and an MS in Computer Engineering.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@CloudExpo Stories
One of biggest questions about Big Data is “How do we harness all that information for business use quickly and effectively?” Geographic Information Systems (GIS) or spatial technology is about more than making maps, but adding critical context and meaning to data of all types, coming from all different channels – even sensors. In his session at @ThingsExpo, William (Bill) Meehan, director of utility solutions for Esri, will take a closer look at the current state of spatial technology and ar...
According to Forrester Research, every business will become either a digital predator or digital prey by 2020. To avoid demise, organizations must rapidly create new sources of value in their end-to-end customer experiences. True digital predators also must break down information and process silos and extend digital transformation initiatives to empower employees with the digital resources needed to win, serve, and retain customers.
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, wh...
19th Cloud Expo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterpri...
Complete Internet of Things (IoT) embedded device security is not just about the device but involves the entire product’s identity, data and control integrity, and services traversing the cloud. A device can no longer be looked at as an island; it is a part of a system. In fact, given the cross-domain interactions enabled by IoT it could be a part of many systems. Also, depending on where the device is deployed, for example, in the office building versus a factory floor or oil field, security ha...
Ask someone to architect an Internet of Things (IoT) solution and you are guaranteed to see a reference to the cloud. This would lead you to believe that IoT requires the cloud to exist. However, there are many IoT use cases where the cloud is not feasible or desirable. In his session at @ThingsExpo, Dave McCarthy, Director of Products at Bsquare Corporation, will discuss the strategies that exist to extend intelligence directly to IoT devices and sensors, freeing them from the constraints of ...
Let’s face it, embracing new storage technologies, capabilities and upgrading to new hardware often adds complexity and increases costs. In his session at 18th Cloud Expo, Seth Oxenhorn, Vice President of Business Development & Alliances at FalconStor, discussed how a truly heterogeneous software-defined storage approach can add value to legacy platforms and heterogeneous environments. The result reduces complexity, significantly lowers cost, and provides IT organizations with improved efficienc...
Businesses are struggling to manage the information flow and interactions between all of these new devices and things jumping on their network, and the apps and IT systems they control. The data businesses gather is only helpful if they can do something with it. In his session at @ThingsExpo, Chris Witeck, Principal Technology Strategist at Citrix, will discuss how different the impact of IoT will be for large businesses, expanding how IoT will allow large organizations to make their legacy ap...
Why do your mobile transformations need to happen today? Mobile is the strategy that enterprise transformation centers on to drive customer engagement. In his general session at @ThingsExpo, Roger Woods, Director, Mobile Product & Strategy – Adobe Marketing Cloud, covered key IoT and mobile trends that are forcing mobile transformation, key components of a solid mobile strategy and explored how brands are effectively driving mobile change throughout the enterprise.
SYS-CON Events announced today that CDS Global Cloud, an Infrastructure as a Service provider, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. CDS Global Cloud is an IaaS (Infrastructure as a Service) provider specializing in solutions for e-commerce, internet gaming, online education and other internet applications. With a growing number of data centers and network points around the world, ...
In this strange new world where more and more power is drawn from business technology, companies are effectively straddling two paths on the road to innovation and transformation into digital enterprises. The first path is the heritage trail – with “legacy” technology forming the background. Here, extant technologies are transformed by core IT teams to provide more API-driven approaches. Legacy systems can restrict companies that are transitioning into digital enterprises. To truly become a lea...
24Notion is full-service global creative digital marketing, technology and lifestyle agency that combines strategic ideas with customized tactical execution. With a broad understand of the art of traditional marketing, new media, communications and social influence, 24Notion uniquely understands how to connect your brand strategy with the right consumer. 24Notion ranked #12 on Corporate Social Responsibility - Book of List.
SYS-CON Events announced today that LeaseWeb USA, a cloud Infrastructure-as-a-Service (IaaS) provider, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. LeaseWeb is one of the world's largest hosting brands. The company helps customers define, develop and deploy IT infrastructure tailored to their exact business needs, by combining various kinds cloud solutions.
Technology vendors and analysts are eager to paint a rosy picture of how wonderful IoT is and why your deployment will be great with the use of their products and services. While it is easy to showcase successful IoT solutions, identifying IoT systems that missed the mark or failed can often provide more in the way of key lessons learned. In his session at @ThingsExpo, Peter Vanderminden, Principal Industry Analyst for IoT & Digital Supply Chain to Flatiron Strategies, will focus on how IoT de...
Big Data has been changing the world. IoT fuels the further transformation recently. How are Big Data and IoT related? In his session at @BigDataExpo, Tony Shan, a renowned visionary and thought leader, will explore the interplay of Big Data and IoT. He will anatomize Big Data and IoT separately in terms of what, which, why, where, when, who, how and how much. He will then analyze the relationship between IoT and Big Data, specifically the drilldown of how the 4Vs of Big Data (Volume, Variety,...
Cloud computing is being adopted in one form or another by 94% of enterprises today. Tens of billions of new devices are being connected to The Internet of Things. And Big Data is driving this bus. An exponential increase is expected in the amount of information being processed, managed, analyzed, and acted upon by enterprise IT. This amazing is not part of some distant future - it is happening today. One report shows a 650% increase in enterprise data by 2020. Other estimates are even higher....
The Internet of Things can drive efficiency for airlines and airports. In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect with GE, and Sudip Majumder, senior director of development at Oracle, will discuss the technical details of the connected airline baggage and related social media solutions. These IoT applications will enhance travelers' journey experience and drive efficiency for the airlines and the airports. The session will include a working demo and a technical d...
DevOps at Cloud Expo – being held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real results. Am...
There is little doubt that Big Data solutions will have an increasing role in the Enterprise IT mainstream over time. Big Data at Cloud Expo - to be held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA - has announced its Call for Papers is open. Cloud computing is being adopted in one form or another by 94% of enterprises today. Tens of billions of new devices are being connected to The Internet of Things. And Big Data is driving this bus. An exponential increase is...
Adobe is changing the world though digital experiences. Adobe helps customers develop and deliver high-impact experiences that differentiate brands, build loyalty, and drive revenue across every screen, including smartphones, computers, tablets and TVs. Adobe content solutions are used daily by millions of companies worldwide-from publishers and broadcasters, to enterprises, marketing agencies and household-name brands. Building on its established design leadership, Adobe enables customers not o...