Welcome!

Cloud Expo Authors: Jeremy Geelan, Elizabeth White, Sue Poremba, Pat Romanski, Patrick Burke

Related Topics: AJAX & REA, Java, SOA & WOA

AJAX & REA: Article

Performance and Architecture Validation with Existing Unit Tests

The big picture

Agile Development Practices have widely been adopted in R&D Organizations. A core component is Continuous Integration where code changes are continuously integrated and tested to achieve the goal of having "potential shippable code" at the end of every Sprint/Iteration.

In order to verify code changes agile team members write Unit or Functional Tests that get executed against every build and every milestone. The results of these tests tell the team whether the functionality of all features is still given and that the recent code changes have not introduced a regression.

Verify for Performance, Scalability and Architecture
Now we got all these JUnit, NUnit, TestNG, Selenium, WebDriver, Silk or QTP tests that verify the functionality of the code. By adding dynaTrace to the Continuous Integration Process these existing tests automatically verify performance, scalability and architectural rules. Besides knowing that the Product Search feature returns the correct result we want to know:

  • How much CPU and Network bandwidth it takes to execute the search query?
  • How many database statements are executed to retrieve the search result?
  • Whether product images will be cached on the browser?
  • How JavaScript impacts the Page Load Time in the browser?
  • Whether the last code change affected any of these Performance, Scalability or Architectural Rules?

dynaTrace analyzes all Unit and Browser Tests and validates execution characteristics such as number of database statements, transferred bytes, cache settings ... against previous builds and test runs. In case there is a change (regression) the developers are notified about what has actually changed.

dynaTrace automatically detects abnormal behavior on all subscribed measures, e.g.: execution time, number of database statements, number of JavaScript files, ...

Automatically validate rules such as number of remoting calls or number of bytes transferred

Compare the difference between the problematic and the Last Known Good run

Besides being confident about functionality these additional checks ensure that the current code base performs, scales and adheres to architectural rules.

Step-by-Step Guide to enable dynaTrace in CI
In order to integrate dynaTrace we need to modify the CI Process. The following is a High-Level Step-By-Step Guide that explains all steps in a typical CI environment. When a new build is triggered the Build Server executes an Ant, NAnt, Maven (or any other type of automation script) script that will execute the following tasks:

  1. Check-out current Code Base
  2. Generate new Buildnumber and then Compile Code
  3. (dynaTrace) Start Session Recording (through REST)
  4. (dynaTrace) Set Test Meta Data Information (through REST)
  5. Execute JUnit, Selenium, ... Tests - (dynaTrace Agent gets loaded into Test Execution Process)
  6. (dynaTrace) Stop Session Recording (through REST)
  7. Generate Test Reports including dynaTrace Results (through REST)

dynaTrace provides an Automation Library that provides both a Java and .NET Implementation to call the dynaTrace Server REST Services. It also includes Ant, NAnt and Maven Tasks that make it easy to add the necessary calls to dynaTrace. The Demo Application includes a fully configured sample including Ant, JUnit and Selenium.

Once these steps are done dynaTrace will automatically

  • identify Unit and Browser Tests
  • learn the expected behavior of each individual test
  • raise an incident when tests start behaving unexpected

Captured results of tests are stored in individual dynaTrace Sessions which makes it easy to compare and share.
More details on Test Automation in the Online Documentation.

Conclusion
Take your Continuous Integration Process to the next level by adding Performance, Scalability and Architectural Rule Validations without needing to write any additional tests. This allows you to find more problems earlier in the development lifecycle which will reduce the time spent in load testing and minimizes the risk of production problems.

Related reading:

  1. Week 6 – How to Make Developers Write Performance Tests I had an interesting conversation with our Test Automation team...
  2. How to use your Selenium Tests for automated JavaScript/AJAX Performance Analysis With more than 3 million downloads, Selenium is the world’s...
  3. Web Performance Optimization Use Cases – Part 3 Automation In the first and second post of this series I...
  4. Hands On Webinar with dynaTrace Dev-Edition on March 30th My colleague Ardeshir Arfaian (aka Ardy) is hosting a Webinar...

More Stories By Andreas Grabner

Andreas has over a decade of experience as an architect and developer, and currently works as a senior performance architect and technology strategist for dynaTrace Software, where he influences product strategy and works closely with customers in implementing performance management solutions across the application life cycle. He is a regular speaker at software conferences, writes for a number of technology publications, and blogs at http://blog.dynatrace.com

Cloud Expo Breaking News
What do the CTO of the U.S. Dept. of Justice and the CIO of the National Reconnaissance Office have in common with the CEOs of Eucalyptus, GoGrid, ActiveState, Appcara, OpSource and Nortonworks, the CTOs of Rackspace, SoftLayer and AppZero, the Founder & General Manager of Dell Boomi, the VP of Big Data & Streams at IBM and the Chief Strategy Officer at Pacific Controls? Answer: all are shortly to present breakout sessions as members of the distinguished Speaker Faculty of Cloud Expo New York, ...
The cloud has many benefits, but when it comes to application development, how does the cloud help enterprises and development teams create custom software and applications that end users actually care about? Using real world examples from Adobe, Herff Jones and Navy Federal Credit Union, this session will highlight the advantages cloud computing provides for quickly developing custom software and applications with compelling user experiences. In their general session at the 10th International ...
Nearly every enterprise is evaluating cloud computing solutions either today or in the near term. Many have already made the leap, and many more are getting close to putting that first toe in the water. But there are key considerations that should be made, questions to be asked, and designs to consider before you can feel secure with your provider. In his session at the 10th International Cloud Expo, David Gulick, Product Manager, Hosting Product Management at Savvis, will help give you food f...
With Cloud Expo 2012 New York (10th Cloud Expo) now under four weeks away, what better time to introduce you in greater detail to the distinguished individuals in our incredible Speaker Faculty for the technical and strategy sessions at the conference... We have technical and strategy sessions for you dealing with every nook and cranny of Cloud Computing, but what of those who are presenting? Who are they, where do they work, what else have they written and/or said about the Cloud that is t...
SYS-CON Events announced today that Super Micro Computer, Inc., a global leader in high-performance, high-efficiency server technology and green computing, will exhibit at SYS-CON's 10th International Cloud Expo, which will take place on June 11–14, 2012, at the Javits Center in New York City, New York. Supermicro (NASDAQ: SMCI), the leading innovator in high-performance, high-efficiency server technology, is a premier provider of advanced server Building Block Solutions for Embedded Systems, E...
SYS-CON Events announced today that ScaleMP, a leading provider of virtualization solutions for high-end computing, will exhibit at SYS-CON's 10th International Cloud Expo, which will take place on June 11–14, 2012, at the Javits Center in New York City, New York. ScaleMP is the leader in virtualization for high-end computing, providing maximum performance and lower total cost of ownership (TCO). The innovative Versatile SMP (vSMP) architecture aggregates multiple independent systems into a sin...
Come learn real-world examples where cloud and mobile are changing the way business works and the impact they're having on efficiency and productivity. In his session at the 10th International Cloud Expo, Rodrigo Coutinho Senior Product Marketing Manager at OutSystems, will look at how mobile and the cloud are interwoven and the wave of change these two 2012 megatrends will bring to your organization. He will also provide a roadmap to assure you can navigate this sea change for business succes...
Enterprise IT organizations want to deploy a virtualized data center fabric that will provide the foundation for agile private cloud computing. Getting there does not have to be difficult, but it does require a new approach to data center infrastructure design – an approach that is non-disruptive, vendor-agnostic, and very adaptable to changing business requirements. In his session at the 10th International Cloud Expo, Bruce Fingles, Chief Information Officer and VP of Product Quality at Xsigo...
With Cloud Expo 2012 New York (10th Cloud Expo) now under four weeks away, what better time to introduce you in greater detail to the distinguished individuals in our incredible Speaker Faculty for the technical and strategy sessions at the conference...
How can businesses harness the power of APIs to reach new customers and markets? In his session at the 10th International Cloud Expo, Alistair Farquharson, CTO at SOA Software, will walk the audience through the growth and evolution of the API, why effective API management is important, and how the game changes when companies expose business applications to the outside world. He will also discuss: A brief history of the API How to use APIs to make money, save money, build brand "Appificatio...