|By Andreas Grabner||
|October 5, 2014 05:30 PM EDT||
Software Quality Metrics for Your Continuous Delivery Pipeline | Part I
How often do you deploy new software? Once a month, once a week or every hour? The more often you deploy the smaller your changes will be. That's good! Why? Because smaller changes tend to be less risky since it's easier to keep track of what has really changed. For developers, it's certainly easier to fix something you worked on three days ago than something you wrote last summer. An analogy from a recent conference talk from AutoScout24 is to think about your release like a container ship, and every one of your changes is a container on that ship:
Your next software release en route to meet its iceberg
If all you know is that you have a problem in one of our containers you'd have to unpack and check all of them. That doesn't seem to make sense for a ship, and neither does it for a release. But that's still what happens quite frequently when a deployment fails and all you get is "it didn't work." In contrast, if you were shipping just a couple of containers you would be able to replace your giant, slow-maneuvering vessel with something faster and more agile - and if you're looking for a problem, you'd only have to inspect a handful of containers. While adopting this practice in the shipping industry would be a rather costly approach, this is exactly what continuous delivery allows us to do: Deploy more often, get faster feedback, and fix problems faster.
A great example is Amazon, who shared their success metrics at Velocity:
Some impressive stats from Amazon showing the success of rapid continuous delivery
However - even small changes can have severe impacts. Examples?
- Memory Leaks in Production: Introduced by a not well tested remote logging framework downloaded on GitHub
- Performance Impact of Exceptions in Ops: Ops and Dev did not follow the same deployment steps (lack of automation scripts) resulting in thousands of exceptions and maxes out CPU on all app servers
Extending Your Delivery Pipeline
Even small changes need to be tracked and their impact on overall software quality must be measured along the delivery pipeline so that your quality gates can stop even the smallest change from causing a huge issue. The three examples above could have been avoided when automatically looking at the following measures across the delivery pipeline and stopping the delivery when "architectural" regressions are detected:
- The number of DOM manipulations
- Memory usage or object churn rate per transaction
- Number of exceptions, number of database queries or number of log entries.
In a series of blog posts I will introduce you to metrics that you have to measure along your pipeline to act as an additional quality measure mechanism in order to prevent problems listed above. It is important that:
- Developers get these measurements in the commit stage
- Automation Engineers need to measure them for the automated unit and integration tests
- Performance Engineers add them to the load testing reports you do in staging
- Operations verify how the real application behaves after a new deployment in production
For each metric I introduce, I'll explain why it is important to monitor it, which types of problems can be detected and how Developers, Testers and Operations can monitor these metrics. To ready more on this, click here for the full article.
SYS-CON Events announced today that IDenticard will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. IDenticard™ is the security division of Brady Corp (NYSE: BRC), a $1.5 billion manufacturer of identification products. We have small-company values with the strength and stability of a major corporation. IDenticard offers local sales, support and service to our customers across the United States and Canada...
Apr. 22, 2015 12:00 AM EDT Reads: 5,039
SOASTA, the leader in performance analytics, today reported record growth of the CloudTest community, exceeding 30,000 registered users of the CloudTest platform in Q1 2015. SOASTA also announced widespread adoption of its Web and mobile testing solutions, with more than 1,600 customers completing more than 285,000 tests using CloudTest during the quarter. This rapid growth shows that DevOps-driven digital businesses are embracing a more continuous approach to testing, and CloudTest is meeting t...
Apr. 22, 2015 12:00 AM EDT Reads: 1,257
Public Cloud IaaS started it's life in the developer and startup communities and has grown rapidly to a $20B+ industry, but it still pales in comparison to how much is spent worldwide on IT: $3.6 trillion. In fact, there are 8.6 million data centers worldwide, the reality is many small and medium sized business have server closets and colocation footprints filled with servers and storage gear. While on-premise environment virtualization may have peaked at 75%, the Public Cloud has lagged in ado...
Apr. 21, 2015 11:45 PM EDT Reads: 872
Apr. 21, 2015 08:45 PM EDT Reads: 830
In this session we look at creating interactive communications via the web by adding messaging, file transfer, and group communication (group chat and audio/video conferencing) into the web experience. We will also discuss potential applications of this technology in areas including B2B, B2C, P2P, and gaming. Peter is Technical Director at Acision. He graduated from The University of Edinburgh in 2000 with a BSc (Hons) in Computer Science. After graduation Peter worked on a PSTN switch dev...
Apr. 21, 2015 08:30 PM EDT Reads: 730
While DevOps most critically and famously fosters collaboration, communication, and integration through cultural change, culture is more of an output than an input. In order to actively drive cultural evolution, organizations must make substantial organizational and process changes, and adopt new technologies, to encourage a DevOps culture. Moderated by Andi Mann, panelists will discuss how to balance these three pillars of DevOps, where to focus attention (and resources), where organizations m...
Apr. 21, 2015 05:00 PM EDT Reads: 1,900
Many of the well-known examples of DevOps success we read in blogs on the Internet paint an idyllic picture of DevOps productivity. A team was facing a stodgy, slow-moving operations department, teams weren’t delivering software on time. Those teams moved to DevOps, became proactive about infrastructure and deployment automation, and an overnight transition to productivity ensues. People are promoted, projects are successful, and developers and system administrators dance hand-in-hand in a final...
Apr. 21, 2015 03:45 PM EDT Reads: 730
SYS-CON Events announced today the IoT Bootcamp – Jumpstart Your IoT Strategy, being held June 9–10, 2015, in conjunction with 16th Cloud Expo and Internet of @ThingsExpo at the Javits Center in New York City. This is your chance to jumpstart your IoT strategy. Combined with real-world scenarios and use cases, the IoT Bootcamp is not just based on presentations but includes hands-on demos and walkthroughs. We will introduce you to a variety of Do-It-Yourself IoT platforms including Arduino, Ras...
Apr. 21, 2015 03:00 PM EDT Reads: 2,764
Health care systems across the globe are under enormous strain, as facilities reach capacity and costs continue to rise. M2M and the Internet of Things have the potential to transform the industry through connected health solutions that can make care more efficient while reducing costs. In fact, Vodafone's annual M2M Barometer Report forecasts M2M applications rising to 57 percent in health care and life sciences by 2016. Lively is one of Vodafone's health care partners, whose solutions enable o...
Apr. 21, 2015 03:00 PM EDT Reads: 1,231
SYS-CON Events announced today the DevOps Foundation Certification Course, being held June ?, 2015, in conjunction with DevOps Summit and 16th Cloud Expo at the Javits Center in New York City, NY. This sixteen (16) hour course provides an introduction to DevOps – the cultural and professional movement that stresses communication, collaboration, integration and automation in order to improve the flow of work between software developers and IT operations professionals. Improved workflows will res...
Apr. 21, 2015 02:00 PM EDT Reads: 2,265
Dave will share his insights on how Internet of Things for Enterprises are transforming and making more productive and efficient operations and maintenance (O&M) procedures in the cleantech industry and beyond. Speaker Bio: Dave Landa is chief operating officer of Cybozu Corp (kintone US). Based in the San Francisco Bay Area, Dave has been on the forefront of the Cloud revolution driving strategic business development on the executive teams of multiple leading Software as a Services (SaaS) ap...
Apr. 21, 2015 02:00 PM EDT Reads: 1,245
SYS-CON Events announced today that Vicom Computer Services, Inc., a provider of technology and service solutions, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. They are located at booth #427. Vicom Computer Services, Inc. is a progressive leader in the technology industry for over 30 years. Headquartered in the NY Metropolitan area. Vicom provides products and services based on today’s requirements...
Apr. 21, 2015 02:00 PM EDT Reads: 1,550
SYS-CON Media announced today that John Treadway’s blog has exceeded 475,000 page views. John Treadway, Vice President at Cloud Technology Partners, has surpassed 475,000 page views on the SYS-CON family of online magazines, which includes Cloud Computing Journal, Internet of Things Journal, Big Data Journal, Microservices Journal, and several others. His blog home page at SYS-CON can be found at JohnTreadway.SYS-CON.com.
Apr. 21, 2015 12:15 PM EDT Reads: 1,514
ProfitBricks has launched its new DevOps Central and REST API, along with support for three multi-cloud libraries and a Python SDK. This, combined with its already existing SOAP API and its new RESTful API, moves ProfitBricks into a position to better serve the DevOps community and provide the ability to automate cloud infrastructure in a multi-cloud world. Following this momentum, ProfitBricks has also introduced several libraries that enable developers to use their favorite language to code ...
Apr. 21, 2015 12:00 PM EDT Reads: 1,503
What exactly is a cognitive application? In her session at 16th Cloud Expo, Ashley Hathaway, Product Manager at IBM Watson, will look at the services being offered by the IBM Watson Developer Cloud and what that means for developers and Big Data. She'll explore how IBM Watson and its partnerships will continue to grow and help define what it means to be a cognitive service, as well as take a look at the offerings on Bluemix. She will also check out how Watson and the Alchemy API team up to off...
Apr. 21, 2015 12:00 PM EDT Reads: 1,654
The IoT Bootcamp is coming to Cloud Expo | @ThingsExpo on June 9-10 at the Javits Center in New York. Instructor. Registration is now available at http://iotbootcamp.sys-con.com/ Instructor Janakiram MSV previously taught the famously successful Multi-Cloud Bootcamp at Cloud Expo | @ThingsExpo in November in Santa Clara. Now he is expanding the focus to Janakiram is the founder and CTO of Get Cloud Ready Consulting, a niche Cloud Migration and Cloud Operations firm that recently got acquir...
Apr. 21, 2015 12:00 PM EDT Reads: 1,413
The 17th International Cloud Expo has announced that its Call for Papers is open. 17th International Cloud Expo, to be held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, APM, APIs, Microservices, Security, Big Data, Internet of Things, DevOps and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding bu...
Apr. 21, 2015 12:00 PM EDT Reads: 2,197
“In the past year we've seen a lot of stabilization of WebRTC. You can now use it in production with a far greater degree of certainty. A lot of the real developments in the past year have been in things like the data channel, which will enable a whole new type of application," explained Peter Dunkley, Technical Director at Acision, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Apr. 21, 2015 12:00 PM EDT Reads: 4,034
In 2015, 4.9 billion connected "things" will be in use. By 2020, Gartner forecasts this amount to be 25 billion, a 410 percent increase in just five years. How will businesses handle this rapid growth of data? Hadoop will continue to improve its technology to meet business demands, by enabling businesses to access/analyze data in real time, when and where they need it. Cloudera's Chief Technologist, Eli Collins, will discuss how Big Data is keeping up with today's data demands and how in t...
Apr. 21, 2015 11:00 AM EDT Reads: 1,129
The best mobile applications are augmented by dedicated servers, the Internet and Cloud services. Mobile developers should focus on one thing: writing the next socially disruptive viral app. Thanks to the cloud, they can focus on the overall solution, not the underlying plumbing. From iOS to Android and Windows, developers can leverage cloud services to create a common cross-platform backend to persist user settings, app data, broadcast notifications, run jobs, etc. This session provide...
Apr. 21, 2015 11:00 AM EDT Reads: 897