Welcome!

Cloud Expo Authors: Jason Bloomberg, Sandi Mappic, Roger Strukhoff, Liz McMillan, Pat Romanski

Related Topics: DevOps Journal, Java, SOA & WOA, .NET, Virtualization, Cloud Expo, Big Data Journal

DevOps Journal: Article

I’m Not Scared of DevOps and You Shouldn’t Be Either

The Foundation of DevOps

DevOps is speeding towards the IT world like a freight train and the hype around it is deafening. There is no reason to be afraid of this change as it is the natural reaction to the agile movement that revolutionized development just a few years ago. By definition, DevOps is the natural alignment of IT performance to business profitability. The relevance of this has yet to be quantified but it has been suggested that the route to the CEO's chair will come from the IT leaders that successfully make the transition to a DevOps model. If this still seems foreign to you, I recommend reading up on DevOps Blog from IT Revolution, the OpsCode Blog, and check out The Phoenix Project.

Despite all the talk around simple monitoring tools, breaking through the walls between Dev and Ops still poses a real challenge. This is because of a misunderstanding around Operations real purpose - extracting real value from its resources. According to Kevin Behr the definition of Operations is the act of harvesting value from IT resources. Anything that prevents this from happening is a detriment to the business. This means that firefighting and war room sessions are a hindrance to the DevOps model. The following screenshots are good examples of a war room scenario.

Unexpected crashes of websites upon new rollouts still lead to "War Room" situations - despite all the good efforts of DevOps and Agile Delivery/Deployment

Successfully riding the DevOps train: Many of our production customers that made it through the firefighting mode applied the principals of DevOps with a special focus on Application Performance. In this article we describe the steps and milestones companies need to go through in order to level-up their Operations and Engineering Teams to provide more value out of the existing resources.

The Foundation of DevOps
CAMS (Culture, Automation, Measurement, Sharing) are four key areas that are core to the DevOps movement. Culture is the hardest to change but is also the most important because it means a change in the way in which the different teams work together and share the responsibility for the end users of their application. It promotes the usage of development practices in operations to automate deployment. It also allows developers to learn from "the real world" Ops experience and with that mutual exchange it breaks down the walls.

The Lack of Performance Focus
An interesting fact
based on the feedback we get from operation teams worldwide: The root cause for about 80% of site crashes or performance problems is related to only about 20% of problem patterns. Want to learn more? Check out blogs such as Top Performance Landmines in Production and I am sure you'll find some issues you already ran into yourself.

Looking at these common problem patterns it is clear that despite all the DevOps efforts lots of performance and scalability-related problems still make it into a release deployment. Why is that? Because our organizations are still very much driven by business requirements that need numerous new features being pushed in ever shorter release cycles. Teams keep growing and are being spread around the world. In order to keep up with the pace, third-party components are included in the code in place of in-house innovation. This "natural" evolution however is also the root cause for firefights and limiting the benefits of DevOps because there is too much focus on pushing functionality through the Deployment Pipeline but not enough focus on Performance.

More developers across more locations including more untested 3rd party code with less time to focus on performance

Plugging Performance into DevOps
In order to focus developers on performance to avoid War Room scenarios you must plug performance into the four pillars of CAMS:

  • Culture: Performance as Key Requirement in Dev, Test and Ops
  • Automation: Automated Performance Tests already in Continuous Integration
  • Measurement: Measure Key Performance Metrics in CI, Test and Ops
  • Sharing: Share the same tools and same performance data across Dev, Test and Ops

There are several key milestones to consider:

Milestone 1: Level-Up Performance to Increase Feedback Between Ops and Development
The first step in any DevOps initiative is to get the Ops teams and the Dev teams talking in order to relieve constraints on the business. This might be easy for small teams to accomplish but the larger the organization, the more difficult it becomes as constraints are greater in larger organizations. Operations has to diminish these constraints on the business. This is where APM solutions can help. Beware as not all solutions are created equal. As mentioned in the previous DevOps blog the drive to diminish constraint needs to be applied across the delivery chain. Monitoring just does not cut it here. There needs to be something that not only starts the process but allows the teams to continue to mature and grow. Simple monitoring tools fall short because they only help extinguish fires in operations. How does this continue to drive down constraints? Remember, firefighting is not a part of operations, which means operations should not be looking at fire extinguishers for their DevOps strategies.

Milestone 2: Level-Up Performance Thinking of Engineering
Both Operations and Test Teams have a good understanding of Performance as they deal with it every day. These teams need to educate engineering on the importance of performance as it is a key requirement to software engineering and how it plays a role in large-scale environments under heavy load.

The Ops team shares data with engineering to highlight the performance behavior of their applications under real production load. This helps engineers to prevent these top performance problems from entering production and with that eliminating the need for firefights.

The test teams do their share by providing automated performance test frameworks and educating engineering on how to automate testing for these performance problem patterns.

Milestone 3: Level-Up Load and Capacity Testing
With development executing its own performance tests it's time to level up the test team as well. On one side there is more time to focus on large-scale load tests that need to be executed in a production- like environment. This helps to find any "data-driven", scalability, and "third-party impacted" performance problems. Close collaboration with Ops ensures that tests can be executed either in the prod environment or in a staged environment that mirrors production. Executing these tests in collaboration with Ops allows the teams to become more confident when releasing a new version and also helps with proper capacity planning steps.

Running tests against the production system gives better input for capacity planning and uncovers heavy load application issues

Milestone 4: Level-Up Performance Test Automation
The "traditional" testing teams are used to execute performance and scalability tests in their own environments at the end of a milestone. The goal is to provide these test frameworks and environments to engineering so that these basic performance tests can be executed automatically in the CI environment. In order for this to work you need to make sure that:

  1. These test frameworks are easy to use and accepted by developers
  2. Deliver performance metrics to detect the common problem patterns
  3. These are fully integrated into continuous integration

Automatic Integration Tests run in C/I to detect performance regressions on metrics such as # of SQL Calls, Page Load Time, # of JS files or Images ...

What's Next? Build a Performance Center of Excellence
Many of our customers who jumped on the DevOps train a while back are now promoting a performance culture in their organizations. In the next few blogs we will cover their best practices and tips on either building a separate "Performance Center of Excellence" Team or up-leveling the existing DevOps teams to deliver software with high confidence and fewer War Room weekends.

More Stories By Andreas Grabner

Andreas Grabner has more than a decade of experience as an architect and developer in the Java and .NET space. In his current role, Andi works as a Technology Strategist for Compuware and leads the Compuware APM Center of Excellence team. In his role he influences the Compuware APM product strategy and works closely with customers in implementing performance management solutions across the entire application lifecycle. He is a frequent speaker at technology conferences on performance and architecture-related topics, and regularly authors articles offering business and technology advice for Compuware’s About:Performance blog.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Cloud Expo Breaking News
There will be 50 billion Internet connected devices by 2020. Today, every manufacturer has a propriety protocol and an app. How do we securely integrate these "things" into our lives and businesses in a way that we can easily control and manage? Even better, how do we integrate these "things" so that they control and manage each other so our lives become more convenient or our businesses become more profitable and/or safe? We have heard that the best interface is no interface. In his session at Internet of @ThingsExpo, Chris Matthieu, Co-Founder & CTO at Octoblu, Inc., will discuss how these devices generate enough data to learn our behaviors and simplify/improve our lives. What if we could connect everything to everything? I'm not only talking about connecting things to things but also systems, cloud services, and people. Add in a little machine learning and artificial intelligence and now we have something interesting...
SYS-CON Events announced today that Esri has been named “Bronze Sponsor” of SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Esri inspires and enables people to positively impact the future through a deeper, geographic understanding of the changing world around them. For more information, visit http://www.esri.com.
The Internet of Things is not new. Historically, smart businesses have used its basic concept of leveraging data to drive better decision making and have capitalized on those insights to realize additional revenue opportunities. So, what has changed to make the Internet of Things one of the hottest topics in tech? In his session at Internet of @ThingsExpo, Chris Gray, Director, Embedded and Internet of Things, will discuss the underlying factors that are driving the economics of intelligent systems. Discover how hardware commoditization, the ubiquitous nature of connectivity, and the emergence of Big Data and analysis are providing the pull to meet customer expectations of a widely connected, multi-dimensional universe of people, things, and information.
After a couple of false starts, cloud-based desktop solutions are picking up steam, driven by trends such as BYOD and pervasive high-speed connectivity. In his session at 15th Cloud Expo, Seth Bostock, CEO of IndependenceIT, cuts through the hype and the acronyms, and discusses the emergence of full-featured cloud workspaces that do for the desktop what cloud infrastructure did for the server. He’ll discuss VDI vs DaaS, implementation strategies and evaluation criteria.
Cloud computing started a technology revolution; now DevOps is driving that revolution forward. By enabling new approaches to service delivery, cloud and DevOps together are delivering even greater speed, agility, and efficiency. No wonder leading innovators are adopting DevOps and cloud together! In his session at DevOps Summit, Andi Mann, Vice President of Strategic Solutions at CA Technologies, will explore the synergies in these two approaches, with practical tips, techniques, research data, war stories, case studies, and recommendations.
SYS-CON Events announced today that Cloudian, Inc., the leading provider of hybrid cloud storage solutions, has been named “Bronze Sponsor” of SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Cloudian is a Foster City, Calif.-based software company specializing in cloud storage. Cloudian HyperStore® is an S3-compatible cloud object storage platform that enables service providers and enterprises to build reliable, affordable and scalable hybrid cloud storage solutions. Cloudian actively partners with leading cloud computing environments including Amazon Web Services, Citrix Cloud Platform, Apache CloudStack, OpenStack and the vast ecosystem of S3 compatible tools and applications. Cloudian's customers include Vodafone, Nextel, NTT, Nifty, and LunaCloud. The company has additional offices in China and Japan.
Cloud Computing is evolving into a Big Three of Amazon Web Services, Google Cloud, and Microsoft Azure. Cloud 360: Multi-Cloud Bootcamp, being held Nov 4–5, 2014, in conjunction with 15th Cloud Expo in Santa Clara, CA, delivers a real-world demonstration of how to deploy and configure a scalable and available web application on all three platforms. The Cloud 360 Bootcamp, led by Janakiram MSV, an analyst with Gigaom Research, is the first bootcamp that introduces the core concepts of Infrastructure as a Service (IaaS) based on the workings of the Big Three platforms – Amazon EC2, Google Compute Engine, and Azure VMs. Bootcamp attendees will get to see the big picture and also receive the knowledge needed to make the best cloud decisions for their business applications and entire enterprise IT organization.
“Distrix fits into the overall cloud and IoT model around software-defined networking. There’s a broad category around software-defined networking that’s focused on data center, and we focus on the WAN,” explained Jay Friedman, President of Distrix, in this SYS-CON.tv interview at the Internet of @ThingsExpo, held June 10-12, 2014, at the Javits Center in New York City. Internet of @ThingsExpo 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading IoT industry players in the world.
The Internet of Things promises to transform businesses (and lives), but navigating the business and technical path to success can be difficult to understand. In his session at 15th Internet of @ThingsExpo, Chad Jones, Vice President, Product Strategy of LogMeIn's Xively IoT Platform, will show you how to approach creating broadly successful connected customer solutions using real world business transformation studies including New England BioLabs and more.
Scott Jenson leads a project called The Physical Web within the Chrome team at Google. Project members are working to take the scalability and openness of the web and use it to talk to the exponentially exploding range of smart devices. Nearly every company today working on the IoT comes up with the same basic solution: use my server and you'll be fine. But if we really believe there will be trillions of these devices, that just can't scale. We need a system that is open a scalable and by using the URL as a basic building block, we open this up and get the same resilience that the web enjoys.
“The Internet of Things is a wave that has arrived and it’s growing really fast. The concern at Aria Systems is making sure that people understand the ramifications of their attempts to monetize whatever it is they build on the Internet of Things," explained C Brendan O’Brien, Co-founder and Chief Architect at Aria Systems, in this SYS-CON.tv interview at the Internet of @ThingsExpo, held June 10-12, 2014, at the Javits Center in New York City. Internet of @ThingsExpo 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading IoT industry players in the world.
The Internet of Things is a natural complement to the cloud and related technologies such as Big Data, analytics, and mobility. In his session at Internet of @ThingsExpo, Joe Weinman will lay out four generic strategies – digital disciplines – to exploit emerging digital technologies for strategic advantage. Joe Weinman has held executive leadership positions at Bell Labs, AT&T, Hewlett-Packard, and Telx, in areas such as corporate strategy, business development, product management, operations, and R&D.
SYS-CON Events announced today that DevOps.com has been named “Media Sponsor” of SYS-CON's “DevOps Summit at Cloud Expo,” which will take place on June 10–12, 2014, at the Javits Center in New York City, New York. DevOps.com is where the world meets DevOps. It is the largest collection of original content relating to DevOps on the web today Featuring up-to-the-minute news, feature stories, blogs, bylined articles and more, DevOps.com is where the thought leaders of the DevOps movement make their ideas known.
There are 182 billion emails sent every day, generating a lot of data about how recipients and ISPs respond. Many marketers take a more-is-better approach to stats, preferring to have the ability to slice and dice their email lists based numerous arbitrary stats. However, fundamentally what really matters is whether or not sending an email to a particular recipient will generate value. Data Scientists can design high-level insights such as engagement prediction models and content clusters that allow marketers to cut through the noise and design their campaigns around strong, predictive signals, rather than arbitrary statistics. SendGrid sends up to half a billion emails a day for customers such as Pinterest and GitHub. All this email adds up to more text than produced in the entire twitterverse. We track events like clicks, opens and deliveries to help improve deliverability for our customers – adding up to over 50 billion useful events every month. While SendGrid data covers only abo...
SYS-CON Events announced today that the Web Host Industry Review has been named “Media Sponsor” of SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Since 2000, The Web Host Industry Review has made a name for itself as the foremost authority of the Web hosting industry providing reliable, insightful and comprehensive news, reviews and resources to the hosting community. TheWHIR Blogs provides a community of expert industry perspectives. The Web Host Industry Review Magazine also offers a business-minded, issue-driven perspective of interest to executives and decision-makers. WHIR TV offers on demand web hosting video interviews and web hosting video features of the key persons and events of the web hosting industry. WHIR Events brings together like-minded hosting industry professionals and decision-makers in local communities. TheWHIR is an iNET Interactive property.