Welcome!

@CloudExpo Authors: Mauro Carniel, William Schmarzo, Mehdi Daoudi, Pat Romanski, Elizabeth White

Related Topics: @CloudExpo, Java IoT, Microservices Expo, @DXWorldExpo

@CloudExpo: Blog Post

Four Ways Cloud Has Influenced Application Troubleshooting By @Stackify | @CloudExpo [#Cloud]

As with any new, disruptive technology, new challenges are also par for the course

Four Ways Cloud Has Influenced Application Troubleshooting

The rise of cloud computing has ushered in an era of unprecedented productivity for developers over the past several years. For those who have embraced this new world order, gone are the days of long lead times for hardware procurement and installation, architecture defined by slow-moving hardware upgrades, hardware-constrained scalability and flexibility, and a world where only sys admins have access to the infrastructure. But, as the barriers between development and delivery disappear, new challenges have emerged that can disrupt the lives of developers and slow down delivery of new products and features, giving back some of the efficiency gains that the Software-Defined Data Center (SDDC) created.

Whether you're new to the cloud or you've been around since before cloud was cool, you are likely to see four common challenges emerge that can make troubleshooting your applications in the cloud more difficult. Let's take a closer look at these common pain points first to help build awareness around the challenges, and then I'll offer some suggestions for how to prevent these hurdles from tripping you and your team up when it comes time to unravel an application troubleshooting mystery.

Shifting Ownership
If you're adopting the cloud with limited support from an operations team, or perhaps you're one of the growing numbers of DevOps or even no-ops teams who find themselves bridging both the operations and development worlds, you will find that the responsibility of operating and supporting both your app and your infrastructure, at some level at least, will introduce a new dynamic that you may not have contemplated.

True, the ability to roll your own architecture without the burden of dealing with physical devices is liberating and far more efficient. But, as developer tools, deployment tools, and cloud operations tools become inextricably linked to one another, the old boundaries between who is dev and who is ops become blurred or even get removed altogether. This means the dev team is suddenly an integral part of operations, whether by design or by default, adding yet another responsibility for developers whose chief mandate is often to go faster. The more time you spend in the operations realm, especially in troubleshooting your app or the cloud resources it depends on, the less time you are able to devote to adding new value through code.

Lack of Transparency and Burden of Proof
While it's true that having the full benefits of the cloud available at the press of a button is awesome, you wouldn't be faulted for having a bit of nostalgia about the "good old days" of being able to have a conversation with a real live person down the hall about real physical hardware that's either healthy or isn't (along with the ability to actually lay hands on it). An old familiar refrain when something went wrong with an app in production was for the burden of proof to rest initially with the ops team - prove the hardware is working, the network is healthy, and the SAN hasn't lost disks before making the dev team dig in. Honestly, everyone was just hoping against hope that it was something "easy" in the infrastructure, because when it was the app, that's when things got hard. Well, that script has been reversed with the cloud: now the burden of proof is on the dev team, because what's really hard is finding a problem that originates with someone else's complex, abstracted, virtualized data center.

App returning a 500 error or performing poorly? If you're using something delivered as-a-Service, such as database, queues, cache and the like, you won't really have any visibility into health other than the cloud provider's status page and whatever you can directly observe. It's either working correctly and is speedy, or it isn't; if it isn't, life gets a lot murkier. Likewise, servers can be monitored, but you can't really tell why your virtual resource's performance has trailed off if you are the victim of something environmental that's out of your control.

No matter how good the support team is at your favorite cloud provider, it's rare that they will be as responsive to your requests for more information on an issue as your own in-house ops team could be, and they won't be as well versed on your architecture. To varying degrees, you're at the mercy of the cloud provider for consistent, reliable services, and it's also up to them to offer timely insight when issues arise with the services you depend on. Your mileage may vary, of course, as to whether your cloud provider offers this level of communication and transparency. But, if they don't, then the burden of proof rests squarely with you to show that the issue isn't in your app. Quite a reversal of fortunes, isn't it?

Easy Complexity
Compounding the challenge of sorting through infrastructure issues vs. code issues is the simple fact that applications are becoming far more complex and, in many cases, portions of the overall architecture may be transient in nature. Combine complexity with impermanence and you have a recipe for some real Sherlock Holmes-caliber mysteries at times.

The incredible thing about an SDDC is that you can create nearly any kind of architecture required to support your application stack's needs, all relatively easily - if you can dream it, you can build it. Want to cobble together .NET, Java, PHP, Node.js, Ruby, Database-as-a-Service for SQL and NoSQL, Message-Queues-as-a-Service, and Search-as-a-Service? From a cloud deployment perspective, it's been made devilishly easy to deploy and get started. But with that ultra-polyglot approach and a heavy reliance on software-defined services comes a new set of challenges:

  • First, you have a variety of services that are black boxes to you. Each of these services comes with its own set of tricks for gaining insight into performance and availability, but each one may be different in how you monitor and troubleshoot.
  • Learning how to support a variety of different technologies creates drag on your delivery velocity. It's hard enough learning the performance and reliability tricks for a few technologies; trying it for a wide variety can draw focus from the real goals of building new value through software and making the business more successful.
  • Not every monitoring tool can support every technology stack, and the wider you cast the technology net, the harder it can become to support your full stack from a single monitoring tool.
  • If you're using dynamic (transient) resources, such as scale-on-demand servers, you are quite likely to lose critical data that you need when troubleshooting a problem if you haven't given thought to how you preserve critical insights that disappear with the server when it's de-provisioned.

More Frequent Change
Finally, we come to the double-edged sword that brought this all about in the first place: going faster! The increased agility that the cloud brings, especially when coupled with dev tools that are integrated into the delivery cycle (think PaaS environments), has a way of shortening delivery cycle times and increasing the number of releases crammed into a given week, month, and year. This is especially true in organizations that have also adopted agile development practices. Code can flow to production smoothly with greater frequency, and architecture changes can be made far more swiftly and easily. Unfortunately, with more frequent code releases and architecture changes comes more frequent opportunities to break something.

A big part of the movement toward Agile and Lean is also the notion of always moving forward - rather than rolling back a release in the event of an issue, detect problems early and patch them quickly. To enable this mandate, however, requires two things that are often missing if you are coming from a slower moving environment or from a more traditional hosting model:

  1. Developer visibility into a baseline of behavior telemetry to know what "good" looks like historically
  2. Instant feedback on the health of the application post-release relative to that healthy baseline

Without this, it's hard to know if you've made gains or losses with your release - your users are often your only real barometer.

So... How Do I Code More and Support Less?
There's no denying that the cloud has impacted the life of many developers, mostly in a very positive way. Of course, with new technologies and capabilities always comes a new set of challenges to overcome. In the case of cloud-hosted applications, this includes challenges to effectively and efficiently support those applications in their new environments so that the gains in productivity aren't given back in support of the application.

What can development teams to do adapt to and overcome these challenges?

There are three basic steps that every development team should take to make supporting cloud-based applications easier.

1. Establish Access, Process, and Protocol: The first order of business for helping developers support their cloud-based apps more effectively is giving them safe access to the information and resources they need. Unfortunately, all too often in cloud environments this is an all-or-nothing proposition - full login rights to servers and even potentially full rights to the management portal, or no access at all. Make sure to establish the correct access methods to your developers so that they have the visibility and access they need, without handing over so much control that it increases the likelihood of accidents.

2. Design Supportability Into the Application: Once your application is in production, there are several common questions that you will need to be able to answer at a moment's notice about your application: Is it (and everything it depends on) running? Are users satisfied with the performance? Is anything silently failing and frustrating users without setting off alarms? If something failed, who was impacted, and what caused the issue?

There are also some things that simply cannot be measured and monitored from outside the application, but which speak directly to the health and well being of your application. To enable you to quickly answer the inevitable questions, consider incorporating the following:

  • If it moves, measure it. Report application metrics and KPIs from within your code in order to see events and data that would otherwise be locked away from you. Some events and metrics only you, the developer, have the power to expose. Knowing how your app behaves at a core level can provide levels of insight that prove invaluable when searching for troubleshooting clues. If you can configure monitoring and alerts for those metrics, even better. We've elaborate on this subject in this article Errors & Logs: putting the data to work
  • Log often, and log meaningfully. If you only report errors, you will lack the critical insights necessary to help point to the root cause of the error. By logging at, say, info or debug instead of just warn or error, you will have the breadcrumb trail you need to find it. It's impossible to get the state of the system after the fact - you need to have logged it at the time of the event.
  • Centralize your insights. Remembering that life in the cloud can be both quite distributed, and quite transient, it's always good to bring everything - logs, errors, custom metrics, and other telemetry - into a central location for normalization, correlation, and continuity. You may need the data, and what it tells you, well beyond the ephemeral life span of your cloud resource.

3. Identify Health Baselines Early: Key information like message queue length, average request time, app pool resource utilization, custom metrics values, log and error rates, and more can all be charted for your application these days - monitoring and charting isn't just the domain of ops tools any longer. Understand what your app looks like both when healthy and unhealthy, preferably starting with pre-production environments even, so that you can see how your application morphs from release to release as well as with different loads and as your architecture evolves. By baselining as far back as dev and QA, you can often catch problems well before they impact customers and send you and your team scrambling.

Conclusion
There's no denying the cloud brings incredible capabilities to the lives of developers: speed, agility, flexibility, scalability, and more. As with any new, disruptive technology, new challenges are also par for the course. By applying some basic strategies for application management, monitoring and troubleshooting, you can have all of the advantages of the cloud without giving back the gains during those critical support engagements, and have happier team members and end users as well.

At Stackify we believe we offer a solution to the issues presented in this article learn more at www.stackify.com

More Stories By Stackify Blog

Stackify offers the only developers-friendly solution that fully integrates error and log management with application performance monitoring and management. Allowing you to easily isolate issues, identify what needs to be fixed quicker and focus your efforts – Support less, Code more. Stackify provides software developers, operations and support managers with an innovative cloud based solution that gives them DevOps insight and allows them to monitor, detect and resolve application issues before they affect the business to ensure a better end user experience. Start your free trial now stackify.com

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@CloudExpo Stories
"Infoblox does DNS, DHCP and IP address management for not only enterprise networks but cloud networks as well. Customers are looking for a single platform that can extend not only in their private enterprise environment but private cloud, public cloud, tracking all the IP space and everything that is going on in that environment," explained Steve Salo, Principal Systems Engineer at Infoblox, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Conventio...
"Cloud Academy is an enterprise training platform for the cloud, specifically public clouds. We offer guided learning experiences on AWS, Azure, Google Cloud and all the surrounding methodologies and technologies that you need to know and your teams need to know in order to leverage the full benefits of the cloud," explained Alex Brower, VP of Marketing at Cloud Academy, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clar...
In his session at 21st Cloud Expo, Carl J. Levine, Senior Technical Evangelist for NS1, will objectively discuss how DNS is used to solve Digital Transformation challenges in large SaaS applications, CDNs, AdTech platforms, and other demanding use cases. Carl J. Levine is the Senior Technical Evangelist for NS1. A veteran of the Internet Infrastructure space, he has over a decade of experience with startups, networking protocols and Internet infrastructure, combined with the unique ability to it...
The question before companies today is not whether to become intelligent, it’s a question of how and how fast. The key is to adopt and deploy an intelligent application strategy while simultaneously preparing to scale that intelligence. In her session at 21st Cloud Expo, Sangeeta Chakraborty, Chief Customer Officer at Ayasdi, provided a tactical framework to become a truly intelligent enterprise, including how to identify the right applications for AI, how to build a Center of Excellence to oper...
"IBM is really all in on blockchain. We take a look at sort of the history of blockchain ledger technologies. It started out with bitcoin, Ethereum, and IBM evaluated these particular blockchain technologies and found they were anonymous and permissionless and that many companies were looking for permissioned blockchain," stated René Bostic, Technical VP of the IBM Cloud Unit in North America, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Conventi...
Gemini is Yahoo’s native and search advertising platform. To ensure the quality of a complex distributed system that spans multiple products and components and across various desktop websites and mobile app and web experiences – both Yahoo owned and operated and third-party syndication (supply), with complex interaction with more than a billion users and numerous advertisers globally (demand) – it becomes imperative to automate a set of end-to-end tests 24x7 to detect bugs and regression. In th...
In his session at 21st Cloud Expo, James Henry, Co-CEO/CTO of Calgary Scientific Inc., introduced you to the challenges, solutions and benefits of training AI systems to solve visual problems with an emphasis on improving AIs with continuous training in the field. He explored applications in several industries and discussed technologies that allow the deployment of advanced visualization solutions to the cloud.
Agile has finally jumped the technology shark, expanding outside the software world. Enterprises are now increasingly adopting Agile practices across their organizations in order to successfully navigate the disruptive waters that threaten to drown them. In our quest for establishing change as a core competency in our organizations, this business-centric notion of Agile is an essential component of Agile Digital Transformation. In the years since the publication of the Agile Manifesto, the conn...
"MobiDev is a software development company and we do complex, custom software development for everybody from entrepreneurs to large enterprises," explained Alan Winters, U.S. Head of Business Development at MobiDev, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Large industrial manufacturing organizations are adopting the agile principles of cloud software companies. The industrial manufacturing development process has not scaled over time. Now that design CAD teams are geographically distributed, centralizing their work is key. With large multi-gigabyte projects, outdated tools have stifled industrial team agility, time-to-market milestones, and impacted P&L stakeholders.
"ZeroStack is a startup in Silicon Valley. We're solving a very interesting problem around bringing public cloud convenience with private cloud control for enterprises and mid-size companies," explained Kamesh Pemmaraju, VP of Product Management at ZeroStack, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Enterprises are adopting Kubernetes to accelerate the development and the delivery of cloud-native applications. However, sharing a Kubernetes cluster between members of the same team can be challenging. And, sharing clusters across multiple teams is even harder. Kubernetes offers several constructs to help implement segmentation and isolation. However, these primitives can be complex to understand and apply. As a result, it’s becoming common for enterprises to end up with several clusters. Thi...
"Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"Codigm is based on the cloud and we are here to explore marketing opportunities in America. Our mission is to make an ecosystem of the SW environment that anyone can understand, learn, teach, and develop the SW on the cloud," explained Sung Tae Ryu, CEO of Codigm, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
High-velocity engineering teams are applying not only continuous delivery processes, but also lessons in experimentation from established leaders like Amazon, Netflix, and Facebook. These companies have made experimentation a foundation for their release processes, allowing them to try out major feature releases and redesigns within smaller groups before making them broadly available. In his session at 21st Cloud Expo, Brian Lucas, Senior Staff Engineer at Optimizely, discussed how by using ne...
Vulnerability management is vital for large companies that need to secure containers across thousands of hosts, but many struggle to understand how exposed they are when they discover a new high security vulnerability. In his session at 21st Cloud Expo, John Morello, CTO of Twistlock, addressed this pressing concern by introducing the concept of the “Vulnerability Risk Tree API,” which brings all the data together in a simple REST endpoint, allowing companies to easily grasp the severity of the ...
While some developers care passionately about how data centers and clouds are architected, for most, it is only the end result that matters. To the majority of companies, technology exists to solve a business problem, and only delivers value when it is solving that problem. 2017 brings the mainstream adoption of containers for production workloads. In his session at 21st Cloud Expo, Ben McCormack, VP of Operations at Evernote, discussed how data centers of the future will be managed, how the p...
"NetApp is known as a data management leader but we do a lot more than just data management on-prem with the data centers of our customers. We're also big in the hybrid cloud," explained Wes Talbert, Principal Architect at NetApp, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...
"We're focused on how to get some of the attributes that you would expect from an Amazon, Azure, Google, and doing that on-prem. We believe today that you can actually get those types of things done with certain architectures available in the market today," explained Steve Conner, VP of Sales at Cloudistics, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.