Welcome!

@CloudExpo Authors: Liz McMillan, Pat Romanski, Flint Brenton, Elizabeth White, Cameron Van Orman

Related Topics: @CloudExpo, Microservices Expo, Containers Expo Blog

@CloudExpo: Article

Do-It-Yourself Cloud Computing Management – Is It Worth It?

Do all the math, not just some of it

Does your IT or devops team run a lot of do-it-yourself projects? DIY is tempting, isn't it? You have smart engineers, easy access to storage and computing power, and a mandate this year to start moving some of (or more of) your applications to the cloud. Why not do it yourself?

I'm not saying that you shouldn't try DIY cloud computing management, but if you're going that route, you should know what you're in for. In particular, you should keep a sharp lookout for the point of no (easy) return.

The Five Cs of DIY
First of all, what compels people to try to roll their own end-to-end cloud management strategy? There are five arguments we often hear for DIY (all of which begin with "C"), depicted in Figure 1.

  1. Control: When you do it yourself, you're able to control everything, right, wrong or indifferent. If there are problems, you can handle them internally. If your business requires custom scripts or exporting usage data into your own warehouse, you'll want to be able flip the levers and configure it yourself. And the images will be yours, all yours.
  2. Complexity: "What's to manage?" you ask. "We're just spinning up a dozen or so dev and stage servers in an Amazon EC2 public cloud to get our feet wet. Plenty of companies do this without all the fuss of additional management tools." When your initial objectives are modest and you want your DevOps team to learn how things work first, cloud computing management looks like just a nice-to-have.
  3. Conquest: Speaking of DevOps, plenty of them want the challenge of understanding what's possible and how to build it. Or some of them say, "If I can't get approval, I'll just do it myself on the side," and they jump in. Because we are still in the early stages of cloud uptake, your engineers and managers are experimenting to discover just how far DIY and cobbling different tool sets together will take them, as opposed to researching what is available in the market. Who can blame them? The cloud presents itself as one of the most powerful APIs ever, and what self-respecting developer or IT department can resist that?
  4. Cost: Everybody's eye is on cost. "For about $200 a month, I can get five large, on-demand Linux instances for 100 hours, 50GB of storage and 10GB of data in and out. With pricing like that, we'll figure out the management piece ourselves. Most of what we'd buy in a cloud management product is stuff we can do ourselves anyway." It certainly looks like that at first glance, and if the budget is tight and your cloud-based DIY deployment is humming along, you probably can't make the case for spending money on management expertise that you're convinced you can provide on your own.
  5. Convenience: It's convenient and easy to think about DIY cloud management because Amazon, for instance, gives you so many options and tools to start with. If there were many long-standing public clouds from which to choose, the water would be muddier, you'd use a sharper pencil in the vetting process, and doing it yourself would seem like less of a slam dunk.

People stack these five Cs against off-the-shelf cloud management offerings all the time. Our industry spends a lot of time hearing these arguments, patiently nodding our heads and repeating the counter-arguments, based on our own experience with customers:

If you really need control, you'd be surprised by how much control and customization you can have with a cloud management product, even when you start with pre-configured images and templates.

Cloud computing management is complex, but cloud management products are designed to shield you from most of the complexity. Besides, the sooner you see how simple it is to automate the management of five or ten servers, the sooner you can get up to the 50 or 100 your business really needs.

There's plenty of technology to conquer in cloud computing management, but the industry has already conquered most of it, which is why off-the-shelf products are so comprehensive and accessible.

The cost of DIY is usually a lot more than your monthly fees. There's recruiting, training, non-recurring engineering expenses, headcount and the risk involved in building and maintaining your team. People don't always think that far down the road.

The flip-side of convenience is lock-in. It's hard to resist the ease of spinning up a cloud with just a browser and a credit card; however, is the product you conveniently start with today going to grow and scale gracefully with your business? Do you want all of your eggs in one basket? You've got to ask yourself, "Do I feel lucky?"

Let's examine the technology more closely.

Eyes Wide Open: Know What You're in For
What does cloud computing management mean? How many different layers are there to it, and when will you hit each one? Figure 2 depicts the layers of cloud computing management you'll traverse eventually, whether you DIY or license a product.

Basic cloud offerings from Amazon and open source tools can cover the entry-level echelon of service (bottom of Figure 2). A combination of DIY and open source is not especially dangerous at these levels:

  1. You provision basic or pre-configured images to meet your specifications and the needs of your users and customers, with memory, computing power, storage, OS and geographic proximity.
  2. Once you've installed applications in your cloud, you'll want to monitor them. Are they running properly? Have any of them gone down? Can you get alerts if something goes wrong?
  3. After you've tweaked your images, you'll want to clone them, say, for development/staging/production, for multiple developers or to meet increasing demand and traffic.
    Once you've gotten this far, you can also try products for cloud auto-scaling (originally invented by RightScale). With a few months and several servers under your belt, you arrive at the advanced echelon of service in need of yet another tool, because entry-level products don't cover these (middle of Figure 2):
  4. It's easy to take snapshots of images; sometimes too easy. You forget why you created them, what's inside them and whether anybody is still using them. You need configuration management to deal with image sprawl. Once you have that under control, you also need configuration management to create repeatable applications and services (for your own mini Platforms-as-a-Service).

With people all over the organization clamoring for cloud-based apps, you're ready for user management to set permissions and audit activity.

The cloud may be inexpensive, but it isn't free. When Finance asks you about cost allocation and ROI by department, project and region, you need tools that can break out your expenses and revenues from activity in the cloud.

The last three items are the state of the art in cloud computing management, and if you can get that far on DIY, you deserve a big raise. On the horizon is a final layer, still in its infancy (top of Figure 2).

Most of your cloud assets need to work together; e.g., start the database server first, then start up the application tier, then start the proxy, run some tests to make sure the whole app is working, and then turn on the website. Orchestration and workflow automation will soon allow you to code how your system should operate so that you don't need to intervene.

That's the long view of cloud computing management. Set your expectations accordingly.

Are You Close to the Tipping Point?
The market is torn at the moment. On the DIY side, cloud computing is not very old, and lots of organizations are scrambling to figure out what it can do for them and their business. There are plenty of eager, curious engineers ready to dive in, fire up a few servers, cut their teeth on cloud computing and a few open source tools, and do their company (and their résumé) some good.

On the off-the-shelf side, the market is filled with entrants and it's growing up fast. Cloud management products are feature-rich because those of us who focus on them have already hit most of the roadblocks.

For some companies, the choice of off-the-shelf is obvious. They've looked at Figure 2 and decided that they don't want to have to do it themselves. They tried rolling their own CRM until they saw how effortless Salesforce.com made it, and they remember tinkering for ages with their own Web servers until they realized Apache had nailed it. They've learned that lesson.

What's the tipping point for everybody else? When do features, capabilities and price point tip in favor of off-the-shelf products? From our experience, here are three telling metrics:

  1. Forty images - Once people try to keep track of more than 40 images, DIY cloud management tools start to get creaky. "I see a whole slew of servers, and some have really short names...I've forgotten what that one does..."
  2. Fifteen users - If they have 15 people operating in the cloud after only a couple of months, they're liable to be at 200 in a year. User management tools need to scale and work with all the other DIY tools in use.
  3. Five accounts - To sort out their billing and ROI headaches, they write discovery apps or go through the Amazon API trying to figure out which instances are in use and which images they come from so they can allocate costs correctly.

Alas, some organizations stick with DIY past that tipping point, to the point of no (easy) return. They've made it well into the advanced echelon, but only by cobbling together a patchwork of three to four disparate tool sets and growing their DevOps teams to 50 or more. Or, perhaps they decide to move all their DIY stuff to a different public cloud provider, or to their own private cloud. An off-the-shelf cloud computing management can still help them when they get to these points, but the effort will cost much more time and money than if they had started there in the first place.

Do All the Math, Not Just Some of It
Many customers come to us after they've outgrown their DIY efforts. Eventually, they discover that there are too many things to stitch together: configuration management, systems automation, monitoring, application automation, provisioning, user permissions, reporting and more.

Even if you're happy with the DIY cloud computing management you've put in place, are you really sure that it's worth the investment in time, money and manpower, compared to an off-the-shelf offering? Not only that, but are you sure you're far enough away from the tipping point that DIY will still look good a year from now?


Cloud Management - Obstacles Overcome in Off-the-Shelf Products

  • Removal of a single user's SSH key from all managed instances
  • Volume striping for better database performance
  • Image fingerprinting to identify pre-rolled images in private clouds accurately

More Stories By Darryl Eaton

Darryl Eaton is the director of product management at RightScale Inc. Not only does he ship products that are ahead of their time and blog about it, but he has also been known to play the Carillon, the largest and loudest musical instrument in the world. He wants you to make the right decision about DIY and off-the-shelf, so contact him at [email protected] and find out more about cloud management and the Five Cloud Commandments in RightScale’s library.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@CloudExpo Stories
Join IBM November 1 at 21st Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA, and learn how IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Cognitive analysis impacts today’s systems with unparalleled ability that were previously available only to manned, back-end operations. Thanks to cloud processing, IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Imagine a robot vacuum that becomes your personal assistant th...
Today traditional IT approaches leverage well-architected compute/networking domains to control what applications can access what data, and how. DevOps includes rapid application development/deployment leveraging concepts like containerization, third-party sourced applications and databases. Such applications need access to production data for its test and iteration cycles. Data Security? That sounds like a roadblock to DevOps vs. protecting the crown jewels to those in IT.
SYS-CON Events announced today that B2Cloud will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. B2Cloud specializes in IoT devices for preventive and predictive maintenance in any kind of equipment retrieving data like Energy consumption, working time, temperature, humidity, pressure, etc.
SYS-CON Events announced today that Cedexis will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Cedexis is the leader in data-driven enterprise global traffic management. Whether optimizing traffic through datacenters, clouds, CDNs, or any combination, Cedexis solutions drive quality and cost-effectiveness.
Many companies start their journey to the cloud in the DevOps environment, where software engineers want self-service access to the custom tools and frameworks they need. Machine learning technology can help IT departments keep up with these demands. In his session at 21st Cloud Expo, Ajay Gulati, Co-Founder, CTO and Board Member at ZeroStack, will discuss the use of machine learning for automating provisioning of DevOps resources, taking the burden off IT teams.
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp em...
Is advanced scheduling in Kubernetes achievable? Yes, however, how do you properly accommodate every real-life scenario that a Kubernetes user might encounter? How do you leverage advanced scheduling techniques to shape and describe each scenario in easy-to-use rules and configurations? In his session at @DevOpsSummit at 21st Cloud Expo, Oleg Chunikhin, CTO at Kublr, will answer these questions and demonstrate techniques for implementing advanced scheduling. For example, using spot instances ...
Why Federal cloud? What is in Federal Clouds and integrations? This session will identify the process and the FedRAMP initiative. But is it sufficient? What is the remedy for keeping abreast of cutting-edge technology? In his session at 21st Cloud Expo, Rasananda Behera will examine the proposed solutions: Private or public or hybrid cloud Responsible governing bodies How can we accomplish?
SYS-CON Events announced today that Ryobi Systems will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Ryobi Systems Co., Ltd., as an information service company, specialized in business support for local governments and medical industry. We are challenging to achive the precision farming with AI. For more information, visit http:...
Cloud-based disaster recovery is critical to any production environment and is a high priority for many enterprise organizations today. Nearly 40% of organizations have had to execute their BCDR plan due to a service disruption in the past two years. Zerto on IBM Cloud offer VMware and Microsoft customers simple, automated recovery of on-premise VMware and Microsoft workloads to IBM Cloud data centers.
SYS-CON Events announced today that mruby Forum will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. mruby is the lightweight implementation of the Ruby language. We introduce mruby and the mruby IoT framework that enhances development productivity. For more information, visit http://forum.mruby.org/.
Cloud applications are seeing a deluge of requests to support the exploding advanced analytics market. “Open analytics” is the emerging strategy to deliver that data through an open data access layer, in the cloud, to be directly consumed by external analytics tools and popular programming languages. An increasing number of data engineers and data scientists use a variety of platforms and advanced analytics languages such as SAS, R, Python and Java, as well as frameworks such as Hadoop and Spark...
SYS-CON Events announced today that Mobile Create USA will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Mobile Create USA Inc. is an MVNO-based business model that uses portable communication devices and cellular-based infrastructure in the development, sales, operation and mobile communications systems incorporating GPS capabi...
SYS-CON Events announced today that Enroute Lab will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enroute Lab is an industrial design, research and development company of unmanned robotic vehicle system. For more information, please visit http://elab.co.jp/.
SYS-CON Events announced today that Nihon Micron will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Nihon Micron Co., Ltd. strives for technological innovation to establish high-density, high-precision processing technology for providing printed circuit board and metal mount RFID tags used for communication devices. For more inf...
SYS-CON Events announced today that Suzuki Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Suzuki Inc. is a semiconductor-related business, including sales of consuming parts, parts repair, and maintenance for semiconductor manufacturing machines, etc. It is also a health care business providing experimental research for...
IBM helps FinTechs and financial services companies build and monetize cognitive-enabled financial services apps quickly and at scale. Hosted on IBM Bluemix, IBM’s platform builds in customer insights, regulatory compliance analytics and security to help reduce development time and testing. In his session at 21st Cloud Expo, Lennart Frantzell, a Developer Advocate with IBM, will discuss how these tools simplify the time-consuming tasks of selection, mapping and data integration, allowing devel...
SYS-CON Events announced today that N3N will exhibit at SYS-CON's @ThingsExpo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. N3N’s solutions increase the effectiveness of operations and control centers, increase the value of IoT investments, and facilitate real-time operational decision making. N3N enables operations teams with a four dimensional digital “big board” that consolidates real-time live video feeds alongside IoT sensor data a...
Today most companies are adopting or evaluating container technology - Docker in particular - to speed up application deployment, drive down cost, ease management and make application delivery more flexible overall. As with most new architectures, this dream takes significant work to become a reality. Even when you do get your application componentized enough and packaged properly, there are still challenges for DevOps teams to making the shift to continuous delivery and achieving that reducti...
Mobile device usage has increased exponentially during the past several years, as consumers rely on handhelds for everything from news and weather to banking and purchases. What can we expect in the next few years? The way in which we interact with our devices will fundamentally change, as businesses leverage Artificial Intelligence. We already see this taking shape as businesses leverage AI for cost savings and customer responsiveness. This trend will continue, as AI is used for more sophistica...