Welcome!

@CloudExpo Authors: Pat Romanski, Elizabeth White, Liz McMillan, Mehdi Daoudi, Rene Buest

Related Topics: @CloudExpo, Microservices Expo, Open Source Cloud

@CloudExpo: Article

Tracing the IT Evolution from the Big Bang to the Big Crunch

How enterprises are progressing from overgrown, difficult-to-manage IT systems to high performance open source infrastructure

Over the history of computing, we can trace a pattern of continuous decomposition, from a single system into disparate components. Early on, these individual parts made it easier to design, program and maintain systems, and meet the fast-growing demand for more power and more capacity.

The industry began with the mainframe, where the entire stack from hardware to application logic was contained in a single box. The next phase was the move from mainframe to client-server. This was followed by SOA (service-oriented architecture). This process of decomposition is a natural byproduct of growth in scale. As we consume increasingly more computing and storage, efficiency gains are achieved through specialization.

Such continuous decomposition is a typical pattern of many industries. Several centuries ago, the model was subsistence farming, where every family as a single unit grew all of their own crops. Today, food production has decomposed into a collection of highly specialized industries.

However, this process of decomposition in IT injects complexity. At a certain scale, highly decomposed systems become extremely challenging to manage. This then drives a pressing need to abstract away from some of the individual components to a higher level. This is largely what we are observing today with infrastructure computing. The complex mammoth of enterprise IT, today comprised of a spaghetti mix of application servers, relational and noSQL databases, messaging queues, caching and search services, etc., is no longer manageable.

Gartner labeled 2011 as the year of cloud platforms or PaaS. Thinking of PaaS, we intuitively think Heroku, Force.com, and Google App Engine, all off-premise cloud platforms. But the cloud movement is not just about on-premise versus off-premise. It's about creating an effective means to abstract away from application infrastructure complexity. As mainframes exploded into myriad sub-components, we experienced sort of a Big Bang in enterprise IT. What we are starting to observe now is the Big Crunch, turning application infrastructure back into a more unified, manageable artifact.

OpenStack
OpenStack is one of the most interesting initiatives topping the headlines during the last several months, and it's directly related to the Big Crunch. An open source project with the promise to help consolidate the many disparate components of application infrastructure, OpenStack is only a year old and is far from fulfilling this promise today. However, I believe that OpenStack for application infrastructure will eventually become what Linux became to application logic many years ago - a single interface unifying all application infrastructure components and exposing a standardized set of APIs to applications running on top of it.

Open Source Cloud Projects and How They Differ
OpenStack is not the first open source cloud project. Eucalyptus, OpenNebula, and Cloud.com all emerged before OpenStack and all of them are still very much alive. However, OpenStack is different from these others because it's the only one that has gained enough critical mass to get on a steady course to mass adoption.

What enabled OpenStack to reach this point was not an accident, but a clever strategy by RackSpace and other founding members. Rather than following a more common, vendor-centric approach to building an open source community, like Eucalyptus and Cloud.com did, RackSpace quickly figured out that getting a "cloud operating system" to mass adoption would require more marketing muscle then any single vendor has. So it positioned OpenStack as a decentralized, community-driven project from the very beginning and set out to get the support of big players in the application infrastructure space, namely Dell, Cisco, and Citrix. It didn't go after just any infrastructure player, but specifically focused on those who were arguably late to the cloud game and aching to make up the distance they lost to the likes of VMware and IBM. Ultimately, OpenStack's blitz to success is a result of unleashing an enormous amount of marketing energy in a short period of time, carefully coordinated between a number of application infrastructure power houses.

Following Amazon to Open Source Infrastructure
Today, OpenStack is focused on low level infrastructure services - compute, storage, image service, etc., and much work still remains to be done by the community in that area. However, we know the trend and have already seen it with Amazon Web Services (AWS). AWS initially started as Infrastructure as a Service (IaaS) with EC2 and S3 offerings; it then evolved into a fully blown Platform as a Service (PaaS). The value in solving application infrastructure complexity in a broader sense, by embedding higher level services like automated deployment, message queues, map reduce, and monitoring, is simply too compelling. At some point, we expect to see OpenStack creeping into the PaaS space, the same way AWS is doing today.

This gradual transition from simply being a compute and storage infrastructure orchestrator into a complete cloud operating system will happen naturally for OpenStack. It will be driven by infrastructure vendors of all sizes that are looking to plug their solutions into the OpenStack ecosystem. With more than  100 member companies on board already today, we see various announcements to this effect right and left: Gluster contributes its file system, Dell builds a deployment services, CloudCruiser builds a cost management solution, etc.

What's Ahead for OpenStack
The openness and decentralized nature of OpenStack is central to the realization of its vision of the cloud operating system. Instead of trying to solve all application infrastructure complexity inside one monolithic system, such as with the VMware stack, OpenStack harnesses the naturally occurring decomposition in the infrastructure space. This is the Big Bang in infrastructure we've all experienced. Individual vendors with competence in one particular area of application infrastructure can plug their solutions (storage, caching, monitoring, etc.) into OpenStack. As OpenStack continues to gain adoption, it will become a channel for infrastructure vendors to sell their offerings in the same way that the Apple app store is a channel for mobile app developers. At the same time, OpenStack will help abstract end users and resident applications away from the complexity of disparate infrastructure solutions.

Today we are still in the early days of OpenStack. It's far from being the ultimate platform. It may also be less feature-rich than competing offerings from Microsoft or VMware. However, this is unimportant today. What's important is that the need for the Big Crunch that will decrease application infrastructure complexity is obvious. The magnitude of effort required to make this happen is not something any single vendor could credibly pull off. Ultimately, it's not OpenStack features that matter, but the "idea" behind this project and the degree of uptake it has already received in the community. When many people come together to realize a sensible vision, that vision inevitably becomes a reality.

More Stories By Boris Renski

Boris is the co-founder of Mirantis and is currently a board member at the OpenStack foundation. He was one of the key influencers to convince all internal Mirantis stakeholders to make the bet on OpenStack services. Today Boris remains as Mirantis’ public face in the OpenStack community. Operationally responsible for Mirantis’ marketing strategy, Boris works to ensure that the marketing dollars the company spends don’t just promote Mirantis’ brand, but also help further global awareness of the OpenStack community as a whole. To that effect Mirantis is a regular sponsor of OpenStack summits, promotes OpenStack at unaffiliated trade shows such as CloudConnect and OSCON and helps facilitate regional meet-ups in the Bay Area, Washington DC, Russia and Ukraine. Boris continuously helps evangelize OpenStack community by regularly blogging about it (blog.mirantis.com), talking on behalf of the community at various conferences and engaging in business development activities to attract new member companies and contributors. During 15 years of his professional career Boris held several executive positions with technology companies he helped establish. He was a founder and CEO of Selectosa Systems – an IT consulting company which was subsequently acquired in 2006; and is a co-founder and angel investor at AGroup.lv – now a venture backed enterprise software company headquartered in Europe.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@CloudExpo Stories
The goal of Continuous Testing is to shift testing left to find defects earlier and release software faster. This can be achieved by integrating a set of open source functional and performance testing tools in the early stages of your software delivery lifecycle. There is one process that binds all application delivery stages together into one well-orchestrated machine: Continuous Testing. Continuous Testing is the conveyer belt between the Software Factory and production stages. Artifacts are m...
When shopping for a new data processing platform for IoT solutions, many development teams want to be able to test-drive options before making a choice. Yet when evaluating an IoT solution, it’s simply not feasible to do so at scale with physical devices. Building a sensor simulator is the next best choice; however, generating a realistic simulation at very high TPS with ease of configurability is a formidable challenge. When dealing with multiple application or transport protocols, you would be...
Cloud resources, although available in abundance, are inherently volatile. For transactional computing, like ERP and most enterprise software, this is a challenge as transactional integrity and data fidelity is paramount – making it a challenge to create cloud native applications while relying on RDBMS. In his session at 21st Cloud Expo, Claus Jepsen, Chief Architect and Head of Innovation Labs at Unit4, will explore that in order to create distributed and scalable solutions ensuring high availa...
An increasing number of companies are creating products that combine data with analytical capabilities. Running interactive queries on Big Data requires complex architectures to store and query data effectively, typically involving data streams, an choosing efficient file format/database and multiple independent systems that are tied together through custom-engineered pipelines. In his session at @BigDataExpo at @ThingsExpo, Tomer Levi, a senior software engineer at Intel’s Advanced Analytics ...
"We're a cybersecurity firm that specializes in engineering security solutions both at the software and hardware level. Security cannot be an after-the-fact afterthought, which is what it's become," stated Richard Blech, Chief Executive Officer at Secure Channels, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Enterprise architects are increasingly adopting multi-cloud strategies as they seek to utilize existing data center assets, leverage the advantages of cloud computing and avoid cloud vendor lock-in. This requires a globally aware traffic management strategy that can monitor infrastructure health across data centers and end-user experience globally, while responding to control changes and system specification at the speed of today’s DevOps teams. In his session at 20th Cloud Expo, Josh Gray, Chie...
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. Jack Norris reviews best practices to show how companies develop, deploy, and dynamically update these applications and how this data-first...
Intelligent Automation is now one of the key business imperatives for CIOs and CISOs impacting all areas of business today. In his session at 21st Cloud Expo, Brian Boeggeman, VP Alliances & Partnerships at Ayehu, will talk about how business value is created and delivered through intelligent automation to today’s enterprises. The open ecosystem platform approach toward Intelligent Automation that Ayehu delivers to the market is core to enabling the creation of the self-driving enterprise.
"We're here to tell the world about our cloud-scale infrastructure that we have at Juniper combined with the world-class security that we put into the cloud," explained Lisa Guess, VP of Systems Engineering at Juniper Networks, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
Historically, some banking activities such as trading have been relying heavily on analytics and cutting edge algorithmic tools. The coming of age of powerful data analytics solutions combined with the development of intelligent algorithms have created new opportunities for financial institutions. In his session at 20th Cloud Expo, Sebastien Meunier, Head of Digital for North America at Chappuis Halder & Co., discussed how these tools can be leveraged to develop a lasting competitive advantage ...
As businesses adopt functionalities in cloud computing, it’s imperative that IT operations consistently ensure cloud systems work correctly – all of the time, and to their best capabilities. In his session at @BigDataExpo, Bernd Harzog, CEO and founder of OpsDataStore, presented an industry answer to the common question, “Are you running IT operations as efficiently and as cost effectively as you need to?” He then expounded on the industry issues he frequently came up against as an analyst, and ...
The question before companies today is not whether to become intelligent, it’s a question of how and how fast. The key is to adopt and deploy an intelligent application strategy while simultaneously preparing to scale that intelligence. In her session at 21st Cloud Expo, Sangeeta Chakraborty, Chief Customer Officer at Ayasdi, will provide a tactical framework to become a truly intelligent enterprise, including how to identify the right applications for AI, how to build a Center of Excellence to ...
In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. H...
You know you need the cloud, but you’re hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You’re looking at private cloud solutions based on hyperconverged infrastructure, but you’re concerned with the limits inherent in those technologies.
SYS-CON Events announced today that Massive Networks will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Massive Networks mission is simple. To help your business operate seamlessly with fast, reliable, and secure internet and network solutions. Improve your customer's experience with outstanding connections to your cloud.
DevOps is under attack because developers don’t want to mess with infrastructure. They will happily own their code into production, but want to use platforms instead of raw automation. That’s changing the landscape that we understand as DevOps with both architecture concepts (CloudNative) and process redefinition (SRE). Rob Hirschfeld’s recent work in Kubernetes operations has led to the conclusion that containers and related platforms have changed the way we should be thinking about DevOps and...
Everything run by electricity will eventually be connected to the Internet. Get ahead of the Internet of Things revolution and join Akvelon expert and IoT industry leader, Sergey Grebnov, in his session at @ThingsExpo, for an educational dive into the world of managing your home, workplace and all the devices they contain with the power of machine-based AI and intelligent Bot services for a completely streamlined experience.
Because IoT devices are deployed in mission-critical environments more than ever before, it’s increasingly imperative they be truly smart. IoT sensors simply stockpiling data isn’t useful. IoT must be artificially and naturally intelligent in order to provide more value In his session at @ThingsExpo, John Crupi, Vice President and Engineering System Architect at Greenwave Systems, will discuss how IoT artificial intelligence (AI) can be carried out via edge analytics and machine learning techn...
SYS-CON Events announced today that Datera, that offers a radically new data management architecture, has been named "Exhibitor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Datera is transforming the traditional datacenter model through modern cloud simplicity. The technology industry is at another major inflection point. The rise of mobile, the Internet of Things, data storage and Big...
FinTechs use the cloud to operate at the speed and scale of digital financial activity, but are often hindered by the complexity of managing security and compliance in the cloud. In his session at 20th Cloud Expo, Sesh Murthy, co-founder and CTO of Cloud Raxak, showed how proactive and automated cloud security enables FinTechs to leverage the cloud to achieve their business goals. Through business-driven cloud security, FinTechs can speed time-to-market, diminish risk and costs, maintain continu...