Welcome!

@CloudExpo Authors: Liz McMillan, Elizabeth White, Yeshim Deniz, Pat Romanski, Maria C. Horton

Related Topics: @CloudExpo, Microservices Expo, Containers Expo Blog

@CloudExpo: Article

If Only Your Internal Cloud Had a Brain

The benefits of a more intelligent approach to cloud management can be staggering

Internal clouds are gaining momentum in large organizations, both as a means to drive agility and efficiency and as a stepping stone to hybrid and external cloud models. Although establishing and managing an internal cloud seems simple and elegant in principle, in practice they rarely resemble the simple, fluffy white cartoons seen in presentations. More often than not, today's cloud strategy involves piecing together hypervisor management layers with self-service portals, orchestration solutions, provisioning tools, and other base components. While this will yield a functioning cloud, it tends to operate at a relatively basic level, with little intelligence powering its action. This is a lot like having a body with no brain: if you tap its knee it might kick, but it is only capable of basic, reactive behavior.

Some might argue that this is no different from traditional IT physical environments, where disjoint tools and processes can be effectively used to handle specific aspects of operations and management. But the difference in these new environments is that there is a much tighter inter-relationship between all of the physical and virtual components. Physical complexity is replaced by virtual complexity, and characteristics like multi-tenancy and resource overcommit add a new twist. All of these factors combine to create an interesting challenge in the management of these environments, taking it beyond the realm of what intuition, spreadsheets and simple load-balancers can effectively handle.

In order to host cloud workloads in an efficient and low-risk way, the equivalent of a brain is needed. Like a human brain, it must aggregate and correlate information regarding the configuration and operation of the components in the system in order to maintain a certain level of "situational awareness." But this alone is not sufficient, as it provides visibility but not action, and is reactive, not proactive. To safely, reliably and proactively manage cloud environments there are two equally critical inputs that are required: cloud management policies and future capacity bookings.

Cloud management policies are the rules, constraints and preferences that govern the relationship between capacity and the workloads that run on it. These policies are critical to cloud management, and affect everything from utilization levels to resource overcommit, technical compatibilities and business rules. Interestingly, most of these considerations have existed for quite some time in IT environments, but there was less of a need to formalize them into a true policy until now. This is mainly because physical environments are not flexible enough to warrant it, and early virtual environments have targeted the "low hanging fruit," typically at the departmental level, which tends to be comparatively simple to manage. But a cloud with no policy is not viable, as it by definition hosts disparate applications from different business groups, each having different performance, availability and security requirements. Throwing these together randomly is a recipe for failure, and can lead to many performance, compliance and efficiency issues.

Once policies have been established, the other key ingredient is a representation of the future capacity bookings. This is the piece of the puzzle that allows an organization to attain a level of predictive analysis, which is essential to proactive management. These bookings can be thought of as hotel reservations, and represent the up-coming demands related to new application deployment, migration and consolidation activity, and other requirements. While most think of clouds as a way to get capacity "right now," in many organizations the goal is more strategic than tactical, and booking capacity with some level of advanced notice is far more consistent with the operational model. This provides tremendous advantages in terms of agility by removing the procurement cycle from the critical path, while also providing a level of control over IT assets. Immediate requests can still be accommodated, of course, but may incur a premium due to the volatility they introduce.

With these inputs in mind, let's get back to the brain. Data gathered from the "eyes and ears" of the environment (e.g., monitoring and alerting tools) provides situational awareness through a knowledge of current and historical activity, configurations and status. Leveraging a brain, or applying analytics to interpret this information in the context of the prevailing policy for a cloud or specific workload within it, enables risks to be identified and efficiency quantified, thus providing judgement as to whether action is needed to improve the environment. Factoring in future bookings enables "future situational awareness," where policies can be used to drive predictive analysis of potential risks and inefficiencies. Just like the brain, these current and future actions can then be communicated to the "arms and legs" of the environment (e.g., management, provisioning and orchestration tools) to achieve the desired outcome.

The benefits of a more intelligent approach to cloud management can be staggering. Isolated and reactive tools, however well integrated, are simply not capable of managing IT to a high degree of efficiency. But the combination of these tools with proactive, policy-based analytics provides significantly greater control, yielding higher efficiency while at the same time significantly reducing operational risk. In today's virtual and cloud environments, this translates into better performance and significantly higher density, saving millions of dollars in infrastructure costs.

More Stories By Andrew Hillier

Andrew Hillier is CTO and co-founder of CiRBA, Inc., a data center intelligence analytics software provider that determines optimal workload placements and resource allocations required to safely maximize the efficiency of Cloud, virtual and physical infrastructure. Reach Andrew at [email protected]

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@CloudExpo Stories
Your homes and cars can be automated and self-serviced. Why can't your storage? From simply asking questions to analyze and troubleshoot your infrastructure, to provisioning storage with snapshots, recovery and replication, your wildest sci-fi dream has come true. In his session at @DevOpsSummit at 20th Cloud Expo, Dan Florea, Director of Product Management at Tintri, provided a ChatOps demo where you can talk to your storage and manage it from anywhere, through Slack and similar services with...
Most people haven’t heard the word, “gamification,” even though they probably, and perhaps unwittingly, participate in it every day. Gamification is “the process of adding games or game-like elements to something (as a task) so as to encourage participation.” Further, gamification is about bringing game mechanics – rules, constructs, processes, and methods – into the real world in an effort to engage people. In his session at @ThingsExpo, Robert Endo, owner and engagement manager of Intrepid D...
Recently, WebRTC has a lot of eyes from market. The use cases of WebRTC are expanding - video chat, online education, online health care etc. Not only for human-to-human communication, but also IoT use cases such as machine to human use cases can be seen recently. One of the typical use-case is remote camera monitoring. With WebRTC, people can have interoperability and flexibility for deploying monitoring service. However, the benefit of WebRTC for IoT is not only its convenience and interopera...
Evan Kirstel is an internationally recognized thought leader and social media influencer in IoT (#1 in 2017), Cloud, Data Security (2016), Health Tech (#9 in 2017), Digital Health (#6 in 2016), B2B Marketing (#5 in 2015), AI, Smart Home, Digital (2017), IIoT (#1 in 2017) and Telecom/Wireless/5G. His connections are a "Who's Who" in these technologies, He is in the top 10 most mentioned/re-tweeted by CMOs and CIOs (2016) and have been recently named 5th most influential B2B marketeer in the US. H...
Michael Maximilien, better known as max or Dr. Max, is a computer scientist with IBM. At IBM Research Triangle Park, he was a principal engineer for the worldwide industry point-of-sale standard: JavaPOS. At IBM Research, some highlights include pioneering research on semantic Web services, mashups, and cloud computing, and platform-as-a-service. He joined the IBM Cloud Labs in 2014 and works closely with Pivotal Inc., to help make the Cloud Found the best PaaS.
Companies are harnessing data in ways we once associated with science fiction. Analysts have access to a plethora of visualization and reporting tools, but considering the vast amount of data businesses collect and limitations of CPUs, end users are forced to design their structures and systems with limitations. Until now. As the cloud toolkit to analyze data has evolved, GPUs have stepped in to massively parallel SQL, visualization and machine learning.
"With Digital Experience Monitoring what used to be a simple visit to a web page has exploded into app on phones, data from social media feeds, competitive benchmarking - these are all components that are only available because of some type of digital asset," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"This week we're really focusing on scalability, asset preservation and how do you back up to the cloud and in the cloud with object storage, which is really a new way of attacking dealing with your file, your blocked data, where you put it and how you access it," stated Jeff Greenwald, Senior Director of Market Development at HGST, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
"Venafi has a platform that allows you to manage, centralize and automate the complete life cycle of keys and certificates within the organization," explained Gina Osmond, Sr. Field Marketing Manager at Venafi, in this SYS-CON.tv interview at DevOps at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Creating replica copies to tolerate a certain number of failures is easy, but very expensive at cloud-scale. Conventional RAID has lower overhead, but it is limited in the number of failures it can tolerate. And the management is like herding cats (overseeing capacity, rebuilds, migrations, and degraded performance). In his general session at 18th Cloud Expo, Scott Cleland, Senior Director of Product Marketing for the HGST Cloud Infrastructure Business Unit, discussed how a new approach is neces...
Cloud-enabled transformation has evolved from cost saving measure to business innovation strategy -- one that combines the cloud with cognitive capabilities to drive market disruption. Learn how you can achieve the insight and agility you need to gain a competitive advantage. Industry-acclaimed CTO and cloud expert, Shankar Kalyana presents. Only the most exceptional IBMers are appointed with the rare distinction of IBM Fellow, the highest technical honor in the company. Shankar has also receive...
"We're developing a software that is based on the cloud environment and we are providing those services to corporations and the general public," explained Seungmin Kim, CEO/CTO of SM Systems Inc., in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Business professionals no longer wonder if they'll migrate to the cloud; it's now a matter of when. The cloud environment has proved to be a major force in transitioning to an agile business model that enables quick decisions and fast implementation that solidify customer relationships. And when the cloud is combined with the power of cognitive computing, it drives innovation and transformation that achieves astounding competitive advantage.
Leading companies, from the Global Fortune 500 to the smallest companies, are adopting hybrid cloud as the path to business advantage. Hybrid cloud depends on cloud services and on-premises infrastructure working in unison. Successful implementations require new levels of data mobility, enabled by an automated and seamless flow across on-premises and cloud resources. In his general session at 21st Cloud Expo, Greg Tevis, an IBM Storage Software Technical Strategist and Customer Solution Architec...
In his session at Cloud Expo, Alan Winters, U.S. Head of Business Development at MobiDev, presented a success story of an entrepreneur who has both suffered through and benefited from offshore development across multiple businesses: The smart choice, or how to select the right offshore development partner Warning signs, or how to minimize chances of making the wrong choice Collaboration, or how to establish the most effective work processes Budget control, or how to maximize project result...
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. In his session at @BigDataExpo, Jack Norris, Senior Vice President, Data and Applications at MapR Technologies, reviewed best practices to ...
"Software-defined storage is a big problem in this industry because so many people have different definitions as they see fit to use it," stated Peter McCallum, VP of Datacenter Solutions at FalconStor Software, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Data is the fuel that drives the machine learning algorithmic engines and ultimately provides the business value. In his session at Cloud Expo, Ed Featherston, a director and senior enterprise architect at Collaborative Consulting, discussed the key considerations around quality, volume, timeliness, and pedigree that must be dealt with in order to properly fuel that engine.
"Our strategy is to focus on the hyperscale providers - AWS, Azure, and Google. Over the last year we saw that a lot of developers need to learn how to do their job in the cloud and we see this DevOps movement that we are catering to with our content," stated Alessandro Fasan, Head of Global Sales at Cloud Academy, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
As organizations shift towards IT-as-a-service models, the need for managing and protecting data residing across physical, virtual, and now cloud environments grows with it. Commvault can ensure protection, access and E-Discovery of your data – whether in a private cloud, a Service Provider delivered public cloud, or a hybrid cloud environment – across the heterogeneous enterprise. In his general session at 18th Cloud Expo, Randy De Meno, Chief Technologist - Windows Products and Microsoft Part...