@CloudExpo Authors: Elizabeth White, Zakia Bouachraoui, Liz McMillan, Pat Romanski, Roger Strukhoff

Related Topics: @CloudExpo, Microservices Expo

@CloudExpo: Article

The Cloud and the Mainframe: Bringing the Best of Both Worlds Together

Preparing an IT organization for the push toward the cloud requires an understanding of the existing assets

Mainframe system or cloud computing? IT organizations don't necessarily have to choose. Many mainframe advocates would argue that developing a private cloud would be redundant, as the mainframe provides the flexible, agile and always available environment promised with cloud computing. And those looking to bring cloud computing into their environments shouldn't assume the mainframe won't be a factor.

"We are inventing the new mainframe in a sense. The internal private cloud is taking the application hosting and delivery infrastructure and managing it as a service. It has all the pieces it had before, but now you have to think of it in a service-oriented approach," says Jim Frey, research director at Enterprise Management Associates (EMA). "It is not separate from the mainframe; the mainframe will be included in an evolutionary role. The cloud is forcing organizations to become more cross-domain and service-oriented in their management."

An October 2010 study commissioned by CA Technologies showed that 79 percent of 300 IT decision makers believed the "mainframe is an essential component of their cloud computing strategy." And 70 percent indicated that cloud computing will sustain or extend the mainframe environment.

"Your back-end data is still in the mainframe even if you have a plan for a distributed cloud environment. The data will stay where it is, and you will see cloud-hosted transactions that will continue to need to access that back-end data," explains Wendy Wong, senior software architect for CA Technologies. "Cloud computing may require more exotic access to the mainframe than you would have traditionally, but that data is not going to move anywhere and you have to determine how you want your cloud implementation to access the mainframe and provide visibility for management."

Organizations today can tie a cloud deployment to their distributed and mainframe environments, but the management challenge is exacerbated.

"From the operational aspect, the cloud is lessening the focus on the physical infrastructure and reorienting toward the virtual, making IT managers consider how to track a business service and the infrastructure it was hitting," says George Hamilton, principal analyst at Yankee Group. "From the performance monitoring aspect, the end-user experience becomes much more important and it becomes much more challenging to do."

IT departments must consider how they will be able to track performance across the multi-tiered environment in a consistent manner, as well as determine - if using public cloud resources - how to gain visibility into those external systems and focus on the metrics that matter.

"Now, IT has to be concerned over assets it does not control. To effectively manage in this new environment, tools are needed to monitor and manage from your ‘on premise' assets out to your cloud providers and back," says Scott Fagen, distinguished engineer and CA Technologies subject matter expert for CA Mainframe 2.0. "It's even possible that you'll need to monitor the assets of the ISP in between you and your cloud providers. Understanding and managing these ‘rented assets' is necessary to continue to maintain service-level agreements to your customers."

What are they saying? To keep management on the simpler side, organizations could theoretically choose to develop a private cloud exclusively leveraging their mainframe system. But given the popularity and affordability of x86-based virtualized servers and temptation of elastic infrastructure as a service (IaaS), it is more likely that hybrid cloud adoption will become the norm, requiring the same mature management control typically found in more traditional on-premise environments. Success depends on creating the right connections and enabling the visibility needed to adequately manage the sophisticated, distributed environments for availability, performance and service assurance.

"The critical element for hybrid cloud will be the management platforms that enable apps to be transparently shifted from one environment to another. Hybrid cloud management will augment in-house system management infrastructure with a new layer of capabilities that span private and public cloud environments, allowing them to be managed as a unified infrastructure," reads Forrester Research, Inc.'s December 22, 2010 report "Enterprise Cloud Management Capabilities Road Map."

Navigating the Network Needs for Your Cloud Environment
Talk of cloud computing doesn't immediately conjure up images of the network, but it should. The network is one of three critical components - compute, storage and network - in any cloud computing implementation, and the success or failure of a cloud implementation often lies within the network.

"Mainframe applications are a critical part of the backbone of services offered; the network provides access to these applications," says Denise Kalm, principal of product marketing at CA Technologies. "The network is the nervous system that enables our customers to move their data between their storage and processing capacity," Fagen adds.

Network managers need to get involved in cloud strategy discussions early. The network team will be tasked with updating configuration, planning for capacity and understanding and communicating the new traffic patterns in the cloud-connected environment.

"There are still network activities that need to happen at the access layer and the distribution layer. You still need that aspect of network management and activity, and you still need that expertise on how to plan and optimize the network," says EMA's Jim Frey. "There is still a network link connecting the end users to the cloud services that has to be maintained and protected in terms of quality being assured and capacity being assured as you get further into the cloud infrastructure itself."

The dynamic nature of cloud will challenge existing network practices, which in some cases are based on several elements remaining static and therefore easier to track. For instance, network management tools often track application delivery across servers and network devices by collecting response time and other metrics. Now, if the server environment, as well as the network component, is virtual and dynamic, tracking performance or even traffic patterns becomes exponentially more difficult.

"It is extremely important that the network team is involved in the early stages of planning any cloud implementation. Network managers need to anticipate bandwidth, configuration and automation, for instance. And they need to anticipate what needs to be established to monitor these environments, which is not just a ‘one-time' set up process. The dynamic nature of cloud requires a mechanism that keeps monitoring up-to-date and network managers need to ensure their monitoring tools adapt," says Kathy Hickey, senior director of product management at CA Technologies.

When planning for a cloud implementation, network managers need to:

  • Baseline the performance of existing applications and services to understand what normal performance is, taking into consideration latency differences by day of week, time of day, location and more;
  • Assess the access requirements, capacity and bandwidth requirements, and then configure a network to support them;
  • Understand the data flow requirements and plan capacity to handle fluctuating demands by providing alternate routes and sufficient capacity for peaks;
  • Gain visibility into the whole network, particularly at the boundaries , and not just the part attached to their platform of choice;
  • Consider what might constrain the network, such as legacy Systems Network Architecture (SNA) networks;
  • Ensure they have the appropriate software to manage from both the most granular level up to the enterprise level; and
  • Implement some form of quality of service (QoS) and priority mapping to ensure critical workloads not only have access when needed, but also have optimal routing.

"As part of the integrated discipline of cloud performance, the servers, the storage, the network and the applications must be managed holistically. If any part fails or slows down, the whole is impacted," Kalm adds."Most of the good network practices don't change substantially because a new architecture is in place, but there would probably be additional policies to insure optimum performance of the networks, proactive management and quick intuitive tools to mitigate any downtime."

Overcoming Political Pitfalls and Technology Traps: Understanding the Network Management Maze of the Mainframe
With the adoption of any new technology comes the struggle for widespread acceptance. Cloud computing is no different even with the positive buzz surrounding the technology. IT organizations hoping to capitalize on existing infrastructure, such as the mainframe, could face a "religious war" of sorts with systems experts who don't immediately see the benefits of cloud.

"The most difficult hurdle is political. ‘Who controls the virtual images?' The banks that are doing the rehosting from distributed back to mainframe for Linux (and to some extent, applications that can be migrated from something else to Linux on System z) have been successful because they've placed someone at the apex of the political pyramid between mainframe and distributed who has an affinity for mainframe, but is mostly interested in savings that accrue from simplification and economies of scale," says Fagen.

If the political hurdle is cleared, there are a couple technical matters to consider when it comes to managing mainframe, network and cloud components. According to Wong, the mainframe network looks much different to management tools than the distributed network. While many organizations would want to achieve a single pane of glass approach - seeing the performance and availability of applications, services and transactions from the cloud through the network back to the mainframe - it will not be a simple task.

"The mainframe IT stack can have thousands of IP addresses associated with it and not all of those IP addresses are externally visible, which can make the mainframe network look more complicated to distributed network management tools," Wong says. "Network managers would have to be aware of the complexities within the mainframe environment and determine if they want their distributed tools to connect to the mainframe for a true end-to-end view or continue to manage the environments separately."

Know Your Network: Build in Management for Private Cloud and Beyond
For the network team, the challenges that might arise depend on the cloud implementation the company chooses.

"Network teams need to be included as early as possible in cloud planning," according to Yankee Group's Hamilton. "Cloud has huge implications for the network; everything becomes a network resource, and every virtual instance is just bits on the wire now."

Network administrators will need to be well-versed in virtualized technologies and the move from physical networking to virtual, distributed switches. To start, network managers need to ensure that the tools they traditionally used to manage the physical environment can also provide visibility into the virtual switch and hypervisor technology of the proposed cloud environment.

"Input from our fourth annual State of the Network Global Study shows that the single largest troubleshooting issue the survey respondents face is identifying the source of any particular problem, highlighting that visibility is increasingly becoming an issue as complexity increases," says Brad Reinboldt, senior product manager at Network Instruments, a CA Technologies partner. "Overall, 35 percent of respondents indicated that their ability to troubleshoot network problems in a virtualized environment had worsened. Another issue with virtualization is that server utilization can jump from 15 percent to 80 percent, requiring the network to be re-architected to increase the bandwidth needed to accommodate those servers."

And the inherent, dynamic nature of a cloud environment could pose unanticipated risk to the environment, of which the network team needs to be aware of early.

"The network configuration is such that it can have a pretty significant impact to the services provided to end users. If it is not configured properly, risk is introduced and the need to monitor applications and the underlying infrastructure for performance is hindered," says Hickey. "These cloud environments are very dynamic, constantly changing, which can break some traditional approaches to network management."

For those exploring private clouds, performance continues to be an issue. Two factors could impact performance. If the network is not equipped to be sending as much traffic as needed out of WAN links, then network managers will be hearing about bottlenecks. And if there is not adequate automation technology available, the speed and agility that cloud promises could be lost as manual efforts fail to keep up with all the moving parts.

"Private cloud isn't that big of a transition with the majority of changes being in the data center. But network managers are going to want to optimize resources to deliver applications from the data center to branch and remote offices without requiring a lot of infrastructure at those locations," said Yankee Group's Hamilton in a Service Assurance Daily blog. "For private cloud, it doesn't change all that much. Network managers will have to automate more because there will be multiple data centers and more parts moving around. The network will have to have the intelligence to dynamically move workloads. Doing this manually will be far too inefficient for cloud computing."

The move to a hybrid or public cloud adds additional management considerations. "Moving to a public cloud doesn't lessen a network administrator's responsibility to maintain performance, it just increases the complexity of doing so," says Network instruments' Reinboldt. "Keep in mind that the shift in focus of large bandwidth capacities from the data center of an internal network to the Internet connection for a public cloud increases the importance of WAN optimization to better prioritize the applications running over the cloud. Also, establishing SLAs with the service provider is critical to maintaining performance. Be clear in the performance metrics you require for particular applications, whether it is low latency, dropped frames or other service qualities. And monitoring that SLA with a network analyzer is never a bad idea."

Take on New Technology without Sacrificing Service Quality
Coupling cloud with mainframe, while ramping up the network to support the hybrid environment, will enable IT organizations to deliver more on-demand services for the business. But that will also require companies to manage underlying IT elements to deliver an end-to-end view of IT services across a highly sophisticated environment, which will require management processes and technologies to be updated. IT organizations will want to ensure the end-user experience with IT services doesn't degrade as requests traverse multiple infrastructure and application components from the network to the mainframe to the cloud. IT management approaches will potentially have to explore how to gain visibility into external cloud providers' environments to best track service levels across the entire delivery path of an IT service.

These hybrid cloud environments, according to industry watchers, will require IT departments to think more holistically, abandoning siloed approaches to IT management behind and adopt IT-cloud management technologies that:

  • Virtualize and package applications so that they can be quickly deployed on any infrastructure, and orchestrate the migration, according to Forrester Research[1];
  • Update performance monitoring and service management approaches from IT domain specific to business service focused, EMA's Frey says;
  • Prioritize the end-user experience as one of the most critical performance monitoring metrics, Yankee's Hamilton adds;
  • Relieve human operators from certain tasks and simplify management through automation, "Automation is a must-have," Yankee's Hamilton concurs; and
  • Integrate cloud monitoring efforts with virtual and traditional physical availability and performance monitoring tools, Frey adds.

"In five years, your systems management infrastructure should seamlessly bridge internal and external cloud resources, managing all these environments from one point. Applications will run on internal or external resources, depending on cost, availability and policy controls. The coordination of public and private cloud computing resources will be directed by a class of system management tools called cloud management platforms. Although there will be new cloud management platforms designed from the ground up, this function will also be offered by server virtualization platforms and traditional systems management suites," the Forrester Research report reads. [1]

Conclusion: Bridge the Gap with Management
Cloud computing can significantly benefit an IT organization, if the implementation doesn't impact IT services being delivered to the business - and that requires a proactive management approach. While incorporating existing mainframe systems into a cloud strategy potentially makes the task more daunting, it should be accepted that the value of the mainframe is too much to be left behind for most companies.

"There are a lot of mainframes out there and they are not going to be left behind or abandoned. And for those out there, they are not only running software from 1978. All of these intermediate generations of IT are interwoven with the mainframe and they are all actively part of Web technologies," says Jeff Cobb, senior vice president of business unit strategy for CA Technologies Service Assurance. "Cloud is something everyone needs to pay attention to and it's not because it's trendy or the flavor of the month. There is a fundamental driver that will push the world toward the cloud. The economics of cloud are so obvious that it simply cannot be ignored."

Preparing an IT organization for the push toward the cloud requires an understanding of the existing assets, such as mainframe systems, and advanced knowledge of network operations that will enable a more dynamic environment. Management technologies will play a pivotal role in ensuring the success of any cloud deployment, and IT organizations should consider how to best gain visibility into the end-to-end performance of IT services from the cloud back to the mainframe.

"Cloud promises an elasticity that is not typically found in environments and that could break existing management approaches. IT organizations need to accept this dynamism, that everything is in flux, and change their management approach to be as flexible as the environment," Cobb continues. "The whole point of cloud is to put a shroud between the infrastructure and the quality of service for your customers. Now with cloud, you just know what capacity is needed and the systems take care of it -- ideally. To achieve that, these network managers need to support the old world and adapt to the new world in which things are much more dynamic. They need management systems that can take action dynamically, but that can also communicate clearly what is happening in the environment."


  1. Forrester Research, Inc., "Enterprise Cloud Management Capabilities Road Map", December 22, 2010.

More Stories By Denise Dubie

Denise Dubie (@DDubie) is New Media Principal in CA Technologies Thought Leadership Group. She is charged with creating content relevant to today’s most pressing technology and business trends for industry leaders and IT professionals.

Prior to joining the company in 2010, Dubie spent 12 years of her career at Network World, an IDG company, covering the IT management industry and all of its players (including CA Technologies and its competitors) as well as high-tech careers, technology trends and vendors such as Cisco, HP, IBM and Microsoft. As Senior Editor at Network World, Dubie also authored the publication's twice-weekly Network and Systems Management Alert newsletter and contributed to the Web site's Microsoft Subnet blog. Before IDG, she served as Assistant Managing Editor at Application Development Trends, managing writers and the monthly publication's production process.

Dubie started her professional journalism career as a Staff Writer/Reporter at The Transcript, a small daily paper in Western Massachusetts.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

CloudEXPO Stories
The platform combines the strengths of Singtel's extensive, intelligent network capabilities with Microsoft's cloud expertise to create a unique solution that sets new standards for IoT applications," said Mr Diomedes Kastanis, Head of IoT at Singtel. "Our solution provides speed, transparency and flexibility, paving the way for a more pervasive use of IoT to accelerate enterprises' digitalisation efforts. AI-powered intelligent connectivity over Microsoft Azure will be the fastest connected path for IoT innovators to scale globally, and the smartest path to cross-device synergy in an instrumented, connected world.
There are many examples of disruption in consumer space – Uber disrupting the cab industry, Airbnb disrupting the hospitality industry and so on; but have you wondered who is disrupting support and operations? AISERA helps make businesses and customers successful by offering consumer-like user experience for support and operations. We have built the world’s first AI-driven IT / HR / Cloud / Customer Support and Operations solution.
ScaleMP is presenting at CloudEXPO 2019, held June 24-26 in Santa Clara, and we’d love to see you there. At the conference, we’ll demonstrate how ScaleMP is solving one of the most vexing challenges for cloud — memory cost and limit of scale — and how our innovative vSMP MemoryONE solution provides affordable larger server memory for the private and public cloud. Please visit us at Booth No. 519 to connect with our experts and learn more about vSMP MemoryONE and how it is already serving some of the world’s largest data centers. Click here to schedule a meeting with our experts and executives.
Darktrace is the world's leading AI company for cyber security. Created by mathematicians from the University of Cambridge, Darktrace's Enterprise Immune System is the first non-consumer application of machine learning to work at scale, across all network types, from physical, virtualized, and cloud, through to IoT and industrial control systems. Installed as a self-configuring cyber defense platform, Darktrace continuously learns what is ‘normal' for all devices and users, updating its understanding as the environment changes.
Codete accelerates their clients growth through technological expertise and experience. Codite team works with organizations to meet the challenges that digitalization presents. Their clients include digital start-ups as well as established enterprises in the IT industry. To stay competitive in a highly innovative IT industry, strong R&D departments and bold spin-off initiatives is a must. Codete Data Science and Software Architects teams help corporate clients to stay up to date with the modern business digitalization solutions. Achieve up to 50% early-stage technological process development cost cutdown with science and R&D-driven investment strategy with Codete's support.