Click here to close now.

Welcome!

CloudExpo® Blog Authors: Elizabeth White, Liz McMillan, Lori MacVittie, Pat Romanski, Rex Morrow, Datical

Related Topics: CloudExpo® Blog, Microservices Expo, Microsoft Cloud, Containers Expo Blog, BigDataExpo® Blog, SDN Journal

CloudExpo® Blog: Blog Feed Post

Notes from the Field: Inside a Real World Large-Scale Cloud Deployment

I thought I’d share some important generalities about this type of effort

I’ve been granted an incredible opportunity. Over the past three and a half months I have gotten to lead a real world large-scale delivery of a cloud solution. The final solution will be delivered as Software-as-a-Service (SaaS) to the customer via an on-premise managed service. While I have developed SaaS/PaaS (Platform-as-a-Service) solutions in the past, I was fortunate enough to have been able to build those on public cloud infrastructures. This has been a rare glimpse into the “making of the sausage” having to orchestrate everything from delivery of the hardware into the data center in four countries to testing and integration with the customer environment.

All I can say about this opportunity is that the term, “it takes a village” applies well. I thought I’d share some important generalities about this type of effort. It’s important to note that this is a Global 100 company with data centers around the globe. Regardless of what the public cloud providers are telling the world, this application is not appropriate for public cloud deployment due to the volume of data traversing the network, the amount of storage required, the types of storage required (e.g. Write-Once-Read-Many), level of integration with internal environments and the requirements for failover.

The following are some observations about deploying cloud solutions at this scale:

  • Data Centers. As part of IT-as-a-Service (ITaaS) we talk a lot about convergence, software-defined data centers and general consolidation. All of this has major implications for simplifying management and lowering the total cost of ownership and operations of the data centers. However, we should not forget that it still takes a considerable amount of planning and effort to bring new infrastructure into an existing data center. The most critical of these is that the data center is a living entity that doesn’t stop because work is going on, which means a lot of this effort occurs after hours and in maintenance windows. This particular data center freezes all changes between mid-December till mid-January to ensure that their customers will not have interrupted service during a peak period that includes major holidays and end of year reporting, which had significant impact on attempting to meet certain end-of-year deliverables. On site surveys were critical to planning the organization of the equipment (four racks in total) on the floor to minimize cabling efforts and ensure our equipment was facing in the right direction to meet the needs for hot/cold isles. Additionally, realize that in this type of business, every country may have different rules for accessing, operating in and racking your equipment.
  • Infrastructure. At the end of the day, we can do more with the hardware infrastructure architectures now available. While we leverage virtualization to take advantage of the greater compute power, it does not alleviate the requirements around planning a large-scale virtual environment that must span countries. Sometimes, it’s the smallest details that can be the most difficult to work out, for example, how to manage an on-premise environment, such as this one, as a service. The difficulties here is that the network, power, cooling, etc. are provided for by the customer, which requires considerable efforts to negotiate shared operating procedures, while still attempting to commit to specific service levels. Many of today’s largest businesses do not operate their internal IT organizations with the same penalties for failure to meet a service level agreement (SLA) as they would apply to an external service provider. Hence, service providers that must rely on this foundation face many challenges and hurdles to ensuring their own service levels.
  • Security. Your solution may be reviewed by the internal security team to ensure it is compliant with current security procedures and policies. Since this is most often not the team that procured or built the solution, you should not expect that they will be able to warn you about all the intricacies for deploying a solution for the business. The best advice here would be to ensure you engage the security team early and often once you have completed your design. In US Federal IT, part of deployment usually requires that those implementing the system obtain an Authority to Operate (ATO). Quite often, medium- and large-sized businesses have a similar procedure; it’s just not spelled out so succinctly. Hence, these audits and tests can introduce unexpected expenses due to the need to modify the solution and unexpected delays.
  • Software. Any piece of software can be tested and operated under a modest set of assumptions. When that software must be deployed as part of a service that has requirements to meet certain performance metrics as well as meet certain recovery metrics in the case of an outage, that same software can fall flat on its face. Hence, the long pole in the tent for building out a cloud solution at this scale is testing for disaster recovery and scalability. In addition to requiring time to complete, it often requires a complementary environment for disaster recovery and failover testing, which can be a significant additional cost to the project. I will also note that in a complex environment software license management can become very cumbersome. I recommend starting the license catalog early and ensure that it is maintained throughout the project.
  • Data Flow. A complex cloud-based solution that integrates with existing internal systems operating on different networks across multiple countries will have to cross multiple firewalls, routers and run along paths with varying bandwidth carrying varying levels of traffic. Hence, issues for production operation and remote management can be impacted by multiple factors both during planning and during operation. No matter how much testing is done in a lab, the answer seemingly comes down to, “we’ll just have to see how it performs in production.” So, perhaps, a better title for this bullet might be “Stuff You’re Going To Learn Only After You Start The Engine.” Your team will most likely have a mix of personalities. Some will be okay with this having learned from doing similar projects in their past, others will not be able to get past this point and continually raise objections. Shoot the naysayer! Okay, not really, but seriously, adopt this mandate and make sure everyone on the team understands it.
  • Documentation. I cannot say enough about ensuring you document early and often. Once the train is started, it’s infinitely more difficult to catch up. Start with good highly-reviewed requirements. Review them with the customer. Call to order the ARB and have them review and sign off. This is a complex environment with a lot of interdependencies. It’s not going to be simple to change one link without it affecting many others. The more changes you can avoid the more smoothly the process of getting a system into production will be.

Most importantly, and I cannot stress this enough, is the importance in building a team environment to accomplish the mission. Transforming a concept into a production-ready operational system requires a large number of people to cooperatively work together to address the hurdles. The solution as designed on paper will hardly ever match perfectly what is deployed in the field for the reasons stated above. This project is heavily reliant upon a Program Management Organization with representatives from engineering, managed services, field services, product and executive leadership to stay on track. Developing the sense of team within this group is critical to providing the appropriate leadership to the project as a whole. Subsequently, we also formed an Architecture Review Board (ARB) comprised of key technical individuals related to each aspect of the solution to address and find solutions for major technical issues that emerged throughout the project. In this way we ensure the responses were holistic in nature, not just focused on the specific problem, but also provided alternatives that would work within the scope of the entire project.

More Stories By JP Morgenthal

JP Morgenthal is an internationally renowned thought leader in the areas of IT transformation, modernization, and cloud computing. JP has served in executive roles within major software companies and technology startups. Areas of expertise include strategy, architecture, application development, infrastructure and operations, cloud computing, DevOps, and integration. He routinely advises C-level executives on the best ways to use technology to derive business value. JP is a published author with four trade publications with his most recent being “Cloud Computing: Assessing the Risks”. JP holds both a Masters and Bachelors of Science in Computer Science from Hofstra University.

@CloudExpo Stories
Most companies hope for rapid growth so it's important to invest in scalable core technologies that won't demand a complete overhaul when a business goes through a growth spurt. Cloud technology enables previously difficult-to-scale solutions like phone, network infrastructure or billing systems to automatically scale based on demand. For example, with a virtual PBX service, a single-user cloud phone service can easily transition into an advanced VoIP system that supports hundreds of phones and ...
Containers Expo Blog covers the world of containers, as this lightweight alternative to virtual machines enables developers to work with identical dev environments and stacks. Containers Expo Blog offers top articles, news stories, and blog posts from the world's well-known experts and guarantees better exposure for its authors than any other publication. Bookmark Containers Expo Blog ▸ Here Follow new article posts on Twitter at @ContainersExpo
Disruptive macro trends in technology are impacting and dramatically changing the "art of the possible" relative to supply chain management practices through the innovative use of IoT, cloud, machine learning and Big Data to enable connected ecosystems of engagement. Enterprise informatics can now move beyond point solutions that merely monitor the past and implement integrated enterprise fabrics that enable end-to-end supply chain visibility to improve customer service delivery and optimize sup...
Over the last few years the healthcare ecosystem has revolved around innovations in Electronic Health Record (HER) based systems. This evolution has helped us achieve much desired interoperability. Now the focus is shifting to other equally important aspects - scalability and performance. While applying cloud computing environments to the EHR systems, a special consideration needs to be given to the cloud enablement of Veterans Health Information Systems and Technology Architecture (VistA), i.e....
SYS-CON Events announced today that MetraTech, now part of Ericsson, has been named “Silver Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York, NY. Ericsson is the driving force behind the Networked Society- a world leader in communications infrastructure, software and services. Some 40% of the world’s mobile traffic runs through networks Ericsson has supplied, serving more than 2.5 billion subscribers.
From telemedicine to smart cars, digital homes and industrial monitoring, the explosive growth of IoT has created exciting new business opportunities for real time calls and messaging. In his session at @ThingsExpo, Ivelin Ivanov, CEO and Co-Founder of Telestax, shared some of the new revenue sources that IoT created for Restcomm – the open source telephony platform from Telestax. Ivelin Ivanov is a technology entrepreneur who founded Mobicents, an Open Source VoIP Platform, to help create, de...
The Internet of Things (IoT) promises to evolve the way the world does business; however, understanding how to apply it to your company can be a mystery. Most people struggle with understanding the potential business uses or tend to get caught up in the technology, resulting in solutions that fail to meet even minimum business goals. In his session at @ThingsExpo, Jesse Shiah, CEO / President / Co-Founder of AgilePoint Inc., showed what is needed to leverage the IoT to transform your business. ...
Grow your business with enterprise wearable apps using SAP Platforms and Google Glass. SAP and Google just launched the SAP and Google Glass Challenge, an opportunity for you to innovate and develop the best Enterprise Wearable App using SAP Platforms and Google Glass and gain valuable market exposure. In his session at @ThingsExpo, Brian McPhail, Senior Director of Business Development, ISVs & Digital Commerce at SAP, outlined the timeline of the SAP Google Glass Challenge and the opportunity...
As enterprises look to take advantage of the cloud, they need to understand the importance of safeguarding their confidential and sensitive data in cloud environments. Enterprises must protect their data from (i) system administrators who don't need to see the data in the clear and (ii) adversaries who become system administrators from stolen credentials. In short, enterprises must take control of their data: The best way to do this is by using advanced encryption, centralized key management and...
What are the benefits of using an enterprise-grade orchestration platform? In their session at 15th Cloud Expo, Nate Gordon, Director of Technology at Appcore, and Kedar Poduri, Senior Director of Product Management at Citrix Systems, took a closer look at the architectural design factors needed to support diverse workloads and how to run these workloads efficiently as a service provider. They also discussed how to deploy private cloud environments in 15 minutes or less.
In his session at DevOps Summit, Andrei Yurkevich, CTO at Altoros, provided an overview of all the benefits and opportunities, as well as drawbacks of deploying Cloud Foundry PaaS with Juju and compared it to BOSH. Discover the features that overlap, and understand what Juju Charm is, what it is not, where you use one or the other or where you use both BOSH and Juju Charms together. Andrei Yurkevich is Cloud Foundry protagonist and CTO at Altoros. Under his supervision, the Altoros engineering ...
Cultural, regulatory, environmental, political and economic (CREPE) conditions over the past decade are creating cross-industry solution spaces that require processes and technologies from both the Internet of Things (IoT), and Data Management and Analytics (DMA). These solution spaces are evolving into Sensor Analytics Ecosystems (SAE) that represent significant new opportunities for organizations of all types. Public Utilities throughout the world, providing electricity, natural gas and water,...
The Internet of Things will put IT to its ultimate test by creating infinite new opportunities to digitize products and services, generate and analyze new data to improve customer satisfaction, and discover new ways to gain a competitive advantage across nearly every industry. In order to help corporate business units to capitalize on the rapidly evolving IoT opportunities, IT must stand up to a new set of challenges. In his session at @ThingsExpo, Jeff Kaplan, Managing Director of THINKstrateg...
Can call centers hang up the phones for good? Intuitive Solutions did. WebRTC enabled this contact center provider to eliminate antiquated telephony and desktop phone infrastructure with a pure web-based solution, allowing them to expand beyond brick-and-mortar confines to a home-based agent model. It also ensured scalability and better service for customers, including MUY! Companies, one of the country's largest franchise restaurant companies with 232 Pizza Hut locations. This is one example of...
One of the biggest challenges when developing connected devices is identifying user value and delivering it through successful user experiences. In his session at Internet of @ThingsExpo, Mike Kuniavsky, Principal Scientist, Innovation Services at PARC, described an IoT-specific approach to user experience design that combines approaches from interaction design, industrial design and service design to create experiences that go beyond simple connected gadgets to create lasting, multi-device exp...
There is little doubt that Big Data solutions will have an increasing role in the Enterprise IT mainstream over time. 8th International Big Data Expo, co-located with 17th International Cloud Expo - to be held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA - has announced its Call for Papers is open. As advanced data storage, access and analytics technologies aimed at handling high-volume and/or fast moving data all move center stage, aided by the cloud computing bo...
Some developers believe that monitoring is a function of the operations team. Some operations teams firmly believe that monitoring the systems they maintain is sufficient to run the business successfully. Most of them are wrong. The complexity of today's applications have gone far and beyond the capabilities of "traditional" system-level monitoring tools and approaches and requires much broader knowledge of business and applications as a whole. The goal of DevOps is to connect all aspects of app...
DevOps is all about agility. However, you don't want to be on a high-speed bus to nowhere. The right DevOps approach controls velocity with a tight feedback loop that not only consists of operational data but also incorporates business context. With a business context in the decision making, the right business priorities are incorporated, which results in a higher value creation. In his session at DevOps Summit, Todd Rader, Solutions Architect at AppDynamics, discussed key monitoring techniques...
The Internet of Things will greatly expand the opportunities for data collection and new business models driven off of that data. In her session at @ThingsExpo, Esmeralda Swartz, CMO of MetraTech, discussed how for this to be effective you not only need to have infrastructure and operational models capable of utilizing this new phenomenon, but increasingly service providers will need to convince a skeptical public to participate. Get ready to show them the money!
Want to enable self-service provisioning of application environments in minutes that mirror production? Can you automatically provide rich data with code-level detail back to the developers when issues occur in production? In his session at DevOps Summit, David Tesar, Microsoft Technical Evangelist on Microsoft Azure and DevOps, will discuss how to accomplish this and more utilizing technologies such as Microsoft Azure, Visual Studio online, and Application Insights in this demo-heavy session.