Welcome!

@CloudExpo Authors: Liz McMillan, Elizabeth White, Pat Romanski, William Schmarzo, Madhavan Krishnan, VP, Cloud Solutions, Virtusa

Related Topics: @CloudExpo, Microservices Expo, Containers Expo Blog, Agile Computing, Apache

@CloudExpo: Blog Feed Post

APIs: Essential for Delivering Storage in Enterprise Cloud Infrastructures

As new storage platforms evolve, native API support is a must

It’s pretty easy to pick holes in the current legacy storage products, especially when it comes to integration within both public and private cloud deployments.  However it’s worth discussing exactly what is required when implementing cloud frameworks, as the way in which storage is deployed is radically different from the traditional model of storage operations.  In this post we will look at why traditional methods of storage management need to change and how that affects the way in which the hardware itself is used.  This leads to a discussion on APIs and how they are essential to drive cloud deployments effectively.

The Legacy View

Legacy Provisioning Process
For the last 10 years or so, the traditional view of  storage management has consisted of a number of Storage Administrators using a GUI, CLIs and/or scripts to process storage requests as they are generated by the business user.  The process is highly manual, with lots of interactions between the requestor, the storage admin delivering the work and other intermediaries to cover things like billing, change control, capacity management and workload scheduling.  This made the overall process pretty people intensive and not surprisingly elongated the delivery time.  Many end users will also have recollections of asking for their specific requirement to be told they can only have something “off the shelf” – i.e. storage to a standard LUN size and with a specific RAID protection.

This was done for obvious reasons; firstly the configuration of large arrays was predicated on pre-planning and a fixed design, usually created at hardware installation time.  Once defined and in use, it couldn’t be changed (or at least couldn’t be changed without significant impact and cost).  Second, it makes sense to reduce requirements into a smaller subset to make the provisioning process easier.  As well as being rigid in configuration, many legacy arrays assume the creation and provisioning of LUNs is an infrequent task.  Many require requests to be packaged and executed in batch and certainly can’t cope easily with concurrent requests.  Although it is possible to automate some provisioning processes using CLIs and scripts, this doesn’t address the real requirements in creating an on-demand model.

The New World

API Provisioning Process
As we scale up to ever large IT deployments and especially within service-based or “cloud” configurations, the idea of having large amounts of human intervention in the provisioning process simply doesn’t work.  Instead, we need to move to a model of “storage on demand” where an external agent – user or orchestration software – can request storage as part of a portal and see the request actioned in real-time or at least within a matter of minutes or hours.  This kind of operation can only be delivered where the hardware has been designed for the purpose.  Where previously storage administrators were involved in every provisioning request, those requests will be actioned within a provisioning framework, defined by the administrator or a storage architect.

Framework
What do we mean by framework?  Well, it’s all about setting a set of parameters around which allocations take place.  This could include:

  • LUN Size
  • Resiliency/Availability
  • Performance
  • Security credentials
  • Snapshot policy
  • Capacity on demand LUN
The architect chooses which specific hardware components are used to meet the requirements.  There are also operational limitations:
  • Maximum number of concurrent requests
  • Maximum number of provisioning requests per hour
  • Ability to suspend or reject provisioning requests by array
  • Restrict requests by array capacity
  • Restrict requests by user based on capacity guidelines
The provisioning framework also needs the ability to work asynchronously and autonomously; that is, to accept, process and acknowledge provisioning requests without the requestor having to maintain a permanent session to the array.  Once requests are completed, the requestor is alerted via a callback mechanism or by manually checking whether a request has completed.  Obviously there is a need for integration into monitoring frameworks, in order to track hardware and performance issues.

Designed for API
Of course there’s a big question around whether APIs can be retro-fitted to existing storage.  In classic IT tradition the answer is “it depends”.  Without a doubt no new storage array should be released without native API functionality.  However fitting an API to existing technology will depend on how flexible the existing configuration process is.  It’s possible to create an API wrapper and build automation into a middleware layer.  This is how products such as iWave’s Storage Automator work.  However adding these features to existing storage products could be costly and still be an imperfect solution.

The Storage Architect Take
As new storage platforms evolve, native API support is a must. The Storage Administrator will simply be required to deploy the infrastructure and plug it into a higher framework from where provisioning will be entirely automated.  Vendors offering this level of functionality will be the most attractive to service providers, looking to make the cost of acquiring and managing storage as cheap as possible. We’re about to see a paradigm shift in the way in which storage is managed and possibly an end to the storage administrator.

Read the original blog entry...

@CloudExpo Stories
SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5–7, 2018, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buye...
The question before companies today is not whether to become intelligent, it’s a question of how and how fast. The key is to adopt and deploy an intelligent application strategy while simultaneously preparing to scale that intelligence. In her session at 21st Cloud Expo, Sangeeta Chakraborty, Chief Customer Officer at Ayasdi, provided a tactical framework to become a truly intelligent enterprise, including how to identify the right applications for AI, how to build a Center of Excellence to oper...
"IBM is really all in on blockchain. We take a look at sort of the history of blockchain ledger technologies. It started out with bitcoin, Ethereum, and IBM evaluated these particular blockchain technologies and found they were anonymous and permissionless and that many companies were looking for permissioned blockchain," stated René Bostic, Technical VP of the IBM Cloud Unit in North America, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Conventi...
SYS-CON Events announced today that Telecom Reseller has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
While some developers care passionately about how data centers and clouds are architected, for most, it is only the end result that matters. To the majority of companies, technology exists to solve a business problem, and only delivers value when it is solving that problem. 2017 brings the mainstream adoption of containers for production workloads. In his session at 21st Cloud Expo, Ben McCormack, VP of Operations at Evernote, discussed how data centers of the future will be managed, how the p...
In his session at 21st Cloud Expo, James Henry, Co-CEO/CTO of Calgary Scientific Inc., introduced you to the challenges, solutions and benefits of training AI systems to solve visual problems with an emphasis on improving AIs with continuous training in the field. He explored applications in several industries and discussed technologies that allow the deployment of advanced visualization solutions to the cloud.
Large industrial manufacturing organizations are adopting the agile principles of cloud software companies. The industrial manufacturing development process has not scaled over time. Now that design CAD teams are geographically distributed, centralizing their work is key. With large multi-gigabyte projects, outdated tools have stifled industrial team agility, time-to-market milestones, and impacted P&L stakeholders.
Enterprises are adopting Kubernetes to accelerate the development and the delivery of cloud-native applications. However, sharing a Kubernetes cluster between members of the same team can be challenging. And, sharing clusters across multiple teams is even harder. Kubernetes offers several constructs to help implement segmentation and isolation. However, these primitives can be complex to understand and apply. As a result, it’s becoming common for enterprises to end up with several clusters. Thi...
"NetApp is known as a data management leader but we do a lot more than just data management on-prem with the data centers of our customers. We're also big in the hybrid cloud," explained Wes Talbert, Principal Architect at NetApp, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"We're focused on how to get some of the attributes that you would expect from an Amazon, Azure, Google, and doing that on-prem. We believe today that you can actually get those types of things done with certain architectures available in the market today," explained Steve Conner, VP of Sales at Cloudistics, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...
"Cloud Academy is an enterprise training platform for the cloud, specifically public clouds. We offer guided learning experiences on AWS, Azure, Google Cloud and all the surrounding methodologies and technologies that you need to know and your teams need to know in order to leverage the full benefits of the cloud," explained Alex Brower, VP of Marketing at Cloud Academy, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clar...
In his session at 21st Cloud Expo, Carl J. Levine, Senior Technical Evangelist for NS1, will objectively discuss how DNS is used to solve Digital Transformation challenges in large SaaS applications, CDNs, AdTech platforms, and other demanding use cases. Carl J. Levine is the Senior Technical Evangelist for NS1. A veteran of the Internet Infrastructure space, he has over a decade of experience with startups, networking protocols and Internet infrastructure, combined with the unique ability to it...
It is of utmost importance for the future success of WebRTC to ensure that interoperability is operational between web browsers and any WebRTC-compliant client. To be guaranteed as operational and effective, interoperability must be tested extensively by establishing WebRTC data and media connections between different web browsers running on different devices and operating systems. In his session at WebRTC Summit at @ThingsExpo, Dr. Alex Gouaillard, CEO and Founder of CoSMo Software, presented ...
"There's plenty of bandwidth out there but it's never in the right place. So what Cedexis does is uses data to work out the best pathways to get data from the origin to the person who wants to get it," explained Simon Jones, Evangelist and Head of Marketing at Cedexis, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
WebRTC is great technology to build your own communication tools. It will be even more exciting experience it with advanced devices, such as a 360 Camera, 360 microphone, and a depth sensor camera. In his session at @ThingsExpo, Masashi Ganeko, a manager at INFOCOM Corporation, introduced two experimental projects from his team and what they learned from them. "Shotoku Tamago" uses the robot audition software HARK to track speakers in 360 video of a remote party. "Virtual Teleport" uses a multip...
"We work around really protecting the confidentiality of information, and by doing so we've developed implementations of encryption through a patented process that is known as superencipherment," explained Richard Blech, CEO of Secure Channels Inc., in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
High-velocity engineering teams are applying not only continuous delivery processes, but also lessons in experimentation from established leaders like Amazon, Netflix, and Facebook. These companies have made experimentation a foundation for their release processes, allowing them to try out major feature releases and redesigns within smaller groups before making them broadly available. In his session at 21st Cloud Expo, Brian Lucas, Senior Staff Engineer at Optimizely, discussed how by using ne...
Gemini is Yahoo’s native and search advertising platform. To ensure the quality of a complex distributed system that spans multiple products and components and across various desktop websites and mobile app and web experiences – both Yahoo owned and operated and third-party syndication (supply), with complex interaction with more than a billion users and numerous advertisers globally (demand) – it becomes imperative to automate a set of end-to-end tests 24x7 to detect bugs and regression. In th...