Welcome!

@CloudExpo Authors: Elizabeth White, Pat Romanski, Yeshim Deniz, Liz McMillan, Aruna Ravichandran

Related Topics: @CloudExpo, Microservices Expo, Containers Expo Blog, Agile Computing, Apache

@CloudExpo: Blog Feed Post

APIs: Essential for Delivering Storage in Enterprise Cloud Infrastructures

As new storage platforms evolve, native API support is a must

It’s pretty easy to pick holes in the current legacy storage products, especially when it comes to integration within both public and private cloud deployments.  However it’s worth discussing exactly what is required when implementing cloud frameworks, as the way in which storage is deployed is radically different from the traditional model of storage operations.  In this post we will look at why traditional methods of storage management need to change and how that affects the way in which the hardware itself is used.  This leads to a discussion on APIs and how they are essential to drive cloud deployments effectively.

The Legacy View

Legacy Provisioning Process
For the last 10 years or so, the traditional view of  storage management has consisted of a number of Storage Administrators using a GUI, CLIs and/or scripts to process storage requests as they are generated by the business user.  The process is highly manual, with lots of interactions between the requestor, the storage admin delivering the work and other intermediaries to cover things like billing, change control, capacity management and workload scheduling.  This made the overall process pretty people intensive and not surprisingly elongated the delivery time.  Many end users will also have recollections of asking for their specific requirement to be told they can only have something “off the shelf” – i.e. storage to a standard LUN size and with a specific RAID protection.

This was done for obvious reasons; firstly the configuration of large arrays was predicated on pre-planning and a fixed design, usually created at hardware installation time.  Once defined and in use, it couldn’t be changed (or at least couldn’t be changed without significant impact and cost).  Second, it makes sense to reduce requirements into a smaller subset to make the provisioning process easier.  As well as being rigid in configuration, many legacy arrays assume the creation and provisioning of LUNs is an infrequent task.  Many require requests to be packaged and executed in batch and certainly can’t cope easily with concurrent requests.  Although it is possible to automate some provisioning processes using CLIs and scripts, this doesn’t address the real requirements in creating an on-demand model.

The New World

API Provisioning Process
As we scale up to ever large IT deployments and especially within service-based or “cloud” configurations, the idea of having large amounts of human intervention in the provisioning process simply doesn’t work.  Instead, we need to move to a model of “storage on demand” where an external agent – user or orchestration software – can request storage as part of a portal and see the request actioned in real-time or at least within a matter of minutes or hours.  This kind of operation can only be delivered where the hardware has been designed for the purpose.  Where previously storage administrators were involved in every provisioning request, those requests will be actioned within a provisioning framework, defined by the administrator or a storage architect.

Framework
What do we mean by framework?  Well, it’s all about setting a set of parameters around which allocations take place.  This could include:

  • LUN Size
  • Resiliency/Availability
  • Performance
  • Security credentials
  • Snapshot policy
  • Capacity on demand LUN
The architect chooses which specific hardware components are used to meet the requirements.  There are also operational limitations:
  • Maximum number of concurrent requests
  • Maximum number of provisioning requests per hour
  • Ability to suspend or reject provisioning requests by array
  • Restrict requests by array capacity
  • Restrict requests by user based on capacity guidelines
The provisioning framework also needs the ability to work asynchronously and autonomously; that is, to accept, process and acknowledge provisioning requests without the requestor having to maintain a permanent session to the array.  Once requests are completed, the requestor is alerted via a callback mechanism or by manually checking whether a request has completed.  Obviously there is a need for integration into monitoring frameworks, in order to track hardware and performance issues.

Designed for API
Of course there’s a big question around whether APIs can be retro-fitted to existing storage.  In classic IT tradition the answer is “it depends”.  Without a doubt no new storage array should be released without native API functionality.  However fitting an API to existing technology will depend on how flexible the existing configuration process is.  It’s possible to create an API wrapper and build automation into a middleware layer.  This is how products such as iWave’s Storage Automator work.  However adding these features to existing storage products could be costly and still be an imperfect solution.

The Storage Architect Take
As new storage platforms evolve, native API support is a must. The Storage Administrator will simply be required to deploy the infrastructure and plug it into a higher framework from where provisioning will be entirely automated.  Vendors offering this level of functionality will be the most attractive to service providers, looking to make the cost of acquiring and managing storage as cheap as possible. We’re about to see a paradigm shift in the way in which storage is managed and possibly an end to the storage administrator.

Read the original blog entry...

@CloudExpo Stories
Is advanced scheduling in Kubernetes achievable? Yes, however, how do you properly accommodate every real-life scenario that a Kubernetes user might encounter? How do you leverage advanced scheduling techniques to shape and describe each scenario in easy-to-use rules and configurations? In his session at @DevOpsSummit at 21st Cloud Expo, Oleg Chunikhin, CTO at Kublr, will answer these questions and demonstrate techniques for implementing advanced scheduling. For example, using spot instances ...
SYS-CON Events announced today that Yuasa System will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Yuasa System is introducing a multi-purpose endurance testing system for flexible displays, OLED devices, flexible substrates, flat cables, and films in smartphones, wearables, automobiles, and healthcare.
SYS-CON Events announced today that Taica will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Taica manufacturers Alpha-GEL brand silicone components and materials, which maintain outstanding performance over a wide temperature range -40C to +200C. For more information, visit http://www.taica.co.jp/english/.
As hybrid cloud becomes the de-facto standard mode of operation for most enterprises, new challenges arise on how to efficiently and economically share data across environments. In his session at 21st Cloud Expo, Dr. Allon Cohen, VP of Product at Elastifile, will explore new techniques and best practices that help enterprise IT benefit from the advantages of hybrid cloud environments by enabling data availability for both legacy enterprise and cloud-native mission critical applications. By rev...
When it comes to cloud computing, the ability to turn massive amounts of compute cores on and off on demand sounds attractive to IT staff, who need to manage peaks and valleys in user activity. With cloud bursting, the majority of the data can stay on premises while tapping into compute from public cloud providers, reducing risk and minimizing need to move large files. In his session at 18th Cloud Expo, Scott Jeschonek, Director of Product Management at Avere Systems, discussed the IT and busine...
Organizations do not need a Big Data strategy; they need a business strategy that incorporates Big Data. Most organizations lack a road map for using Big Data to optimize key business processes, deliver a differentiated customer experience, or uncover new business opportunities. They do not understand what’s possible with respect to integrating Big Data into the business model.
Companies are harnessing data in ways we once associated with science fiction. Analysts have access to a plethora of visualization and reporting tools, but considering the vast amount of data businesses collect and limitations of CPUs, end users are forced to design their structures and systems with limitations. Until now. As the cloud toolkit to analyze data has evolved, GPUs have stepped in to massively parallel SQL, visualization and machine learning.
Recently, REAN Cloud built a digital concierge for a North Carolina hospital that had observed that most patient call button questions were repetitive. In addition, the paper-based process used to measure patient health metrics was laborious, not in real-time and sometimes error-prone. In their session at 21st Cloud Expo, Sean Finnerty, Executive Director, Practice Lead, Health Care & Life Science at REAN Cloud, and Dr. S.P.T. Krishnan, Principal Architect at REAN Cloud, will discuss how they b...
Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities – ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups. As a result, many firms employ new business models that place enormous impor...
The next XaaS is CICDaaS. Why? Because CICD saves developers a huge amount of time. CD is an especially great option for projects that require multiple and frequent contributions to be integrated. But… securing CICD best practices is an emerging, essential, yet little understood practice for DevOps teams and their Cloud Service Providers. The only way to get CICD to work in a highly secure environment takes collaboration, patience and persistence. Building CICD in the cloud requires rigorous a...
Data scientists must access high-performance computing resources across a wide-area network. To achieve cloud-based HPC visualization, researchers must transfer datasets and visualization results efficiently. HPC clusters now compute GPU-accelerated visualization in the cloud cluster. To efficiently display results remotely, a high-performance, low-latency protocol transfers the display from the cluster to a remote desktop. Further, tools to easily mount remote datasets and efficiently transfer...
SYS-CON Events announced today that Dasher Technologies will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Dasher Technologies, Inc. ® is a premier IT solution provider that delivers expert technical resources along with trusted account executives to architect and deliver complete IT solutions and services to help our clients execute their goals, plans and objectives. Since 1999, we'v...
SYS-CON Events announced today that MIRAI Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MIRAI Inc. are IT consultants from the public sector whose mission is to solve social issues by technology and innovation and to create a meaningful future for people.
In his session at 21st Cloud Expo, Raju Shreewastava, founder of Big Data Trunk, will provide a fun and simple way to introduce Machine Leaning to anyone and everyone. Together we will solve a machine learning problem and find an easy way to be able to do machine learning without even coding. Raju Shreewastava is the founder of Big Data Trunk (www.BigDataTrunk.com), a Big Data Training and consulting firm with offices in the United States. He previously led the data warehouse/business intellige...
SYS-CON Events announced today that TidalScale, a leading provider of systems and services, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale has been involved in shaping the computing landscape. They've designed, developed and deployed some of the most important and successful systems and services in the history of the computing industry - internet, Ethernet, operating s...
SYS-CON Events announced today that TidalScale will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale is the leading provider of Software-Defined Servers that bring flexibility to modern data centers by right-sizing servers on the fly to fit any data set or workload. TidalScale’s award-winning inverse hypervisor technology combines multiple commodity servers (including their ass...
Gemini is Yahoo’s native and search advertising platform. To ensure the quality of a complex distributed system that spans multiple products and components and across various desktop websites and mobile app and web experiences – both Yahoo owned and operated and third-party syndication (supply), with complex interaction with more than a billion users and numerous advertisers globally (demand) – it becomes imperative to automate a set of end-to-end tests 24x7 to detect bugs and regression. In th...
Amazon is pursuing new markets and disrupting industries at an incredible pace. Almost every industry seems to be in its crosshairs. Companies and industries that once thought they were safe are now worried about being “Amazoned.”. The new watch word should be “Be afraid. Be very afraid.” In his session 21st Cloud Expo, Chris Kocher, a co-founder of Grey Heron, will address questions such as: What new areas is Amazon disrupting? How are they doing this? Where are they likely to go? What are th...
SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
We all know that end users experience the Internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices – not doing so will be a path to eventual b...