Welcome!

Cloud Expo Authors: Pat Romanski, Liz McMillan, Elizabeth White, Kira Makagon, Sematext Blog

Related Topics: Cloud Expo

Cloud Expo: Article

Cloud Database Design, Scale Out Using Shared Nothing Pattern

Virtual storage and database partitions

Abstraction Vs Flexbility :
It is evident that storage plays a major part in the data center and for cloud services. The storage virtualization plays a key part in the dynamic infrastructure attribute of Cloud Computing. Which means the storage is provisioned and de-allocated on demand and usage needs. The good part is that this complex stuff is hidden from the cloud consumer.

However while the storage allocation is abstracted it also brings in performance concerns in a multi tenant cloud environment where by most of the cloud consumers are geographically dispersed and a good amount of data retrieval and manipulation at stake.

Currently most cloud storage virtualization platforms rely on the hardware options like fiber channel, Ethernet switch, network attached storage (NAS) and iSCSI as the implementation medium.

Currently Cloud platforms have very little support for database design related virtualization enhancements. But in future designing databases specific for Cloud especially for private clouds in large enterprises is a sure possibility. In this context the following design principles are important when you design database applications which needs to be delivered using Cloud platform.

Shared Nothing Vs Shared Every Thing
Shared resource and shared nothing represent forms of data access architectures. In a shared (every thing) resource model various processes in the DBMS have access to all system resources, including the data. In the shared-nothing environment, separate DBMS resources divide up the workload, rach responsible for its own data, memory locations, and other resources.

Most popular databases support a shared-nothing model using different strategies. For example,

  • Oracle : Utilizes Range & Hash portioning of tables towards shared nothing . Here it is a table and table space level segregation of data based either on range of key values or a hashing algorithm. (note : Oracle Real Application clusters at the instance level adopt a Shared Every Thing Model)
  • DB2 UDB follows a shared nothing model. Data is partitioned according to a partition key. Rows are assigned to a partition, and each partition has total control of that row. If another partition wants to read or update a row, it must send the request to the owning partition. The owning partition then executes the command on behalf of the requestor.
  • Sql Server : Utilizes Distributed Partitioned views to implement Shared nothing architecture.

Shared nothing model greatly simplifies things like resource contention, including memory, locks and processors . Implemented properly , it offers unlimited scalability. As new rows or data sources are added, more partitions can be added. The workload on any individual partition remains the same.

Storage Virtualization & Shared Nothing Meet Each Other

While the dynamic infrastructure tenant of cloud computing is satisfied by the current classic model as seen in the picture , but the load balancing and scalability needs are fully satisfied by hardware at this time. However implementing the shared nothing data access architecture with a proper access key will highly improve the scalability and performance of cloud databases which in turn increases the QoS of the Cloud platform and make the XaaS a real success providing the best of cost efficiency and improved performance.

View Of a Shared Nothing Cloud Database

As evident implementing a Shared Nothing Cloud Database Model does not replace Storage Virtualization, but compliments the scalability further by splitting the data within multiple virtual storage drives. As seen in the diagram,

Load balancing algorithms implemented at the shared nothing cloud databases further augment the Load balancing options provided by the Cloud Platform.

Shared Nothing Cloud Database Design Considerations

  • Implement Hashing or Ranging Keys as part of the tables which logically separate the data based on the Cloud platform needs. For example if in a SaaS platform multi tenancy is the need then having the CUSTOMER ID or CUSTOMER KEY as a Hashing key for the shared nothing cloud databases
  • Increase the scalability further by implementing a Geography or Time based partition which scales up well depending on the situations
  • Ensure that the queries utilize the logical partitions established as part of the shared nothing architecture. For example if you can query a customer by SSN in a cloud database which has implemented a shared nothing architecture using Geography, then utilize the geography also part of the query filter conditions
  • Have an option to rebalance the data within the virtualized storage if they started becoming skewed due to the way the hashing algorithm is built in

What is expected from Cloud Platform Providers & Database Vendors

  • Expecting the database vendors provide shared nothing algorithms tailored for virtual storage, especially on the rebalancing based on the disk usage
  • New cloud aware partitioning techniques
  • Complimenting load balancing algorithm to augment what is currently available at the hardware
  • Ability to PIN SQLs to a virtual machine based on dynamically queried resource usage

Over all Private Clouds at Enterprise need to satisfy still more SLA or QoS as part of their service and full support from database engines is key to the success. Share Nothing Architecture is one important thing to achieve these non functional needs of the Cloud Platform.

More Stories By Srinivasan Sundara Rajan

Srinivasan is passionate about ownership and driving things on his own, with his breadth and depth on Enterprise Technology he could run any aspect of IT Industry and make it a success.

He is a seasoned Enterprise IT Expert, mainly in the areas of Solution, Integration and Architecture, across Structured, Unstructured data sources, especially in manufacturing domain.

He currently works as Technology Head For GAVS Technologies.

@CloudExpo Stories
The Internet of Things is tied together with a thin strand that is known as time. Coincidentally, at the core of nearly all data analytics is a timestamp. When working with time series data there are a few core principles that everyone should consider, especially across datasets where time is the common boundary. In his session at Internet of @ThingsExpo, Jim Scott, Director of Enterprise Strategy & Architecture at MapR Technologies, discussed single-value, geo-spatial, and log time series dat...
Cultural, regulatory, environmental, political and economic (CREPE) conditions over the past decade are creating cross-industry solution spaces that require processes and technologies from both the Internet of Things (IoT), and Data Management and Analytics (DMA). These solution spaces are evolving into Sensor Analytics Ecosystems (SAE) that represent significant new opportunities for organizations of all types. Public Utilities throughout the world, providing electricity, natural gas and water,...
In her General Session at 15th Cloud Expo, Anne Plese, Senior Consultant, Cloud Product Marketing, at Verizon Enterprise, focused on finding the right mix of renting vs. buying Oracle capacity to scale to meet business demands, and offer validated Oracle database TCO models for Oracle development and testing environments. Anne Plese is a marketing and technology enthusiast/realist with over 19+ years in high tech. At Verizon Enterprise, she focuses on driving growth for the Verizon Cloud platfo...
Between the compelling mockups and specs produced by your analysts and designers, and the resulting application built by your developers, there is a gulf where projects fail, costs spiral out of control, and applications fall short of requirements. In his session at DevOps Summit, Charles Kendrick, CTO and Chief Architect at Isomorphic Software, will present a new approach where business and development users collaborate – each using tools appropriate to their goals and expertise – to build mo...
There is no doubt that Big Data is here and getting bigger every day. Building a Big Data infrastructure today is no easy task. There are an enormous number of choices for database engines and technologies. To make things even more challenging, requirements are getting more sophisticated, and the standard paradigm of supporting historical analytics queries is often just one facet of what is needed. As Big Data growth continues, organizations are demanding real-time access to data, allowing immed...
Scott Jenson leads a project called The Physical Web within the Chrome team at Google. Project members are working to take the scalability and openness of the web and use it to talk to the exponentially exploding range of smart devices. Nearly every company today working on the IoT comes up with the same basic solution: use my server and you'll be fine. But if we really believe there will be trillions of these devices, that just can't scale. We need a system that is open a scalable and by using ...
15th Cloud Expo, which took place Nov. 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA, expanded the conference content of @ThingsExpo, Big Data Expo, and DevOps Summit to include two developer events. IBM held a Bluemix Developer Playground on November 5 and ElasticBox held a Hackathon on November 6. Both events took place on the expo floor. The Bluemix Developer Playground, for developers of all levels, highlighted the ease of use of Bluemix, its services and functionalit...
"ElasticBox is an enterprise company that makes it very easy for developers and IT ops to collaborate to develop, build and deploy applications on any cloud - private, public or hybrid," stated Monish Sharma, VP of Customer Success at ElasticBox, in this SYS-CON.tv interview at DevOps Summit, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
"For the past 4 years we have been working mainly to export. For the last 3 or 4 years the main market was Russia. In the past year we have been working to expand our footprint in Europe and the United States," explained Andris Gailitis, CEO of DEAC, in this SYS-CON.tv interview at Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
The Internet of Things will greatly expand the opportunities for data collection and new business models driven off of that data. In her session at @ThingsExpo, Esmeralda Swartz, CMO of MetraTech, discussed how for this to be effective you not only need to have infrastructure and operational models capable of utilizing this new phenomenon, but increasingly service providers will need to convince a skeptical public to participate. Get ready to show them the money!
The Industrial Internet revolution is now underway, enabled by connected machines and billions of devices that communicate and collaborate. The massive amounts of Big Data requiring real-time analysis is flooding legacy IT systems and giving way to cloud environments that can handle the unpredictable workloads. Yet many barriers remain until we can fully realize the opportunities and benefits from the convergence of machines and devices with Big Data and the cloud, including interoperability, ...
At 15th Cloud Expo, Shrikant Pattathil, Executive Vice President at Harbinger Systems, demos a video delivery platform that helps you do interactive videos. He discusses how Harbinger is accomplishing it in the cloud world, the problems they faced and the choices they made to get around these problems.
“DevOps is really about the business. The business is under pressure today, competitively in the marketplace to respond to the expectations of the customer. The business is driving IT and the problem is that IT isn't responding fast enough," explained Mark Levy, Senior Product Marketing Manager at Serena Software, in this SYS-CON.tv interview at DevOps Summit, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
“The year of the cloud – we have no idea when it's really happening but we think it's happening now. For those technology providers like Zentera that are helping enterprises move to the cloud - it's been fun to watch," noted Mike Loftus, VP Product Management and Marketing at Zentera Systems, in this SYS-CON.tv interview at Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
The 3rd International Internet of @ThingsExpo, co-located with the 16th International Cloud Expo - to be held June 9-11, 2015, at the Javits Center in New York City, NY - announces that its Call for Papers is now open. The Internet of Things (IoT) is the biggest idea since the creation of the Worldwide Web more than 20 years ago.
Want to enable self-service provisioning of application environments in minutes that mirror production? Can you automatically provide rich data with code-level detail back to the developers when issues occur in production? In his session at DevOps Summit, David Tesar, Microsoft Technical Evangelist on Microsoft Azure and DevOps, will discuss how to accomplish this and more utilizing technologies such as Microsoft Azure, Visual Studio online, and Application Insights in this demo-heavy session.
Entuity®, a provider of enterprise-class network management solutions, today announced that it solidifies its position as a market leader through global enterprise customer acquisitions and a refined channel strategy. In 2014, Entuity increased new license revenues in EMEA by over 75 percent, and LATAM by over 125 percent as customers embraced Entuity for its highly automated solution and unified architecture. Entuity’s refined channel strategy focuses on even deeper strategic alignment with ke...
We are all here because we are sold on the transformative promise of The Cloud. But what good is all of this ephemeral, on-demand infrastructure if your usage doesn't actually improve the agility and speed of your business? How must Operations adapt in order to avoid stifling your Cloud initiative? In his session at DevOps Summit, Damon Edwards, co-founder and managing partner of the DTO Solutions, will highlight the successful organizational, process, and tooling patterns of high-performing c...
The 4th International DevOps Summit, co-located with16th International Cloud Expo – being held June 9-11, 2015, at the Javits Center in New York City, NY – announces that its Call for Papers is now open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's large...
Cloud Expo 2014 TV commercials will feature @ThingsExpo, which was launched in June, 2014 at New York City's Javits Center as the largest 'Internet of Things' event in the world.