Welcome!

Cloud Expo Authors: Pat Romanski, Ignacio M. Llorente, Haim Koshchitzky, Liz McMillan, Gary Kaiser

Related Topics: Cloud Expo

Cloud Expo: Article

Cloud Database Design, Scale Out Using Shared Nothing Pattern

Virtual storage and database partitions

Abstraction Vs Flexbility :
It is evident that storage plays a major part in the data center and for cloud services. The storage virtualization plays a key part in the dynamic infrastructure attribute of Cloud Computing. Which means the storage is provisioned and de-allocated on demand and usage needs. The good part is that this complex stuff is hidden from the cloud consumer.

However while the storage allocation is abstracted it also brings in performance concerns in a multi tenant cloud environment where by most of the cloud consumers are geographically dispersed and a good amount of data retrieval and manipulation at stake.

Currently most cloud storage virtualization platforms rely on the hardware options like fiber channel, Ethernet switch, network attached storage (NAS) and iSCSI as the implementation medium.

Currently Cloud platforms have very little support for database design related virtualization enhancements. But in future designing databases specific for Cloud especially for private clouds in large enterprises is a sure possibility. In this context the following design principles are important when you design database applications which needs to be delivered using Cloud platform.

Shared Nothing Vs Shared Every Thing
Shared resource and shared nothing represent forms of data access architectures. In a shared (every thing) resource model various processes in the DBMS have access to all system resources, including the data. In the shared-nothing environment, separate DBMS resources divide up the workload, rach responsible for its own data, memory locations, and other resources.

Most popular databases support a shared-nothing model using different strategies. For example,

  • Oracle : Utilizes Range & Hash portioning of tables towards shared nothing . Here it is a table and table space level segregation of data based either on range of key values or a hashing algorithm. (note : Oracle Real Application clusters at the instance level adopt a Shared Every Thing Model)
  • DB2 UDB follows a shared nothing model. Data is partitioned according to a partition key. Rows are assigned to a partition, and each partition has total control of that row. If another partition wants to read or update a row, it must send the request to the owning partition. The owning partition then executes the command on behalf of the requestor.
  • Sql Server : Utilizes Distributed Partitioned views to implement Shared nothing architecture.

Shared nothing model greatly simplifies things like resource contention, including memory, locks and processors . Implemented properly , it offers unlimited scalability. As new rows or data sources are added, more partitions can be added. The workload on any individual partition remains the same.

Storage Virtualization & Shared Nothing Meet Each Other

While the dynamic infrastructure tenant of cloud computing is satisfied by the current classic model as seen in the picture , but the load balancing and scalability needs are fully satisfied by hardware at this time. However implementing the shared nothing data access architecture with a proper access key will highly improve the scalability and performance of cloud databases which in turn increases the QoS of the Cloud platform and make the XaaS a real success providing the best of cost efficiency and improved performance.

View Of a Shared Nothing Cloud Database

As evident implementing a Shared Nothing Cloud Database Model does not replace Storage Virtualization, but compliments the scalability further by splitting the data within multiple virtual storage drives. As seen in the diagram,

Load balancing algorithms implemented at the shared nothing cloud databases further augment the Load balancing options provided by the Cloud Platform.

Shared Nothing Cloud Database Design Considerations

  • Implement Hashing or Ranging Keys as part of the tables which logically separate the data based on the Cloud platform needs. For example if in a SaaS platform multi tenancy is the need then having the CUSTOMER ID or CUSTOMER KEY as a Hashing key for the shared nothing cloud databases
  • Increase the scalability further by implementing a Geography or Time based partition which scales up well depending on the situations
  • Ensure that the queries utilize the logical partitions established as part of the shared nothing architecture. For example if you can query a customer by SSN in a cloud database which has implemented a shared nothing architecture using Geography, then utilize the geography also part of the query filter conditions
  • Have an option to rebalance the data within the virtualized storage if they started becoming skewed due to the way the hashing algorithm is built in

What is expected from Cloud Platform Providers & Database Vendors

  • Expecting the database vendors provide shared nothing algorithms tailored for virtual storage, especially on the rebalancing based on the disk usage
  • New cloud aware partitioning techniques
  • Complimenting load balancing algorithm to augment what is currently available at the hardware
  • Ability to PIN SQLs to a virtual machine based on dynamically queried resource usage

Over all Private Clouds at Enterprise need to satisfy still more SLA or QoS as part of their service and full support from database engines is key to the success. Share Nothing Architecture is one important thing to achieve these non functional needs of the Cloud Platform.

More Stories By Srinivasan Sundara Rajan

I am passionate about ownership and driving things on my own, with my breadth and depth on Enterprise Technology I could run any aspect of IT Industry and make it a success. I am a Seasoned Enterprise IT Expert, mainly in the areas of Solution,Integration and Architecture, across Structured, Unstructured data sources, especially in manufacturing domain. My recent work is on Natural Language Processing, Semantic Enrichment of Unstructured Data, Data Mining and Predictive Analytics. However I have a strong footing across all tiers of Enterprise IT spectrum. I am geared to handle the massive flow of data by Internet Of Things with appropriate platform, tools and processes.

Cloud Expo Breaking News
Cloud Computing is evolving into a Big Three of Amazon Web Services, Google Cloud, and Microsoft Azure. Cloud 360: Multi-Cloud Bootcamp, being held Nov 4–5, 2014, in conjunction with 15th Cloud Expo in Santa Clara, CA, delivers a real-world demonstration of how to deploy and configure a scalable and available web application on all three platforms. The Cloud 360 Bootcamp, led by Janakiram MSV, an analyst with Gigaom Research, is the first bootcamp that introduces the core concepts of Infrastructure as a Service (IaaS) based on the workings of the Big Three platforms – Amazon EC2, Google Compute Engine, and Azure VMs. Bootcamp attendees will get to see the big picture and also receive the knowledge needed to make the best cloud decisions for their business applications and entire enterprise IT organization.
Scott Jenson leads a project called The Physical Web within the Chrome team at Google. Project members are working to take the scalability and openness of the web and use it to talk to the exponentially exploding range of smart devices. Nearly every company today working on the IoT comes up with the same basic solution: use my server and you'll be fine. But if we really believe there will be trillions of these devices, that just can't scale. We need a system that is open a scalable and by using the URL as a basic building block, we open this up and get the same resilience that the web enjoys.
The Internet of Things is a natural complement to the cloud and related technologies such as Big Data, analytics, and mobility. In his session at Internet of @ThingsExpo, Joe Weinman will lay out four generic strategies – digital disciplines – to exploit emerging digital technologies for strategic advantage. Joe Weinman has held executive leadership positions at Bell Labs, AT&T, Hewlett-Packard, and Telx, in areas such as corporate strategy, business development, product management, operations, and R&D.
SYS-CON Events announced today that DevOps.com has been named “Media Sponsor” of SYS-CON's “DevOps Summit at Cloud Expo,” which will take place on June 10–12, 2014, at the Javits Center in New York City, New York. DevOps.com is where the world meets DevOps. It is the largest collection of original content relating to DevOps on the web today Featuring up-to-the-minute news, feature stories, blogs, bylined articles and more, DevOps.com is where the thought leaders of the DevOps movement make their ideas known.
There are 182 billion emails sent every day, generating a lot of data about how recipients and ISPs respond. Many marketers take a more-is-better approach to stats, preferring to have the ability to slice and dice their email lists based numerous arbitrary stats. However, fundamentally what really matters is whether or not sending an email to a particular recipient will generate value. Data Scientists can design high-level insights such as engagement prediction models and content clusters that allow marketers to cut through the noise and design their campaigns around strong, predictive signals, rather than arbitrary statistics. SendGrid sends up to half a billion emails a day for customers such as Pinterest and GitHub. All this email adds up to more text than produced in the entire twitterverse. We track events like clicks, opens and deliveries to help improve deliverability for our customers – adding up to over 50 billion useful events every month. While SendGrid data covers only abo...
SYS-CON Events announced today that the Web Host Industry Review has been named “Media Sponsor” of SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Since 2000, The Web Host Industry Review has made a name for itself as the foremost authority of the Web hosting industry providing reliable, insightful and comprehensive news, reviews and resources to the hosting community. TheWHIR Blogs provides a community of expert industry perspectives. The Web Host Industry Review Magazine also offers a business-minded, issue-driven perspective of interest to executives and decision-makers. WHIR TV offers on demand web hosting video interviews and web hosting video features of the key persons and events of the web hosting industry. WHIR Events brings together like-minded hosting industry professionals and decision-makers in local communities. TheWHIR is an iNET Interactive property.
SYS-CON Events announced today that O'Reilly Media has been named “Media Sponsor” of SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. O'Reilly Media spreads the knowledge of innovators through its books, online services, magazines, and conferences. Since 1978, O'Reilly Media has been a chronicler and catalyst of cutting-edge development, homing in on the technology trends that really matter and spurring their adoption by amplifying "faint signals" from the alpha geeks who are creating the future. An active participant in the technology community, the company has a long history of advocacy, meme-making, and evangelism.
SYS-CON Events announced today that Verizon has been named “Gold Sponsor” of SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Verizon Enterprise Solutions creates global connections that generate growth, drive business innovation and move society forward. With industry-specific solutions and a full range of global wholesale offerings provided over the company's secure mobility, cloud, strategic networking and advanced communications platforms, Verizon Enterprise Solutions helps open new opportunities around the world for innovation, investment and business transformation. Visit verizonenterprise.com to learn more.
SYS-CON Events announced today that TMCnet has been named “Media Sponsor” of SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Technology Marketing Corporation (TMC) is the world's leading business to business and integrated marketing media company, servicing niche markets within the communications and technology industries.
"In my session I spoke about enterprise cloud analytics and how we can leverage analytics as a service," explained Ajay Budhraja, CTO at the Department of Justice, in this SYS-CON.tv interview at the 14th International Cloud Expo®, held June 10-12, 2014, at the Javits Center in New York City. Cloud Expo® 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading Cloud industry players in the world.
"We are automated capacity control software, which basically looks at all the supply and demand and running a virtual cloud environment and does a deep analysis of that and says where should things go," explained Andrew Hillier, Co-founder & CTO of CiRBA, in this SYS-CON.tv interview at the 14th International Cloud Expo®, held June 10-12, 2014, at the Javits Center in New York City. Cloud Expo® 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading Cloud industry players in the world.
“We are starting to see people move beyond the commodity cloud and enterprises need to start focusing on additional value added services in order to really drive their adoption," explained Jason Mondanaro, Director of Product Management at MetraTech, in this SYS-CON.tv interview at the 14th International Cloud Expo®, held June 10-12, 2014, at the Javits Center in New York City. Cloud Expo® 2014 Silicon Valley, November 4–6, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading Cloud industry players in the world.
Almost everyone sees the potential of Internet of Things but how can businesses truly unlock that potential. The key will be in the ability to discover business insight in the midst of an ocean of Big Data generated from billions of embedded devices via Systems of Discover. Businesses will also need to ensure that they can sustain that insight by leveraging the cloud for global reach, scale and elasticity. In his session at Internet of @ThingsExpo, Mac Devine, Distinguished Engineer at IBM, will discuss bringing these three elements together via Systems of Discover.
The Internet of Things promises to transform businesses (and lives), but navigating the business and technical path to success can be difficult to understand. In his session at 15th Internet of @ThingsExpo, Chad Jones, Vice President, Product Strategy of LogMeIn's Xively IoT Platform, will show you how to approach creating broadly successful connected customer solutions using real world business transformation studies including New England BioLabs and more.
All too many discussions about DevOps conclude that the solution is an all-purpose player: developer and operations guru, complete with pager for round-the-clock duty. For most organizations that is not the way forward. In his session at DevOps Summit, Bernard Golden, Vice President of Strategy at ActiveState, will discuss how to achieve the agility and speed of end-to-end automation without requiring an organization stocked with Supermen and Superwomen.