With the right tools, file storage today can be in the cloud or on-premise, with seamless and secure access and publishing regardless of location. Novel, high-speed transport technologies that alleviate the bottlenecks and limitations of traditional data movement protocols are now intrinsically enabled for cloud object storage, such as Amazon Web Services S3 and Microsoft Windows Azure. Companies of all types and sizes can use the latest technology to ingest and distribute large media files to a...| By Greg Schulz | Article Rating: |
|
| November 27, 2012 11:30 AM EST | Reads: |
3,265 |
Recently while I was in Europe presenting some sessions at conferences and doing some seminars, I was invited by Ed Saipetch (@edsai) of Inktank.com to attend the first Ceph Day in Amsterdam.
As luck or fate would turn out, I was in Nijkerk which is about an hour train ride from Amsterdam central station plus a free day in my schedule. After a morning train ride and nice walk from Amsterdam Central I arrived at the Tobacco Theatre (a former tobacco trading venue) where Ceph Day was underway, and in time for lunch of Krokettens sandwich.
Let's take a quick step back and address for those not familiar what is Ceph (Cephalanthera) and why it was worth spending a day to attend this event. Ceph is an open source distributed object scale out (e.g. cluster or grid) software platform running on industry standard hardware.
Ceph is used for deploying object storage, cloud storage and managed services, general purpose storage for research, commercial, scientific, high performance computing (HPC) or high productivity computing (commercial) along with backup or data protection and archiving destinations. Other software similar in functionality or capabilities to Ceph include OpenStack Swift, Basho Riak CS, Cleversafe, Scality and Caringo among others. There are also the tin wrapped software (e.g. appliances or pre-packaged) solutions such as Dell DX (Caringo), DataDirect Networks (DDN) WOS, EMC ATMOS and Centera, Amplidata and HDS HCP among others. From a service standpoint, these solutions can be used to build services similar Amazon S3 and Glacier, Rackspace Cloud files and Cloud Block, DreamHost DreamObject and HP Cloud storage among others.
At the heart of Ceph is RADOS a distributed object store that consists of peer nodes functioning as object storage devices (OSD). Data can be accessed via REST (Amazon S3 like) APIs, Libraries, CEPHFS and gateway with information being spread across nodes and OSDs using a CRUSH based algorithm (note Sage Weil is one of the authors of CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data). Ceph is scalable in terms of performance, availability and capacity by adding extra nodes with hard disk drives (HDD) or solid state devices (SSDs). One of the presentations pertained to DreamHost that was an early adopter of Ceph to make their DreamObjects (cloud storage) offering.
In addition to storage nodes, there are also an odd number of monitor nodes to coordinate and manage the Ceph cluster along with optional gateways for file access. In the above figure (via DreamHost), load balancers sit in front of gateways that interact with the storage nodes. The storage node in this example is a physical server with 12 x 3TB HDDs each configured as a OSD.
In the DreamHost example above, there are 90 storage nodes plus 3 management nodes, the total raw storage capacity (no RAID) is about 3PB (12 x 3TB = 36TB x 90 = 3.24PB). Instead of using RAID or mirroring, each objects data is replicated or copied to three (e.g. N=3) different OSDs (on separate nodes), where N is adjustable for a given level of data protection, for a usable storage capacity of about 1PB.
Note that for more usable capacity and lower availability, N could be set lower, or a larger value of N would give more durability or data protection at higher storage capacity overhead cost. In addition to using JBOD configurations with replication, Ceph can also be configured with a combination of RAID and replication providing more flexibility for larger environments to balance performance, availability, capacity and economics.
One of the benefits of Ceph is the flexibility to configure it how you want or need for different applications. This can be in a cost-effective hardware light configuration using JBOD or internal HDDs in small form factor generally available servers, or high density servers and storage enclosures with optional RAID adapters along with SSD. This flexibility is different from some cloud and object storage systems or software tools which take a stance of not using or avoiding RAID vs. providing options and flexibility to configure and use the technology how you see fit.
Here are some links to presentations from Ceph Day:
Introduction and Welcome by Wido den Hollander
Ceph: A Unified Distributed Storage System by Sage Weil
Ceph in the Cloud by Wido den Hollander
DreamObjects: Cloud Object Storage with Ceph by Ross Turk
Cluster Design and Deployment by Greg Farnum
Notes on Librados by Sage Weil
While at Ceph day, I was able to spend a few minutes with Sage Weil Ceph creator and founder of inktank.com to record a pod cast (listen here) about what Ceph is, where and when to use it, along with other related topics. Also while at the event I had a chance to sit down with Curtis (aka Mr. Backup) Preston where we did a simulcast video and pod cast. The simulcast involved Curtis recording this video with me as a guest discussing Ceph, cloud and object storage, backup, data protection and related themes while I recorded this pod cast.
One of the interesting things I heard, or actually did not hear while at the Ceph Day event that I tend to hear at related conferences such as SNW is a focus on where and how to use, configure and deploy Ceph along with various configuration options, replication or copy modes as opposed to going off on erasure codes or other tangents. In other words, instead of focusing on the data protection protocol and algorithms, or what is wrong with the competition or other architectures, the Ceph Day focused was removing cloud and object storage objections and enablement.
Where do you get Ceph? You can get it here, as well as via 42on.com and inktank.com.
Thanks again to Sage Weil for taking time out of his busy schedule to record a pod cast talking about Ceph, as well 42on.com and inktank for hosting, and the invitation to attend the first Ceph Day in Amsterdam.

Returning to Amsterdam central station after Ceph Day
Ok, nuff said.
Cheers Gs
Greg Schulz - Author Cloud and Virtual Data Storage Networking (CRC Press, 2011), The Green and Virtual Data Center (CRC Press, 2009), and Resilient Storage Networks (Elsevier, 2004)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2012 StorageIO All Rights Reserved
Read the original blog entry...
Published November 27, 2012 Reads 3,265
Copyright © 2012 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.
More Stories By Greg Schulz
Greg Schulz is founder of the Server and StorageIO (StorageIO) Group, an IT industry analyst and consultancy firm. Greg has worked with various server operating systems along with storage and networking software tools, hardware and services. Greg has worked as a programmer, systems administrator, disaster recovery consultant, and storage and capacity planner for various IT organizations. He has worked for various vendors before joining an industry analyst firm and later forming StorageIO.
In addition to his analyst and consulting research duties, Schulz has published over a thousand articles, tips, reports and white papers and is a sought after popular speaker at events around the world. Greg is also author of the books Resilient Storage Network (Elsevier) and The Green and Virtual Data Center (CRC). His blog is at www.storageioblog.com and he can also be found on twitter @storageio.
With the right tools, file storage today can be in the cloud or on-premise, with seamless and secure access and publishing regardless of location. Novel, high-speed transport technologies that alleviate the bottlenecks and limitations of traditional data movement protocols are now intrinsically enabled for cloud object storage, such as Amazon Web Services S3 and Microsoft Windows Azure. Companies of all types and sizes can use the latest technology to ingest and distribute large media files to a...May. 18, 2013 10:45 AM EDT Reads: 1,165 |
By Jeremy Geelan Organizations across the world are increasingly starting to see the benefits of moving more and more services to the cloud. The focus on the cost-saving potential of cloud is rapidly shifting to completely transforming the business with cloud. As organizations are investing enormous sums on technology they are starting to realize that in order to maximize the return on investment and accelerate the business transformation process the first area of focus should be people. By ensuring the organiza...May. 18, 2013 10:45 AM EDT Reads: 1,240 |
By Liz McMillan Enterprise cloud adoption revolves around pushing the BYOD movement and focusing on data security.
In his session at the 12th International Cloud Expo, Ross Brouse, COO and President of Solar VPS, will cover how cloud adoption is driven by consumerism, humanity’s need to socialize, our addiction to new gadgets and the ability of data to stay secure in a growing collaborative world. The cloud is a drug and we’re just getting hooked.
Ross Brouse is the COO and President of Solar VPS. He is a tr...May. 18, 2013 10:00 AM EDT Reads: 1,006 |
By Jeremy Geelan New, "Super-Sized" 4-Day Cloud Computing Bootcamp is a brief introduction to cloud computing carefully created and devised to help you keep up with evolving trends like Big Data, PaaS, APIs, Mobile, Social and Data Analytics. Solutions built around these topics require a sound cloud computing infrastructure to be successful while assisting customers harvest real benefits from this transformational change that is happening in the IT ecosystem.May. 18, 2013 10:00 AM EDT Reads: 717 |
By Jeremy Geelan The world’s first vendor neutral marketplace for IaaS (Infrastructure as a Service) cloud computing is being built. This marketplace fills the current gap in the value chain by offering standardized products and by addressing the needs of providers and consumers of cloud computing resources. Zimory is the technical partner for the settlement process of this project.
In his session at 12th Cloud Expo | Cloud Expo New York [June 10-13, 2013], Zimory CEO Rüdiger Baumann session will introduce th...May. 18, 2013 10:00 AM EDT Reads: 3,189 |
By Jeremy Geelan Companies around the world are moving into on-premise private cloud environments. Many connect their private cloud to their public cloud service providers. In his session at 12th Cloud Expo | Cloud Expo New York [June 10-13], Brian Patrick Donaghy will talk about examples of what worked, what failed and why we should think about this evolution.May. 18, 2013 10:00 AM EDT Reads: 1,807 |
By Jeremy Geelan May. 18, 2013 09:00 AM EDT Reads: 3,318 |
By Jeremy Geelan Companies around the world are collecting massive amounts of data everyday that’s sitting around and not being utilized. Take for example the fact that companies collect demographic and location-based data via mobile devices all the time, but have to figure out how to monetize that data. In this session, Joyent CTO and founder Jason Hoffman will examine the state of Big Data, taking a look at what we're doing now to discussing what's on the horizon, as companies prepare and realign their busines...May. 18, 2013 08:00 AM EDT Reads: 965 |
By Jeremy Geelan Planning scalable environments isn't terribly difficult, but it does require a change of perspective. During this session we'll broaden our views to think on an Internet Scale by dissecting a video publishing application built with The SoftLayer Platform, Message Queuing, Object Storage, and Drupal. By examining a scalable modular application build that can handle unpredictable traffic, you'll be able to grow your development arsenal and pick up a few strategies to apply to your own projects. May. 18, 2013 06:00 AM EDT Reads: 2,205 |
By Liz McMillan Learn about the complex regulations surrounding HIPAA compliance and other considerations for running sensitive data in the Cloud.
In their session at the 12th International Cloud Expo, Ken Ziegler, CEO of Logicworks, and Frank Nydam, Director of Healthcare Solutions at VMware, will discuss the best practices for leveraging virtualization and cloud technologies without sacrificing security or compliance. Care providers, State and Federal entities, integrators and SaaS providers large and small...May. 18, 2013 06:00 AM EDT Reads: 1,895 |
- Cloud People: A Who's Who of Cloud Computing
- Cloud Expo New York Speaker Profile: Dave Linthicum – Cloud Technology Partners
- Cloud Expo New York: Cloud Is Changing the Economics of Business
- Cloud Expo New York Speaker Profile: Nicos Vekiarides – TwinStrata
- AMD and Adobe Collaborate on Upcoming Version of Adobe Premiere Pro Software to Enable Breakthrough Video Editing Performance Through Open Standards
- Windows Azure IaaS Reaches General Availability
- State and Local Governments Adopt Microsoft Dynamics CRM to Improve Citizen Service Delivery
- New Relic Q1 2013 Blazes Past Growth Targets and Reaches 40,000 Active Customer Accounts
- Enterasys Spotlights SDN's Impact on Traditional Networking in Upcoming Webinar
- Best CIO Practices Shared from SHI’s Customers
- Cloud Expo New York: Delivering Digital Marketing on the Cloud
- Cloud Expo New York: Deploying Hybrid Cloud for Performance and Uptime
- Cloud People: A Who's Who of Cloud Computing
- Cloud Expo New York: Best CIO Practices Shared from SHI’s Customers
- Cloud Expo New York Speaker Profile: Dave Linthicum – Cloud Technology Partners
- Cloud Expo New York Speaker Profile: Jill T. Singer – NRO
- Cloud Expo New York Speaker Profile: Greg O'Connor – AppZero
- Examining the True Cost of Big Data
- Cloud Expo New York: Cloud Is Changing the Economics of Business
- Cloud Expo New York: How to Use Google Apps Script
- Cloud Expo New York Speaker Profile: Nicos Vekiarides – TwinStrata
- Cloud Computing Bootcamp at Cloud Expo New York
- AMD and Adobe Collaborate on Upcoming Version of Adobe Premiere Pro Software to Enable Breakthrough Video Editing Performance Through Open Standards
- Windows Azure IaaS Reaches General Availability
- The Top 150 Players in Cloud Computing
- What is Cloud Computing?
- Six Benefits of Cloud Computing
- The Top 250 Players in the Cloud Computing Ecosystem
- Twenty-One Experts Define Cloud Computing
- What's the Difference Between Cloud Computing and SaaS?
- Virtualization Conference Keynote Webcast Live on SYS-CON.TV
- The Future of Cloud Computing
- A Brief History of Cloud Computing: Is the Cloud There Yet?
- GDS International: Global Warming Scam?
- Cloud Expo Europe 2009 in Prague: Themes & Topics
- Cloud Computing Expo 2009 West: Call for Papers Now Closed

















Organizations across the world are increasingly starting to see the benefits of moving more and more services to the cloud. The focus on the cost-saving potential of cloud is rapidly shifting to completely transforming the business with cloud. As organizations are investing enormous sums on technology they are starting to realize that in order to maximize the return on investment and accelerate the business transformation process the first area of focus should be people. By ensuring the organiza...
Enterprise cloud adoption revolves around pushing the BYOD movement and focusing on data security.
In his session at the 12th International Cloud Expo, Ross Brouse, COO and President of Solar VPS, will cover how cloud adoption is driven by consumerism, humanity’s need to socialize, our addiction to new gadgets and the ability of data to stay secure in a growing collaborative world. The cloud is a drug and we’re just getting hooked.
Ross Brouse is the COO and President of Solar VPS. He is a tr...
New, "Super-Sized" 4-Day Cloud Computing Bootcamp is a brief introduction to cloud computing carefully created and devised to help you keep up with evolving trends like Big Data, PaaS, APIs, Mobile, Social and Data Analytics. Solutions built around these topics require a sound cloud computing infrastructure to be successful while assisting customers harvest real benefits from this transformational change that is happening in the IT ecosystem.
The world’s first vendor neutral marketplace for IaaS (Infrastructure as a Service) cloud computing is being built. This marketplace fills the current gap in the value chain by offering standardized products and by addressing the needs of providers and consumers of cloud computing resources. Zimory is the technical partner for the settlement process of this project.
In his session at 12th Cloud Expo | Cloud Expo New York [June 10-13, 2013], Zimory CEO Rüdiger Baumann session will introduce th...
Companies around the world are moving into on-premise private cloud environments. Many connect their private cloud to their public cloud service providers. In his session at 12th Cloud Expo | Cloud Expo New York [June 10-13], Brian Patrick Donaghy will talk about examples of what worked, what failed and why we should think about this evolution.
Companies around the world are collecting massive amounts of data everyday that’s sitting around and not being utilized. Take for example the fact that companies collect demographic and location-based data via mobile devices all the time, but have to figure out how to monetize that data. In this session, Joyent CTO and founder Jason Hoffman will examine the state of Big Data, taking a look at what we're doing now to discussing what's on the horizon, as companies prepare and realign their busines...
Planning scalable environments isn't terribly difficult, but it does require a change of perspective. During this session we'll broaden our views to think on an Internet Scale by dissecting a video publishing application built with The SoftLayer Platform, Message Queuing, Object Storage, and Drupal. By examining a scalable modular application build that can handle unpredictable traffic, you'll be able to grow your development arsenal and pick up a few strategies to apply to your own projects.
Learn about the complex regulations surrounding HIPAA compliance and other considerations for running sensitive data in the Cloud.
In their session at the 12th International Cloud Expo, Ken Ziegler, CEO of Logicworks, and Frank Nydam, Director of Healthcare Solutions at VMware, will discuss the best practices for leveraging virtualization and cloud technologies without sacrificing security or compliance. Care providers, State and Federal entities, integrators and SaaS providers large and small...
A recent Gartner study states that the function of the modern CIO is in flux and that his or her future focus must incorporate digital assets (aka cloud-based data and applications) to remain relevant. Towards the goal of riding the sea change a compiler of stacks to a broker of business needs, secu...
New technologies allow schools, colleges and universities to analyze absolutely everything that happens. From student behavior, testing results, career development of students as well as educational needs based on changing societies. A lot of this data has already been stored and is used for statist...
In the coming years, big data will change the way organisations and societies are operated and managed. Big data however, is not the only trend that will impact significantly how organisations operate. Another major trend at the moment is gamification. Gamification will change the way organisations ...
We all talk about cloud differently, but is there a way we should be speaking about this tech?
Cloud computing is now a widely reported, if not accepted, IT movement that, depending on who you talk to, has changed or is changing the way businesses utilize infrastructure.
The age of data center automation is upon us. Whether it's cloud or SDN or devops in general, automation as a means to achieve efficiency and, one hopes, free up resources that can be then redirected to focus on innovation.
As is always the case when we begin to move further upwards, abstracting ...
Windows Azure Virtual Networks offers the power to open up several cross-premises use case scenarios, including Active Directory Disaster Recovery, SQL Database Replication, Windows Server 2012 DFS-R File Replication, Accelerated Cloud File Services with BranchCache, Hybrid Web Applications and MORE...
As the infrastructure cloud market (IaaS and PaaS) continues to grow rapidly, we are seeing quite a few customers who are delivering an application – whether it is a mission-critical or SaaS application – and basing their solution on VMware.
VMware Security Cloud Encryption cloud keyboard Cloud Enc...
Have you heard of products like IBM’s InfoSphere Streams, Tibco’s Event Processing product, or Oracle’s CEP product? All good examples of commercially available stream processing technologies which help you process events in real-time.
I’ve been asked what I consider as “Big Data” versus “Small Dat...
My fellow Technical Evangelists and I have authored a content series that steps through building your very own Private Cloud by leveraging Windows Server 2012, our FREE Hyper-V Server 2012, Windows Azure Infrastructure Services ( IaaS ) and System Center 2012 Service Pack 1.
Week-by-week, we walk ...











