|By Tom Major||
|February 22, 2013 09:00 AM EST||
I have been working for technology companies for 33 years now, so I don't know why I'm always surprised at the technology myths that proliferate. For example, there is a popular notion running around that Solid State Drives (SSDs) will replace Hard Disk Drives (HDDs) as the dominant storage media. So, which will win? Let me give you my answer upfront before I take you through my arguments: both win, and the market size for both continues to grow.
As I offer up this data and this perspective, I'll repeat some advice I received early in my career. A mentor once told me, when searching for the truth in business and technology "look to the economics" (maybe my version of "go to the mattresses" for you "Godfather" fans).
Here is what I read and hear on a regular basis regarding the battle of HDDs and SSDs:
- SSDs run circles around HDDs for performance
- SSDs will soon replace HDDs
- The improving density of SSDs will collapse the current cost premium to HDDs
Sound familiar? Well, as with most claims of this ilk, there are some elements of truth to them but each of these statements goes too far when viewed in light of some data.
Yes, it's true that SSDs are fundamentally faster than HDDs. This is particularly true of random I/O data reading (as with databases). With large block sequential data (i.e., rich media like video), however, this difference tends to be very small.
Here's a claim where the data just doesn't add up. While the attractiveness of the SSD technology would lead one to believe this, the economics really don't hold up. I should probably point more specifically to NAND flash here (the "solid state" in SSDs) because another solid state technology might actually achieve this, but that appears to be years and tens of billions of dollars away.
Let's talk for a minute about the world's demand for storage. Last year approximately 2500 exabytes of data was created and/or replicated. And...this is doubling about every two years. This needs to be serviced largely by HDDs and SSDs. Last year, the NAND flash industry produced somewhere between 30 and 40 exabytes of storage; with somewhere around 3.5 exabytes finding its way into SSDs (the balance in phones, tablets, cameras, etc.). What is that, 1.4% of our total storage need? But production is ramping up... at a capital cost of about $1.5 billion per Exabyte (semiconductor fabs are expensive)! So, from an available supply perspective, SSDs replacing HDDs seems implausible.
Improving Technology Makes SSDs Cheaper
Agree. The NAND flash technology point is at 21nm line widths with plans to move to 19, then 16 nm. Storage density is further improving with the use of multi-level cell (MLC) capabilities vs single-level cell (SLC). This is bringing down the cost of solid state storage in much the same way the areal density increases seen in HDDs brought down the cost of hard drive storage.
Notice how I said "brought" down (i.e., past tense)? Because a real density growth in the hard drive industry has slowed to a crawl, the rapid erosion of cost/Gb has also slowed to a crawl. Now, the HDD industry needs to move to its next technology (HAMR?) to continue to take cost out and HAMR is a number of years away. HAMR will require significant capital investment by the HDD companies. Significant capital investment will be required anyway to keep up with storage demand (even more so with slowing areal density growth). This all spells out a flattening of HDD costs for the foreseeable future. Some would speculate that a renewed interest in improving the utilization of HDD capacity is an artifact of these economics.
Here's what's preventing the complete collapse of the price difference between the two technologies. The NAND flash suppliers have a similar problem as the HDD manufacturers... the implications of shrinking technology geometries. As the line widths shrink, the ability of flash to sustain multiple write-erase cycles declines. To make up for this deficiency, sophisticated error-correction algorithms and "brute force" overprovisioning (to allow some cells to wear out) are being used. This and the above mentioned semiconductor fab costs tend to mitigate the progress SSDs are making in closing the cost gap to HDDs.
Having made my arguments that SSDs won't take over the world, I have to say that IT solutions need this technology. I think you can see that SSDs aren't the answer to all our storage needs, but they allow us to address a crying storage requirement. At a high level, storage is called on to produce two key deliverables:
- Make data available to an application or user in an appropriate timeframe (i.e., performance) and in today's environment this need is growing.
- Store data reliably (i.e., capacity) and in today's environment this need is growing
By and large, to date, systems with hard drives have been architected to deliver both of these capabilities. It's hard to argue that hard drives haven't done a good job of delivering affordable capacity. But to deliver against the performance requirements of IT solutions, hard drives:
- Have been developed with higher speeds, but arguably little progress has been made in the last ten years
- Have been "short stroked" (a technique limiting the stroke of the actuator to improve performance)
- Have been grouped together to allow striping of data across a large number of drives to aggregate their performance.
With each of these approaches the user suffers from higher power requirements, and with the last two approaches their system has been overprovisioned (leaving stranded capacity) to deliver performance. This all adds up to significant system cost that can be avoided with new storage architectures.
Here's where SSDs come in. SSDs are proving to be a technology answer to the new generation of storage needs, both the growing performance and the growing capacity requirements. Here's the simple, logical way to think about SSDs and HDDs and their role in storage solutions. Use the right tool for the job. That is, take advantage of SSDs for performance (particularly small block, random I/O) and HDDs for capacity.
You might challenge me now and ask, "Does adding SSDs (Flash) to IT solutions make economic sense?" SSDs are expensive, but, used appropriately, SSDs can also minimize the number of HDDs required in a given solution. The secret is in optimizing the use of both, that is, avoid overprovisioning of both. Thin provisioning of capacity has become popular as a cost saver. Similarly, thin provisioning of performance is a similar cost saver. (This is a benefit of virtualized caching or tiering capabilities of newer storage solutions.)
I'll close with a few proof points. Look to the latest economically sensible storage solutions that are answering today's performance and capacity calls. The vast majority of affordable "Ultrabooks" incorporate both flash and hard drive technology, as does Apple's latest "Fusion" drive.
The definition of IoT is not new, in fact it’s been around for over a decade. What has changed is the public's awareness that the technology we use on a daily basis has caught up on the vision of an always on, always connected world. If you look into the details of what comprises the IoT, you’ll see that it includes everything from cloud computing, Big Data analytics, “Things,” Web communication, applications, network, storage, etc. It is essentially including everything connected online from ha...
Nov. 22, 2014 10:00 PM EST Reads: 955
We are all here because we are sold on the transformative promise of The Cloud. But what good is all of this ephemeral, on-demand infrastructure if your usage doesn't actually improve the agility and speed of your business? How must Operations adapt in order to avoid stifling your Cloud initiative? In his session at DevOps Summit, Damon Edwards, co-founder and managing partner of the DTO Solutions, will highlight the successful organizational, process, and tooling patterns of high-performing c...
Nov. 22, 2014 08:45 PM EST Reads: 880
Software-driven innovation is becoming a primary approach to how businesses create and deliver new value to customers. A survey of 400 business and IT executives by the IBM Institute for Business Value showed businesses that are more effective at software delivery are also more profitable than their peers nearly 70 percent of the time (1). DevOps provides a way for businesses to remain competitive, applying lean and agile principles to software development to speed the delivery of software that ...
Nov. 22, 2014 08:00 PM EST Reads: 1,408
Docker offers a new, lightweight approach to application portability. Applications are shipped using a common container format and managed with a high-level API. Their processes run within isolated namespaces that abstract the operating environment independently of the distribution, versions, network setup, and other details of this environment. This "containerization" has often been nicknamed "the new virtualization." But containers are more than lightweight virtual machines. Beyond their small...
Nov. 22, 2014 06:45 PM EST Reads: 1,092
The move in recent years to cloud computing services and architectures has added significant pace to the application development and deployment environment. When enterprise IT can spin up large computing instances in just minutes, developers can also design and deploy in small time frames that were unimaginable a few years ago. The consequent move toward lean, agile, and fast development leads to the need for the development and operations sides to work very closely together. Thus, DevOps become...
Nov. 22, 2014 05:45 PM EST Reads: 1,174
Cloud Expo 2014 TV commercials will feature @ThingsExpo, which was launched in June, 2014 at New York City's Javits Center as the largest 'Internet of Things' event in the world.
Nov. 22, 2014 05:30 PM EST Reads: 1,242
An entirely new security model is needed for the Internet of Things, or is it? Can we save some old and tested controls for this new and different environment? In his session at @ThingsExpo, New York's at the Javits Center, Davi Ottenheimer, EMC Senior Director of Trust, reviewed hands-on lessons with IoT devices and reveal a new risk balance you might not expect. Davi Ottenheimer, EMC Senior Director of Trust, has more than nineteen years' experience managing global security operations and asse...
Nov. 22, 2014 05:30 PM EST Reads: 1,025
The major cloud platforms defy a simple, side-by-side analysis. Each of the major IaaS public-cloud platforms offers their own unique strengths and functionality. Options for on-site private cloud are diverse as well, and must be designed and deployed while taking existing legacy architecture and infrastructure into account. Then the reality is that most enterprises are embarking on a hybrid cloud strategy and programs. In this Power Panel at 15th Cloud Expo (http://www.CloudComputingExpo.com...
Nov. 22, 2014 07:00 AM EST Reads: 1,323
Explosive growth in connected devices. Enormous amounts of data for collection and analysis. Critical use of data for split-second decision making and actionable information. All three are factors in making the Internet of Things a reality. Yet, any one factor would have an IT organization pondering its infrastructure strategy. How should your organization enhance its IT framework to enable an Internet of Things implementation? In his session at Internet of @ThingsExpo, James Kirkland, Chief Ar...
Nov. 21, 2014 09:15 PM EST Reads: 1,243
Leysin American School is an exclusive, private boarding school located in Leysin, Switzerland. Leysin selected an OpenStack-powered, private cloud as a service to manage multiple applications and provide development environments for students across the institution. Seeking to meet rigid data sovereignty and data integrity requirements while offering flexible, on-demand cloud resources to users, Leysin identified OpenStack as the clear choice to round out the school's cloud strategy. Additional...
Nov. 21, 2014 08:45 PM EST Reads: 1,209
The security devil is always in the details of the attack: the ones you've endured, the ones you prepare yourself to fend off, and the ones that, you fear, will catch you completely unaware and defenseless. The Internet of Things (IoT) is nothing if not an endless proliferation of details. It's the vision of a world in which continuous Internet connectivity and addressability is embedded into a growing range of human artifacts, into the natural world, and even into our smartphones, appliances, a...
Nov. 21, 2014 08:00 PM EST Reads: 1,260
Technology is enabling a new approach to collecting and using data. This approach, commonly referred to as the "Internet of Things" (IoT), enables businesses to use real-time data from all sorts of things including machines, devices and sensors to make better decisions, improve customer service, and lower the risk in the creation of new revenue opportunities. In his General Session at Internet of @ThingsExpo, Dave Wagstaff, Vice President and Chief Architect at BSQUARE Corporation, discuss the ...
Nov. 21, 2014 08:00 PM EST Reads: 1,323
"BSQUARE is in the business of selling software solutions for smart connected devices. It's obvious that IoT has moved from being a technology to being a fundamental part of business, and in the last 18 months people have said let's figure out how to do it and let's put some focus on it, " explained Dave Wagstaff, VP & Chief Architect, at BSQUARE Corporation, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Nov. 21, 2014 07:00 PM EST Reads: 1,199
The 4th International DevOps Summit, co-located with16th International Cloud Expo – being held June 9-11, 2015, at the Javits Center in New York City, NY – announces that its Call for Papers is now open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's large...
Nov. 21, 2014 06:45 PM EST Reads: 1,165
Verizon Enterprise Solutions is simplifying the cloud-purchasing experience for its clients, with the launch of Verizon Cloud Marketplace, a key foundational component of the company's robust ecosystem of enterprise-class technologies. The online storefront will initially feature pre-built cloud-based services from AppDynamics, Hitachi Data Systems, Juniper Networks, PfSense and Tervela. Available globally to enterprises using Verizon Cloud, Verizon Cloud Marketplace provides a one-stop shop fo...
Nov. 21, 2014 05:30 PM EST Reads: 1,131
"Our premise is Docker is not enough. That's not a bad thing - we actually love Docker. At ActiveState all our products are based on open source technology and Docker is an up-and-coming piece of open source technology," explained Bart Copeland, President & CEO of ActiveState Software, in this SYS-CON.tv interview at DevOps Summit at Cloud Expo®, held Nov 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Nov. 20, 2014 10:30 PM EST Reads: 1,269
DevOps Summit 2015 New York, co-located with the 16th International Cloud Expo - to be held June 9-11, 2015, at the Javits Center in New York City, NY - announces that it is now accepting Keynote Proposals. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce software that is obsolete...
Nov. 20, 2014 09:00 PM EST Reads: 1,459
Infor has announced a new feature Infor CloudSuite™ Aerospace & Defense (A&D) to aid compliance with International Traffic in Arms Regulations (ITAR). The ITAR function will serve as a complementary function for new or existing Infor CloudSuite A&D customers, to facilitate compliance for Infor customers that are creating a US defense article or performing a US defense service and wish to benefit from cloud-services. The ITAR regulation serves to manage handling and access requirements for dat...
Nov. 20, 2014 07:00 PM EST Reads: 1,279
What do a firewall and a fortress have in common? They are no longer strong enough to protect the valuables housed inside. Like the walls of an old fortress, the cracks in the firewall are allowing the bad guys to slip in - unannounced and unnoticed. By the time these thieves get in, the damage is already done and the network is already compromised. Intellectual property is easily slipped out the back door leaving no trace of forced entry. If we want to reign in on these cybercriminals, it's hig...
Nov. 20, 2014 07:00 PM EST Reads: 1,155