The growth of storage capacity led to an attitude that storage was too cheap to manage, but this didn't last long. Before we knew it, IT was faced with a flood of data, easily too much to manage. Faced with limits to their ability to control data growth, IT tried to get the business interested in information lifecycle management (ILM). When this didn't work, they opted for cheaper capacity. Today, we look into the outcome of this tiered storage adventure.

Why isn't storage getting cheaper? This series of articles attempts to answer this question:

1. Too Cheap to Manage
2. Too Much to Manage
3. Tiered Storage
4. The Glass Floor
5. Storage as a Service

Tiered Storage

The uncontrolled growth of unstructured data left IT managers trying to reduce storage costs at the wrong end: Frantically adding cheaper capacity and trying to lower overall cost with tiered storage. This is simply the only option they had, after the business failed to focus on data classification, content management, and information lifecycle management concepts.

Classical tiered storage assumed that there was flexibility in the relationship
between capacity, performance, and cost

Tiered storage was essentially information lifecycle management (ILM) without the lifecycle: It assumed that high-performance storage is scarce, expensive, and ought to be reserved for high-value applications. Since most enterprise storage was taken as high-performance, the implementation process generally revolved around adding cheap "bulk storage" and migrating data down the pyramid.

Tiered storage was supposed to dramatically reduce cost, but it has not helped all that much. There are a few reasons for this:

1. Automation has been something of a holy grail for tiered storage, with data classification engines algorithmically assigning value and data movers migrating content between tiers. But neither element worked all that well in practice, so most tiered storage architectures lack automation: Two or three tiers of storage was installed, and whole applications were manually placed where the administrators thought appropriate.
2. A lack of automation implies a lack of granularity of data placement. Most require entire LUNs (also called drives, drive letters, or volumes) to be manually placed on initial creation. This unit of management is simply too large, placing data on the wrong storage tier.
3. Some storage hardware offers automated internal tiering to various degrees, but these tend to be inherently expensive. If a high-end enterprise storage array is required to implement tiers, the difference in cost between one disk type and another is unlikely to make a significant difference.
4. The expense and effort of migration and management further reduces the impact of even the best automated tiered storage approaches. This is something I will get into in the next article in this series.
5. Some placed tape in the tiered storage pyramid, but the offline nature of content on tape causes serious issues. Unless an integrated hierarchical storage management (HSM) application is used (as in the case with many mainframe systems), tape really is not a tier of online or primary storage. It's an alternate location for data protection.

Solid-state disk and cloud storage can complement traditional on-site disk,
delivering real performance and cost advantages

All is not lost, however. We can try to address each of these deficiencies, and many storage vendors are focused on doing just this. Hu Yoshida of HDS suggests that flash makes up the apex, and most of the pyramid will be taken up by disk. IBM's Barry Whyte even suggests that perhaps the tip of the pyramid might actually expand into an egg timer shape. Regardless, both agree that tiered storage is changing, especially with regard to what we can expect from conventional hard disk drives. But neither talk about where this storage will reside. Considering the massive cost of on-site disk storage, cloud storage looks like a great alternative for lower-tier data.

Tomorrow we will take a deeper look into the reason that reducing the cost of disk capacity has not impacted overall cost of enterprise storage.