On the Path to Sustainable Enterprise Storage

drought tolerant yardWith the drought here in California, we replaced our grass lawn with drought tolerant plants. It requires a lot less water, but the best thing about it is that we have a new patio to hang out on and watch the world go past.

A few days ago I was smugly sipping a Quivera Sauvignon Blanc on our new patio, looking at our drought-tolerant garden and thinking about posting on “green enterprise storage” – a concept that always seemed like a stretch, considering the massive energy drain and short life cycles of these machines (~ 3.5 years). Even the comparatively small enterprise storage systems I used to work with at equallogic back power cablesEqualLogic and StorSimple used a lot of power and generated a lot of heat. The much larger 3PAR systems I worked with drew massive amounts of power and were anything but green, even though they operated at higher utilization rates than most of their big gladiator-class competitors.

Enterprise storage is designed for hard use and it tends to get exhausted in short order. Organizations depend heavily on their enterprise storage systems and have little motivation to take chances by extending their longevity. No amount of greenwashing is going to change that dynamic.

While green enterprise storage is not exactly credible,sustainable computing word cloud the concept of sustainability in enterprise storage is. Sustainability is not an assessment of a product’s environmental impact as much as it is a long-term approach to reducing resource consumption and undesirable waste. In other words, the enterprise arrays used by a business might not necessarily be “green”, but the IT organization can have a goal of becoming more sustainable over time. When you see something that is so obviously wasteful – as enterprise storage is – it isn’t difficult to believe that progress can be made.

Flash storage is the game changer for sustainability

new flash trajectoryThe undisputable champion of sustainable enterprise storage is flash. Flash has changed the trajectory of the entire industry and everybody involved is developing products and strategies to exploit it. While most of the focus has been on performance, there are clear cost and sustainability wins too.  For starters, flash SSDs consume less energy and run cooler than disk drives, things that are key to lowering TCO and improving sustainability. Flash SSDs also wear out more slowly than disk drives because they do not have moving, mechanical parts. Even in hybrid arrays combining flash SSDs and disk drives, the disk drives are accessed far less frequently, reducing the heat they generate. Arrays that require far less energy for cooling over their lifetime improve sustainability.

Arrays that last longer and necessitate less frequent swap-outs do too. It is not evident across all vendors yet, but the warranties of flash-based arrays appear to be longer than non-flash arrays and the expected life cycles of flash-based arrays will likely prove to be longer as well, adding years to technology refresh cycles. When this trend is realized throughout the industry it will be another flash-based boost for sustainability.

It’s not clear to me what the manufacturing, distribution and end-of-life waste elements are for flash SSDs and disk drives. Perhaps that is something that will come to light in the future to help guide further discussions and comparisons of disk and flash storage.

The synergy of virtualization, consolidation & data reduction

Considering that data is stored under the direction of the operating system and hypervisor, there is a clear synergy between servers and storage, including the potential to improve sustainability. It follows that improvements generated by servers can be compounded by improvements generated by storage. For example, combining server virtualization and consolidation with data reduction in storage creates a very efficient stack, as illustrated below:

virtualization consolidation and data reductionMany VMs are consolidated onto a much smaller number of physical machines, which use a single enterprise storage array to store data (represented by the steamroller). The array employs data reduction to eliminate statistical redundancies and duplicate copies of data that exist across all the virtual machines.

flash IOPS comparisonIn the same way that virtualization and consolidation work hand-in-hand to improve enterprise storage sustainability, data reduction and flash-based storage are similarly aligned. Prior to the availability of flash-based storage, virtualization experts warned that applications could become starved for IOPs if too many VMs were accessing data on a single storage array. In other words, the scarcity of IOPs was limiting VM consolidation ratios – and further improvements in sustainability. Fortunately, flash-based arrays provide an abundance of IOPs, significantly expanding throughput and multiplying VM consolidation ratios several times over. It is still possible to oversubscribe a storage array with virtual machines, but the point is that flash-based arrays can support many more VMs than non-flash arrays. Increased VM density equates to fewer arrays purchased, less resources consumed and less waste at the end-of-life. When you factor in reduced energy consumption and longer array life-cycles, sustainability is increased even more.

increasing capacity returns of dedupeThe sustainability benefits of deduplication improve even further when similar VMs are aggregated together on a single array. For example, using a single array for a large number of Windows Server VMs creates an environment with a great deal of data commonality, where each additional VM added to the array has increased overlapping data that is deduplicated thereby consuming fewer array resources than the VMs that preceded it.

Small improvements matter

Enterprise storage arrays need to provide high level of performance and availability, which means they will never become sustainable in the way that organic farming is. That said, there are well-defined and achievable ways to improve the sustainability of enterprise storage using flash-based and data reduction technologies. There are a wide range of products and prices, from server software solutions, to hybrid arrays and all-flash arrays. The good news is that improving enterprise storage sustainability is easily done without restructuring budgets by replacing end-of-life non-flash arrays with more sustainable flash-based arrays that cost approximately the same, or even less than the arrays that they are replacing – and have lower operating costs. As they say, YMMV and there are many options and ways to proceed.

Disclaimer: The company I work for, Tegile Systems, designs, manufactures and sells both hybrid and all-flash enterprise storage arrays.

Flash is Disrupting the Benchmark Game too

reindeer raceA couple days ago, Lou Lydiksen from Pure Storage posted a blog titled “What’s wrong with using 100% non-reducible data?” The post was better than the title suggests because it exposes a problem storage buyers have today, which is validating the performance of flash storage systems using benchmarks developed for disk-based systems.

Independent storage performance TALK-NERDY-TO-MEbenchmarks have been nerdy infotainment for many years due to the work that often goes into getting the best possible results. Storage systems are usually configured and tuned a certain way to get the best results. (An example of a benchmark that appears to break this rule is the recent SPC benchmark from Kaminario). There is nothing wrong with optimizing application performance, even if that application is a benchmark, but it obviously makes sense to use benchmarks that model realistic, production workloads.

New architecturesThe larger question is what happens when new technologies with new architectures come to market?  For example, the SPC1 benchmark for transaction processing accommodates flash storage technology, but it does not allow the use of compression and deduplication. That’s a big deal because both are important cost-saving features of enterprise flash systems that most customers want but have no way to predict if, or how, they will impact performance. The lack of a feature-comprehensive benchmark does not appear to be slowing the growth of flash storage systems, but that is beside the point – customers would benefit. Other realistic tests would be running the benchmark in a virtual environment alongside simulated background processes and pinning the benchmark application in flash while background processes run in disk or some combination of disk and+flash.

The SPC has indicated that they are sorting some of this out and are developing a way to include compression (first) and deduplication (later) rainbow in skittlesin their benchmarks. I don’t know where they are with respect to virtualization and hybrid designs but in my opinion, storage benchmarking is suddenly a green-field opportunity again – brought about by the rise of enterprise flash storage systems. Will we see enterprising analyst/entrepreneurs rise up to grab the brass ring and establish clear thought leadership amidst the chaos? Are there any takers?

Skimming content from Calvin’s blog – my homies!


Two of my buddies from HP, Calvin Zito and Michael Haag recorded a podcast at SNW last week.  The topic is Building Storage for Virtualization.  In addition to having good content, this podcast is very well recorded – good job Calvin.  Of course, the discussion moved to 3PAR systems and although I no longer work for HP/3PAR, I still like the 3PAR system and its architecture.  I’m glad that StorSimple does not compete with 3PAR because I have so many good friends over there.

At one point Calvin suggests that Michael sounds like JR (Jim Richardson, legendary 3PAR employee).  No, Michael you don’t, thank goodness.  I can’t imagine two JRs breathing the same air – something would have to give.

At the end of the podcast, they talk about storage efficiency for virtual servers. The two technologies mentioned were thin provisioning and deduplication.  While StorSimple is now becoming known for it’s cloud integration, I always like reminding people that our systems use both thin provisioning and deduplication technologies with primary storage.

Not dead yet, but when will you get rid of tape?

Do you have any more tapes you want to get rid of?

People have predicted the ending of tape as a storage medium since the first rotating storage drums were made by wrapping recording tape around modified washing machine drums. Too cumbersome and too error prone, tape has survived because people use it for archiving and off-site DR storage. It has always been the storage backstop for all the other things that can go wrong – from human error to combinations of calamities that are stranger than fiction.

But tape itself has been a big problem. It is a byzantine technology with impressive data fast transfer rates, but is saddled by cumbersome management that requires many touch points where things can go wrong. Restoring from multiple tapes is time consuming and unnerving, but considered normal. Contrast that with using dedupe technology that can access and restore  data much more quickly.  The main problem with dedupe is it’s cost. The most popular disk-based dedupe systems are not necessarily cheap. The other problem is that many customers still use tape with dedupe for DR purposes. Used this way, tape it is less intrusive, but it still is a pain.

Disk-based dedupe has taken a big bite out of tape’s business, but yet tape has continued limping along like an unkillable zombie. Now with cloud backup looking like it could take even more out of tape’s market, is tape going to finally keel over?

Tape is tired


Putting tape backups on less expensive virtual tape cloud storage could look like an obvious solution, but like all things in storage, initial impressions are usually misleading. While cloud storage can be made to look like a big disk or tape drive in the sky, it is much slower than the old frenemy tape. The difference is most pronounced when you want it to be most transparent – during restores. Technologies for data reduction, such as deduplication and compression help, but the fastest restores from the cloud will use technologies like thin restores that were developed by StorSimple. Why restore data that you probably won’t need again? Just leave it in the cloud.

But getting back to tape, the cloud industry is making enormous investments in service offerings, including storage services, which will continue to be improved and expanded.  The cloud service providers are not stupid. They want your data so they can get your computing business when you are ready to start doing that.

Tape technology vendors do not have the marketing muscle to protect their install base, regardless of how entrenched those customers may appear. The fact is, only the largest IT shops have the resources to “do tape” well. Everybody else struggles with the stuff and will happy to jettison it as soon as they can.

So will tape disappear completely if most of the market goes away? Probably not, for starters cloud storage service providers will probably use a lot of tape, and large customers that know how to make it work will continue to want it.

My guess is that tape will follow the path of mainframe technologies into the mostly invisible corners of the technology industry where vendors are few and margins are high. Tape won’t die, it will only seem like it did.