Are you prepared for the future of data? Here’s some guidelines

The next leap forward in enterprise storage speed and consolidation is upon us.

The potential of data is huge – the surface has hardly been scratched – and it's clear that it is and likely always will be integral to the digital economy.

Enter stage left, the future. NVMe is a high-performance technology that you will already find in laptops and phones. It's faster and more parallel that SAS, which can deliver a significant increase in performance and density for storage architectures – provided they're ready for it of course.

And herein lies the problem. There is a crisis in performance density, as today's fastest all-flash storage architectures are fundamentally limited by SAS-based serial connectivity – which is inevitably a bottleneck.

It doesn't matter how many CPU cores you possess or how dense your flash is, at the end of the day all data must move serially.

SAS uses the legacy protocol SCSI, which was designed for disk, yet presents a serial bottleneck for communicating to flash. Each connection from CPU core to flash is limited by the SAS host bus adapter (HBA) and synchronized locking. This serial bottleneck creates performance bottlenecks within the back-end of all-flash arrays.

What's more, big SSDs are only exacerbating the issue, as large amounts of flash behind the SAS-drives delivers rack density at the expense of dismal performance density (IOPS/GB).

The NVMe (over PCIe) protocol holds the promise to eliminate the SAS (SCSI) bottleneck. NVMe brings massive parallelism with up to 64K queues and lockless connections that can provide each CPU core with dedicated queue access to each SSD.

The massive parallelism unlocked by NVMe will be required to take advantage of future technological advances (super-dense SSDs, modern multi-core CPUs, new memory technologies and high-speed interconnects).

NVMe is not only poised to unlock the next generation of performance and density while obliterating the bottlenecks of old, but also looks set to represent a technology that will have a profound impact on all-flash arrays in the coming years.

With the NVMe transition starting in 2017, if you're buying a storage array now that has no non-disruptive upgrade path to full NVMe, your investment is likely being thrown away and the benefits of NVMe foregone or deferred by several years.

Here are some of the main benefits that NVMe will bring to customers' applications and data centers:

• Better performance, including increases in bandwidth and consistently low latency
• Higher density – 1PB of usable capacity from Pure Storage would have spanned 6 racks in 2012, and 1PB occupies just 5U now. You can bet that with NVMe, it will be even denser in 2017.
• Performance density that will offer compelling performance, even from very small arrays
• Larger Flash modules without a performance penalty
• Greater consolidation