Story image

Project Denali aims to define SSDs for cloud-scale applications

Last September, at SNIA’s Storage Developer’s Conference, Microsoft presented a prototype of the Project Denali SSD. 

Project Denali drives provide the flexibility needed to optimize for the workloads of a wide variety of cloud applications, the simplicity to keep pace with rapid innovations in NAND flash memory and application design, and the scale required for multitenant hardware that is so common in the cloud. 

This month, Azure Storage senior software engineer Laura Caulfield is attending the Open Compute Project (OCP) U.S. Summit 2018 to begin formalization of the specification that will define the interface to Project Denali drives. 

Once in place, the specification will allow both hardware vendors and cloud providers to build and release their final products. 

The specification defines a new abstraction, which separates the roles of managing NAND and managing data placement. 

The former will remain in the hardware, close to the NAND and in the product that reinvents itself with every new generation of NAND. 

The latter, once separated from the NAND management algorithms, will be allowed to follow its own schedule for innovation, and won’t be prone to bugs introduced by product cycles that track solely with NAND generations.

Caulfield states in a blog post, “One of the primary drivers behind the Project Denali architecture is the mismatch of write and reclaim size between the hardware and the application. 

“Cloud-scale applications tend to scale out the number of tenants on a machine, whereas the hardware scales up the size of its architectural parameters, as the number of cores in a server increases, a single machine can support more VMs.”

“When storage servers increase their capacity, they typically increase the number of tenants using each as a back-end, while there are some notable exceptions, there is still a need for cloud hardware to provide enough flexibility to efficiently serve these multi-tenant designs.”

SSDs’ caching further increases this divide. 

Caulfield continues, “To make the best use of the SSD’s parallelism, its controller collects enough writes to fill one flash page in every plane, about 4MB in our current drives.

“If the data comes from multiple applications, the drive mixes their data in a way that is difficult to predict or control, this becomes a problem when the device performs garbage collection (GC).”

By this time, some applications may have freed their data, or updated it at a faster rate than other applications. 

For efficient GC, the ideal is to free or update the data in a single block. 

But when caching brings the effective block size to 1GB, it’s very unlikely that the host can issue enough data to the drive for one tenant before servicing writes for the next tenant.

The final nail in the coffin is in how flash block sizes change over time. 

Advances in the density of NAND flash typically increase the size of the block. 

For this reason, it will be even more important for the host to have access to place its data on the native block size in future designs.

Caulfield concludes, “To address the challenges described above, we took a closer look at the algorithms in SSD’s flash translation layers, and in cloud applications.

“We look forward to finalizing the Denali specification in the months ahead through the Denali group and plan to make this specification available broadly later this year.”

“Refactoring the flash translation layer will promote innovation across the storage stack."

Lenovo DCG moves Knight into A/NZ general manager role
Knight will now relocate to Sydney where he will be tasked with managing and growing the company’s data centre business across A/NZ.
The key to financial institutions’ path to digital dominance
By 2020, about 1.7 megabytes a second of new information will be created for every human being on the planet.
Is Supermicro innocent? 3rd party test finds no malicious hardware
One of the larger scandals within IT circles took place this year with Bloomberg firing shots at Supermicro - now Supermicro is firing back.
Record revenues from servers selling like hot cakes
The relentless demand for data has resulted in another robust quarter for the global server market with impressive growth.
Opinion: Critical data centre operations is just like F1
Schneider's David Gentry believes critical data centre operations share many parallels to a formula 1 race car team.
MulteFire announces industrial IoT network specification
The specification aims to deliver robust wireless network capabilities for Industrial IoT and enterprises.
Google Cloud, Palo Alto Networks extend partnership
Google Cloud and Palo Alto Networks have extended their partnership to include more security features and customer support for all major public clouds.
DigiCert conquers Google's distrust of Symantec certs
“This could have been an extremely disruptive event to online commerce," comments DigiCert CEO John Merrill.