Storage Developer Conference

Two emerging interconnect efforts, OpenCAPI and Gen-Z, are top-of-mind today in system architecture and storage, including advanced platform support of Persistent Memory. Come hear distinguished leads from each project give an update on the goals, progress, and futures of each. Expect to learn both the differences and commonalities between these potentially symbiotic standards, and how they may enable new platforms and new storage solutions as they move forward.

Kubernetes supports external storage through volume plugins, however, the framework was built using generic concepts so as to work with all types of storage including legacy SANs as well as newer Software Defined Storage (SDS) solutions. As a result of this approach, Kubernetes cannot natively take advantage of some of the benefits of cloud-native storage systems like hyperconvergence, and also lacks native support for advanced storage monitoring and snapshots. Dinesh will present STORK (STorage Orchestrator Runtime for Kubernetes) a new open source extension for Kubernetes, that provides additional intelligence and control for your storage. He will show how STORK can be used to co-locate pods with their data, deal with storage failures seamlessly and take application consistent volume snapshots through Kubernetes. This talk will show how users can overcome limitations of stateful services and bulletproof them in Kubernetes using STORK. Dinesh will demonstrate how, using STORK, Kubernetes based applications

Today’s NVMe storage platforms now support massive quantities of NVMe devices, upwards of 96 drives per system. With all this growth comes new issues around Power delivery, airflow and aggregate performance. While newer architectures like NVMe-oF and in-memory platforms are being developed, they still treat the storage elements as simply ‘holding locations.’ Creating a paradigm shift with In-Storage Compute is a simple, scalable, low power solution that provides developers with opportunities to have intelligent storage and break the boundaries still being hung onto via traditional rotating media architectures. Learning Objectives: 1) In-Storage Compute value; 2) How to deploy at scale; 3) The ease of implementation with existing architectures.

Cinder has long been a key component in OpenStack for providing block storage. It provides support for a broad variety of storage backends ranging from LVM, NFS shares, FC and iSCSI SANs, and more. Yet with all these options, Cinder provides one consistent API abstraction that can be used to manage those varied storage options. But what many don’t realize is – Cinder can be used for much more than OpenStack. Over the last several releases, there has been a focus on making Cinder a viable stand alone storage management interface. Today, Cinder can be used not just in OpenStack, but in container environments such as Kubernetes. It can be used as a scriptable interface for devops tooling and automation. And with clients and SDKs available for Python, Golang, Java, and more, it can provide a simple, programmatic way to build storage management into CI/CD pipelines and other custom workflows. This session demonstrates how to leverage the capabilities of Cinder to meet many of your storage provisioning needs. Le

As storage devices get faster, data management tasks rob the host of CPU cycles and DDR bandwidth. In this session, we examine a new interface to storage devices that can leverage existing and new CPU and DRAM resources to take over data management tasks like availability, recovery, and migrations. This new interface provides a roadmap for device-to-device interactions and more powerful storage devices capable of providing in-store compute services that can dramatically improve performance. We call such storage devices “eusocial” because we are inspired by eusocial insects like ants, termites, and bees, which as individuals are primitive but collectively accomplish amazing things.

Today’s exponential growth of Big-Data makes lossless compression one of the most important operations in a data center as it expands storage capacity, reduces storage costs and speeds up data access. Off-loading compression from processors to FPGAs can free up valuable CPU time while reducing compression time and power consumption. Moreover, sharing accelerators across a data center can further improve resource utilization and lower operation costs. Here, we present an efficient FPGA implementation of a GZIP/ZLIB accelerator which can compress the data at up to 1.7GB/s using about 30KLUTs on a Virtex-7 device. This accelerator is presented as an NVMe namespace via the Eideticom NoLoad platform which allows users to access it locally or across the network using standard NVMe and NVMe over Fabrics drivers available in major operating systems. We also present the software API to access this accelerator which relies on the high-performance inbox NVMe drivers and uses a user-space framework to provide both local

Our human brain can be modeled as two distinctly different systems: a real time intuition system, and a background reasoning system. As we move into the AI compute era, Emerging Memory technologies play an increasingly important role in overcoming the limitations of DRAM and NAND. The commercialization of Emerging Memory will therefore accelerate our realization of powerful AI systems. In this talk, the speaker will: • Demonstrate the human brain has two distinct systems • Discuss the best fit compute architecture to model each AI system • Articulate how DRAM and NAND are applied to the compute architectures • Identify the key limitations of DRAM and NAND • Present and classify some Emerging Memory alternatives • Discuss the Emerging Memory system enhancements • Identify who is doing what (by when) in the Emerging Memory landscape • Articulate the unique challenges in realizing an AI intuition system • Propose how Emerging Memory may solve some intuition problems • Point to the future of AI systems based on

This presentation will consider the various architectures for All Flash Solutions. Considering the limitations in PCIe lanes, tradeoffs need to be made depending on the customer’s application. These tradeoffs will be discussed. Learning Objectives: 1) What limits an all flash solution?; 2) What are the tradeoffs that can be made?; 3) How do these tradeoffs impact the customer’s application?

Kubernetes streamlines the deployment and orchestration of clustered workloads, both on-premise and in the cloud. The Container Storage Interface (CSI) is a newly-defined model for defining templates for mountable volumes and file systems, instantiating them, and binding them to individual containers. In this talk, we will present an overview of the CSI and how it enables rapid migration of workloads to cloud. We will also discuss future directions for extending Kubernetes storage interfaces. Learning Objectives: 1) How Container Storage Interface (CSI) enables container workloads to easily migrate from on-premise to cloud; 2) How to control volume attributes such as redundancy and location; 3) Opportunities to extend Kubernetes to support new storage technologies and use cases.

Serverless computing is becoming increasingly popular, enabling users to quickly launch thousands of short-lived tasks in the cloud with high elasticity and fine-grain billing. While these properties make serverless computing appealing for interactive data analytics, a key challenge is managing intermediate data shared between tasks. Since communicating directly between short-lived serverless tasks is difficult, the natural approach is to store such ephemeral data in a common remote data store. However, existing storage systems are not designed to meet the demands of serverless applications in terms of elasticity, performance and cost. We present Pocket, a distributed data store that elastically scales to automatically provide applications with desired performance at low cost. Pocket dynamically rightsizes storage cluster resource allocations across multiple dimensions (storage capacity, network bandwidth and CPU cores) as application load varies. We show that Pocket cost-effectively rightsizes the type and n

PCIe devices continue to get faster and more powerful. At this point building systems where all DMA traffic must pass through the system memory of the host CPU is becoming problematic. For this reason, there has been considerable work done on both hardware standardization and software frameworks to enable Peer-2-Peer (P2P) DMAs between PCIe End Points (EPs). In this talk we will present an update on the P2PDMA ecosystem and include performance results gathered from systems that have been designed to utilize P2PDMA. We will show how such systems can outperform their conventional counterparts and lead to lower-cost and lower-power designs that still obtain optimal performance. We will also discuss the future for P2PDMA and what work is ongoing to extend the framework. Learning Objectives: 1) Learn new types of data flow within PCIe based storage systems; 2) NVMe CMBs and how they can be used in interesting ways; 3) How software frameworks like SPDK and Linux are being updated to enable these new data flows; 4)

Building on the concepts presented in the Introduction to Swordfish sessions, this session will go into more detail on the new Swordfish Scalable Storage Management API specification, including details of the Swordfish Class of Service concepts, structure and usage. It will also cover details of constructing file vs block systems. The deep-dive will also provide a look at the schema, and the ability to support both RESTful and OData clients.

The SNIA’s Scalable Storage Management Technical Work Group (SSM TWG) has developed an open industry standard specification that provides a unified approach for the management of storage systems and data services. This specification extends the DMTF’s Redfish specification using RESTful methods and JSON formatting. This session will present an overview of the specification. This session will also describe the positioning of the specification developed by the SSM TWG vs SMI-S as well as the base Redfish specification.

Many software infrastructures used in modern data centers and storage appliances rely on key value abstraction to access underlying storage and memory systems. However, the existing devices only provide a block interface via storage protocols. As a result, an S/W translation layer such as memcached, LevelDB, RocksDB, extent layer, etc is used to bridge this gap. However, the high overhead of such software layers impact on performance, reliability, resource utilization, and scalability of infrastructure. In this special session of key value SSD, we introduce a comprehensive introduction to key value SSD, covering the concept, device, software ecosystem, system proof point, performance, and standard activity.

In this tutorial, we will introduce the audience to the lunatic fringe of extreme high-performance computing and its storage systems. The most difficult challenge in HPC storage is caused by millions (soon to be billions) of simultaneously writing threads. Although cloud providers handle workloads of comparable, or larger, aggregate scale, the HPC challenge is unique because the concurrent writers are modifying shared data. We will begin with a brief history of HPC computing covering the previous few decades, bringing us into the petaflop era which started in 2009. Then we will discuss the unique computational science in HPC so that the audience can understand the unavoidability of its unique storage challenges. We will then move into a discussion of archival storage and the hardware and software technologies needed to store today’s exabytes of data forever. From archive we will move into the parallel file systems of today and will end the lecture portion of the tutorial with a discussion of anticipated HPC

Caches in modern storage systems lack the ability to adapt automatically and optimize for dynamic workload mixes. Despite the potential for huge improvements in cost, performance, and predictability, such adaptability is extremely challenging, due to inherently complex, non-linear, and workload-dependent behavior. Even when manually-tunable controls are provided to support dynamic cache sizing, partitioning, and parameter tuning, administrators simply don’t have the information required to make good decisions. In this talk, we will present an overview of the significant opportunity for self-optimizing caches by examining several examples from production systems. We will review recently-published research in this area, including robust, general methods for efficient cache modeling. Optimizations that leverage these models promise to improve the performance of most workloads and cache policies automatically.

The Storage Performance Development Kit (SPDK) is an open source set of tools and libraries for writing high performance, scalable, user-mode storage applications. It achieves high performance by moving all of the necessary drivers into userspace and operating in a polled mode instead of relying on interrupts. The Blobstore is a relative newcomer to SPDK and provides local, persistent, power-fail safe block allocator designed to replace filesystem usage in many popular databases. Most importantly, the Blobstore has been designed for the properties of flash and next-generation media from the start and directly leverages NVMe features. The team has already ported a popular embedded key/value database, RocksDB, to use the Blobstore which demonstrated a significant improvement for database queries under common workloads. In this session we will explore the basics of the Blobstore and review some of the latest exciting performance data!

The Windows Server 2016 release contains support for Persistent Memory including the introduction of DAX volumes. This presentation will discuss the PM improvemenetns in Windows since this release. We will also review Windows support of the NVML library.

Open Channel describes a new interface to Solid State Drives (SSDs) which promises to increase usage of SSDs’ raw bandwidth from 40% to 95%, increase user-visible flash capacity from 50%-70% to 99%, increase I/O bandwidth by 3x and reduce per-GB hardware cost by 50%. Despite many proposals and implementations proving these benefits, industry has seen limited adoption and no standards body has integrated the concept. One of the largest hurdles to adoption is that the proposed changes permeate every layer in the storage stack, from device firmware to application. To reap the benefits, we need not only an end goal, but a pragmatic approach to introducing these changes to one or two layers at a time. We present relevant information about host and drive architecture, the expected use cases for Open Channel, and a general-purpose, maintainable end target for Open Channel. The final architecture refactors Flash Translation Layer into Log-Management, handled in the host, and NAND Management, handled in the drive. Thi

PCIe devices such as GPGPUs, FPGA accerlators, RDMA enabled NICs and NVM Express SSDs are placing a huge strain on the IO subsystem of the CPUs they are connected too. With the emergence of NVMe over Fabrics and heterogeneous compute there is often a desire to move large stream of data between these endpoints without CPU intervention. In this presentation we give an overview of the latest work we have done to enable this PCIe Peer-2-Peer (P2P) communication in the Linux kernel. We also cover a detailed performance comparison between normal data flows and those that avail of P2P. We show how the P2P data-flows offload the IO subsystem of the CPU and lead to better throughput, latency and Quality of Service.