Accelerating operations on persistent memory device via hardware based memory offloading technique

With more and more fast devices (especially persistent memory, aka. PMEM) equipped in data center, there is great pressure on CPU to drive those devices (e.g., Intel Optane DC persistent memory) for persistency purpose under heavy workloads. Because there is no DMA related capability provided by persistent memory compared with those hdds and SSD, thus CPU needs to participate the data operations on the persistent memory all the time. In this talk, we would like to mitigate the CPU pressure via hardware based memory offloading devices (e.g., Intel's IOAT and DSA). To demonstrate our work, we introduce the memory offloading device solution in two kinds of workloads on using PMEM, i.e., (1) storage applications based on SPDK framework; (2) leveraging DSA in Ceph's bluestore with PMEM device. In SPDK side, we introduce how to design and implement a new block device based on persistent memory (e.g., Intel Optane DC persistent memory) and SPDK acceleration framework (which leverages IOAT or DSA). In Ceph side, we describe the main change in bluestore's pmem module (i.e., src/blk/pmemdevice.cc) and state how we can achieve the offloading via DSA including the challenges. With our approach, we do see the performance improvement with the workloads on the PMEM device, .e.g., IOPS increasing and latency decreasing with storage application based on SPDK framework. And we will have some early performance results if Intel's SPR platform is available in public.