DDS: Linux/POSIX storage engine for DPUs

DDS is a research storage engine that exposes a Linux/POSIX-style file API on the host and executes storage operations through a DPU backend. The host and DPU communicate over RDMA; the DPU accesses the storage target through SPDK NVMe-oF.

This repository contains the storage-engine artifact accompanying the DDS paper. Network-engine, general offload, DPKernels, decrypt/decompress, and unrelated application code are intentionally excluded.

Components

Path	Purpose
StorageEngine/DDSPosixInterface/	Linux/POSIX-style host API and benchmark applications
StorageEngine/DDSFrontEnd/	Asynchronous host frontend and RDMA bridge
StorageEngine/DDSApplicationLinux/	Native asynchronous DDS benchmark application
StorageEngine/DDSBackEndDPUService/	DPU backend, SPDK configuration, and build/run scripts
Common/	Shared protocol and host/DPU support code
config/	Host-side SPDK NVMe-oF target configuration
scripts/	Repository-level build and launch wrappers

DDSPosix is a namespaced C++ API with familiar file operations; it is not a transparent libc interposer. The current implementation targets Linux hosts and NVIDIA BlueField DPUs.

Prerequisites

• Meson and Ninja for the host build.
• RDMA development libraries on the host and DPU.
• SPDK on the host for nvmf_tgt.
• The pinned SPDK submodule and DPU toolchain for the backend build.
• Hugepages configured as described below.

Initialize dependencies after cloning:

git submodule update --init --recursive

Build

Build the host POSIX applications and native asynchronous application in release mode:

./scripts/build-host.sh
./scripts/build-native-app.sh

On the DPU, build the backend in release mode:

./scripts/build-dpu.sh release

The DPU build also retains its original SPDK setup and patching procedure under StorageEngine/DDSBackEndDPUService/ and Scripts/InstallSPDK.sh.

Run

The supplied workflow uses a 3-GiB malloc-backed NVMe-oF namespace and a two-core DPU topology. Start the components in order from three terminals.

On the host, launch the SPDK NVMe-oF target:

NVMF_TGT_BIN=/path/to/spdk/build/bin/nvmf_tgt \
  ./scripts/run-nvmf-target-3g.sh /tmp/dds-run

The wrapper reserves 2,560 node-0 2-MiB hugepages, mounts a run-scoped hugetlbfs, derives the 3-GiB target configuration, and restores the previous hugepage count when the target exits normally or receives SIGINT/SIGTERM.

On the DPU, launch the backend:

./scripts/run-dpu.sh

Finally, run one host application:

# Concurrent Linux/POSIX-style read/write workload
./scripts/run-concurrent-posix.sh

# Or the native asynchronous DDS workload
./scripts/run-native-app.sh

Use a fresh target/backend pair before switching applications because each workload initializes its own file and metadata. Stop the DPU backend before stopping the host target.

Configuration

The default files describe the validated lab topology:

• config/nvmf_tgt.json exports malloc bdev Malloc0 through nqn.2016-06.io.spdk:bf3_malloc at 10.10.2.101:4420.
• StorageEngine/DDSBackEndDPUService/SpdkDev.json connects from the DPU at 10.10.2.201 and exposes the target as nvme0n1.
• The DPU backend uses one SPDK reactor/storage core and one DMA core.

Adjust the addresses, device configuration, and CPU placement for another environment. The checked-in target configuration is malloc-backed; replace its bdev definition to use physical storage. Keep workloads within the capacity of the exported namespace.

License

DDS is released under the MIT License.