Quick Start · NBD usage · Capabilities · Architecture · Storage Backends · Spinifex Integration · Development · Docs

---

Viperblock: WAL-backed block storage for QEMU and KVM

Viperblock is a block-storage engine for virtual machines running on QEMU and KVM. It provides durable writes, snapshots, caching and pluggable storage backends through an NBD interface.

Viperblock is the storage engine used by Spinifex for EBS-compatible volumes.

Quick Start

Dependencies

sudo apt install nbdkit nbdkit-plugin-dev

Build

git clone https://github.com/mulgadc/viperblock.git
cd viperblock
make build

The build produces:

• bin/sfs — Simple File System demonstration
• bin/vblock — volume-management CLI
• lib/nbdkit-viperblock-plugin.so — nbdkit plugin

Run an NBD volume

Viperblock volumes are served to QEMU/KVM through nbdkit. Spinifex manages this automatically; for standalone use:

nbdkit --filter=blocksize ./lib/nbdkit-viperblock-plugin.so \
  volume=my-volume \
  size=$((10*1024*1024*1024)) \
  base_dir=/data/viperblock \
  cache_size=20

Plugin parameters:

Parameter	Description
size	Volume size in bytes
volume	Volume name
base_dir	Local storage directory (file backend)
bucket	S3 bucket (S3 backend)
host	S3 endpoint URL (S3 backend)
region	AWS region (S3 backend)
access_key / secret_key	S3 credentials (S3 backend)
cache_size	LRU cache as percentage of system memory
shardwal	Enable sharded WAL (true/false)
gc_enabled	Enable chunk garbage collection (true/false, default false)

Capabilities

• Durable write-ahead logging
• Sixteen sharded WAL writers
• Extent-based allocation and constant-time lookup
• Copy-on-write snapshots and clones
• File, memory and S3-compatible storage backends
• NBD access through nbdkit
• LRU read caching
• Arena-based memory allocation
• CRC32 integrity checks

Architecture

See DESIGN.md for detailed write path, read path, WAL format, chunk format, and block mapping internals.

Storage Backends

Backend	Intended use
Memory	Tests and temporary volumes
Filesystem	Local development and standalone storage
S3-compatible	Distributed or remote persistent storage

SFS Demonstration

# File backend
./bin/sfs -btype file -dir /path/to/data -vol my-volume -voldata /tmp/vb

# S3-compatible backend
AWS_HOST="https://localhost:8443/" \
AWS_BUCKET="viperblock" \
AWS_ACCESS_KEY="EXAMPLEKEY" \
AWS_SECRET_KEY="EXAMPLEKEY" \
./bin/sfs -btype s3 -dir /path/to/data -vol my-volume -voldata /tmp/vb

SFS Options

Flag	Description	Default
-btype	Backend type: file, memory, s3	file
-vol	Volume name
-size	Volume size in bytes	524288
-dir	Directory to read into volume
-voldata	Local directory for volume data
-createvol	Initialize a new volume
-vbstate	Viperblock state file path
-sfsstate	SFS state file path

Design Decisions

A summary of the key design choices. See DESIGN.md for the full treatment.

WAL on fast local storage, chunks on S3. Writes are acknowledged from memory and durably flushed to a local WAL (NVMe recommended). WAL entries are then consolidated into 4 MB chunks and uploaded to the backend. This separates write latency (local NVMe speed) from storage durability (S3 replication).

Extent-based block mapping. Rather than one index entry per 4 KB block, consecutive blocks are merged into extents (inspired by ext4). A 10-block sequential write produces one extent entry instead of ten, reducing memory usage and speeding up lookups.

16-way sharded locking. Both the UnifiedBlockStore and the sharded WAL use 16 shards keyed by blockNum & 0xF. Concurrent writes to different blocks never contend on the same lock.

Copy-on-write snapshots. CreateSnapshot freezes the block-to-object mapping without copying any data. Clones created from a snapshot read unmodified blocks from the source volume's chunks and only allocate new storage for blocks that are overwritten. This makes snapshot creation instant and clone creation near-instant regardless of volume size.

CRC32 checksums everywhere. Every WAL record includes a CRC32 checksum validated during replay and consolidation. Corrupt records are detected before they reach chunk storage.

Spinifex Integration

Within Spinifex, Viperblock handles EBS-compatible volume storage and snapshots. It subscribes to volume-lifecycle NATS subjects, starts nbdkit for an attached volume, and returns an NBD URI for QEMU.

Component	Role
Spinifex	VM orchestration (EC2-compatible)
Viperblock	Block storage (EBS-compatible)
Predastore	Object storage (S3-compatible)
Northstar	Authoritative DNS (Route53-compatible)

Viperblock can also be used standalone for any application that needs durable block storage with S3 as a backend tier.

Development

make test
make preflight

Research

The following papers informed the design of Viperblock:

• Hajkazemi, M. H. et al. "Beating the I/O bottleneck: a case for log-structured virtual disks." EuroSys 2022. https://doi.org/10.1145/3492321.3524271
• Zhou, D. et al. "Enabling high-performance and secure userspace NVM file systems with the trio architecture." SOSP 2023. https://doi.org/10.1145/3600006.3613171
• Li, H. et al. "Ursa: Hybrid block storage for cloud-scale virtual disks." EuroSys 2019. https://doi.org/10.1145/3302424.3303967

Trademarks

Amazon Web Services, AWS and Amazon EBS are trademarks of Amazon.com, Inc. or its affiliates. Viperblock is not affiliated with or endorsed by Amazon Web Services.

License

Viperblock is licensed under the GNU Affero General Public License v3.0 (AGPLv3) license.