There's plenty of sage advice out there about how to run Rust services in production at scale — config cascades, structured logging, masking secrets, multi-backend secrets management, Prometheus, OpenTelemetry, Kafka transports, tiered disk-spillover sinks, adaptive worker pools, graceful shutdown — but almost none of it as code you can just install and use.
This is that code.
Opinionated, drop-in, working out of the box. The patterns from blog posts, watercooler chats and beers with your Google mates as actual library — not a framework you assemble from twenty crates and 8 weeks of munging.
Built as the foundation for HyperI's PB/hr data services. Generic enough that you don't need to be at HyperI to use it.
This module exists because of this — https://www.youtube.com/watch?v=xE9W9Ghe4Jk — but for the backend. And of course, no microservices.
[dependencies]
hyperi-rustlib = "2"Default features: config, logger. Add the others you want explicitly.
use hyperi_rustlib::{config, logger, env};fnmain() -> anyhow::Result<()>{let environment = env::Environment::detect();
logger::setup_default()?;
config::setup(config::ConfigOptions{env_prefix:"MYAPP".into(),
..Default::default()})?;
tracing::info!("Running in {environment:?}");Ok(())}Pick the slice you need; pay only for what you use.
| Feature | Description |
|---|---|
env | Environment detection (K8s, Docker, Container, BareMetal) |
runtime | Runtime path resolution (XDG/container-aware) |
config | 8-layer config cascade (figment-based) |
config-reload | SharedConfig<T> + ConfigReloader hot-reload |
config-postgres | PostgreSQL config source |
logger | Structured logging, JSON/text auto-detect, sensitive-field masking |
metrics | Prometheus metrics + process/container metrics |
otel-metrics | OpenTelemetry metrics export (OTLP) |
otel-tracing | OpenTelemetry distributed tracing |
http | HTTP client with retry middleware (reqwest) |
http-server | Axum HTTP server with health probe trinity (/healthz/{startup,live,ready}) |
transport-kafka | Kafka transport (rdkafka, dynamic-linking) |
transport-grpc | gRPC transport (tonic/prost) |
transport-memory | In-memory transport (testing/dev) |
transport-grpc-vector-compat | Vector wire-protocol compatibility |
spool | Disk-backed async FIFO queue (yaque + zstd) |
tiered-sink | Resilient delivery: hot buffer + circuit breaker + disk spillover |
secrets | Secrets management core (file backend) |
secrets-vault | OpenBao / HashiCorp Vault provider |
secrets-aws | AWS Secrets Manager provider |
directory-config | YAML directory-backed config store |
directory-config-git | Git integration for directory-config (git2) |
scaling | Back-pressure / scaling-pressure primitives |
cli | Standard CLI framework (clap) |
top | TUI metrics dashboard (ratatui) |
io | File rotation, NDJSON writer |
dlq | Dead-letter queue (file backend) |
dlq-kafka | DLQ Kafka backend |
output-file | File output sink |
expression | CEL expression evaluation |
deployment | Deployment-contract validation |
version-check | Optional startup version check |
resilience | Circuit breaker, retry, bulkhead (tower-resilience) |
full | Everything |
This crate dynamically links against system C libraries for several features. Both build hosts and deployment targets need the appropriate packages.
| Feature | Crate | Build Package | Notes |
|---|---|---|---|
transport-kafka | rdkafka-sys | librdkafka-dev (>= 2.12.1) | Requires Confluent APT repo — Ubuntu's default is too old |
directory-config-git | libgit2-sys | libgit2-dev, libssh2-1-dev | System lib avoids vendored C build |
spool, tiered-sink | zstd-sys | libzstd-dev | System lib avoids vendored C build |
| (transitive) | libz-sys | zlib1g-dev | Used by multiple deps |
| (transitive) | openssl-sys | libssl-dev | Dynamic linking via pkg-config |
secrets-aws | aws-lc-sys | — | C/C++ compiled from source (no system lib available); ~20–30s first build, cached by sccache |
For librdkafka-dev >= 2.12.1, add the Confluent APT repo:
curl -fsSL https://packages.confluent.io/clients/deb/archive.key \
| sudo gpg --dearmor -o /usr/share/keyrings/confluent-clients.gpg
echo"deb [signed-by=/usr/share/keyrings/confluent-clients.gpg] \ https://packages.confluent.io/clients/deb noble main" \
| sudo tee /etc/apt/sources.list.d/confluent-clients.list
sudo apt-get update
sudo apt-get install -y librdkafka-dev libssl-dev libsasl2-dev pkg-configThe compiled binary links against .so files at runtime. Install the
runtime packages (not -dev) on deployment hosts or in Docker images.
| Feature | Runtime Package | Shared Object |
|---|---|---|
transport-kafka | librdkafka1 (from Confluent repo) | librdkafka.so.1 |
directory-config-git | libgit2-1.7 (or matching version) | libgit2.so |
spool, tiered-sink | libzstd1 | libzstd.so.1 |
| (transitive) | zlib1g | libz.so.1 |
| (transitive) | libssl3 | libssl.so.3 |
Only install what you use. Check the features your binary enables to determine which runtime packages are needed.
# Build stageFROM rust:1 AS builder
RUN apt-get update && apt-get install -y \
pkg-config libssl-dev librdkafka-dev libgit2-dev libzstd-dev
COPY . .
RUN cargo build --release
# Runtime stageFROM ubuntu:24.04
RUN apt-get update && apt-get install -y --no-install-recommends \
librdkafka1 libssl3 libgit2-1.7 libzstd1 ca-certificates \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /app/target/release/myapp /usr/local/bin/For librdkafka1, add the Confluent APT repo to both build and runtime stages.
For services deployed to Kubernetes, the http-server feature provides
the three K8s probe types:
| Probe | Path | Checks | On failure |
|---|---|---|---|
| Startup | /healthz/startup | Init complete | K8s waits, then restarts |
| Liveness | /healthz/live | Process not deadlocked | Restart pod |
| Readiness | /healthz/ready | Deps healthy + ready flag set | Stop routing traffic |
Liveness MUST NEVER check downstream dependencies (a DB outage shouldn't
restart your replicas). Readiness checks dependencies AND requires an
explicit set_ready() call — cleared during graceful shutdown.
A rustlib data-plane app regulates its own intake. Sized for steady state, a
pod slows down or speeds up WITHIN itself first -- the default, fast, local
response to a burst, a stalled upstream, or a transform that balloons memory.
Only when that vertical headroom is exhausted does it escalate to horizontal
scale (KEDA adding pods), driven by the same pressure signal. Memory is the
hard, never-OOM authority; CPU is left to the kernel scheduler (CFS), which
the byte-budget loop reads through longer process times. It is ON by default
and opt-out via self_regulation.enabled = false. See
docs/SELF-REGULATION.md.
See docs/ for the full documentation index — docs/ARCHITECTURE.md for the module map and layering, and docs/core-pillars/CONFIG.md for the 8-layer config cascade reference.
BUSL-1.1 — Business Source License 1.1, transitions to Apache 2.0 after 3 years.
- hyperi-pylib — sister library for Python services. Same opinions, same patterns, expressive Python ergonomics for control planes, APIs, and integration layers.