The fastest derive-macro framework for encoding and decoding Key-Length-Value
(KLV) binary streams, built on
winnow parser combinators.
KLV (a generic Tag-Length-Value framing) is the backbone of telemetry packets,
video metadata streams, IoT sensor framing, and most custom binary protocols
that evolve without breaking older parsers. tinyklv is protocol-agnostic and
ships no baked-in standards: you declare your keys, length encoding, sentinel,
and per-field codecs as attributes on a struct, and #[derive(Klv)] generates
the encoder and decoder in a single pass.
cargo add tinyklvuse tinyklv::Klv;use tinyklv::prelude::*;use tinyklv::dec::binary as decb;use tinyklv::enc::binary as encb;#[derive(Klv,Debug,PartialEq)]#[klv( stream = &[u8], sentinel = b"\x47\x48", key(dec = decb::u8, enc = encb::u8), len(dec = decb::u8_as_usize, enc = encb::u8_from_usize),)]structHeartbeat{#[klv( key = 0x01, dec = decb::u8, enc = *encb::u8,)]sequence:u8,#[klv( key = 0x02, dec = decb::be_u16, enc = *encb::be_u16,)]temperature_centideg:u16,}fnmain(){let original = Heartbeat{sequence:42,temperature_centideg:2350,};letmut frame = Vec::new();
original.encode_frame(&mut frame);let decoded = Heartbeat::decode_frame(&mut frame.as_slice()).unwrap();assert_eq!(decoded, original);}Full annotated version: examples/01_hello_world.rs.
Across decode and encode, flat and nested, clean and framed-over-noise,
tinyklv is the fastest derive-based KLV framework - several times faster than
serde_klv, an order of magnitude faster than tlv_parser, and within striking
distance of hand-rolled parsing. It is also the only one that handles
nested KLV without hand-written glue.
The suite also pits all four KLV approaches against four protobuf stacks
(prost, quick_protobuf, rust_protobuf, micropb) encoding the same records
from .proto schemas - a different wire format (length-prefixed tag-value rather
than KLV), measured the same way.
A third record is built from common native Rust types (chrono::DateTime,
std::net::Ipv4Addr/Ipv6Addr, std::time::Duration, NaiveDate/NaiveTime,
char, NonZeroU32, bool, String). tinyklv decodes straight into them via
its bench-gated DecodeValue/EncodeValue impls; every other approach must
convert from a raw scalar - the KLV libraries field-by-field, and the protobuf
crates over a whole generated struct (a generated message can never be a native
type), each using whatever native conversions the crate genuinely provides.
The benchmark suite, the eight competing implementations (four KLV libraries and
four protobuf crates), and the one-command chart reproduction
(benches/scripts/gencharts.sh) all live in
benches/.
Benchmark machine:
- OS: Ubuntu 24.04.4 LTS, Linux
6.17.0-35-generic - CPU: AMD Ryzen 9 9900X
- Host: B850 AI TOP -CF-WCP-ADO
See the full chart set: KLV/TLV · protobuf
The same benchmark records also put tinyklv within protobuf territory, and in several measured encode/decode cases ahead of the protobuf stacks:
The only thing faster per packet is the manual bar - and it is the least
maintainable code in the suite. A hand-rolled decoder is a panic-adjacent
slice-indexing loop with one Option per field to juggle, repeated for every
record shape, plus a second hand-written sub-parser for every level of nesting.
From the benchmark's manual nested decoder
(benches/suite/approaches/manual/nested.rs):
fndecode(body:&[u8]) -> Option<Platform>{letmut id = None;letmut coord = None;// ...one `let mut <field> = None;` for every one of nine fields...letmut sensors = None;letmut j = 0;while j + 2 <= body.len(){// bounds checklet tag = body[j];// allow clippy slice indexinglet len = usize::from(body[j + 1]);
j += 2;let val = body.get(j..j + len)?;// hand-rolled bounds math
j += len;match tag {// manually define keys for each struct
key::ID => id = Some(u32::from_be_bytes(val.try_into().ok()?)),
key::COORD => coord = Some(decode_coord(val)?),// a SECOND hand-written loop// ...one arm per field, each with its own try_into().ok()? dance...
_ => {}}}Some(Platform{id: id?,coord: coord?,/* ...unwrap all nine... */sensors: sensors? })}tinyklv collapses the whole thing - both directions, nesting included - into
attributes on the struct:
#[derive(Klv)]#[klv( stream = &[u8], sentinel = b"\x47\x48", key(dec = decb::u8, enc = encb::u8), len(dec = decb::u8_as_usize, enc = encb::u8_from_usize), trait_fallback,)]structPlatform{#[klv(key = 0x01, dec = decb::be_u32, enc = *encb::be_u32)]id:u32,#[klv(key = 0x02)]coord:GpsCoord,// where this struct has `#[derive(Klv)]`// ...the remaining fields, one attribute line each...}- Built-in codecs: binary (native/BE/LE for
u8..u128,i8..i128,f32/f64), BER length, BER-OID keys, UTF-8 / UTF-16 / ASCII strings - Sentinel seeking - resync on noisy byte streams
- Streaming partial packets on noisy or incomplete streams
- Repeated decode with user-defined break conditions
- Nested
Klvstructs - compose packets from sub-packets - Generic structs and lifetimes,
Option<T>fields, per-field/container defaults - Stream type is user-selected, where any
winnow::Streamworks
Issues and pull requests welcome at
https://github.com/arpadav/tinyklv. Run cargo test --all and
cargo clippy --all-targets -- -D warnings before opening a PR.
Licensed under the MIT License. See LICENSE for details.
If tinyklv is useful to you:
- Buy Me a Coffee
- Bitcoin:
bc1q5stdywthj254agv80s5gky6440xy73cpqgv0q7