Skip to content

🐛 Bug Report — Runtime APIs: node:zlib zstd ignores the dictionary option (silent on compress, "Data corruption detected" on decompress) #6967

Description

@oddharsh

node:zlib's zstd functions accept a dictionary option and drop it. On compression
that's silent, and on decompression it surfaces as a corruption error against bytes
that aren't corrupt.

Node grew this option in nodejs/node@201304537e
and workerd's zstd bindings (#6007) predate it, so I assume this is a gap rather than a
regression.

Versions

workerd 1.20260801.1, via wrangler 4.120.0 / miniflare 5.20260801.1-alpha.
compatibility_date = "2026-08-01", compatibility_flags = ["nodejs_compat"].
Compared against node v26.7.0 running the identical program.

Repro

import{zstdCompressSync,zstdDecompressSync,constants}from"node:zlib";// A frame produced by node with a dictionary, base64'd in so the worker can try to read it.constFRAME="<node's dictionary-compressed output, see below>";exportdefault{fetch(){constprobe=Buffer.from("the quick brown fox jumps over the lazy dog ".repeat(200));constwrong=Buffer.from("completely unrelated filler bytes ".repeat(200));constlvl={[constants.ZSTD_c_compressionLevel]: 19};constout={};out.compress={none: zstdCompressSync(probe,{params: lvl}).length,good: zstdCompressSync(probe,{dictionary: probe,params: lvl}).length,wrong: zstdCompressSync(probe,{dictionary: wrong,params: lvl}).length,};try{constback=zstdDecompressSync(Buffer.from(FRAME,"base64"),{dictionary: probe});out.decompress={ok: true,bytes: back.length,matches: back.equals(probe)};}catch(e){out.decompress={ok: false,error: String(e?.message??e)};}returnResponse.json(out);},};

Generate FRAME under node:

const{ zstdCompressSync, constants }=require("node:zlib");constprobe=Buffer.from("the quick brown fox jumps over the lazy dog ".repeat(200));zstdCompressSync(probe,{dictionary: probe,params: {[constants.ZSTD_c_compressionLevel]: 19}}).toString("base64");// 19 bytes

Results

node v26.7.0workerd 1.20260801.1
compress, no dictionary6363
compress, correct dictionary1963
compress, wrong dictionary6363
decompress a dictionary-compressed frameok, 8800 bytes, byte-exactthrows

workerd's decompress error:

Zstd decompression failed: Data corruption detected

Why this isn't just a missing-feature note

On compress, nothing tells you. A frame compressed without a dictionary decodes
perfectly with one, so no consumer errors and no log line looks unusual. The only
signal is a byte count that never shrank. The only reliable detection I've found is a
feature-detect that compresses a buffer against itself and asserts the output collapses.

On decompress, the error blames the wrong thing. "Data corruption detected" points
at the payload. The payload is fine. Anyone debugging that will go looking at their
storage or their transport before they suspect that the option they passed was dropped.

The practical consequence is that a Worker can't participate in
RFC 9842 Compression Dictionary Transport
in either direction: it can't produce a dcz delta, and it can't read one. That's a
shame specifically for Workers, since computing a delta against a dictionary the client
already holds is an edge-shaped job.

Suggested fix

  1. Wire ZSTD_CCtx_loadDictionary / ZSTD_DCtx_loadDictionary (and the _refPrefix
    variants, if you want raw-content dictionaries, which is what RFC 9842 uses) into the
    bindings from Add ZSTD bindings to the node:zlib package. #6007, matching node's implementation linked above.
  2. If that's not near-term, please throw on a dictionary option the binding can't
    honour, rather than accepting it. ERR_INVALID_ARG_VALUE at the call site is far
    cheaper to debug than either symptom above.

I'm more than happy to take a stab via a PR for (2) if that's a useful stopgap.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions