feat: add sampled Redis shadow validation

[lan17]

Summary

Adds opt-in semantic shadow validation for sensitive, invalidation-tracked Redis cache hits. A selected hit returns its cached value normally, then detached best-effort work re-reads the source of truth (SoT), independently deserializes the retained Redis payload, and compares the two application values.

Closes#104.

Architecture

flowchart LR
A[Tracked Redis hit] --> B[Return cached value]
A --> C{shadowRamp selects key?}
C -- no --> X[No shadow work]
C -- yes --> D{Metric hook and capacity?}
D -- no --> X
D -- yes --> E[Reserve exact-key flight]
E --> F[Unref setImmediate]
F --> G[Read SoT with DialCache disabled]
G --> H[serializer.load retained payload]
H --> I[Default or custom value comparator]
I --> J[Bounded outcome metric]

Loading

The hit path performs eligibility checks and slot reservation only. It does no SoT read, extra deserialization, deep comparison, payload-size-linear copy, repair, or cache mutation. All comparison work begins from an unreferenced immediate and is not awaited by the caller.

Public API and configuration

• Adds DialCacheKeyConfig.shadowRamp?: number.
  • Omitted or 0 disables validation.
  • 100 selects every otherwise eligible exact key.
  • Partial ramps use a stable exact-key cohort with an independent :shadow discriminator.
  • Sparse runtime overlays inherit the static baseline; malformed values skip shadow work and emit the existing remote config_resolution error without changing the hit.
• Adds DialCacheConfig.shadowMaxInFlight?: number, a positive safe integer defaulting to 1 per instance.
  • There is no queue.
  • Exact-key duplicates and global-cap overflow emit dropped.
  • Scheduled and timed-out-but-still-running work retains its slot until the underlying promise settles.
• Adds the root-exported synchronous type ShadowComparator<T> and optional shadowComparator on cached() / getOrLoad() options.
• Extends DialCacheMetricsAdapter with an optional shadowValidation hook, preserving existing custom adapters.

exporttypeShadowComparator<T>=(cachedValue: T,sourceValue: T,)=>boolean;

Comparison and data boundaries

• Eligibility requires an actual successful Redis hit with trackForInvalidation: true; request-local/process-local hits, misses, read errors, and initial deserialize failures do not validate.
• Redis reads retain the client-returned semantic string | Buffer payload internally alongside the decoded value.
• Detached work reads the raw SoT value, then runs the same effective serializer's load() again on the retained payload to create an independent cached snapshot.
• The object already returned to the caller is never compared, so caller mutation after the hit cannot contaminate validation.
• The comparator receives two values of T, never a Redis payload.
• The default is Node's util.isDeepStrictEqual; an optional per-operation comparator defines use-case-specific equality.
• SoT is intentionally not dump/load normalized. Lossy serialization remains visible unless a custom comparator explicitly treats the normalized values as equivalent.
• A comparator must synchronously return a boolean and must be deterministic, side-effect-free, non-mutating, and bounded. Throws and non-boolean returns emit comparison_error.
• An accidental Promise from untyped JavaScript is consumed safely but never accepted as a comparison result; it retains the flight slot until settlement.
• Redis framing, timestamps, TTLs, watermarks, Lua, keys, and bundled client protocol remain unchanged.

Detachment, liveness, and safety

• Source execution starts from an unreferenced setImmediate and runs under dialcache.disable(...) so it cannot recursively satisfy itself from the same cache.
• One monotonic, unreferenced deadline covers the source read, detached serializer load, and comparison.
• A finite fallbackTimeoutMs is reused; fallbackTimeoutMs: null keeps normal fallbacks unbounded but gives shadow work the internal 60-second default.
• On timeout, DialCache releases its retained payload reference and suppresses later phases. Already-running application/serializer/comparator work cannot be cancelled and continues to occupy its slot until settlement.
• Scheduler and deadline handles do not keep an otherwise idle process alive. Completion during shutdown remains best effort.
• Detached execution retains original cached() argument references or a getOrLoad() closure; docs require relevant inputs/captures to be immutable or snapshotted.
• Validation is observational only: no repair, write, TTL refresh, invalidation, local eviction, payload logging, or change to the returned value.

Observability

Bounded outcomes: match, mismatch, source_error, deserialization_error, comparison_error, timeout, and dropped.

Labels/tags are limited to cache namespace, use case, key type, and outcome. IDs, cached/SoT values, payloads, Redis keys, and raw exception messages are excluded.

Built-in adapters add:

• Prometheus: dialcache_shadow_validation_counter
• Datadog: dialcache.shadow.count

An adapter without the optional hook disables shadow execution so DialCache never performs an unobservable SoT read. Runtime thenable rejections from the hook are consumed without being awaited.

Compatibility and operations

• The feature is default-off and all new configuration/operation fields and the metrics hook are optional.
• Existing cache behavior, Redis wire format, Lua scripts, keys, watermarks, serializers, and package export paths remain unchanged.
• This is suitable for a minor release and rolling upgrade; enabling it is an operational rollout because it adds sampled SoT reads plus detached serializer/comparison CPU.
• shadowMaxInFlight bounds one instance, not the fleet. Roll out shadowRamp gradually and monitor match, mismatch, timeout, source_error, comparison_error, and dropped.
• The Prometheus adapter registers one additive collector family; applications that already own the exact same prefixed name with an incompatible schema must resolve that registry collision.
• README documents serializer repeatability, custom Redis payload ownership, detached input lifetime, concurrency budgeting, shutdown behavior, and privacy boundaries.

Change footprint

The PR is test-led rather than carrying duplicate first-pass production code:

• 22 files, +1,992 / -38 against current main
• Tests and package-consumer proofs: +1,364 / -2 across 7 files (68% of additions)
• Production source: +453 / -25 across 13 files (23% of additions)
• Benchmark tooling: +92
• README: +83 / -11

The dedicated shadow suite contains 36 scenarios covering the detached lifecycle and failure matrix. Shared setup is factored through test-only helpers; no obsolete serialized-payload comparison implementation remains.

Validation

Run on Node.js 22.22.0 at 4b8f7e4, including current main@500d5e7:

• corepack pnpm check
  • 341 unit tests passed
  • coverage: 97.04% statements, 93.65% branches, 98.47% functions, 97.16% lines
  • typecheck, build, and packed-consumer test passed
• corepack pnpm test:integration
  • 81 tests passed across Redis 6.2 and Valkey 8 with node-redis and GLIDE
  • includes real tracked-value semantic match/mismatch coverage with no repair
• DIALCACHE_BENCH_ITERATIONS=20000 DIALCACHE_BENCH_FANOUT=10000 corepack pnpm benchmark:request-local
  • all eight semantic scenarios passed
  • includes tracked Redis hits with shadow omitted and deterministically ramped out
• corepack pnpm audit --prod: no known production-dependency vulnerabilities
• GitHub Actions: CI, CodeQL, and PR-title validation passed on 4b8f7e4
• git diff --check

Focused shadow tests cover detachment, semantic default/custom equality, caller-mutation isolation, serializer ownership/repeatability, runtime ramp overlays, comparator type/runtime failures, monotonic deadlines, timeout slot retention, exact-key/global drops, coalescing, disabled contexts, metric failure isolation, and unreferenced handles.

Review history

• A broad production review covered concurrency/lifecycle, API compatibility, rollout/configuration, observability, tests, and operations.
• The cleanup round fixed low-severity coverage gaps for packed public APIs, runtime ramp enable/disable overlays, synchronous comparator deadline crossing, custom Buffer ownership, and honest benchmark reporting.
• Two fresh post-fix reviewers independently confirmed the feature and its integration with current main with no remaining findings.

[lan17]

Summary

Adds opt-in semantic shadow validation for sensitive, invalidation-tracked Redis cache hits. A selected hit returns its cached value normally, then detached best-effort work re-reads the source of truth (SoT), independently deserializes the retained Redis payload, and compares the two application values.

Closes#104.

Architecture

flowchart LR
A[Tracked Redis hit] --> B[Return cached value]
A --> C{shadowRamp selects key?}
C -- no --> X[No shadow work]
C -- yes --> D{Metric hook and capacity?}
D -- no --> X
D -- yes --> E[Reserve exact-key flight]
E --> F[Unref setImmediate]
F --> G[Read SoT with DialCache disabled]
G --> H[serializer.load retained payload]
H --> I[Default or custom value comparator]
I --> J[Bounded outcome metric]

Loading

The hit path performs eligibility checks and slot reservation only. It does no SoT read, extra deserialization, deep comparison, payload-size-linear copy, repair, or cache mutation. All comparison work begins from an unreferenced immediate and is not awaited by the caller.

Public API and configuration

• Adds DialCacheKeyConfig.shadowRamp?: number.
  • Omitted or 0 disables validation.
  • 100 selects every otherwise eligible exact key.
  • Partial ramps use a stable exact-key cohort with an independent :shadow discriminator.
  • Sparse runtime overlays inherit the static baseline; malformed values skip shadow work and emit the existing remote config_resolution error without changing the hit.
• Adds DialCacheConfig.shadowMaxInFlight?: number, a positive safe integer defaulting to 1 per instance.
  • There is no queue.
  • Exact-key duplicates and global-cap overflow emit dropped.
  • Scheduled and timed-out-but-still-running work retains its slot until the underlying promise settles.
• Adds the root-exported synchronous type ShadowComparator<T> and optional shadowComparator on cached() / getOrLoad() options.
• Extends DialCacheMetricsAdapter with an optional shadowValidation hook, preserving existing custom adapters.

exporttypeShadowComparator<T>=(cachedValue: T,sourceValue: T,)=>boolean;

Comparison and data boundaries

• Eligibility requires an actual successful Redis hit with trackForInvalidation: true; request-local/process-local hits, misses, read errors, and initial deserialize failures do not validate.
• Redis reads retain the client-returned semantic string | Buffer payload internally alongside the decoded value.
• Detached work reads the raw SoT value, then runs the same effective serializer's load() again on the retained payload to create an independent cached snapshot.
• The object already returned to the caller is never compared, so caller mutation after the hit cannot contaminate validation.
• The comparator receives two values of T, never a Redis payload.
• The default is Node's util.isDeepStrictEqual; an optional per-operation comparator defines use-case-specific equality.
• SoT is intentionally not dump/load normalized. Lossy serialization remains visible unless a custom comparator explicitly treats the normalized values as equivalent.
• A comparator must synchronously return a boolean and must be deterministic, side-effect-free, non-mutating, and bounded. Throws and non-boolean returns emit comparison_error.
• An accidental Promise from untyped JavaScript is consumed safely but never accepted as a comparison result; it retains the flight slot until settlement.
• Redis framing, timestamps, TTLs, watermarks, Lua, keys, and bundled client protocol remain unchanged.

Detachment, liveness, and safety

• Source execution starts from an unreferenced setImmediate and runs under dialcache.disable(...) so it cannot recursively satisfy itself from the same cache.
• One monotonic, unreferenced deadline covers the source read, detached serializer load, and comparison.
• A finite fallbackTimeoutMs is reused; fallbackTimeoutMs: null keeps normal fallbacks unbounded but gives shadow work the internal 60-second default.
• On timeout, DialCache releases its retained payload reference and suppresses later phases. Already-running application/serializer/comparator work cannot be cancelled and continues to occupy its slot until settlement.
• Scheduler and deadline handles do not keep an otherwise idle process alive. Completion during shutdown remains best effort.
• Detached execution retains original cached() argument references or a getOrLoad() closure; docs require relevant inputs/captures to be immutable or snapshotted.
• Validation is observational only: no repair, write, TTL refresh, invalidation, local eviction, payload logging, or change to the returned value.

Observability

Bounded outcomes: match, mismatch, source_error, deserialization_error, comparison_error, timeout, and dropped.

Labels/tags are limited to cache namespace, use case, key type, and outcome. IDs, cached/SoT values, payloads, Redis keys, and raw exception messages are excluded.

Built-in adapters add:

• Prometheus: dialcache_shadow_validation_counter
• Datadog: dialcache.shadow.count

An adapter without the optional hook disables shadow execution so DialCache never performs an unobservable SoT read. Runtime thenable rejections from the hook are consumed without being awaited.

Compatibility and operations

• The feature is default-off and all new configuration/operation fields and the metrics hook are optional.
• Existing cache behavior, Redis wire format, Lua scripts, keys, watermarks, serializers, and package export paths remain unchanged.
• This is suitable for a minor release and rolling upgrade; enabling it is an operational rollout because it adds sampled SoT reads plus detached serializer/comparison CPU.
• shadowMaxInFlight bounds one instance, not the fleet. Roll out shadowRamp gradually and monitor match, mismatch, timeout, source_error, comparison_error, and dropped.
• The Prometheus adapter registers one additive collector family; applications that already own the exact same prefixed name with an incompatible schema must resolve that registry collision.
• README documents serializer repeatability, custom Redis payload ownership, detached input lifetime, concurrency budgeting, shutdown behavior, and privacy boundaries.

Change footprint

The PR is test-led rather than carrying duplicate first-pass production code:

• 22 files, +1,992 / -38 against current main
• Tests and package-consumer proofs: +1,364 / -2 across 7 files (68% of additions)
• Production source: +453 / -25 across 13 files (23% of additions)
• Benchmark tooling: +92
• README: +83 / -11

The dedicated shadow suite contains 36 scenarios covering the detached lifecycle and failure matrix. Shared setup is factored through test-only helpers; no obsolete serialized-payload comparison implementation remains.

Validation

Run on Node.js 22.22.0 at 4b8f7e4, including current main@500d5e7:

• corepack pnpm check
  • 341 unit tests passed
  • coverage: 97.04% statements, 93.65% branches, 98.47% functions, 97.16% lines
  • typecheck, build, and packed-consumer test passed
• corepack pnpm test:integration
  • 81 tests passed across Redis 6.2 and Valkey 8 with node-redis and GLIDE
  • includes real tracked-value semantic match/mismatch coverage with no repair
• DIALCACHE_BENCH_ITERATIONS=20000 DIALCACHE_BENCH_FANOUT=10000 corepack pnpm benchmark:request-local
  • all eight semantic scenarios passed
  • includes tracked Redis hits with shadow omitted and deterministically ramped out
• corepack pnpm audit --prod: no known production-dependency vulnerabilities
• GitHub Actions: CI, CodeQL, and PR-title validation passed on 4b8f7e4
• git diff --check

Focused shadow tests cover detachment, semantic default/custom equality, caller-mutation isolation, serializer ownership/repeatability, runtime ramp overlays, comparator type/runtime failures, monotonic deadlines, timeout slot retention, exact-key/global drops, coalescing, disabled contexts, metric failure isolation, and unreferenced handles.

Review history

• A broad production review covered concurrency/lifecycle, API compatibility, rollout/configuration, observability, tests, and operations.
• The cleanup round fixed low-severity coverage gaps for packed public APIs, runtime ramp enable/disable overlays, synchronous comparator deadline crossing, custom Buffer ownership, and honest benchmark reporting.
• Two fresh post-fix reviewers independently confirmed the feature and its integration with current main with no remaining findings.

[lan17]

Summary

Adds opt-in semantic shadow validation for sensitive, invalidation-tracked Redis cache hits. A selected hit returns its cached value normally, then detached best-effort work re-reads the source of truth (SoT), independently deserializes the retained Redis payload, and compares the two application values.

Closes#104.

Architecture

flowchart LR
A[Tracked Redis hit] --> B[Return cached value]
A --> C{shadowRamp selects key?}
C -- no --> X[No shadow work]
C -- yes --> D{Metric hook and capacity?}
D -- no --> X
D -- yes --> E[Reserve exact-key flight]
E --> F[Unref setImmediate]
F --> G[Read SoT with DialCache disabled]
G --> H[serializer.load retained payload]
H --> I[Default or custom value comparator]
I --> J[Bounded outcome metric]

Loading

The hit path performs eligibility checks and slot reservation only. It does no SoT read, extra deserialization, deep comparison, payload-size-linear copy, repair, or cache mutation. All comparison work begins from an unreferenced immediate and is not awaited by the caller.

Public API and configuration

• Adds DialCacheKeyConfig.shadowRamp?: number.
  • Omitted or 0 disables validation.
  • 100 selects every otherwise eligible exact key.
  • Partial ramps use a stable exact-key cohort with an independent :shadow discriminator.
  • Sparse runtime overlays inherit the static baseline; malformed values skip shadow work and emit the existing remote config_resolution error without changing the hit.
• Adds DialCacheConfig.shadowMaxInFlight?: number, a positive safe integer defaulting to 1 per instance.
  • There is no queue.
  • Exact-key duplicates and global-cap overflow emit dropped.
  • Scheduled and timed-out-but-still-running work retains its slot until the underlying promise settles.
• Adds the root-exported synchronous type ShadowComparator<T> and optional shadowComparator on cached() / getOrLoad() options.
• Extends DialCacheMetricsAdapter with an optional shadowValidation hook, preserving existing custom adapters.

exporttypeShadowComparator<T>=(cachedValue: T,sourceValue: T,)=>boolean;

Comparison and data boundaries

• Eligibility requires an actual successful Redis hit with trackForInvalidation: true; request-local/process-local hits, misses, read errors, and initial deserialize failures do not validate.
• Redis reads retain the client-returned semantic string | Buffer payload internally alongside the decoded value.
• Detached work reads the raw SoT value, then runs the same effective serializer's load() again on the retained payload to create an independent cached snapshot.
• The object already returned to the caller is never compared, so caller mutation after the hit cannot contaminate validation.
• The comparator receives two values of T, never a Redis payload.
• The default is Node's util.isDeepStrictEqual; an optional per-operation comparator defines use-case-specific equality.
• SoT is intentionally not dump/load normalized. Lossy serialization remains visible unless a custom comparator explicitly treats the normalized values as equivalent.
• A comparator must synchronously return a boolean and must be deterministic, side-effect-free, non-mutating, and bounded. Throws and non-boolean returns emit comparison_error.
• An accidental Promise from untyped JavaScript is consumed safely but never accepted as a comparison result; it retains the flight slot until settlement.
• Redis framing, timestamps, TTLs, watermarks, Lua, keys, and bundled client protocol remain unchanged.

Detachment, liveness, and safety

• Source execution starts from an unreferenced setImmediate and runs under dialcache.disable(...) so it cannot recursively satisfy itself from the same cache.
• One monotonic, unreferenced deadline covers the source read, detached serializer load, and comparison.
• A finite fallbackTimeoutMs is reused; fallbackTimeoutMs: null keeps normal fallbacks unbounded but gives shadow work the internal 60-second default.
• On timeout, DialCache releases its retained payload reference and suppresses later phases. Already-running application/serializer/comparator work cannot be cancelled and continues to occupy its slot until settlement.
• Scheduler and deadline handles do not keep an otherwise idle process alive. Completion during shutdown remains best effort.
• Detached execution retains original cached() argument references or a getOrLoad() closure; docs require relevant inputs/captures to be immutable or snapshotted.
• Validation is observational only: no repair, write, TTL refresh, invalidation, local eviction, payload logging, or change to the returned value.

Observability

Bounded outcomes: match, mismatch, source_error, deserialization_error, comparison_error, timeout, and dropped.

Labels/tags are limited to cache namespace, use case, key type, and outcome. IDs, cached/SoT values, payloads, Redis keys, and raw exception messages are excluded.

Built-in adapters add:

• Prometheus: dialcache_shadow_validation_counter
• Datadog: dialcache.shadow.count

An adapter without the optional hook disables shadow execution so DialCache never performs an unobservable SoT read. Runtime thenable rejections from the hook are consumed without being awaited.

Compatibility and operations

• The feature is default-off and all new configuration/operation fields and the metrics hook are optional.
• Existing cache behavior, Redis wire format, Lua scripts, keys, watermarks, serializers, and package export paths remain unchanged.
• This is suitable for a minor release and rolling upgrade; enabling it is an operational rollout because it adds sampled SoT reads plus detached serializer/comparison CPU.
• shadowMaxInFlight bounds one instance, not the fleet. Roll out shadowRamp gradually and monitor match, mismatch, timeout, source_error, comparison_error, and dropped.
• The Prometheus adapter registers one additive collector family; applications that already own the exact same prefixed name with an incompatible schema must resolve that registry collision.
• README documents serializer repeatability, custom Redis payload ownership, detached input lifetime, concurrency budgeting, shutdown behavior, and privacy boundaries.

Change footprint

The PR is test-led rather than carrying duplicate first-pass production code:

• 22 files, +1,992 / -38 against current main
• Tests and package-consumer proofs: +1,364 / -2 across 7 files (68% of additions)
• Production source: +453 / -25 across 13 files (23% of additions)
• Benchmark tooling: +92
• README: +83 / -11

The dedicated shadow suite contains 36 scenarios covering the detached lifecycle and failure matrix. Shared setup is factored through test-only helpers; no obsolete serialized-payload comparison implementation remains.

Validation

Run on Node.js 22.22.0 at 4b8f7e4, including current main@500d5e7:

• corepack pnpm check
  • 341 unit tests passed
  • coverage: 97.04% statements, 93.65% branches, 98.47% functions, 97.16% lines
  • typecheck, build, and packed-consumer test passed
• corepack pnpm test:integration
  • 81 tests passed across Redis 6.2 and Valkey 8 with node-redis and GLIDE
  • includes real tracked-value semantic match/mismatch coverage with no repair
• DIALCACHE_BENCH_ITERATIONS=20000 DIALCACHE_BENCH_FANOUT=10000 corepack pnpm benchmark:request-local
  • all eight semantic scenarios passed
  • includes tracked Redis hits with shadow omitted and deterministically ramped out
• corepack pnpm audit --prod: no known production-dependency vulnerabilities
• GitHub Actions: CI, CodeQL, and PR-title validation passed on 4b8f7e4
• git diff --check

Focused shadow tests cover detachment, semantic default/custom equality, caller-mutation isolation, serializer ownership/repeatability, runtime ramp overlays, comparator type/runtime failures, monotonic deadlines, timeout slot retention, exact-key/global drops, coalescing, disabled contexts, metric failure isolation, and unreferenced handles.

Review history

• A broad production review covered concurrency/lifecycle, API compatibility, rollout/configuration, observability, tests, and operations.
• The cleanup round fixed low-severity coverage gaps for packed public APIs, runtime ramp enable/disable overlays, synchronous comparator deadline crossing, custom Buffer ownership, and honest benchmark reporting.
• Two fresh post-fix reviewers independently confirmed the feature and its integration with current main with no remaining findings.

[lan17]

Summary

Adds opt-in semantic shadow validation for sensitive, invalidation-tracked Redis cache hits. A selected hit returns its cached value normally, then detached best-effort work re-reads the source of truth (SoT), independently deserializes the retained Redis payload, and compares the two application values.

Closes#104.

Architecture

flowchart LR
A[Tracked Redis hit] --> B[Return cached value]
A --> C{shadowRamp selects key?}
C -- no --> X[No shadow work]
C -- yes --> D{Metric hook and capacity?}
D -- no --> X
D -- yes --> E[Reserve exact-key flight]
E --> F[Unref setImmediate]
F --> G[Read SoT with DialCache disabled]
G --> H[serializer.load retained payload]
H --> I[Default or custom value comparator]
I --> J[Bounded outcome metric]

Loading

The hit path performs eligibility checks and slot reservation only. It does no SoT read, extra deserialization, deep comparison, payload-size-linear copy, repair, or cache mutation. All comparison work begins from an unreferenced immediate and is not awaited by the caller.

Public API and configuration

• Adds DialCacheKeyConfig.shadowRamp?: number.
  • Omitted or 0 disables validation.
  • 100 selects every otherwise eligible exact key.
  • Partial ramps use a stable exact-key cohort with an independent :shadow discriminator.
  • Sparse runtime overlays inherit the static baseline; malformed values skip shadow work and emit the existing remote config_resolution error without changing the hit.
• Adds DialCacheConfig.shadowMaxInFlight?: number, a positive safe integer defaulting to 1 per instance.
  • There is no queue.
  • Exact-key duplicates and global-cap overflow emit dropped.
  • Scheduled and timed-out-but-still-running work retains its slot until the underlying promise settles.
• Adds the root-exported synchronous type ShadowComparator<T> and optional shadowComparator on cached() / getOrLoad() options.
• Extends DialCacheMetricsAdapter with an optional shadowValidation hook, preserving existing custom adapters.

exporttypeShadowComparator<T>=(cachedValue: T,sourceValue: T,)=>boolean;

Comparison and data boundaries

• Eligibility requires an actual successful Redis hit with trackForInvalidation: true; request-local/process-local hits, misses, read errors, and initial deserialize failures do not validate.
• Redis reads retain the client-returned semantic string | Buffer payload internally alongside the decoded value.
• Detached work reads the raw SoT value, then runs the same effective serializer's load() again on the retained payload to create an independent cached snapshot.
• The object already returned to the caller is never compared, so caller mutation after the hit cannot contaminate validation.
• The comparator receives two values of T, never a Redis payload.
• The default is Node's util.isDeepStrictEqual; an optional per-operation comparator defines use-case-specific equality.
• SoT is intentionally not dump/load normalized. Lossy serialization remains visible unless a custom comparator explicitly treats the normalized values as equivalent.
• A comparator must synchronously return a boolean and must be deterministic, side-effect-free, non-mutating, and bounded. Throws and non-boolean returns emit comparison_error.
• An accidental Promise from untyped JavaScript is consumed safely but never accepted as a comparison result; it retains the flight slot until settlement.
• Redis framing, timestamps, TTLs, watermarks, Lua, keys, and bundled client protocol remain unchanged.

Detachment, liveness, and safety

• Source execution starts from an unreferenced setImmediate and runs under dialcache.disable(...) so it cannot recursively satisfy itself from the same cache.
• One monotonic, unreferenced deadline covers the source read, detached serializer load, and comparison.
• A finite fallbackTimeoutMs is reused; fallbackTimeoutMs: null keeps normal fallbacks unbounded but gives shadow work the internal 60-second default.
• On timeout, DialCache releases its retained payload reference and suppresses later phases. Already-running application/serializer/comparator work cannot be cancelled and continues to occupy its slot until settlement.
• Scheduler and deadline handles do not keep an otherwise idle process alive. Completion during shutdown remains best effort.
• Detached execution retains original cached() argument references or a getOrLoad() closure; docs require relevant inputs/captures to be immutable or snapshotted.
• Validation is observational only: no repair, write, TTL refresh, invalidation, local eviction, payload logging, or change to the returned value.

Observability

Bounded outcomes: match, mismatch, source_error, deserialization_error, comparison_error, timeout, and dropped.

Labels/tags are limited to cache namespace, use case, key type, and outcome. IDs, cached/SoT values, payloads, Redis keys, and raw exception messages are excluded.

Built-in adapters add:

• Prometheus: dialcache_shadow_validation_counter
• Datadog: dialcache.shadow.count

An adapter without the optional hook disables shadow execution so DialCache never performs an unobservable SoT read. Runtime thenable rejections from the hook are consumed without being awaited.

Compatibility and operations

• The feature is default-off and all new configuration/operation fields and the metrics hook are optional.
• Existing cache behavior, Redis wire format, Lua scripts, keys, watermarks, serializers, and package export paths remain unchanged.
• This is suitable for a minor release and rolling upgrade; enabling it is an operational rollout because it adds sampled SoT reads plus detached serializer/comparison CPU.
• shadowMaxInFlight bounds one instance, not the fleet. Roll out shadowRamp gradually and monitor match, mismatch, timeout, source_error, comparison_error, and dropped.
• The Prometheus adapter registers one additive collector family; applications that already own the exact same prefixed name with an incompatible schema must resolve that registry collision.
• README documents serializer repeatability, custom Redis payload ownership, detached input lifetime, concurrency budgeting, shutdown behavior, and privacy boundaries.

Change footprint

The PR is test-led rather than carrying duplicate first-pass production code:

• 22 files, +1,992 / -38 against current main
• Tests and package-consumer proofs: +1,364 / -2 across 7 files (68% of additions)
• Production source: +453 / -25 across 13 files (23% of additions)
• Benchmark tooling: +92
• README: +83 / -11

The dedicated shadow suite contains 36 scenarios covering the detached lifecycle and failure matrix. Shared setup is factored through test-only helpers; no obsolete serialized-payload comparison implementation remains.

Validation

Run on Node.js 22.22.0 at 4b8f7e4, including current main@500d5e7:

• corepack pnpm check
  • 341 unit tests passed
  • coverage: 97.04% statements, 93.65% branches, 98.47% functions, 97.16% lines
  • typecheck, build, and packed-consumer test passed
• corepack pnpm test:integration
  • 81 tests passed across Redis 6.2 and Valkey 8 with node-redis and GLIDE
  • includes real tracked-value semantic match/mismatch coverage with no repair
• DIALCACHE_BENCH_ITERATIONS=20000 DIALCACHE_BENCH_FANOUT=10000 corepack pnpm benchmark:request-local
  • all eight semantic scenarios passed
  • includes tracked Redis hits with shadow omitted and deterministically ramped out
• corepack pnpm audit --prod: no known production-dependency vulnerabilities
• GitHub Actions: CI, CodeQL, and PR-title validation passed on 4b8f7e4
• git diff --check

Focused shadow tests cover detachment, semantic default/custom equality, caller-mutation isolation, serializer ownership/repeatability, runtime ramp overlays, comparator type/runtime failures, monotonic deadlines, timeout slot retention, exact-key/global drops, coalescing, disabled contexts, metric failure isolation, and unreferenced handles.

Review history

• A broad production review covered concurrency/lifecycle, API compatibility, rollout/configuration, observability, tests, and operations.
• The cleanup round fixed low-severity coverage gaps for packed public APIs, runtime ramp enable/disable overlays, synchronous comparator deadline crossing, custom Buffer ownership, and honest benchmark reporting.
• Two fresh post-fix reviewers independently confirmed the feature and its integration with current main with no remaining findings.

[lan17]

Summary

Adds opt-in semantic shadow validation for sensitive, invalidation-tracked Redis cache hits. A selected hit returns its cached value normally, then detached best-effort work re-reads the source of truth (SoT), independently deserializes the retained Redis payload, and compares the two application values.

Closes#104.

Architecture

flowchart LR
A[Tracked Redis hit] --> B[Return cached value]
A --> C{shadowRamp selects key?}
C -- no --> X[No shadow work]
C -- yes --> D{Metric hook and capacity?}
D -- no --> X
D -- yes --> E[Reserve exact-key flight]
E --> F[Unref setImmediate]
F --> G[Read SoT with DialCache disabled]
G --> H[serializer.load retained payload]
H --> I[Default or custom value comparator]
I --> J[Bounded outcome metric]

Loading

The hit path performs eligibility checks and slot reservation only. It does no SoT read, extra deserialization, deep comparison, payload-size-linear copy, repair, or cache mutation. All comparison work begins from an unreferenced immediate and is not awaited by the caller.

Public API and configuration

• Adds DialCacheKeyConfig.shadowRamp?: number.
  • Omitted or 0 disables validation.
  • 100 selects every otherwise eligible exact key.
  • Partial ramps use a stable exact-key cohort with an independent :shadow discriminator.
  • Sparse runtime overlays inherit the static baseline; malformed values skip shadow work and emit the existing remote config_resolution error without changing the hit.
• Adds DialCacheConfig.shadowMaxInFlight?: number, a positive safe integer defaulting to 1 per instance.
  • There is no queue.
  • Exact-key duplicates and global-cap overflow emit dropped.
  • Scheduled and timed-out-but-still-running work retains its slot until the underlying promise settles.
• Adds the root-exported synchronous type ShadowComparator<T> and optional shadowComparator on cached() / getOrLoad() options.
• Extends DialCacheMetricsAdapter with an optional shadowValidation hook, preserving existing custom adapters.

exporttypeShadowComparator<T>=(cachedValue: T,sourceValue: T,)=>boolean;

Comparison and data boundaries

• Eligibility requires an actual successful Redis hit with trackForInvalidation: true; request-local/process-local hits, misses, read errors, and initial deserialize failures do not validate.
• Redis reads retain the client-returned semantic string | Buffer payload internally alongside the decoded value.
• Detached work reads the raw SoT value, then runs the same effective serializer's load() again on the retained payload to create an independent cached snapshot.
• The object already returned to the caller is never compared, so caller mutation after the hit cannot contaminate validation.
• The comparator receives two values of T, never a Redis payload.
• The default is Node's util.isDeepStrictEqual; an optional per-operation comparator defines use-case-specific equality.
• SoT is intentionally not dump/load normalized. Lossy serialization remains visible unless a custom comparator explicitly treats the normalized values as equivalent.
• A comparator must synchronously return a boolean and must be deterministic, side-effect-free, non-mutating, and bounded. Throws and non-boolean returns emit comparison_error.
• An accidental Promise from untyped JavaScript is consumed safely but never accepted as a comparison result; it retains the flight slot until settlement.
• Redis framing, timestamps, TTLs, watermarks, Lua, keys, and bundled client protocol remain unchanged.

Detachment, liveness, and safety

• Source execution starts from an unreferenced setImmediate and runs under dialcache.disable(...) so it cannot recursively satisfy itself from the same cache.
• One monotonic, unreferenced deadline covers the source read, detached serializer load, and comparison.
• A finite fallbackTimeoutMs is reused; fallbackTimeoutMs: null keeps normal fallbacks unbounded but gives shadow work the internal 60-second default.
• On timeout, DialCache releases its retained payload reference and suppresses later phases. Already-running application/serializer/comparator work cannot be cancelled and continues to occupy its slot until settlement.
• Scheduler and deadline handles do not keep an otherwise idle process alive. Completion during shutdown remains best effort.
• Detached execution retains original cached() argument references or a getOrLoad() closure; docs require relevant inputs/captures to be immutable or snapshotted.
• Validation is observational only: no repair, write, TTL refresh, invalidation, local eviction, payload logging, or change to the returned value.

Observability

Bounded outcomes: match, mismatch, source_error, deserialization_error, comparison_error, timeout, and dropped.

Labels/tags are limited to cache namespace, use case, key type, and outcome. IDs, cached/SoT values, payloads, Redis keys, and raw exception messages are excluded.

Built-in adapters add:

• Prometheus: dialcache_shadow_validation_counter
• Datadog: dialcache.shadow.count

An adapter without the optional hook disables shadow execution so DialCache never performs an unobservable SoT read. Runtime thenable rejections from the hook are consumed without being awaited.

Compatibility and operations

• The feature is default-off and all new configuration/operation fields and the metrics hook are optional.
• Existing cache behavior, Redis wire format, Lua scripts, keys, watermarks, serializers, and package export paths remain unchanged.
• This is suitable for a minor release and rolling upgrade; enabling it is an operational rollout because it adds sampled SoT reads plus detached serializer/comparison CPU.
• shadowMaxInFlight bounds one instance, not the fleet. Roll out shadowRamp gradually and monitor match, mismatch, timeout, source_error, comparison_error, and dropped.
• The Prometheus adapter registers one additive collector family; applications that already own the exact same prefixed name with an incompatible schema must resolve that registry collision.
• README documents serializer repeatability, custom Redis payload ownership, detached input lifetime, concurrency budgeting, shutdown behavior, and privacy boundaries.

Change footprint

The PR is test-led rather than carrying duplicate first-pass production code:

• 22 files, +1,992 / -38 against current main
• Tests and package-consumer proofs: +1,364 / -2 across 7 files (68% of additions)
• Production source: +453 / -25 across 13 files (23% of additions)
• Benchmark tooling: +92
• README: +83 / -11

The dedicated shadow suite contains 36 scenarios covering the detached lifecycle and failure matrix. Shared setup is factored through test-only helpers; no obsolete serialized-payload comparison implementation remains.

Validation

Run on Node.js 22.22.0 at 4b8f7e4, including current main@500d5e7:

• corepack pnpm check
  • 341 unit tests passed
  • coverage: 97.04% statements, 93.65% branches, 98.47% functions, 97.16% lines
  • typecheck, build, and packed-consumer test passed
• corepack pnpm test:integration
  • 81 tests passed across Redis 6.2 and Valkey 8 with node-redis and GLIDE
  • includes real tracked-value semantic match/mismatch coverage with no repair
• DIALCACHE_BENCH_ITERATIONS=20000 DIALCACHE_BENCH_FANOUT=10000 corepack pnpm benchmark:request-local
  • all eight semantic scenarios passed
  • includes tracked Redis hits with shadow omitted and deterministically ramped out
• corepack pnpm audit --prod: no known production-dependency vulnerabilities
• GitHub Actions: CI, CodeQL, and PR-title validation passed on 4b8f7e4
• git diff --check

Focused shadow tests cover detachment, semantic default/custom equality, caller-mutation isolation, serializer ownership/repeatability, runtime ramp overlays, comparator type/runtime failures, monotonic deadlines, timeout slot retention, exact-key/global drops, coalescing, disabled contexts, metric failure isolation, and unreferenced handles.

Review history

• A broad production review covered concurrency/lifecycle, API compatibility, rollout/configuration, observability, tests, and operations.
• The cleanup round fixed low-severity coverage gaps for packed public APIs, runtime ramp enable/disable overlays, synchronous comparator deadline crossing, custom Buffer ownership, and honest benchmark reporting.
• Two fresh post-fix reviewers independently confirmed the feature and its integration with current main with no remaining findings.

[lan17]

Summary

Adds opt-in semantic shadow validation for sensitive, invalidation-tracked Redis cache hits. A selected hit returns its cached value normally, then detached best-effort work re-reads the source of truth (SoT), independently deserializes the retained Redis payload, and compares the two application values.

Closes#104.

Architecture

flowchart LR
A[Tracked Redis hit] --> B[Return cached value]
A --> C{shadowRamp selects key?}
C -- no --> X[No shadow work]
C -- yes --> D{Metric hook and capacity?}
D -- no --> X
D -- yes --> E[Reserve exact-key flight]
E --> F[Unref setImmediate]
F --> G[Read SoT with DialCache disabled]
G --> H[serializer.load retained payload]
H --> I[Default or custom value comparator]
I --> J[Bounded outcome metric]

Loading

The hit path performs eligibility checks and slot reservation only. It does no SoT read, extra deserialization, deep comparison, payload-size-linear copy, repair, or cache mutation. All comparison work begins from an unreferenced immediate and is not awaited by the caller.

Public API and configuration

• Adds DialCacheKeyConfig.shadowRamp?: number.
  • Omitted or 0 disables validation.
  • 100 selects every otherwise eligible exact key.
  • Partial ramps use a stable exact-key cohort with an independent :shadow discriminator.
  • Sparse runtime overlays inherit the static baseline; malformed values skip shadow work and emit the existing remote config_resolution error without changing the hit.
• Adds DialCacheConfig.shadowMaxInFlight?: number, a positive safe integer defaulting to 1 per instance.
  • There is no queue.
  • Exact-key duplicates and global-cap overflow emit dropped.
  • Scheduled and timed-out-but-still-running work retains its slot until the underlying promise settles.
• Adds the root-exported synchronous type ShadowComparator<T> and optional shadowComparator on cached() / getOrLoad() options.
• Extends DialCacheMetricsAdapter with an optional shadowValidation hook, preserving existing custom adapters.

exporttypeShadowComparator<T>=(cachedValue: T,sourceValue: T,)=>boolean;

Comparison and data boundaries

• Eligibility requires an actual successful Redis hit with trackForInvalidation: true; request-local/process-local hits, misses, read errors, and initial deserialize failures do not validate.
• Redis reads retain the client-returned semantic string | Buffer payload internally alongside the decoded value.
• Detached work reads the raw SoT value, then runs the same effective serializer's load() again on the retained payload to create an independent cached snapshot.
• The object already returned to the caller is never compared, so caller mutation after the hit cannot contaminate validation.
• The comparator receives two values of T, never a Redis payload.
• The default is Node's util.isDeepStrictEqual; an optional per-operation comparator defines use-case-specific equality.
• SoT is intentionally not dump/load normalized. Lossy serialization remains visible unless a custom comparator explicitly treats the normalized values as equivalent.
• A comparator must synchronously return a boolean and must be deterministic, side-effect-free, non-mutating, and bounded. Throws and non-boolean returns emit comparison_error.
• An accidental Promise from untyped JavaScript is consumed safely but never accepted as a comparison result; it retains the flight slot until settlement.
• Redis framing, timestamps, TTLs, watermarks, Lua, keys, and bundled client protocol remain unchanged.

Detachment, liveness, and safety

• Source execution starts from an unreferenced setImmediate and runs under dialcache.disable(...) so it cannot recursively satisfy itself from the same cache.
• One monotonic, unreferenced deadline covers the source read, detached serializer load, and comparison.
• A finite fallbackTimeoutMs is reused; fallbackTimeoutMs: null keeps normal fallbacks unbounded but gives shadow work the internal 60-second default.
• On timeout, DialCache releases its retained payload reference and suppresses later phases. Already-running application/serializer/comparator work cannot be cancelled and continues to occupy its slot until settlement.
• Scheduler and deadline handles do not keep an otherwise idle process alive. Completion during shutdown remains best effort.
• Detached execution retains original cached() argument references or a getOrLoad() closure; docs require relevant inputs/captures to be immutable or snapshotted.
• Validation is observational only: no repair, write, TTL refresh, invalidation, local eviction, payload logging, or change to the returned value.

Observability

Bounded outcomes: match, mismatch, source_error, deserialization_error, comparison_error, timeout, and dropped.

Labels/tags are limited to cache namespace, use case, key type, and outcome. IDs, cached/SoT values, payloads, Redis keys, and raw exception messages are excluded.

Built-in adapters add:

• Prometheus: dialcache_shadow_validation_counter
• Datadog: dialcache.shadow.count

An adapter without the optional hook disables shadow execution so DialCache never performs an unobservable SoT read. Runtime thenable rejections from the hook are consumed without being awaited.

Compatibility and operations

• The feature is default-off and all new configuration/operation fields and the metrics hook are optional.
• Existing cache behavior, Redis wire format, Lua scripts, keys, watermarks, serializers, and package export paths remain unchanged.
• This is suitable for a minor release and rolling upgrade; enabling it is an operational rollout because it adds sampled SoT reads plus detached serializer/comparison CPU.
• shadowMaxInFlight bounds one instance, not the fleet. Roll out shadowRamp gradually and monitor match, mismatch, timeout, source_error, comparison_error, and dropped.
• The Prometheus adapter registers one additive collector family; applications that already own the exact same prefixed name with an incompatible schema must resolve that registry collision.
• README documents serializer repeatability, custom Redis payload ownership, detached input lifetime, concurrency budgeting, shutdown behavior, and privacy boundaries.

Change footprint

The PR is test-led rather than carrying duplicate first-pass production code:

• 22 files, +1,992 / -38 against current main
• Tests and package-consumer proofs: +1,364 / -2 across 7 files (68% of additions)
• Production source: +453 / -25 across 13 files (23% of additions)
• Benchmark tooling: +92
• README: +83 / -11

The dedicated shadow suite contains 36 scenarios covering the detached lifecycle and failure matrix. Shared setup is factored through test-only helpers; no obsolete serialized-payload comparison implementation remains.

Validation

Run on Node.js 22.22.0 at 4b8f7e4, including current main@500d5e7:

• corepack pnpm check
  • 341 unit tests passed
  • coverage: 97.04% statements, 93.65% branches, 98.47% functions, 97.16% lines
  • typecheck, build, and packed-consumer test passed
• corepack pnpm test:integration
  • 81 tests passed across Redis 6.2 and Valkey 8 with node-redis and GLIDE
  • includes real tracked-value semantic match/mismatch coverage with no repair
• DIALCACHE_BENCH_ITERATIONS=20000 DIALCACHE_BENCH_FANOUT=10000 corepack pnpm benchmark:request-local
  • all eight semantic scenarios passed
  • includes tracked Redis hits with shadow omitted and deterministically ramped out
• corepack pnpm audit --prod: no known production-dependency vulnerabilities
• GitHub Actions: CI, CodeQL, and PR-title validation passed on 4b8f7e4
• git diff --check

Focused shadow tests cover detachment, semantic default/custom equality, caller-mutation isolation, serializer ownership/repeatability, runtime ramp overlays, comparator type/runtime failures, monotonic deadlines, timeout slot retention, exact-key/global drops, coalescing, disabled contexts, metric failure isolation, and unreferenced handles.

Review history

• A broad production review covered concurrency/lifecycle, API compatibility, rollout/configuration, observability, tests, and operations.
• The cleanup round fixed low-severity coverage gaps for packed public APIs, runtime ramp enable/disable overlays, synchronous comparator deadline crossing, custom Buffer ownership, and honest benchmark reporting.
• Two fresh post-fix reviewers independently confirmed the feature and its integration with current main with no remaining findings.

[lan17]

Summary

Adds opt-in semantic shadow validation for sensitive, invalidation-tracked Redis cache hits. A selected hit returns its cached value normally, then detached best-effort work re-reads the source of truth (SoT), independently deserializes the retained Redis payload, and compares the two application values.

Closes#104.

Architecture

flowchart LR
A[Tracked Redis hit] --> B[Return cached value]
A --> C{shadowRamp selects key?}
C -- no --> X[No shadow work]
C -- yes --> D{Metric hook and capacity?}
D -- no --> X
D -- yes --> E[Reserve exact-key flight]
E --> F[Unref setImmediate]
F --> G[Read SoT with DialCache disabled]
G --> H[serializer.load retained payload]
H --> I[Default or custom value comparator]
I --> J[Bounded outcome metric]

Loading

The hit path performs eligibility checks and slot reservation only. It does no SoT read, extra deserialization, deep comparison, payload-size-linear copy, repair, or cache mutation. All comparison work begins from an unreferenced immediate and is not awaited by the caller.

Public API and configuration

• Adds DialCacheKeyConfig.shadowRamp?: number.
  • Omitted or 0 disables validation.
  • 100 selects every otherwise eligible exact key.
  • Partial ramps use a stable exact-key cohort with an independent :shadow discriminator.
  • Sparse runtime overlays inherit the static baseline; malformed values skip shadow work and emit the existing remote config_resolution error without changing the hit.
• Adds DialCacheConfig.shadowMaxInFlight?: number, a positive safe integer defaulting to 1 per instance.
  • There is no queue.
  • Exact-key duplicates and global-cap overflow emit dropped.
  • Scheduled and timed-out-but-still-running work retains its slot until the underlying promise settles.
• Adds the root-exported synchronous type ShadowComparator<T> and optional shadowComparator on cached() / getOrLoad() options.
• Extends DialCacheMetricsAdapter with an optional shadowValidation hook, preserving existing custom adapters.

exporttypeShadowComparator<T>=(cachedValue: T,sourceValue: T,)=>boolean;

Comparison and data boundaries

• Eligibility requires an actual successful Redis hit with trackForInvalidation: true; request-local/process-local hits, misses, read errors, and initial deserialize failures do not validate.
• Redis reads retain the client-returned semantic string | Buffer payload internally alongside the decoded value.
• Detached work reads the raw SoT value, then runs the same effective serializer's load() again on the retained payload to create an independent cached snapshot.
• The object already returned to the caller is never compared, so caller mutation after the hit cannot contaminate validation.
• The comparator receives two values of T, never a Redis payload.
• The default is Node's util.isDeepStrictEqual; an optional per-operation comparator defines use-case-specific equality.
• SoT is intentionally not dump/load normalized. Lossy serialization remains visible unless a custom comparator explicitly treats the normalized values as equivalent.
• A comparator must synchronously return a boolean and must be deterministic, side-effect-free, non-mutating, and bounded. Throws and non-boolean returns emit comparison_error.
• An accidental Promise from untyped JavaScript is consumed safely but never accepted as a comparison result; it retains the flight slot until settlement.
• Redis framing, timestamps, TTLs, watermarks, Lua, keys, and bundled client protocol remain unchanged.

Detachment, liveness, and safety

• Source execution starts from an unreferenced setImmediate and runs under dialcache.disable(...) so it cannot recursively satisfy itself from the same cache.
• One monotonic, unreferenced deadline covers the source read, detached serializer load, and comparison.
• A finite fallbackTimeoutMs is reused; fallbackTimeoutMs: null keeps normal fallbacks unbounded but gives shadow work the internal 60-second default.
• On timeout, DialCache releases its retained payload reference and suppresses later phases. Already-running application/serializer/comparator work cannot be cancelled and continues to occupy its slot until settlement.
• Scheduler and deadline handles do not keep an otherwise idle process alive. Completion during shutdown remains best effort.
• Detached execution retains original cached() argument references or a getOrLoad() closure; docs require relevant inputs/captures to be immutable or snapshotted.
• Validation is observational only: no repair, write, TTL refresh, invalidation, local eviction, payload logging, or change to the returned value.

Observability

Bounded outcomes: match, mismatch, source_error, deserialization_error, comparison_error, timeout, and dropped.

Labels/tags are limited to cache namespace, use case, key type, and outcome. IDs, cached/SoT values, payloads, Redis keys, and raw exception messages are excluded.

Built-in adapters add:

• Prometheus: dialcache_shadow_validation_counter
• Datadog: dialcache.shadow.count

An adapter without the optional hook disables shadow execution so DialCache never performs an unobservable SoT read. Runtime thenable rejections from the hook are consumed without being awaited.

Compatibility and operations

• The feature is default-off and all new configuration/operation fields and the metrics hook are optional.
• Existing cache behavior, Redis wire format, Lua scripts, keys, watermarks, serializers, and package export paths remain unchanged.
• This is suitable for a minor release and rolling upgrade; enabling it is an operational rollout because it adds sampled SoT reads plus detached serializer/comparison CPU.
• shadowMaxInFlight bounds one instance, not the fleet. Roll out shadowRamp gradually and monitor match, mismatch, timeout, source_error, comparison_error, and dropped.
• The Prometheus adapter registers one additive collector family; applications that already own the exact same prefixed name with an incompatible schema must resolve that registry collision.
• README documents serializer repeatability, custom Redis payload ownership, detached input lifetime, concurrency budgeting, shutdown behavior, and privacy boundaries.

Change footprint

The PR is test-led rather than carrying duplicate first-pass production code:

• 22 files, +1,992 / -38 against current main
• Tests and package-consumer proofs: +1,364 / -2 across 7 files (68% of additions)
• Production source: +453 / -25 across 13 files (23% of additions)
• Benchmark tooling: +92
• README: +83 / -11

The dedicated shadow suite contains 36 scenarios covering the detached lifecycle and failure matrix. Shared setup is factored through test-only helpers; no obsolete serialized-payload comparison implementation remains.

Validation

Run on Node.js 22.22.0 at 4b8f7e4, including current main@500d5e7:

• corepack pnpm check
  • 341 unit tests passed
  • coverage: 97.04% statements, 93.65% branches, 98.47% functions, 97.16% lines
  • typecheck, build, and packed-consumer test passed
• corepack pnpm test:integration
  • 81 tests passed across Redis 6.2 and Valkey 8 with node-redis and GLIDE
  • includes real tracked-value semantic match/mismatch coverage with no repair
• DIALCACHE_BENCH_ITERATIONS=20000 DIALCACHE_BENCH_FANOUT=10000 corepack pnpm benchmark:request-local
  • all eight semantic scenarios passed
  • includes tracked Redis hits with shadow omitted and deterministically ramped out
• corepack pnpm audit --prod: no known production-dependency vulnerabilities
• GitHub Actions: CI, CodeQL, and PR-title validation passed on 4b8f7e4
• git diff --check

Focused shadow tests cover detachment, semantic default/custom equality, caller-mutation isolation, serializer ownership/repeatability, runtime ramp overlays, comparator type/runtime failures, monotonic deadlines, timeout slot retention, exact-key/global drops, coalescing, disabled contexts, metric failure isolation, and unreferenced handles.

Review history

• A broad production review covered concurrency/lifecycle, API compatibility, rollout/configuration, observability, tests, and operations.
• The cleanup round fixed low-severity coverage gaps for packed public APIs, runtime ramp enable/disable overlays, synchronous comparator deadline crossing, custom Buffer ownership, and honest benchmark reporting.
• Two fresh post-fix reviewers independently confirmed the feature and its integration with current main with no remaining findings.

[lan17]

Summary

Adds opt-in semantic shadow validation for sensitive, invalidation-tracked Redis cache hits. A selected hit returns its cached value normally, then detached best-effort work re-reads the source of truth (SoT), independently deserializes the retained Redis payload, and compares the two application values.

Closes#104.

Architecture

flowchart LR
A[Tracked Redis hit] --> B[Return cached value]
A --> C{shadowRamp selects key?}
C -- no --> X[No shadow work]
C -- yes --> D{Metric hook and capacity?}
D -- no --> X
D -- yes --> E[Reserve exact-key flight]
E --> F[Unref setImmediate]
F --> G[Read SoT with DialCache disabled]
G --> H[serializer.load retained payload]
H --> I[Default or custom value comparator]
I --> J[Bounded outcome metric]

Loading

The hit path performs eligibility checks and slot reservation only. It does no SoT read, extra deserialization, deep comparison, payload-size-linear copy, repair, or cache mutation. All comparison work begins from an unreferenced immediate and is not awaited by the caller.

Public API and configuration

• Adds DialCacheKeyConfig.shadowRamp?: number.
  • Omitted or 0 disables validation.
  • 100 selects every otherwise eligible exact key.
  • Partial ramps use a stable exact-key cohort with an independent :shadow discriminator.
  • Sparse runtime overlays inherit the static baseline; malformed values skip shadow work and emit the existing remote config_resolution error without changing the hit.
• Adds DialCacheConfig.shadowMaxInFlight?: number, a positive safe integer defaulting to 1 per instance.
  • There is no queue.
  • Exact-key duplicates and global-cap overflow emit dropped.
  • Scheduled and timed-out-but-still-running work retains its slot until the underlying promise settles.
• Adds the root-exported synchronous type ShadowComparator<T> and optional shadowComparator on cached() / getOrLoad() options.
• Extends DialCacheMetricsAdapter with an optional shadowValidation hook, preserving existing custom adapters.

exporttypeShadowComparator<T>=(cachedValue: T,sourceValue: T,)=>boolean;

Comparison and data boundaries

• Eligibility requires an actual successful Redis hit with trackForInvalidation: true; request-local/process-local hits, misses, read errors, and initial deserialize failures do not validate.
• Redis reads retain the client-returned semantic string | Buffer payload internally alongside the decoded value.
• Detached work reads the raw SoT value, then runs the same effective serializer's load() again on the retained payload to create an independent cached snapshot.
• The object already returned to the caller is never compared, so caller mutation after the hit cannot contaminate validation.
• The comparator receives two values of T, never a Redis payload.
• The default is Node's util.isDeepStrictEqual; an optional per-operation comparator defines use-case-specific equality.
• SoT is intentionally not dump/load normalized. Lossy serialization remains visible unless a custom comparator explicitly treats the normalized values as equivalent.
• A comparator must synchronously return a boolean and must be deterministic, side-effect-free, non-mutating, and bounded. Throws and non-boolean returns emit comparison_error.
• An accidental Promise from untyped JavaScript is consumed safely but never accepted as a comparison result; it retains the flight slot until settlement.
• Redis framing, timestamps, TTLs, watermarks, Lua, keys, and bundled client protocol remain unchanged.

Detachment, liveness, and safety

• Source execution starts from an unreferenced setImmediate and runs under dialcache.disable(...) so it cannot recursively satisfy itself from the same cache.
• One monotonic, unreferenced deadline covers the source read, detached serializer load, and comparison.
• A finite fallbackTimeoutMs is reused; fallbackTimeoutMs: null keeps normal fallbacks unbounded but gives shadow work the internal 60-second default.
• On timeout, DialCache releases its retained payload reference and suppresses later phases. Already-running application/serializer/comparator work cannot be cancelled and continues to occupy its slot until settlement.
• Scheduler and deadline handles do not keep an otherwise idle process alive. Completion during shutdown remains best effort.
• Detached execution retains original cached() argument references or a getOrLoad() closure; docs require relevant inputs/captures to be immutable or snapshotted.
• Validation is observational only: no repair, write, TTL refresh, invalidation, local eviction, payload logging, or change to the returned value.

Observability

Bounded outcomes: match, mismatch, source_error, deserialization_error, comparison_error, timeout, and dropped.

Labels/tags are limited to cache namespace, use case, key type, and outcome. IDs, cached/SoT values, payloads, Redis keys, and raw exception messages are excluded.

Built-in adapters add:

• Prometheus: dialcache_shadow_validation_counter
• Datadog: dialcache.shadow.count

An adapter without the optional hook disables shadow execution so DialCache never performs an unobservable SoT read. Runtime thenable rejections from the hook are consumed without being awaited.

Compatibility and operations

• The feature is default-off and all new configuration/operation fields and the metrics hook are optional.
• Existing cache behavior, Redis wire format, Lua scripts, keys, watermarks, serializers, and package export paths remain unchanged.
• This is suitable for a minor release and rolling upgrade; enabling it is an operational rollout because it adds sampled SoT reads plus detached serializer/comparison CPU.
• shadowMaxInFlight bounds one instance, not the fleet. Roll out shadowRamp gradually and monitor match, mismatch, timeout, source_error, comparison_error, and dropped.
• The Prometheus adapter registers one additive collector family; applications that already own the exact same prefixed name with an incompatible schema must resolve that registry collision.
• README documents serializer repeatability, custom Redis payload ownership, detached input lifetime, concurrency budgeting, shutdown behavior, and privacy boundaries.

Change footprint

The PR is test-led rather than carrying duplicate first-pass production code:

• 22 files, +1,992 / -38 against current main
• Tests and package-consumer proofs: +1,364 / -2 across 7 files (68% of additions)
• Production source: +453 / -25 across 13 files (23% of additions)
• Benchmark tooling: +92
• README: +83 / -11

The dedicated shadow suite contains 36 scenarios covering the detached lifecycle and failure matrix. Shared setup is factored through test-only helpers; no obsolete serialized-payload comparison implementation remains.

Validation

Run on Node.js 22.22.0 at 4b8f7e4, including current main@500d5e7:

• corepack pnpm check
  • 341 unit tests passed
  • coverage: 97.04% statements, 93.65% branches, 98.47% functions, 97.16% lines
  • typecheck, build, and packed-consumer test passed
• corepack pnpm test:integration
  • 81 tests passed across Redis 6.2 and Valkey 8 with node-redis and GLIDE
  • includes real tracked-value semantic match/mismatch coverage with no repair
• DIALCACHE_BENCH_ITERATIONS=20000 DIALCACHE_BENCH_FANOUT=10000 corepack pnpm benchmark:request-local
  • all eight semantic scenarios passed
  • includes tracked Redis hits with shadow omitted and deterministically ramped out
• corepack pnpm audit --prod: no known production-dependency vulnerabilities
• GitHub Actions: CI, CodeQL, and PR-title validation passed on 4b8f7e4
• git diff --check

Focused shadow tests cover detachment, semantic default/custom equality, caller-mutation isolation, serializer ownership/repeatability, runtime ramp overlays, comparator type/runtime failures, monotonic deadlines, timeout slot retention, exact-key/global drops, coalescing, disabled contexts, metric failure isolation, and unreferenced handles.

Review history

• A broad production review covered concurrency/lifecycle, API compatibility, rollout/configuration, observability, tests, and operations.
• The cleanup round fixed low-severity coverage gaps for packed public APIs, runtime ramp enable/disable overlays, synchronous comparator deadline crossing, custom Buffer ownership, and honest benchmark reporting.
• Two fresh post-fix reviewers independently confirmed the feature and its integration with current main with no remaining findings.