Uh oh!
There was an error while loading. Please reload this page.
fix(dig-node): drain the manifest memo on cache.clear + cap dig.getMetadata (#2145) - #185
Merged
Merged
Conversation
…tadata (#2145) The decoded-manifest memo is already byte-budgeted (MANIFEST_MEMO_MAX_BYTES / MANIFEST_ENTRY_MAX_BYTES with LRU eviction), but two anonymous-reachable DoS gaps remained: - cache.clear did not drain the memo, so an operator could not reclaim its process-lifetime RAM. Add ManifestMemo::clear + clear_manifest_memo() and wire it into the cache.clear dispatch beside clear_cache/clear_content_cache. - dig.getMetadata rendered a whole publisher metadata section into ONE response with no ceiling. Its custom/links are publisher-controlled and it cannot be windowed like getContent/getCapsule, so a ~200-byte anonymous request could return ~100 MB. Refuse a section over METADATA_RESPONSE_MAX_BYTES (3 MiB, the WINDOW ceiling) with a bounded METADATA_TOO_LARGE (-32015); read the rendered length before parsing so the oversized body is never re-serialized. Normal (kilobyte) metadata serves unchanged. Regression tests: cache.clear drains the memo (per-capsule, race-free); an oversized metadata section is refused with a bounded error; a normal section serves unchanged; ManifestMemo::clear resets entries+bytes. SPEC.md + docs.dig.net error catalog document the byte budget, the response ceiling, and -32015. Closes #2145 Co-Authored-By: Claude <noreply@anthropic.com>
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes dig_ecosystem#2145. Hardens two DoS gaps on the anonymous, internet-reachable public-manifest read path.
§2.0 already-shipped note
The ticket's fix#1 — the manifest-memo byte budget — is already on
main:MANIFEST_MEMO_MAX_BYTES = 32 MiB+MANIFEST_ENTRY_MAX_BYTES = 4 MiBwith LRU eviction and exact rendered-JSON byte retention (not a hand-estimated tree size). So this PR closes only the two remaining gaps the ticket named:Fix
cache.clearnow drains the memo. It previously cleared the content/disk caches but left the process-lifetime manifest memo intact, so an operator couldn't reclaim it.cache.clearnow also callsclear_manifest_memo().dig.getMetadatagets a response ceiling. It was the only public-tier read with no response cap —getContent/getCapsulewindow at 3 MiB, but this rendered the whole metadata section into one JSON-RPC response (~200 B in → up to ~100 MB out). A whole JSON object can't be windowed, so an oversized section is now refused with a bounded-32015 METADATA_TOO_LARGErather than rendered.read_metadata_manifest_jsonreturns the renderedArc<str>soget_metadatachecks the raw length before parsing — the ~100 MB body is never parsed + re-serialized. Normal (kilobyte) metadata serves byte-identically.METADATA_RESPONSE_MAX_BYTES = WINDOW = 3 MiB— pinned to the same hard RAM ceiling the sibling window reads use, not scaled by publisher input.How verified (TDD)
dig_get_metadata_refuses_a_section_over_the_response_ceiling— non-vacuous (neutering the check to> usize::MAXfails the test).cache_clear_drains_the_manifest_memo— non-vacuous (removingclear_manifest_memo()fails it).dig_get_metadata_serves_a_normal_section_unchanged(no-behaviour-change guard) +clearing_the_manifest_memo_resets_its_entries_and_byte_total.cargo test -p dig-node-core --lib→ 775 passed, 0 failed; fmt +clippy -D warningsclean; both crates build.Cross-repo coherence (§1.3b)
The new
-32015error is catalogued in docs.dig.net (companion PR — specdig-spec.mjs, error-codes + methods pages, regenerated OpenRPC/error-code JSON; machine-spec drift gate green). SPEC.md (§ memo byte-budget + getMetadata ceiling + the-32015row) + DEVELOPMENT_LOG updated here.Blast radius
read_metadata_manifest_json→ single callerget_metadata(return-type change fully contained);ManifestMemo::clear/clear_manifest_memoadditive;cache.cleargains one additive call; the new consts +-32015have no prior consumers. (gitnexus per-repo runner is absent in dig-node — scoped by grep/call-graph; filing a ticket to reprovision it.)Version
root
[workspace.package].version0.99.1 → 0.99.2 (patch,fix:);Cargo.lockregenerated.Generated by Claude Code