Uh oh!
There was an error while loading. Please reload this page.
feat: serve dig.getManifest locally from the embedded public manifest - #1
Merged
Conversation
#176 Phase C: dig.getManifest moves from a blind-passthrough alias to a LOCAL read, resolving the normalized PublicManifest (.dig data-section id 13, Phase A) embedded in a held capsule's compiled module. - dig-node-core: new Node::get_manifest + read_public_manifest_blocking (spawn_blocking, mirrors serve_local_blocking). Params {store_id, root} (a capsule identifier, no retrieval_key needed - the manifest is public, unencrypted data). A held module with no manifest section (an older .dig, or a private store) returns `result: null`, never an error (store-format backwards-compatibility rule); a capsule not held at all reports -32004, matching dig.fetchRange's miss code. - dig.stage now sets FinalizeOptions.include_public_manifest for PUBLIC stores, matching the digstore CLI's own commit rule (a new required field added by the digstore rev bump below). - Bumped the pinned digstore rev (dig-node-core + dig-wallet, kept in sync) to the tip carrying the public-manifest format extension, and added digstore-compiler as a direct dependency for extract_data_section_blob (already a transitive dep via digstore-stage). - dig-node-service: meta.rs catalogue entry flipped to served: "local" with an updated summary; doc comments in server.rs/rpc.rs updated to match. - SPEC.md/README.md/USER_JOURNEY.md updated to describe the new local behavior instead of the old passthrough alias. - Tests: dig_get_manifest_returns_embedded_manifest_json_when_present, dig_get_manifest_returns_null_when_section_absent, dig_get_manifest_reports_unavailable_when_capsule_not_held, dig_get_manifest_rejects_malformed_params_without_touching_disk. The existing passthrough_alias_methods_are_method_not_found_on_the_node test and the openrpc_drift_guard suite were updated/re-verified against the new classification. Verified: cargo fmt --all -- --check, cargo clippy --workspace --all-targets --all-features -- -D warnings, and cargo test --workspace all green (dig-node-core 193, dig-node-service 92, dig-wallet 71, dig-runtime 3 - zero failures).
Uh oh!
There was an error while loading. Please reload this page.
MichaelTaylor3d added a commit
that referenced
this pull request
Jul 10, 2026
…#1) #176 Phase C: dig.getManifest moves from a blind-passthrough alias to a LOCAL read, resolving the normalized PublicManifest (.dig data-section id 13, Phase A) embedded in a held capsule's compiled module. - dig-node-core: new Node::get_manifest + read_public_manifest_blocking (spawn_blocking, mirrors serve_local_blocking). Params {store_id, root} (a capsule identifier, no retrieval_key needed - the manifest is public, unencrypted data). A held module with no manifest section (an older .dig, or a private store) returns `result: null`, never an error (store-format backwards-compatibility rule); a capsule not held at all reports -32004, matching dig.fetchRange's miss code. - dig.stage now sets FinalizeOptions.include_public_manifest for PUBLIC stores, matching the digstore CLI's own commit rule (a new required field added by the digstore rev bump below). - Bumped the pinned digstore rev (dig-node-core + dig-wallet, kept in sync) to the tip carrying the public-manifest format extension, and added digstore-compiler as a direct dependency for extract_data_section_blob (already a transitive dep via digstore-stage). - dig-node-service: meta.rs catalogue entry flipped to served: "local" with an updated summary; doc comments in server.rs/rpc.rs updated to match. - SPEC.md/README.md/USER_JOURNEY.md updated to describe the new local behavior instead of the old passthrough alias. - Tests: dig_get_manifest_returns_embedded_manifest_json_when_present, dig_get_manifest_returns_null_when_section_absent, dig_get_manifest_reports_unavailable_when_capsule_not_held, dig_get_manifest_rejects_malformed_params_without_touching_disk. The existing passthrough_alias_methods_are_method_not_found_on_the_node test and the openrpc_drift_guard suite were updated/re-verified against the new classification. Verified: cargo fmt --all -- --check, cargo clippy --workspace --all-targets --all-features -- -D warnings, and cargo test --workspace all green (dig-node-core 193, dig-node-service 92, dig-wallet 71, dig-runtime 3 - zero failures). Co-Authored-By: Claude <noreply@anthropic.com>
MichaelTaylor3d added a commit
that referenced
this pull request
Jul 10, 2026
…#1) #176 Phase C: dig.getManifest moves from a blind-passthrough alias to a LOCAL read, resolving the normalized PublicManifest (.dig data-section id 13, Phase A) embedded in a held capsule's compiled module. - dig-node-core: new Node::get_manifest + read_public_manifest_blocking (spawn_blocking, mirrors serve_local_blocking). Params {store_id, root} (a capsule identifier, no retrieval_key needed - the manifest is public, unencrypted data). A held module with no manifest section (an older .dig, or a private store) returns `result: null`, never an error (store-format backwards-compatibility rule); a capsule not held at all reports -32004, matching dig.fetchRange's miss code. - dig.stage now sets FinalizeOptions.include_public_manifest for PUBLIC stores, matching the digstore CLI's own commit rule (a new required field added by the digstore rev bump below). - Bumped the pinned digstore rev (dig-node-core + dig-wallet, kept in sync) to the tip carrying the public-manifest format extension, and added digstore-compiler as a direct dependency for extract_data_section_blob (already a transitive dep via digstore-stage). - dig-node-service: meta.rs catalogue entry flipped to served: "local" with an updated summary; doc comments in server.rs/rpc.rs updated to match. - SPEC.md/README.md/USER_JOURNEY.md updated to describe the new local behavior instead of the old passthrough alias. - Tests: dig_get_manifest_returns_embedded_manifest_json_when_present, dig_get_manifest_returns_null_when_section_absent, dig_get_manifest_reports_unavailable_when_capsule_not_held, dig_get_manifest_rejects_malformed_params_without_touching_disk. The existing passthrough_alias_methods_are_method_not_found_on_the_node test and the openrpc_drift_guard suite were updated/re-verified against the new classification. Verified: cargo fmt --all -- --check, cargo clippy --workspace --all-targets --all-features -- -D warnings, and cargo test --workspace all green (dig-node-core 193, dig-node-service 92, dig-wallet 71, dig-runtime 3 - zero failures). Co-Authored-By: Claude <noreply@anthropic.com>
This was referenced Jul 20, 2026
MichaelTaylor3d added a commit
that referenced
this pull request
Aug 2, 2026
) * feat(dig-node-core): add pure tier-0 knapsack selector (#1988) Add tier0_selector: a pure, deterministic module that picks which speculative-precache candidates are worth keeping under a small sub-budget of DIG_NODE_CACHE_CAP, reusing relevance()/RelevanceValue/ should_displace from the child-1 relevance module rather than reimplementing scoring or hysteresis. - Candidate{size_bytes, relevance} + select_within_budget: O(n log n) greedy by value-density (relevance/size), never exceeding the given budget. Deliberately greedy, not DP: at GiB-scale budgets a DP table is disproportionate, and greedy under-fills by at most one candidate's size. - tier0_budget_bytes: TIER0_BUDGET_FRACTION (0.10) of the whole cache cap, pure arithmetic over a caller-supplied cap (no I/O in this module). - should_displace_tier0: thin named wrapper over should_displace with the selector's own DEFAULT_HYSTERESIS_MARGIN, so a marginally-better candidate cannot displace an incumbent and thrash fetch/evict/refetch. Also fixes two comment-accuracy nits left over from child #1 in relevance.rs: the xor-weight-dominance comment now states it exceeds only the ATTACKER-GAMEABLE secondaries (scarcity+demand+recency=0.85), excluding the operator-controlled pin/pin_adjacent bonuses; the proximity-map comment now says `hi128 / u128::MAX` to match what the code actually divides by. No behaviour change. Bumps: crates/dig-node-core 0.27.0 -> 0.28.0 (MINOR, new export) and the root workspace [workspace.package].version (the released dig-node binary) 0.76.0 -> 0.77.0 (MINOR), with Cargo.lock regenerated to match. Co-Authored-By: Claude <noreply@anthropic.com> * docs(dig-node): spec the tier-0 knapsack selector + fix rustfmt (#1988) Add SPEC.md §7.10b normativley describing tier0_selector's contract (the 10% sub-budget, greedy-by-density selection, and the hysteresis reuse of should_displace) alongside §7.10a's relevance model. Also applies the rustfmt ordering rustfmt itself wants for the new `pub mod tier0_selector;` declaration in lib.rs. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
MichaelTaylor3d pushed a commit
that referenced
this pull request
Aug 2, 2026
…tion (#1987) Produce the tier-0 precache CANDIDATE SET (epic #1934, child 2/7): random keyspace sampling picks which neighbourhoods to probe, and a pure multi-peer reconciliation admits a content key only on an M-of-N distinct-peer quorum, deriving each key's provider count from the lower median (never a liar's inflated max). Feeds relevance()'s RelevanceInputs; does not score, select, or fetch (children #1/#3/#4). - reconcile(): per-peer dedup then quorum + median aggregate (pure, no I/O) - sample_keyspace_points(): deterministic under an injected SplitMix64 RNG - sample_candidates(): async composition over a NeighbourhoodProbe seam - SPEC.md §7.10c documents the sampling + quorum contract (normative) Bumps dig-node-core 0.28.0 -> 0.29.0 and workspace 0.77.0 -> 0.78.0 (MINOR). Co-Authored-By: Claude <noreply@anthropic.com>
MichaelTaylor3d added a commit
that referenced
this pull request
Aug 2, 2026
…tion (#153) * feat(dig-node): scaffold DHT candidate sampling (#1987) Co-Authored-By: Claude <noreply@anthropic.com> * feat(dig-node): DHT candidate sampling + anti-Sybil quorum reconciliation (#1987) Produce the tier-0 precache CANDIDATE SET (epic #1934, child 2/7): random keyspace sampling picks which neighbourhoods to probe, and a pure multi-peer reconciliation admits a content key only on an M-of-N distinct-peer quorum, deriving each key's provider count from the lower median (never a liar's inflated max). Feeds relevance()'s RelevanceInputs; does not score, select, or fetch (children #1/#3/#4). - reconcile(): per-peer dedup then quorum + median aggregate (pure, no I/O) - sample_keyspace_points(): deterministic under an injected SplitMix64 RNG - sample_candidates(): async composition over a NeighbourhoodProbe seam - SPEC.md §7.10c documents the sampling + quorum contract (normative) Bumps dig-node-core 0.28.0 -> 0.29.0 and workspace 0.77.0 -> 0.78.0 (MINOR). Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
This was referenced Aug 3, 2026
MichaelTaylor3d added a commit
that referenced
this pull request
Aug 6, 2026
…tadata (#185) Closes dig_ecosystem#2145. Hardens two DoS gaps on the anonymous, internet-reachable public-manifest read path. The ticket's fix#1 (the manifest-memo byte budget: MANIFEST_MEMO_MAX_BYTES=32 MiB + MANIFEST_ENTRY_MAX_BYTES=4 MiB + LRU) was already on main; this closes the two remaining gaps: 1. cache.clear now drains the process-lifetime manifest memo (it previously left it intact, so an operator couldn't reclaim it). 2. dig.getMetadata gets a 3 MiB response ceiling. It was the only public-tier read with no response cap (~200 B in -> up to ~100 MB out). A whole JSON object can't be windowed, so an oversized section is refused with a bounded -32015 METADATA_TOO_LARGE. read_metadata_manifest_json returns the rendered Arc<str> so get_metadata checks the raw length BEFORE parsing, so the oversized body is never parsed + re-serialized. Normal metadata serves byte-identically. Both gates PASS: loop-reviewer (correctness) + loop-security (the getMetadata ceiling is a hard pre-parse bound on every render path, no bypass; cache.clear is control-token-gated, not anonymous-reachable; every arm fails closed). The -32015 error is catalogued in docs.dig.net (companion PR #66). Non-gating follow-ups: #2217 (error-code drift), #2218 (gitnexus runner), #2219 (encoded-length pre-filter). root [workspace.package].version 0.99.1 -> 0.99.2 (patch, fix). Closes #2145 Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
#176 Phase C:
dig.getManifestmoves from a blind-passthrough alias to a LOCAL read, resolving the normalizedPublicManifest(.digdata-section id 13, Phase A — already merged in digstore's store-libs) embedded in a held capsule's compiled module.dig-node-core: newNode::get_manifest+read_public_manifest_blocking(runs onspawn_blocking, mirroringserve_local_blocking). Params{store_id, root}(a capsule identifier —storeId:rootHash, matchingdig.getAvailability/dig.fetchRange's shape); noretrieval_keyneeded since the manifest is PUBLIC, unencrypted data.result= the manifest JSON ({schema_version, entries:[...]}, byte-identical toPublicManifest::to_json)..dig, or a private store) →result: null— never an error (store-format §5.1 backwards-compatibility rule).-32004(same codedig.fetchRangeuses on a miss).store_id/root→-32602before any filesystem access.dig.stagenow setsFinalizeOptions.include_public_manifestfor PUBLIC stores (matching the digstore CLI's own commit rule) — required because the digstore rev bump below adds this as a new field on that struct.dig-node-coreanddig-wallet, kept in sync per the existing "bump together" contract) to the tip carrying the public-manifest format extension (digstore-core/-store/-compiler/-stageetc., nodigstore-guestchanges — the vendored guest wasm did not need rebuilding). Addeddigstore-compileras a direct dependency ofdig-node-coreforextract_data_section_blob(a pure wasm-binary-format parse — no wasmtime instantiation needed to read this public section).dig-node-service:meta.rscatalogue entry flippedserved: "passthrough"→"local"with an updated summary; doc comments inserver.rs/rpc.rsupdated to match (a clarifying comment was added torpc.rsexplainingRoute::Passthroughthere is a request-normalization classification, independent of local-vs-relay serving).SPEC.md/README.md/USER_JOURNEY.mdupdated to document the new local behavior (including a new SPEC §5.5.1) instead of describing it as a blind-passthrough alias.Version bump
dig-node-core0.1.0→0.2.0anddig-node-service0.3.0→0.4.0— minor: a new capability (dig.getManifestis now resolved by the node instead of always relaying to the upstream), fully backwards compatible (existing callers of the method still get an answer; everything else is unchanged).Tests (TDD — written before the implementation)
dig_get_manifest_returns_embedded_manifest_json_when_presentdig_get_manifest_returns_null_when_section_absentdig_get_manifest_reports_unavailable_when_capsule_not_helddig_get_manifest_rejects_malformed_params_without_touching_diskBoth manifest-presence tests build a real compiled
.digmodule viadigstore_stage::stage_and_compile(not a mock) so the data-section extraction + decode is exercised end-to-end. The pre-existingpassthrough_alias_methods_are_method_not_found_on_the_nodetest was updated to dropdig.getManifestfrom the passthrough-alias list, and theopenrpc_drift_guardsuite (which dispatches every catalogued method through the realhandle_rpc) was re-verified green against the newserved: "local"classification.How verified
cargo fmt --all -- --check— clean.cargo clippy --workspace --all-targets --all-features -- -D warnings— zero warnings.cargo test --workspace— all green:dig-node-core193 passed,dig-node-service92 passed (incl. the 5openrpc_drift_guardtests + the 25server.rsintegration tests),dig-wallet71 passed,dig-runtime3 passed. Zero failures.digstore-*crate across the whole workspace (dig-wallethad its own independent, now-stale pin — bumped it too to keep the documented "same rev as the node" invariant).Note: this repo's current required PR checks are "Lint commit messages" (commitlint) and "Check version increment" only — there is no
ci.ymlrunningcargo test/clippy/fmtas a required gate yet. I ran all of the above locally per the ecosystem-wide contract; adding a dedicated CI workflow for it would be a separate, repo-wide infra task.