Uh oh!
There was an error while loading. Please reload this page.
feat(dig-node): DHT candidate sampling + anti-Sybil quorum reconciliation - #153
Merged
Conversation
Co-Authored-By: Claude <noreply@anthropic.com>
…tion (#1987) Produce the tier-0 precache CANDIDATE SET (epic #1934, child 2/7): random keyspace sampling picks which neighbourhoods to probe, and a pure multi-peer reconciliation admits a content key only on an M-of-N distinct-peer quorum, deriving each key's provider count from the lower median (never a liar's inflated max). Feeds relevance()'s RelevanceInputs; does not score, select, or fetch (children #1/#3/#4). - reconcile(): per-peer dedup then quorum + median aggregate (pure, no I/O) - sample_keyspace_points(): deterministic under an injected SplitMix64 RNG - sample_candidates(): async composition over a NeighbourhoodProbe seam - SPEC.md §7.10c documents the sampling + quorum contract (normative) Bumps dig-node-core 0.28.0 -> 0.29.0 and workspace 0.77.0 -> 0.78.0 (MINOR). Co-Authored-By: Claude <noreply@anthropic.com>
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes DIG-Network/dig_ecosystem#1987 — child 2/7 of the tiered relevance-cache epic (dig_ecosystem#1934). Produces the candidate set that children 1/3 (already merged) rank + select.
What changed
New pure leaf module
crates/dig-node-core/src/dht_sampling.rs(pub mod dht_sampling;in lib.rs). No existing symbol edited.Fits the real dig-dht v0.11.1 API (investigated first):
find_providers(&ContentId)returns records already aggregated across peers — no per-peer view, so it can't support quorum. The per-peer view isDhtService::provider_snapshot(max_keys)(the RLY-009 shape): a Kademlia node holds records for keys near its own peer-id, so one peer's snapshot = onePeerObservation. That is what reconciliation consumes.find_node/known_closest(accept anyKey), matching the random-sampling intent.Candidate.size_hint: Option<u64>(real size arrives at fetch, child 4).Anti-Sybil design:
reconcile()— dedups per peer first (one peer listing a key many times = one vote), admits a content-id only on ≥ M distinct peers (DEFAULT_QUORUM_MIN_PEERS = 3, overridable viaQuorumPolicy), and setsknown_provider_count= lower median across reporting peers (never max) so a single inflating/deflating liar can't set it. §7.10a[1,32]clamp remains the final defense downstream.sample_keyspace_points()— deterministic under an injectedKeyspaceRngseam; ships a self-contained non-cryptoSplitMix64(no new dep; explicitly not for keys/nonces).sample_candidates()— async composition reconciling all probed regions together (whole-round quorum); the concrete network probe is aNeighbourhoodProbetrait seam, mocked in tests, with live wiring deferred to child 4 (per scope).Scope: candidate discovery + reconciliation only — no selection (child 3), no fetch/prefetch loop (child 4), no live-cache wiring.
How verified
cargo fmt --all -- --check→ 0;cargo clippy -p dig-node-core --all-targets --all-features -- -D warnings→ 0;cargo build -p dig-node-core→ 0.cargo test -p dig-node-core --lib→ 546 passed, 9 failed; the 9 are the known socket-bind sandbox failures (peer::tests::*/seams::dig_peer::net/control_peer_status), unrelated — the 13 newdht_samplingtests all pass (quorum M-1 reject / exactly-M admit, single-peer junk drop, per-peer dedup, median-not-max on inflation AND deflation, size-hint median + absence, deterministic + spread sampling, zero-K, async multi-region). Real CI runners bind sockets, so the 9 run there.Version
dig-node-core0.28.0 → 0.29.0; root[workspace.package].version0.77.0 → 0.78.0 (the gate reads root);Cargo.lockregenerated.SPEC.md§7.10c adds the sampling + quorum contract. No SYSTEM.md change (dig-dht consumed read-only; the provider-snapshot shape already exists).Follow-ups filed for child 4 (#1989)
provider_snapshot(only the local method + the relay RLY-009) — child 4's concreteNeighbourhoodProbeneeds a new peer RPC or relay-routed reads.size_hint: Nonecandidates need size resolved before child 3's knapsack (which requiressize_bytes) can use them — child 4 must resolve size at/ before selection.Generated by Claude Code