Skip to content

feat(dig-node): DHT candidate sampling + anti-Sybil quorum reconciliation - #153

Merged
MichaelTaylor3d merged 2 commits into
mainfrom
feat/dht-candidate-sampling
Aug 2, 2026
Merged

feat(dig-node): DHT candidate sampling + anti-Sybil quorum reconciliation#153
MichaelTaylor3d merged 2 commits into
mainfrom
feat/dht-candidate-sampling

Conversation

@MichaelTaylor3d

Copy link
Copy Markdown
Contributor

Closes DIG-Network/dig_ecosystem#1987 — child 2/7 of the tiered relevance-cache epic (dig_ecosystem#1934). Produces the candidate set that children 1/3 (already merged) rank + select.

What changed

New pure leaf module crates/dig-node-core/src/dht_sampling.rs (pub mod dht_sampling; in lib.rs). No existing symbol edited.

Fits the real dig-dht v0.11.1 API (investigated first):

  • find_providers(&ContentId) returns records already aggregated across peers — no per-peer view, so it can't support quorum. The per-peer view is DhtService::provider_snapshot(max_keys) (the RLY-009 shape): a Kademlia node holds records for keys near its own peer-id, so one peer's snapshot = one PeerObservation. That is what reconciliation consumes.
  • Arbitrary keyspace probing uses find_node/known_closest (accept any Key), matching the random-sampling intent.
  • Snapshots carry no sizeCandidate.size_hint: Option<u64> (real size arrives at fetch, child 4).

Anti-Sybil design:

  • Pure reconcile() — dedups per peer first (one peer listing a key many times = one vote), admits a content-id only on ≥ M distinct peers (DEFAULT_QUORUM_MIN_PEERS = 3, overridable via QuorumPolicy), and sets known_provider_count = lower median across reporting peers (never max) so a single inflating/deflating liar can't set it. §7.10a [1,32] clamp remains the final defense downstream.
  • sample_keyspace_points() — deterministic under an injected KeyspaceRng seam; ships a self-contained non-crypto SplitMix64 (no new dep; explicitly not for keys/nonces).
  • sample_candidates() — async composition reconciling all probed regions together (whole-round quorum); the concrete network probe is a NeighbourhoodProbe trait seam, mocked in tests, with live wiring deferred to child 4 (per scope).
  • Residual model stated honestly in module docs + SPEC: M minted identities still clear the bar — quorum raises the cost, the XOR-distance relevance (child 1) is the ungameable anchor.

Scope: candidate discovery + reconciliation only — no selection (child 3), no fetch/prefetch loop (child 4), no live-cache wiring.

How verified

  • cargo fmt --all -- --check → 0; cargo clippy -p dig-node-core --all-targets --all-features -- -D warnings → 0; cargo build -p dig-node-core → 0.
  • cargo test -p dig-node-core --lib546 passed, 9 failed; the 9 are the known socket-bind sandbox failures (peer::tests::* / seams::dig_peer::net / control_peer_status), unrelated — the 13 new dht_sampling tests all pass (quorum M-1 reject / exactly-M admit, single-peer junk drop, per-peer dedup, median-not-max on inflation AND deflation, size-hint median + absence, deterministic + spread sampling, zero-K, async multi-region). Real CI runners bind sockets, so the 9 run there.

Version

dig-node-core 0.28.0 → 0.29.0; root [workspace.package].version 0.77.0 → 0.78.0 (the gate reads root); Cargo.lock regenerated. SPEC.md §7.10c adds the sampling + quorum contract. No SYSTEM.md change (dig-dht consumed read-only; the provider-snapshot shape already exists).

Follow-ups filed for child 4 (#1989)

  1. There is currently no peer-to-peer RPC to fetch a remote node's provider_snapshot (only the local method + the relay RLY-009) — child 4's concrete NeighbourhoodProbe needs a new peer RPC or relay-routed reads.
  2. size_hint: None candidates need size resolved before child 3's knapsack (which requires size_bytes) can use them — child 4 must resolve size at/ before selection.

Generated by Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
…tion (#1987)
Produce the tier-0 precache CANDIDATE SET (epic #1934, child 2/7): random
keyspace sampling picks which neighbourhoods to probe, and a pure multi-peer
reconciliation admits a content key only on an M-of-N distinct-peer quorum,
deriving each key's provider count from the lower median (never a liar's
inflated max). Feeds relevance()'s RelevanceInputs; does not score, select,
or fetch (children #1/#3/#4).
- reconcile(): per-peer dedup then quorum + median aggregate (pure, no I/O)
- sample_keyspace_points(): deterministic under an injected SplitMix64 RNG
- sample_candidates(): async composition over a NeighbourhoodProbe seam
- SPEC.md §7.10c documents the sampling + quorum contract (normative)
Bumps dig-node-core 0.28.0 -> 0.29.0 and workspace 0.77.0 -> 0.78.0 (MINOR).
Co-Authored-By: Claude <noreply@anthropic.com>
@MichaelTaylor3d
MichaelTaylor3d merged commit e353743 into mainAug 2, 2026
15 checks passed
@MichaelTaylor3d
MichaelTaylor3d deleted the feat/dht-candidate-sampling branch August 2, 2026 22:14
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@MichaelTaylor3d@claude