Uh oh!
There was an error while loading. Please reload this page.
feat: throttle outgoing bandwidth and redirect saturated serves to peers - #3
Merged
Merged
Conversation
Add a configurable outgoing-bandwidth cap (DIG_NODE_MAX_OUTGOING_BYTES_PER_SEC, unlimited by default) on the node's serve path. When serving would push this node's outgoing traffic over the cap, it resolves alternate holders via the DHT and answers with the SAME CONTENT_REDIRECT (-32008) shape and hop-cap discipline the existing #165 redirect-on-miss mechanism uses, instead of serving over-budget. When no alternate holder is known (or no P2P engine is attached), it serves the request anyway - a throttle that redirects when it can, never one that drops or fails closed. Wired into the three surfaces that return locally-held resource bytes: the dig.getContent local-first serve, dig.fetchRange, and the mTLS peer range-stream (the busiest node-to-node egress path multi-source downloaders fan across). New bandwidth module with a fixed-1s-window throttle (unit-tested), plus integration tests proving cap-exceeded-with-provider redirects, no-provider and no-P2P-engine graceful fallback, under-cap non-interference, and hop-cap reuse. SPEC.md gains a new outgoing-bandwidth-throttle section, the DIG_NODE_MAX_OUTGOING_BYTES_PER_SEC variable, and the -32008 error-catalogue entry (previously undocumented).
Uh oh!
There was an error while loading. Please reload this page.
MichaelTaylor3d added a commit
that referenced
this pull request
Jul 10, 2026
…ers (#3) Add a configurable outgoing-bandwidth cap (DIG_NODE_MAX_OUTGOING_BYTES_PER_SEC, unlimited by default) on the node's serve path. When serving would push this node's outgoing traffic over the cap, it resolves alternate holders via the DHT and answers with the SAME CONTENT_REDIRECT (-32008) shape and hop-cap discipline the existing #165 redirect-on-miss mechanism uses, instead of serving over-budget. When no alternate holder is known (or no P2P engine is attached), it serves the request anyway - a throttle that redirects when it can, never one that drops or fails closed. Wired into the three surfaces that return locally-held resource bytes: the dig.getContent local-first serve, dig.fetchRange, and the mTLS peer range-stream (the busiest node-to-node egress path multi-source downloaders fan across). New bandwidth module with a fixed-1s-window throttle (unit-tested), plus integration tests proving cap-exceeded-with-provider redirects, no-provider and no-P2P-engine graceful fallback, under-cap non-interference, and hop-cap reuse. SPEC.md gains a new outgoing-bandwidth-throttle section, the DIG_NODE_MAX_OUTGOING_BYTES_PER_SEC variable, and the -32008 error-catalogue entry (previously undocumented). Co-Authored-By: Claude <noreply@anthropic.com>
MichaelTaylor3d added a commit
that referenced
this pull request
Jul 10, 2026
…ers (#3) Add a configurable outgoing-bandwidth cap (DIG_NODE_MAX_OUTGOING_BYTES_PER_SEC, unlimited by default) on the node's serve path. When serving would push this node's outgoing traffic over the cap, it resolves alternate holders via the DHT and answers with the SAME CONTENT_REDIRECT (-32008) shape and hop-cap discipline the existing #165 redirect-on-miss mechanism uses, instead of serving over-budget. When no alternate holder is known (or no P2P engine is attached), it serves the request anyway - a throttle that redirects when it can, never one that drops or fails closed. Wired into the three surfaces that return locally-held resource bytes: the dig.getContent local-first serve, dig.fetchRange, and the mTLS peer range-stream (the busiest node-to-node egress path multi-source downloaders fan across). New bandwidth module with a fixed-1s-window throttle (unit-tested), plus integration tests proving cap-exceeded-with-provider redirects, no-provider and no-P2P-engine graceful fallback, under-cap non-interference, and hop-cap reuse. SPEC.md gains a new outgoing-bandwidth-throttle section, the DIG_NODE_MAX_OUTGOING_BYTES_PER_SEC variable, and the -32008 error-catalogue entry (previously undocumented). Co-Authored-By: Claude <noreply@anthropic.com>
MichaelTaylor3d added a commit
that referenced
this pull request
Aug 2, 2026
Piece #3 (receiver) + #4 (holder) of store-melt propagation, over spy-testable seams (MeltChain/MeltCache/MeltBroadcast) with the fail-closed NC-9 gate: only a held, on-chain-confirmed melt deletes; Err/Unknown never deletes; held-check precedes any chain read; the TombstoneSet CAS bounds each node to one rebroadcast so the epidemic terminates. All 8 adversarial tests green. Refs #1316 Co-Authored-By: Claude <noreply@anthropic.com>
MichaelTaylor3d pushed a commit
that referenced
this pull request
Aug 2, 2026
…tion (#1987) Produce the tier-0 precache CANDIDATE SET (epic #1934, child 2/7): random keyspace sampling picks which neighbourhoods to probe, and a pure multi-peer reconciliation admits a content key only on an M-of-N distinct-peer quorum, deriving each key's provider count from the lower median (never a liar's inflated max). Feeds relevance()'s RelevanceInputs; does not score, select, or fetch (children #1/#3/#4). - reconcile(): per-peer dedup then quorum + median aggregate (pure, no I/O) - sample_keyspace_points(): deterministic under an injected SplitMix64 RNG - sample_candidates(): async composition over a NeighbourhoodProbe seam - SPEC.md §7.10c documents the sampling + quorum contract (normative) Bumps dig-node-core 0.28.0 -> 0.29.0 and workspace 0.77.0 -> 0.78.0 (MINOR). Co-Authored-By: Claude <noreply@anthropic.com>
MichaelTaylor3d added a commit
that referenced
this pull request
Aug 2, 2026
…tion (#153) * feat(dig-node): scaffold DHT candidate sampling (#1987) Co-Authored-By: Claude <noreply@anthropic.com> * feat(dig-node): DHT candidate sampling + anti-Sybil quorum reconciliation (#1987) Produce the tier-0 precache CANDIDATE SET (epic #1934, child 2/7): random keyspace sampling picks which neighbourhoods to probe, and a pure multi-peer reconciliation admits a content key only on an M-of-N distinct-peer quorum, deriving each key's provider count from the lower median (never a liar's inflated max). Feeds relevance()'s RelevanceInputs; does not score, select, or fetch (children #1/#3/#4). - reconcile(): per-peer dedup then quorum + median aggregate (pure, no I/O) - sample_keyspace_points(): deterministic under an injected SplitMix64 RNG - sample_candidates(): async composition over a NeighbourhoodProbe seam - SPEC.md §7.10c documents the sampling + quorum contract (normative) Bumps dig-node-core 0.28.0 -> 0.29.0 and workspace 0.77.0 -> 0.78.0 (MINOR). Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
MichaelTaylor3d added a commit
that referenced
this pull request
Aug 3, 2026
Piece #3 (receiver) + #4 (holder) of store-melt propagation, over spy-testable seams (MeltChain/MeltCache/MeltBroadcast) with the fail-closed NC-9 gate: only a held, on-chain-confirmed melt deletes; Err/Unknown never deletes; held-check precedes any chain read; the TombstoneSet CAS bounds each node to one rebroadcast so the epidemic terminates. All 8 adversarial tests green. Refs #1316 Co-Authored-By: Claude <noreply@anthropic.com>
MichaelTaylor3d added a commit
that referenced
this pull request
Aug 3, 2026
Piece #3 (receiver) + #4 (holder) of store-melt propagation, over spy-testable seams (MeltChain/MeltCache/MeltBroadcast) with the fail-closed NC-9 gate: only a held, on-chain-confirmed melt deletes; Err/Unknown never deletes; held-check precedes any chain read; the TombstoneSet CAS bounds each node to one rebroadcast so the epidemic terminates. All 8 adversarial tests green. Refs #1316 Co-Authored-By: Claude <noreply@anthropic.com>
MichaelTaylor3d added a commit
that referenced
this pull request
Aug 4, 2026
Piece #3 (receiver) + #4 (holder) of store-melt propagation, over spy-testable seams (MeltChain/MeltCache/MeltBroadcast) with the fail-closed NC-9 gate: only a held, on-chain-confirmed melt deletes; Err/Unknown never deletes; held-check precedes any chain read; the TombstoneSet CAS bounds each node to one rebroadcast so the epidemic terminates. All 8 adversarial tests green. Refs #1316 Co-Authored-By: Claude <noreply@anthropic.com>
MichaelTaylor3d added a commit
that referenced
this pull request
Aug 4, 2026
Piece #3 (receiver) + #4 (holder) of store-melt propagation, over spy-testable seams (MeltChain/MeltCache/MeltBroadcast) with the fail-closed NC-9 gate: only a held, on-chain-confirmed melt deletes; Err/Unknown never deletes; held-check precedes any chain read; the TombstoneSet CAS bounds each node to one rebroadcast so the epidemic terminates. All 8 adversarial tests green. Refs #1316 Co-Authored-By: Claude <noreply@anthropic.com>
MichaelTaylor3d added a commit
that referenced
this pull request
Aug 4, 2026
Piece #3 (receiver) + #4 (holder) of store-melt propagation, over spy-testable seams (MeltChain/MeltCache/MeltBroadcast) with the fail-closed NC-9 gate: only a held, on-chain-confirmed melt deletes; Err/Unknown never deletes; held-check precedes any chain read; the TombstoneSet CAS bounds each node to one rebroadcast so the epidemic terminates. All 8 adversarial tests green. Refs #1316 Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
DIG_NODE_MAX_OUTGOING_BYTES_PER_SEC, unlimited by default) on the node's serve path (newbandwidthmodule,OutgoingThrottle: a fixed-1s-window byte budget).download::NodeContent::find_providers) and answers with the sameCONTENT_REDIRECT(-32008) shape + hop-cap discipline the existing fix(dig-node): gate inbound-demand pull on XOR-proximity admission (anti-amplification) #165 redirect-on-miss mechanism already uses — extending redirect from "not held" to "held but saturated," per dig_ecosystem issue fix(dig-node): correct Discord invite (imposter link -> official) #30.dig.getContent's local-first serve (cold cache hit + post-§21-sync hit),dig.fetchRange, and the mTLS peer range-stream (stream_range— the busiest node-to-node egress path, since multi-source downloaders fan byte-ranges across it).providers[].addressesinherit the DHT's IPv6-first candidate ordering (§5.2) — the throttle does not reorder them.Blast radius (gitnexus impact analysis, §2.0)
Ran
npx gitnexus@latest analyzeon the repo, thengitnexus impact(CLI) on the touched symbols before editing:Nodestruct: exactly 5 direct construction sites, all incrates/dig-node-core/src/lib.rs(thefrom_envproduction constructor + 4 test constructors) — all updated with the newoutgoing_throttlefield.fetch_range_frame: 2 real call sites (lib.rsdispatch,peer.rs::NodeResponder::stream_range) — both updated.stream_range: a trait method (PeerRpcResponder) with independent per-type impls (StubResponder, integration-test responders) — editing the realNodeResponderimpl does not affect the others.redirect_error_object: 1 existing caller — my new call sites are additive.dig-node-service/dig-runtime): both only consumeNodeopaquely + the free functionshandle_rpc/cache_cap_bytes/cache_used_bytes— none of the signatures touched here cross the crate boundary. Risk contained todig-node-core(lib.rs,peer.rs, newbandwidth.rs).gitnexus detect-changes --scope compare --base-ref mainpost-commit reports "critical" risk / 71 affected flows — this is theNodestruct's inherent connectivity (touching its private-field literal necessarily touches the 5 constructor sites and everything the twogap_fill/chain_watchend-to-end tests exercise), not a functional regression: the full workspace test suite (below) is green.How verified
TDD throughout (failing test first, then implementation):
bandwidth.rs: 8 new unit tests for the throttle's window/env-parsing logic (pure, no sleeps —*_atvariants take the instant explicitly).lib.rs: 6 new integration tests over the realdig.getContent/dig.fetchRangedispatch — cap-exceeded+provider→redirect, no-provider fallback, no-P2P-engine fallback, under-cap non-interference, and hop-cap reuse (verified genuinely RED before wiring the dispatch, then GREEN after).peer.rs: 2 new tests over the realNodeResponder::stream_range(viatokio::io::duplex+serve_one_stream) — over-cap+provider→redirect frame, no-provider→streams anyway.cargo fmt --all -- --check: clean.cargo clippy --workspace --all-targets -- -D warnings: clean.cargo build --workspace: clean.cargo test -p dig-node-core(lib +tests/dht_integration.rs+tests/peer_network.rs+ doctests): 209 + 8 + 2 passed, 0 failed.cargo test --workspace(dig-node-service, dig-runtime, dig-wallet): all green, no regressions.Docs / SPEC
SPEC.md: new §17 "Outgoing-bandwidth throttle and redirect-on-saturation" (config, accounting, serve-path integration, hop-cap reuse, graceful fallback);DIG_NODE_MAX_OUTGOING_BYTES_PER_SECadded to the §3.2 variables table;-32008 CONTENT_REDIRECTadded to the §10 error catalogue (previously undocumented despite already being emitted by fix(dig-node): gate inbound-demand pull on XOR-proximity admission (anti-amplification) #165 redirect-on-miss).README.md: new env-var row.Version bump
crates/dig-node-core/Cargo.toml:0.2.0→0.3.0(minor — a new, backwards-compatible, opt-in capability:feat:, matching this repo's precedent, e.g.0.1.0→0.2.0for thedig.getManifestfeature). The workspace rootCargo.tomlhas no[package]/[workspace.package]version field, so theensure-version-incrementgate is a no-op there per its own documented behavior.Test plan (for reviewers)
cargo test -p dig-node-coregreen (209 lib + 8 + 2 integration)cargo test --workspacegreencargo fmt --all -- --checkcleancargo clippy --workspace --all-targets -- -D warningscleancargo build --workspacecleanCloses/references DIG-Network/dig_ecosystem#30.