Skip to content

feat(collateral): declare control.collateral.buffer, the recommended $DIG buffer + funding state - #36

Merged
MichaelTaylor3d merged 1 commit into
mainfrom
loop/35-collateral-buffer
Aug 28, 2026
Merged

feat(collateral): declare control.collateral.buffer, the recommended $DIG buffer + funding state#36
MichaelTaylor3d merged 1 commit into
mainfrom
loop/35-collateral-buffer

Conversation

@MichaelTaylor3d

@MichaelTaylor3dMichaelTaylor3d commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

DO NOT MERGE — gate round pending.Closes#35.
Producer DIG-Network/dig-node#389 · consumer DIG-Network/dig-app#306 (PR #311). Release-first: both wait on this publishing.

The shape decision: a SEPARATE method, control.collateral.buffer

Extending control.collateral.requirement's result was the cheaper option and I rejected it.

  • The two figures have different provenance. The requirement is consensus-derived: every node derives the same per-store figure for an epoch from the same census, and a mirror advertisement counts only if it posts at least that. The buffer is local in four separate ways — the (owner, store, root) pairs this node serves, an operator preference (the safety margin), this node's unreclaimed collateral, and a horizon this node chose. One result object carrying both would put a node's private position inside the value clients read as the network's price.
  • 0.23.0 already decided this case, one field earlier.CollateralRequirementResult's own doc says "The margin is deliberately absent here… the requirement is a consensus-derived value every node derives identically; the margin is a local operator preference that MUST NOT be a consensus input." The buffer is that same argument with more force: it is built from the margin, so anything the margin was excluded for applies to it a fortiori.
  • The unknown taxonomies do not overlap. The requirement can fail for four census reasons; the buffer can additionally fail because the served set, the reclaim state, or the balance is unreadable — and those have different remedies. Extending the existing result would either collapse three new missing-fact answers into not_censused (false and unactionable) or make an already-tagged union carry two independent unknown axes.
  • The cheapness argument is also weaker than it looks.CollateralRequirementResult::Known is exhaustively destructured by consumers; adding eight fields to it is a change every existing reader sees, whereas a new method is invisible to them. Additive-in-SemVer is not the same as additive-in-blast-radius.

Argued against, honestly: two round trips instead of one, and epoch/protocol_version/required_per_store_dig_base_units are duplicated across the two results. I accepted the duplication because those three fields are what make the buffer checkable against the requirement a client may already hold — a buffer quoted against a different epoch than the one on screen is exactly the disagreement worth being able to see.

The declared payload — every unit named

CollateralBufferResult::Known ({"state":"known", …}):

fieldunit
epochepoch number, one-based
protocol_versioncollateral protocol version that computed the epoch
funding_stateshort_now | dangerously_low | below_recommended_buffer | funded
recommended_buffer_dig_base_unitsDIG base units — authoritative total
spendable_dig_base_unitsDIG base units — the balance the verdict was reached against
pairs_served_by_this_nodecount of (owner, store, root) pairs this node serves
required_per_store_dig_base_unitsDIG base units, pre-margin
margin_bpbasis points (100 = +1%), never converted
overlap_dig_base_unitsDIG base units — unreclaimed transition overlap
escalation_headroom_dig_base_unitsDIG base units
horizon_epochscount of future epochs
escalation_ceiling_microsmillionths (1_000_000 = x1.0), a worst case

$DIG has 3 decimals, so one base unit is 0.001 DIG. Not a mojo — that is XCH's base unit at 1e-12 XCH, nine orders of magnitude away. The margin stays in basis points because that is the unit dig_mirror_collateral::apply_safety_margin takes and the one dig-app SPEC.md §3.7b fixes; it is never converted here.

The total is authoritative, the terms are the working. Documented, and specced as a MUST, that a client renders the node's total rather than re-adding the terms — the rounding lives in the node's arithmetic.

The horizon is a required wire field, not an implied default. Escalation is bounded at +12.5%/epoch (UP_STEP_DENOM = 8) and compounds — x1.12 at one epoch, x1.60 at four, x4.62 at thirteen — so horizon_epochs and escalation_ceiling_micros both fail to decode when absent (test below). escalation_ceiling_micros is documented as a worst case, not a forecast: in the controller's dead band the multiplier does not move at all.

The funding state is carried, not re-derived. Every input is in the payload, so a client could threshold it — and two that did would pick different thresholds and disagree. is_shortfall() names the two states in which an epoch is actually uncovered; below_recommended_buffer is deliberately excluded and specced as a readout that MUST NOT be a recurring notification, because a healthy node sits there much of the time. Whether a state interrupts somebody stays the client's call.

Two constants published alongside: DEFAULT_BUFFER_HORIZON_EPOCHS = 4 and ESCALATION_UP_STEP_DENOM = 8, restated rather than imported for the same reason DEFAULT_SAFETY_MARGIN_BP is (dig-mirror-collateral sits at the same crate level; same-level deps are forbidden). DEFAULT_BUFFER_HORIZON_EPOCHS is documented as recognisable, never substitutable — a reader that used it in place of a payload it failed to read would state a claim the node never made.

How unknown is expressed

A tagged variant, {"state":"unknown","reason":…}, with no representable numeric field — not an Option<u64> beside a reason. On .requirement a fabricated zero reads as a free requirement; here it reads as no buffer needed, and an operator acting on it posts nothing and loses the epoch. Four reasons, each naming a different missing fact with a different remedy: requirement_unknown, served_set_unknown, reclaim_state_unknown, balance_unknown. requirement_unknown deliberately does not restate §4.2e's four census reasons — a copy would drift — and points at control.collateral.requirement instead.

Blast radius checked

gitnexus was not used: the global index is the superproject's and a per-worktree analyze on a 12k-line crate is not worth its cost here; blast radius was measured by grep + direct read of every site the catalog touches, and stated rather than assumed.

  • ControlMethod is #[non_exhaustive] and every consumer match must already carry a _ arm, so a new variant is additive by construction. Nothing existing was edited: name(), category() and ALL gained one arm/entry each; routing() falls through its _ => Routing::Owned and requires_auth() through !is_open_read(), both correct without change.
  • ControlHandler gained one required method, which is the only breaking-shaped part of the diff. It is the established pattern for every method this crate has added (each of 0.21.0–0.23.0 did the same) and is why this is a minor on 0.x; the sole implementor in-repo is the KAT mock, and dig-node adopts it as the producer half of the epic.
  • No existing type, field, wire name or predicate was modified.git diff touches only additions plus one superseded SPEC paragraph (below) and the version.
  • The one doc contradiction found and fixed: SPEC §4.2e ended with "A client assembling a recommended-$DIG buffer reads the held-store count from control.hostedStores.list" — the exact client-side derivation Carry the recommended $DIG buffer and the node's funding state — the app cannot derive either #35 exists to forbid. Rewritten to point at control.collateral.buffer.

Tests — and the proof each one is load-bearing

Five new KATs. Each was proved by mutating only the thing it claims to pin and confirming the test goes red (source committed first; restore verified against git status):

mutationresult
#[serde(default)] on horizon_epochs (the optional-with-fallback reading)killed by a_buffer_without_its_horizon_does_not_decode
CollateralBuffer dropped from ControlMethod::ALLkilled by the_buffer_method_is_in_the_catalog_categorised_and_gated
below_recommended_buffer folded into is_shortfall()killed by exactly_two_funding_states_mean_an_epoch_is_uncovered
a defaulted recommended_buffer_dig_base_units added to the Unknown variantkilled by an_unknown_buffer_carries_a_reason_and_never_a_number
the handler answers the census stores count as its served setkilled by the_buffer_read_returns_this_nodes_served_set_not_the_census_count

Two fixture notes, because both are places a green would otherwise have been false:

  • The mock answers 12 pairs while its requirement handler answers a census stores of 4_200. Both are u64 counts in the same domain, so a handler wired to the census figure compiles and looks plausible; only a fixture where the two differ can see the substitution — which is the "confident, badly wrong number" the ticket names.
  • The catalog test resolves via ControlMethod::from_name, not by naming the variant.from_name searches ALL, so a variant added to the enum and forgotten in ALL — invisible to discovery and to every sweep, while compiling everywhere — fails here. A direct CollateralBuffer.name() assertion would have passed.

Instrument honesty: the first mutation run reported all five as SURVIVED. The cause was the harness, not the tests — cargo test --lib <name> -- --exact needs the full kats:: path, so zero tests ran and the run exited 0. The harness now asserts a non-zero test count and reports INSTRUMENT BROKEN rather than SURVIVED for an empty run. A uniform verdict across five unrelated mutations is an instrument claim, not a code claim.

cargo test: 173 lib + 9 doc tests pass. cargo clippy --all-targets -- -D warnings and cargo fmt --check clean.

Version + deps

0.23.0 → 0.24.0 (minor). Additive new capability: a new ControlMethod variant on a #[non_exhaustive] enum, new result/params types, and one new ControlHandler method. No existing wire name, field, type or predicate changed, so nothing published breaks for a reader; the added trait method is why this is not a patch. Tag is cut by the merge — no hand-pushed tag.

§2.4b: this crate declares no dig-* and no chia-* dependencies at allserde, serde_json, async-trait, semver, futures only, all already at their latest majors. Nothing to bring forward, and no git = … dep to check against a branch. Verified by reading Cargo.toml, not from memory.

Crate tarball hygiene

exclude = [".claude", "AGENTS.md", "CLAUDE.md"] added. cargo package --list: 27 entries → 25, with AGENTS.md and CLAUDE.md gone (no .claude dir exists here; the entry is prophylactic). .github/, cliff.toml and commitlint.config.mjs deliberately left in. 0.23.0 ships those two files permanently — a crates.io version is immutable — and this stops 0.24.0 becoming another instance. The ecosystem-wide gate is https://github.com/DIG-Network/dig_ecosystem/issues/3182 and is not taken here.

Stated plainly, not verified

  • Nothing consumes this yet. dig-node#389 is building the calculation and dig-app#306 renders it; that this contract fits both is an argued claim from the ticket, not a measured one.
  • escalation_ceiling_micros = 1_601_806 for a four-epoch horizon appears only in the KAT mock and in prose. It is consistent with (9/8)^4, but this crate declares the contract and does not compute escalation — deliberately, to avoid a rival implementation of dig-mirror-collateral's arithmetic. Nothing here checks the producer's ceiling against the crate's.
  • DEFAULT_BUFFER_HORIZON_EPOCHS = 4 is my judgement, not a value read from dig-mirror-collateral (no such constant exists there). The rationale is in its doc-comment; the node reports what it actually used regardless.

…$DIG buffer + funding state
Declares a SEPARATE collateral method rather than extending
`control.collateral.requirement`. The requirement is consensus-derived and
identical on every node; the buffer is local — it rests on the pairs THIS node
serves, on an operator preference (the safety margin), on unreclaimed collateral,
and on a horizon this node chose. 0.23.0 kept the margin out of `.requirement`
on exactly that grounds, and folding a local position into a consensus result
would make one node's preferences look like the network's price.
The payload carries the recommended buffer, the funding state, and the working
behind both: pairs served by this node, the pre-margin per-store requirement, the
margin in basis points, the unreclaimed transition overlap, the escalation
headroom, the spendable balance the verdict was reached against, and the HORIZON
plus the compounded worst-case ceiling that headroom assumed. Amounts are DIG
base units (3 decimals); a mojo is XCH's unit and is nine orders of magnitude
away. The horizon is required on the wire: escalation compounds at up to +12.5%
per epoch, so the same buffer over a different horizon is a different claim.
`unknown` is a tagged variant with a named reason and no representable number,
because a zero here reads as "no buffer needed" — an operator acting on it posts
nothing and loses the epoch. The funding state is the node's verdict rather than
a threshold each client re-derives; `below_recommended_buffer` is a readout and
never a recurring notification.
Also excludes AGENTS.md and CLAUDE.md from the published tarball (27 -> 25
entries): a crates.io version is immutable, so 0.23.0 carries them permanently.
Closes#35
@MichaelTaylor3d
MichaelTaylor3dforce-pushed the loop/35-collateral-buffer branch from 64ffa45 to aea2298CompareAugust 28, 2026 11:11
@MichaelTaylor3d

Copy link
Copy Markdown
ContributorAuthor

IN PROGRESS — not the verdict. Gate on aea2298.

Confirmed so far, by direct read of the diff at this head:

  • Worktree cut detached at aea2298; git status --porcelain empty; single commit, author michael@michaeltaylor.dev.
  • CollateralBufferResult::Unknown { reason } (src/results.rs:3089) is structurally incapable of carrying a number — the variant has exactly one field, of enum type. Not a convention, a type property.
  • is_shortfall() (src/results.rs:2977) matches only ShortNow | DangerouslyLow; BelowRecommendedBuffer is excluded and documented as a readout.
  • CollateralBuffer added to name(), category(), description() and ALL; routing()/requires_auth() fall through by design.

Still running: test suite, independent re-execution of two mutations, the empty-run guard, cargo package --list diff, the SPEC sweep for surviving client-side-derivation prose.

@MichaelTaylor3dMichaelTaylor3d left a comment

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PASS

Head audited: aea2298ff834298b599c2a7fe0db94bfd17b5209 (resolved from the remote; unchanged throughout).

One correctness gate. No gating findings. Draft, unmerged, no code edited.

Shape decision — judged, not rubber-stamped

Argument 3 holds, and it is the strongest of the three. I read the census taxonomy at src/results.rs:2820-2828. All four reasons — NotCensused, BehindFinalityDepth, RecordUnreadable, NoChainSource — are facts about the census record and the chain view. RecordUnreadable is scoped to the census record specifically, not to arbitrary local state. None can express "I cannot enumerate my own served set", "I cannot read my reclaim state", or "I cannot read my balance": those are facts about node-local bookkeeping that the census taxonomy has no vocabulary for. Collapsing them into not_censused would be false, and also unactionable — the three have different remedies (hosted-store view, transition bookkeeping, chain source). The three reasons are genuinely unrepresentable. Separate taxonomy is correct.

The destructuring claim is verified.src/kats.rs:4636 destructures CollateralRequirementResult::Knownexhaustively, with no .. — all seven fields bound. Adding eight fields there is a compile break for every reader written that way, while a new method is invisible. "Additive in SemVer is not additive in blast radius" is accurate as stated.

The duplicated triple earns its keep.epoch / protocol_version / required_per_store_dig_base_units are what let a client detect that a buffer is quoted against a different epoch, or computed under a different protocol version, than the requirement already on screen. Without them the two results are mutually uncheckable and a stale buffer is indistinguishable from a current one — on a money surface. The two-round-trip cost is real, correctly disclosed, and buys a detectable disagreement.

Honesty properties — all four verified

  • unknown cannot hold a number, structurally.CollateralBufferResult::Unknown { reason: CollateralBufferUnknownReason } (src/results.rs:3089) has exactly one field, of enum type. There is no Option<u64> and no representable numeric state — a type property, not a convention. kats.rs additionally asserts obj.len() == 2 and that no value is a number or null, across every reason.
  • horizon_epochs and escalation_ceiling_micros are required on the wire. Proven by execution, not by reading: I applied #[serde(default)] to horizon_epochs in my own worktree and the test went red at kats.rs:4820 — "horizon_epochs must be REQUIRED". The test itself decodes payloads with each field stripped, with the intact payload as a control.
  • The third funding state cannot be swept into an alert via is_shortfall().src/results.rs:2977 matches only ShortNow | DangerouslyLow. I folded BelowRecommendedBuffer in; the test went red at kats.rs:4899. The KAT writes the shortfall set out literally rather than deriving it from is_shortfall, so it pins the SET and not the implementation's opinion of itself.
  • Units: no conflation found. Every amount field carries _dig_base_units; the two non-amount numerics are pairs_served_by_this_node and horizon_epochs, both named as counts; margin_bp is basis points and escalation_ceiling_micros millionths, both stated at every mention. "mojo" appears in this diff only in the sentence distinguishing it (results.rs:3019, method.rs:588). pairs_served_by_this_node is documented and tested as this node's own set, with the mock answering 12 against a census stores of 4_200 so a substitution is visible.

Instrument verified, not trusted

  • The disclosed harness failure reproduces exactly.cargo test --lib a_buffer_without_its_horizon_does_not_decode -- --exact returns 0 passed; 173 filtered out and exit 0. With kats:: prefixed: 1 passed. The lane's self-report is accurate.
  • I re-executed three of the five mutations independently (not the two required): #[serde(default)] on horizon_epochs → killed; BelowRecommendedBuffer folded into is_shortfall() → killed; CollateralBuffer removed from ControlMethod::ALL → killed at kats.rs:4781, confirming the from_name route catches the omission an ALL-iterating test is structurally blind to. Verdicts were not uniform across the run — each named its own distinct assertion, which is what a code claim looks like rather than an instrument claim.
  • Tree clean.git status --porcelain empty after every restore and at completion; HEAD still aea2298. Suite: 173 lib + 9 doc pass.

Realization 1 — the fix is complete

SPEC §4.2e's superseded paragraph is gone; the replacement (SPEC.md:968-971) points at control.collateral.buffer and pairs_served_by_this_node. I swept the whole SPEC and README rather than spot-checking: hostedStores now appears only in the four catalog table rows (SPEC.md:90-93), never as a buffer input, and no surviving sentence points a client at a locally-assembled buffer or store count. §4.2f restates the prohibition as a MUST.

Also confirmed

  • Every new doc sentence checks out against code in this same diff.DEFAULT_SAFETY_MARGIN_BP exists (params.rs:1779); control.wallet.watched is gated for the cited reason (method.rs:584); (9/8)^4 x 1e6 = 1_601_806.64, so the quoted 1_601_806 is that value truncated, and x1.125 / x4.6236 match "x1.12" and "x4.62". The ceiling is stated as a worst case with the dead-band caveat at every mention — results.rs, method.rs:588, traits.rs, SPEC.md:1011, README.md. No sentence implies a forecast.
  • SemVer minor is right. The new required ControlHandler::collateral_buffer method is a break for any external implementor, and on 0.x a minor is the semver-incompatible bump (^0.23 does not accept 0.24). Patch would have been wrong. Matches the 0.21-0.23 precedent.
  • exclude drops exactly two files and no source. I re-ran cargo package --list with and without the key and diffed: removed = AGENTS.md, CLAUDE.md; added = none; all eight src/*.rs present. 25 entries.
  • DEFAULT_BUFFER_HORIZON_EPOCHS = 4 is defensible. 28 days at ~x1.60 headroom sits between one epoch (x1.12 — one bad epoch from dangerously_low) and thirteen (x4.62 — locking capital against a ceiling the controller reaches only by escalating every epoch, which the dead band makes unlikely). More importantly the constant is not load-bearing: the horizon the node used always travels in the payload, both fields are required on the wire, and the doc marks it recognisable-never-substitutable. A wrong default here cannot produce a wrong claim.
  • Merge preconditions by name: all four required contexts present and SUCCESSFormat / Clippy / Build / Docs, Coverage (>=80% lines), Lint commit messages, Check version increment. unresolvedReviewThreads=0. RESULT: BLOCKED on draft=true alone, correct for a gate round. Single commit, author michael@michaeltaylor.dev — the configured identity, not a fabricated one.

Coverage I did NOT achieve

  • No producer or consumer was exercised. dig-node#389 and dig-app#306 do not exist yet, so the contract's fit with both is argued, not measured. Inherent to release-first; stated, not a defect.
  • I re-executed three of five mutations, not five. The two I did not re-run are the Unknown-variant defaulted-field mutation and the census-substitution handler mutation. Both have assertions I inspected and judged load-bearing by reading (obj.len() == 2 over every reason; assert_ne!(pairs, census_stores) with differing fixture values), but I did not watch them go red myself.
  • No gitnexus index. Blast radius was measured by grep plus direct read of every catalog site, matching the lane's own method. A call-graph tool could surface an external destructuring site this crate cannot see.
  • I did not verify the crates.io publish workflow itself, only the tarball contents it would ship.
  • escalation_ceiling_micros truncation (...806 vs ...806.64) is noted, not raised: it appears only in the mock and prose, this crate computes no escalation by design, and nothing here checks a producer's ceiling. Pre-accepted in the brief.

No inline threads opened — there is nothing gating to anchor one to.

@MichaelTaylor3d
MichaelTaylor3d marked this pull request as ready for review August 28, 2026 13:37
@MichaelTaylor3d
MichaelTaylor3d merged commit 0c54703 into mainAug 28, 2026
9 checks passed
@MichaelTaylor3d
MichaelTaylor3d deleted the loop/35-collateral-buffer branch August 28, 2026 13:37
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Carry the recommended $DIG buffer and the node's funding state — the app cannot derive either

1 participant

@MichaelTaylor3d