IxVM: cheaper shard checking via an addr-first kernel, plus soundness hardening - #529

Merged
johnchandlerburnham merged 19 commits into
mainfrom
ap/ixvm4
Aug 3, 2026
Merged

IxVM: cheaper shard checking via an addr-first kernel, plus soundness hardening#529
johnchandlerburnham merged 19 commits into
mainfrom
ap/ixvm4

Conversation

@arthurpaulino

@arthurpaulinoarthurpaulino commented Jul 31, 2026

Copy link
Copy Markdown
Member

Motivation

Make sharded environment checking cheap. A large environment is partitioned across shards, and each shard proves its own constants well-typed against a thin-frontier assumption cutoff, so every constant is checked in exactly one shard and cross-shard assumption trees stay minimal.

Delivering that required migrating the IxVM Aiur kernel from a positional constant table to a content-addressed, lazily-faulted design — and then auditing the result, which turned up five soundness holes with working exploits, including three separate routes to proving False.

Sharded environment checking

A shard proves CheckEnv{root, assumptions}: root is a merkle root over the shard's owned constants, assumptions a merkle root over the thin frontier — the direct out-of-owned walk edges. The kernel's env_walk starts from the owned set, checks each reachable constant, and stops at a frontier member (assumed well-typed by its owning shard: not re-checked, not recursed into).

  • Sub-frontier constants are never re-checked here; they are covered transitively by the frontier members' own shard claims. This is the cost win over checking each constant's full closure: no cross-shard duplication, and the assumption tree is one layer deep instead of the whole closure.
  • Strict membership: a walk-reachable constant in neither owned nor the frontier is a bad partition and hard-errors, so a shard cannot silently drop coverage of its own closure.
  • Byte scope ≠ check scope: delta unfolding reads through the frontier, so a witness ships the full dependency closure of owned even though only the strict-membership set is checked. The witness IOBuffer is built in Rust (aiur_ixvm_witness) rather than Lean, removing the per-byte boxing that dominated shard wall time.

Addr-first kernel

  • KExprNode.Const / Proj key on a 32-byte content Addr, not a numeric position; KConstantInfoInduct/Ctor/Rec carry a (block_addr, ind_idx) parent pair instead of indices. The positional top: List<KCI> table threaded through the kernel is gone.
  • get_ci(addr) is the sole resolver, Aiur-memoized per addr, faulting constant bytes lazily from an IOBuffer. load_verified_constant re-derives blake3 over the bytes and asserts key == addr; the io_get_info idx/len are unconstrained witness and are never branched on.
  • IOBuffer channels: 0 claim, 1 assumption tree, 2 constant bytes, 3 Defn reducibility hint (advisory), 4 blob bytes. The kernel derives blob-vs-constant from Expr context, so no wire discriminator ships. The two sets are not disjoint — one address can be read both ways — so each channel is seeded independently.
  • Kernel/Inductive.lean and Kernel/Primitive.lean deleted (~6.6k lines); inductive validation absorbed into Kernel/Check.lean, primitives split into Kernel/Klimbs.lean (bignum gadgets) and Kernel/NatPrim.lean (Nat/Str addr dispatch). New Kernel/InferOnly.lean breaks the DefEq↔Infer cycle. Ingress.lean collapsed 1700→298 lines.

Recursor validation

  • build_rec_type reconstructs a recursor's canonical type and asserts it def-eq to the declared type, on both the kind-aware and address-only lookup paths. Without it a recursor could declare ... -> False while its rules still reconstructed correctly.
  • spec_params are lowered from the major-premise frame into the recursor-param frame at extraction: the root cause of 17 nested/aux recursor reconstruction failures, and why the canonical check accepted a superset.
  • k_flag and parent-inductive-shape checks hoisted out of the attacker-selectable is_aux gate; the canonical rules comparison, which formerly never ran, now runs for every recursor.

Pre-existing criticals

IDFix
P1blob-vs-constant derived from Expr context instead of a prover-answered io_get_info probe, retiring the ch-2 len=0 sentinel; a prover can no longer mark a walk root or owned leaf "skip me"
P2Nat.div n 0 pins the quotient to zero (was prover-chosen)
P3klimbs_sub/le corrected on operands with trailing zero limbs; div/mod hints normalized before use
P4is_large_eliminator field-index arithmetic parenthesized (Aiur - is right-associative)
P5Reveal recursor rules assert the claimed ruleIdx equals its position

Robustness

  • L5: UTF-8 string-literal decode validates continuation-byte ranges and per-length overlong minimums, so distinct byte sequences (e.g. overlong NUL C0 80 vs 00) no longer decode to the same codepoint and def-eq as equal.
  • L6: run_claim dispatches on the full tag4 size (was the low byte only), matching Rust Claim::get.
  • L8: the K-recursor synth gate checks the major's inferred inductive against the recursor's own, not against itself, so I.rec no longer strips a major of a different inductive.

Aiur assert_eq! messages and circuit cleanup

  • assert_eq!(a, b, "why") gains an optional diagnostic label, threaded Source.Term → Typed → Simple → Concrete → Bytecode.Op → Rust Op::AssertEq(_, _, Option<String>) → ExecError::AssertEqMismatch. The message is diagnostic-only and never enters the circuit; all ~122 IxVM assert sites are labeled.
  • Unreachable _ => assert_eq!(0, 1) match arms removed: an unmatched runtime value already aborts (the kernel's reject), so the extra arm only widened the circuit.
  • Address equality is a full 32-byte content comparison — never a pointer-value compare, which is unsound for distinct-pointer/same-data — packed as five base-256 field limbs behind a byte-0 prefilter.

Witness construction

witness_scope reuses Env::bfs_closure, which already follows refs, Prj → block, and the derived Muts → member/ctor projection edges, so the wrappers get_ci synthesizes ship as forward edges of a single O(closure) walk with no reverse scan of the env. Primitive addresses are seeded as additional walk roots, because def-eq delta-unfolds primitive bodies whose refs are reachable from nowhere else. The Lean seeder computes the same scope by the same rule, so the two cannot drift.

bench-typecheck's constants metric now reads check_const's unique query count — exactly the constants typechecked — instead of the shipped byte scope, which is a superset and inflated throughput.

Soundness audit

Eight parallel read-only audits covered the pipeline end-to-end (compile, Ixon serde, ingress/convert, type-theory core, declaration checking, claim/witness layer, primitives, entry points), under the zkDSL threat model: a prover authors the data and pays for its own trace, so aborts and rejections are not threats and only a wrong answer counts. Thirty findings; every claim relayed here was re-verified against the code before acting on it, and several agent claims were corrected or withdrawn in the process.

Fixed, with a working exploit

Each of these shipped with a fixture that was red before the fix and green after, paired with a minimal-pair control so the verdict is attributable to one variable.

HoleWhat it bought an attacker
Self-referential definitionstheorem bad : False := bad typechecked. A standalone Defn was handed a recur slot naming itself, Rec erases to an ordinary Const, and k_infer reads a Const's declared type without checking it — so the body discharged its own type by citing itself. Now restricted to unsafe/partial, which is where Lean permits self-reference and where safe code cannot reach
Nested-inductive positivityThe nested branch dropped the parameter arguments and descended into the nested inductive's constructors without substituting them, so the occurrence being hunted was never present. Host | mk : Inner Host → Host was accepted for a negative Inner. Admits False
Recursor rules compared under the prover's contextcompare_rules handed the stored rule's own binder domains to k_is_def_eq as the comparison context. Retyping them to a Prop makes both sides infer to that Prop, proof irrelevance fires, and a recursor with its computation rules swapped between constructors was accepted
Non-canonical computed Nat literalsString.utf8ByteSize built a KLimbs as one unchecked byte, and klimbs_div_mod pinned the quotient's value but not its digits. Since klimbs_eq compares limbs rather than values, the kernel had the polarity backwards: Nat.beq (String.utf8ByteSize "") 0 = false was accepted while the true equation was rejected
Tag0/Tag2 overlong payloadsThree readers, three answers on the same bytes: the circuit discarded bytes past the eighth, the Lean host folded them back in (its shift is mod 64), Rust rejected. One address denoted different constants to different readers

Fixed as defense-in-depth

No exploit demonstrated for these; each commit says so in its first line. They remove a dependence on some other check happening to hold.

  • Struct-eta type equality.try_eta_struct omitted the infer(t) ≟ infer(s) side condition all three references carry, degenerating to t ≡ S.mk … (Proj S i t) … for any t of any type.
  • Recursor rule labels.cidx selects a rule at reduction time but was validated only positionally; nf decides how iota splits the constructor's arguments and was never checked against the constructor. Equivalents existed only in dead code.
  • Iota constructor identity. Rules were selected by index alone, so any inductive's constructor with a matching index fired one. Unreachable through a claim — inference k_checks every argument first — but reduction should not depend on that.
  • Universe cache context.compile_univ memoized Level → Univ while a Level::Param compiles to a position in univ_params, and the cache outlives a constant. Now keyed by (level, univ_params_key), matching what collect_expr_tables already does one level up.

Entry points and coverage

  • verify_const / verify_check / verify_check_env are DEBUG entrypoints that trust their dependencies, and they shipped in the production verifying key alongside verify_claim — leaving ix verify's funcIdx check as the only barrier, untested, with no equivalent in the recursive verifier. The production toplevel is now pruned to verify_claim alone, so such a proof cannot exist under that VK. Zero FFT cost.
  • bench-typecheck reported "status": "ok" and exit 0 when verification failed, and dropped thrown constants without marking them failed.
  • shardsCover skipped constants whose bytes do not parse — assigning them to no shard, counting them as neither owned nor unowned, and still including them in "covers all N consts". Now a hard failure.

Test guards that could not fail

Four of these were found by auditing the tests themselves, and they matter because they were the safety net for everything above.

  • prim-addrs asserted the literal true. Keyed by address, so a permuted primitive table passed — precisely the defect its docstring claimed to catch. Now the address must resolve to the constant the function's name claims; verified by temporarily swapping nat_add_addr to Nat.mul's bytes and watching it go red.
  • The arena negative corpus counted any abort as a correct rejection, so a fixture aborting on invalid IO key — never reaching the code under test — scored as a pass. That is the same failure mode that made two exploit fixtures in this series pass for the wrong reason before it was caught.
  • The shard-pipeline SKIP was a passing test conflating "target absent" with "partition selected zero constants".
  • A --ignored filter naming no runner executed nothing and returned 0, so a typo'd suite name reported success having run no tests.

Adversarial Ixon test suite

New Tests/Ix/IxVM/Exploits.lean, 19 cases, wired into the ixvm suite. The kernel-arena corpus attacks the layer above: its bad_* fixtures are raw Lean.ConstantInfo declarations that skip the elaborator, but they are still compiled by rsCompileEnvFFI, so the Ixon they produce is always well-formed. This suite authors Ixon.Constant values directly, and storeRawAt goes one level lower still, storing hand-authored byte strings — needed because every value-level fixture is canonical by construction and so cannot express a malformed encoding.

Each case records the false statement acceptance would buy, so a failure reads as a security statement. Rejection reasons are printed, which earned its keep repeatedly: fixtures that "passed" via invalid IO key without reaching the checker were caught this way, as was a fixture that silently corrupted nothing because non-mutual recursors are standalone constants rather than Muts members.

Testing

  • lake test -- --ignored ixvm green.
  • cargo test --release across all crates: 1195 passed, 0 failed. This also repaired shard_check_env_claim_is_thin_frontier, red since the claim walk moved to positive constant-use because its fixture's Exprs never referenced their own refs.
  • New prim-addrs parity suite; codegen ↔ bytecode parity gate green; arena negative corpus (77 bad_*) all reject, now only via genuine in-kernel refusals.
  • FFT-cost pins re-pinned per change. Cumulative effect is small (Nat.add_comm 42_747_927 → 42_986_025, +0.6% over the whole series); the largest single contributors are the nested-positivity substitution and the struct-eta guard.
  • ix codegen regenerated whenever the Aiur kernel changed, so the committed Rust kernel never lags the source the parity gate compares against.

Known limitations

Recorded rather than silently dropped.

  • Call-site surgery is not covered by the value pin. The compiler rewrites calls to aux-generated constants — permuting arguments, dropping some, replacing the head and its universe list — and the plans driving it live only in ConstantMeta, which prune_to_closure_anon excludes. So a verified claim says the term at address X is well-typed, not that term is the Lean declaration you wrote; the equivalence rests on the rewrite being faithful, checked today only by decompile round-trip tests. Closing it is a claim-format decision, not a patch.
  • Assumption cycles are not rejected, and a conditional Check claim naming its own target is accepted. That is the specified semantics — some root means "conditional on every leaf", so the claim asserts A → A, vacuous rather than false — and the obligation sits with whoever discharges assumptions. Sharded CheckEnv is not exposed, since shardClaimDigest recomputes the claim from (env, blocks). Pinned as ACCEPT so the boundary cannot move silently.
  • Alpha-collapse comparators ignore safety, so two members agreeing on kind/lvls/typ/value and disagreeing on safety collapse. Deliberately not fixed: a prover authors the environment directly and never goes through the compiler, so hardening it stops no attacker, while adding safety to the ordering would move every address and refusing the collapse would put a hard error in the compiler on the strength of an unverified assumption.
  • klimbs_le / klimbs_eq are in the logical trusted base, not just the performance path: Nat.decLe mints Decidable.isTrue with a proof term whose correctness is asserted rather than checked, so one wrong verdict yields False.
  • check_inductive_shape_ctors omits validate_expr_well_scoped / k_ensure_sort / assert_safety that check_const's Ctor arm performs; only an entirely unreferenced constructor escapes those.

@arthurpaulino
arthurpaulinoforce-pushed the ap/ixvm4 branch 4 times, most recently from 06e0489 to 559ced9CompareAugust 1, 2026 00:53
… hardening
Make sharded environment checking cheap. A large environment is partitioned
across shards, and each shard proves its own constants well-typed against a
thin-frontier assumption cutoff, so every constant is checked in exactly one
shard and cross-shard assumption trees stay minimal. Delivering that required
migrating the IxVM Aiur kernel from a positional constant table to a
content-addressed, lazily-faulted design -- and closing the soundness
regressions that migration introduced along with several pre-existing holes.
Sharded environment checking
- A shard proves `CheckEnv{root, assumptions}`: root is a merkle over the
shard's owned constants, assumptions a merkle over the THIN FRONTIER --
the direct out-of-owned walk edges. The kernel's env_walk starts from the
owned set, checks each reachable constant, and stops at a frontier member
(assumed well-typed by its owning shard, not re-checked, not recursed).
- Sub-frontier constants are never re-checked here; they are covered
transitively by the frontier members' own shard claims. This is the cost
win over checking each constant's full closure: no cross-shard duplication,
and the assumption tree is one layer deep instead of the whole closure.
- Strict membership: a walk-reachable constant in neither owned nor the
frontier is a bad partition and hard-errors, so a shard cannot silently
drop coverage of its own closure.
- Byte scope != check scope: delta unfolding reads through the frontier, so
a witness ships the full walk closure of owned even though only the
strict-membership set is checked. The witness IOBuffer is built in Rust
(aiur_ixvm_witness) rather than Lean, removing the per-byte boxing that
dominated shard wall time.
Addr-first kernel
- KExprNode.Const/Proj key on a 32-byte content Addr, not a numeric
position; KConstantInfo Induct/Ctor/Rec carry a (block_addr, ind_idx)
parent pair instead of indices. The positional `top: List<KCI>` table
threaded through the kernel is gone.
- get_ci(addr) is the sole resolver, Aiur-memoized per addr, faulting
constant bytes lazily from an IOBuffer. load_verified_constant re-derives
blake3 over the bytes and asserts key == addr; the io_get_info idx/len
are unconstrained witness and are never branched on.
- IOBuffer channels: 0 claim, 1 assumption tree, 2 constant bytes, 3 Defn
reducibility hint (advisory), 4 blob bytes. An address is on ch2 iff a
constant and ch4 iff a blob; the kernel derives which from Expr context
(Str/Nat -> blob, Ref/Prj -> constant), so no wire discriminator ships.
- Kernel/Inductive.lean and Kernel/Primitive.lean deleted (~6.6k lines);
inductive validation absorbed into Kernel/Check.lean, primitives split
into Kernel/Klimbs.lean (bignum gadgets) and Kernel/NatPrim.lean (Nat/Str
addr dispatch). New Kernel/InferOnly.lean breaks the DefEq<->Infer cycle.
Ingress.lean collapsed 1700->298 lines. V1/V2 naming removed throughout.
Recursor validation (R1, R2, R3)
- build_rec_type reconstructs a recursor's canonical type and asserts it
def-eq to the declared type, on both the kind-aware and address-only
lookup paths. Without it a recursor could declare `... -> False` while
its rules still reconstructed correctly.
- spec_params are lowered from the major-premise frame into the
recursor-param frame at extraction: the root cause of 17 nested/aux
recursor reconstruction failures, and why the canonical check accepted a
superset.
- k_flag and parent-inductive-shape checks hoisted out of the
attacker-selectable `is_aux` gate; the canonical rules comparison, which
formerly never ran, now runs for every recursor.
Pre-existing criticals (P1-P5)
- P1: blob-vs-constant is derived from Expr context instead of a prover-
answered io_get_info probe, retiring the ch-2 len=0 sentinel; a prover
can no longer mark a walk root or owned leaf "skip me".
- P2: Nat.div n 0 pins the quotient to zero (was prover-chosen).
- P3: klimbs_sub/le corrected on operands with trailing zero limbs; div/mod
hints normalized before use.
- P4: is_large_eliminator field-index arithmetic parenthesized (Aiur `-` is
right-associative).
- P5: Reveal recursor rules assert the claimed ruleIdx equals its position.
Robustness (L5, L6, L8)
- L5: UTF-8 string-literal decode validates continuation-byte ranges and
per-length overlong minimums, so distinct byte sequences (e.g. overlong
NUL C0 80 vs 00) no longer decode to the same codepoint and def-eq as
equal. Roughly zero added FFT cost.
- L6: run_claim dispatches on the full tag4 size (was the low byte only),
matching Rust Claim::get.
- L8: the K-recursor synth gate checks the major's inferred inductive
against the recursor's own (via se_parent_addr), not against itself, so
I.rec no longer strips a major of a different inductive.
Aiur assert_eq! messages + circuit cleanup
- assert_eq!(a, b, "why") gains an optional diagnostic label, threaded
Source.Term -> Typed -> Simple -> Concrete -> Bytecode.Op -> Rust
Op::AssertEq(_, _, Option<String>) -> ExecError::AssertEqMismatch. The
message is diagnostic-only and never enters the circuit; all ~122 IxVM
assert sites are labeled.
- Unreachable `_ => assert_eq!(0, 1)` match arms removed: an unmatched
runtime value already aborts (the kernel's reject), so the extra arm only
widened the circuit.
- address equality is a full 32-byte content comparison (never a pointer-
value compare, which is unsound for distinct-pointer/same-data), packed
as five base-256 field limbs (7+7+7+7+4) behind a byte-0 prefilter and
short-circuiting on the first mismatched group.
Crate reorganization
- PrimAddrs / reserved_marker_name moved verbatim from
crates/kernel/src/primitive.rs to crates/common/src/prim_addrs.rs (kernel
re-exports), so witness builders reach the primitive address table
without an ix-kernel -> ixon -> ix-kernel cycle.
- witness_scope reuses Env::bfs_closure -- the env's own dependency-closure
pass, which already follows refs, Prj->block, and the derived
Muts->member/ctor projection edges -- so the wrappers the kernel's get_ci
synthesizes ship as forward edges of a single O(closure) walk, with no
reverse scan of the whole env. Primitive-address seeding is unioned on
top: primitives are fabricated during reduction and need not appear in a
target's ref closure, so a closure-only scope aborts with `invalid IO
key`.
Tests
- New prim-addrs parity suite (Lean<->Aiur primitive address table).
- Codegen<->bytecode parity gate restored, green (198/198).
- FFT-cost pins re-pinned to current behavior; arena negative-fixture
corpus (77 bad_* constants) all reject.
…onstants
closureFromBase now walks the derived Muts->member/ctor projection
wrapper addresses as forward edges (projWrapperAddr: blake3 over the
serialized proj-only Constant, compile's construction), so closureFrom
and buildShardCheckEnvWitness drop their O(env) reverse scans that
parsed every env constant per witness build. Primitive addresses are
seeded bytes-only after the walk instead of being pushed onto the
worklist, so their transitive closures no longer inflate the byte scope
(a delta-unfold into an unshipped primitive body aborts with invalid IO
key rather than proving anything wrong).
bench-typecheck's constants metric now reads check_const's unique query
count (memoized per (ci, addr) = exactly the constants typechecked)
instead of closureFrom.size, which counted the shipped byte scope.
check_const is resolved up front and both lookups fail with IO.userError
instead of panicking.
Experiment: lake exe bench-typecheck --ixe seed.ixe --consts
Nat.add_comm --execute-only (seed.ixe = ix compile Tests.lean --consts
Nat.add_comm) reports constants=42, matching check_const's height in
lake exe ix check Nat.add_comm; fft-cost unchanged at 42747927.
The claim walk decided which of a constant's `refs` to recurse into by
SKIPPING every index the constant's Exprs used as a blob payload
(`Str(i)` / `Nat(i)`). That rule is unsound: the Exprs are authored by
the prover, and `refs` is one index space shared by `Ref`, `Prj`, `Str`
and `Nat`, so marking an index suppresses the walk of that index used as
a constant everywhere else in the same constant.
The marking need not even be live. `blob_idxs_of` traversed `sharing` in
full, but conversion reaches a `sharing` entry only through an
`Expr.Share` node, and nothing validates the array. A dead `Nat(i)`
entry is therefore free: it is never converted, never typechecked, and
needs no blob on channel 4. It just hides `refs[i]` from the checker.
`k_infer` then reads that dependency's DECLARED type without ever
checking it, so the kernel accepts a constant whose transitive closure
is ill-typed -- a wrong answer, not an abort. Both claim variants were
affected; for CheckEnv the skip happens before `env_walk` is entered, so
the strict owned-membership assert never fires either.
Replace the family with `const_idxs_*`: collect the indices `Ref(i, ..)`
and `Prj(i, ..)` use, and walk `refs[i]` iff `i` is among them. `Rec`
resolves through `recur_addrs` and a projection's block comes from the
ConstantInfo payload, both walked separately, so the positive set is
complete. Same single traversal as before, so no added cost. The failure
direction is now safe: over-approximating walks an address that faults
an unseeded key and aborts, which only costs a dishonest prover.
Adversarial Ixon test infrastructure
- New `Tests/Ix/IxVM/Exploits.lean`, wired into the ixvm suite. The
kernel-arena corpus attacks the layer above this one: its `bad_*`
fixtures are raw `Lean.ConstantInfo` declarations that skip the
elaborator, but they are still compiled by `rsCompileEnvFFI`, so the
Ixon they produce is always well-formed. This suite authors
`Ixon.Constant` values directly, which is what a prover can do.
- Each `ExploitCase` records the false statement acceptance would buy,
so a failure reads as a security statement. Rejection reasons are
printed: a fixture that aborts before reaching the mechanism under
attack is indistinguishable from a kernel that correctly refuses it,
and the first run of this fixture passed for exactly that wrong reason
(`invalid IO key` from an unseeded channel-3 hint, now seeded by
`storeAt`).
- The exploit ships with two controls that differ from it minimally: one
checking the smuggled dependency alone, one checking the same target
without the dead `sharing` entry. Both reject via the intended
in-kernel assert, so the exploit's acceptance was attributable to the
`sharing` entry alone.
- Two further cases probe what the fix does NOT cover. An index used as
both a `Nat` literal and a `Ref` — reachable because blob and constant
addresses share one un-domain-separated blake3 space and
`load_verified_blob` parses nothing — is now walked and rejected,
though by channel fault rather than type error, which is the safe
direction. A conditional Check claim naming its own target as its own
assumption IS accepted, and is pinned that way: `some root` means
"conditional on every leaf being well-typed", so the claim asserts
`A -> A` and is vacuous rather than false. The obligation sits with
whoever discharges assumptions, which must reject such cycles; sharded
CheckEnv is not exposed, since `shardClaimDigest` recomputes the claim
from (env, blocks).
Primitive closures are walked again
- The previous commit seeded primitive addresses as bytes only, without
walking their closures. That is too aggressive: def-eq delta-unfolds
primitive BODIES, and a body's refs are reachable from nowhere else in
the scope, so the `strAppend*` arena fixtures aborted with `invalid IO
key`. Both seeders now pass the primitives as walk ROOTS
(`closureFromRoots` in Lean, `bfs_closure` over target+primitives in
Rust); the Rust side had the same gap latently, since only the Lean
path had arena coverage.
- The removal of the O(env) reverse wrapper scan, which was that
commit's actual win, is unaffected.
Tests
- `lake test -- --ignored ixvm` green. FFT pins re-pinned across the 66
kernel-check entries and the shard pipeline; costs move in both
directions (e.g. `Eq.rec` 1_249_383 -> 1_249_094, `Vector.append`
1_994_697_378 -> 2_010_055_088), the increases coming from the
restored primitive closures.
- `ix codegen` regenerated: the Aiur kernel changed, so the committed
Rust kernel would otherwise be stale and the codegen/bytecode parity
gate would compare against it.
`Expr.Str` conversion wrapped any hash-valid blob in `KLiteral.Str`, and
`k_infer_lit` types a `KLiteral.Str` as `String` without inspecting the
bytes. Nothing on the ordinary checking path ever decodes a literal, so a
prover-authored definition carrying malformed bytes typechecked as
`x : String` even though no Lean `String` corresponds to those bytes. The
reference kernel rejects this at ingress via `String::from_utf8`
(crates/kernel/src/ingress.rs).
The UTF-8 validation added earlier lives in the string DECODER, which is
reached only during string reductions, so it did not cover this path.
`utf8_decode_one` already rejects out-of-range continuation bytes, stray
continuations used as leaders, and overlong forms. Walk the whole blob
with it at conversion (`utf8_validate`), which is the same rejection the
reference kernel performs.
Tests
- `invalid-utf8-accepted-as-string` in the adversarial Ixon suite: a
definition declaring `String` over the blob `[0xFF]`, which no
well-formed UTF-8 can contain. Accepted before this change, now
rejected.
- Paired with `control-valid-utf8-string-literal`, the same shape over
`[0x68, 0x69]`, which must still be accepted — so the verdict is
attributable to the bytes rather than to the fixture's shape. The
fixtures build on a base env supplying the real `String` constant, so
the declared type is the genuine one.
- Six FFT pins move, all string-touching, by between +17 and +66_530
(e.g. `IxVMPrim.str_size_lit` 567_999_042 -> 568_000_142,
`Vector.append` 2_010_055_088 -> 2_010_088_729, +0.0017%). Constants
with no string literals are unchanged, `Nat.add_comm` included.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`add_entries_parallel` returned early after a constant hit, so an address
present in both `env.consts` and `env.blobs` was seeded on channel 2 but
never on channel 4. The Lean seeder walks `consts` and `blobs` in
separate passes and seeds both, so the two builders disagreed, and the
Rust-native check/prove paths aborted with `invalid IO key` on such an
environment.
The two tables are not disjoint. Constant and blob addresses share a hash
codomain, and it is the Expr context — `Ref`/`Prj` versus `Str`/`Nat` —
that says which table a reference means, so the same bytes can legitimately
be read both ways. The kernel faults the two channels separately, so the
witness has to seed them separately. Reaching an address through both
kinds of use became expressible when the claim walk moved to selecting
indices by positive constant-use.
Corrects the module header and the `ChannelEntries` doc, which both
asserted the disjointness this relies on not having.
Tests
- `dual_use_address_is_seeded_on_both_channels`: an address stored as a
constant and as a blob must appear under both `(ch 2, key)` and
`(ch 4, key)` in the built IOBuffer. Fails on the previous code with
the ch-4 entry missing.
- `lake test -- --ignored ixvm` green; clippy `--all-targets` and fmt
clean.
The kernel walks a constant's `refs` by positive constant-use — index `i`
is followed only when some Expr names it via `Ref(i, ..)` or `Prj(i, ..)`
— but both host claim builders still treated every `refs` entry as a walk
edge. The host and the kernel therefore disagreed about the edge
relation.
A `refs` entry that is a real constant but that no Expr uses as one is
never walked in-circuit, so admitting it host-side puts a phantom member
in the thin frontier: a cross-shard assumption the kernel never makes,
and a spurious dependency edge that can close a cycle once assumptions
are discharged. `walk_check_set` likewise stopped computing the set the
verifier actually checks.
Mirror `const_idxs_of` on both hosts and emit an edge only for
positively-used indices. `Env::bfs_closure` is deliberately left alone:
it computes the witness BYTE scope, where over-approximating is correct
(shipping unused bytes is free, missing bytes aborts).
Honest constants are unaffected, since a compiler only puts an address in
`refs` when an Expr references it — the shard FFT pin is unchanged. The
divergence is reachable only through prover-authored `refs`.
Tests
- `thin_frontier_excludes_unused_ref_entries`: `refs` carries two
constants but only index 0 is named by a `Ref`; the frontier must
contain the used one and not the dead one. Verified red against the
previous edge relation ("unused refs entry became a phantom frontier
member").
- `const_with_refs` now names each of its `refs` entries via a `Ref`
node. The old helper left them unreferenced, which under the walk rule
would have produced no edges at all and quietly hollowed out the
existing frontier tests.
- `lake test -- --ignored ixvm` green; clippy `--all-targets` and fmt
clean.
`idx_to_u64` packed a synthesized member/constructor index into the
projection wrapper's `u64` field by populating only the low byte behind a
`u8_range_check`, so a mutual block with more than 256 members — or an
inductive with more than 256 constructors, which is the likelier case —
was rejected outright with `value 256 out of u8 range`.
Field-to-bytes is not free: the bytes have to be supplied as advice and
then pinned. Take them from `unconstrained_g_to_bytes` and discharge the
three obligations its contract names:
- range: four `u8_range_check` calls cover the eight bytes, the byte chip
taking a pair per lookup row;
- recomposition and canonicality: `flatten_u64(bytes) == idx`, since
`flatten_u64` asserts the top byte is zero and sums the low seven.
Canonicality is what makes the decomposition unique, and it is enforced
by RESTRICTING THE DOMAIN rather than by comparing against `p`. Goldilocks
is 2^64 - 2^32 + 1, so an arbitrary eight-byte string is not injective
into the field: any value and that value plus `p` recompose to the same
element, e.g. `[00]*8` and `[01,00,00,00,FF,FF,FF,FF]` both give 0.
Forcing the top byte to zero bounds the value by 2^56 - 1 < `p`, so the
sum cannot wrap and exactly one string encodes each index. That
uniqueness is the soundness property: the projection address is blake3
over these serialized bytes, so a prover free to choose a second string
for one index would obtain a second address for a member, or collide two
members onto one address. The pre-existing `u8_from_field_unsafe`
version did collide members 0 and 256, which the range check had already
fixed by rejecting instead.
Both halves are load-bearing. `flatten_u64` does NOT range-check its
input — `[U8; 8]` is nominal typing and `u8_from_field_unsafe` mints a
`U8` holding any field element — so without the range checks the sum is
satisfiable many ways (`b1 = k`, `b0 = idx - 256k`). Its doc comment now
records that its injectivity is conditional on the caller, and names this
call site.
Indices at or beyond 2^56 still fail the assert rather than truncating.
That is a liveness bound only, 2^48 times the previous cap and far past
any real block or constructor count; buying the remaining `[2^56, p)`
would cost a multi-byte comparison against `p` for no practical gain.
Tests
- `muts-block-past-single-byte-index`: a 257-member block of well-typed
definitions with a projection wrapper for each, which now checks.
Rejected before this change with `value 256 out of u8 range`.
- `control-small-muts-block`: the same construction at two members,
already accepted — so the big-block result is attributable to the index
width alone and not to a malformed fixture.
- `storeAt` also registers a reducibility hint for `DPrj` wrappers, which
`get_ci_dprj` faults under the wrapper's own address; without it these
fixtures aborted with `invalid IO key` before reaching the index logic.
- All 66 kernel-check FFT pins and the shard pin move by +17 to +485
(`Nat.add_comm` 42_967_279 -> 42_967_313, +0.00008%), the cost of the
hint plus four range checks per synthesized index.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`get_ci` handed every standalone `Defn` a one-element `recur_addrs` whose
only entry was its own address. `Rec` converts to a plain `Const`, and
`k_infer` discharges a `Const` against its DECLARED type without checking
it, so a definition could prove its own type by citing itself:
theorem bad : False := bad
typechecked. The ref-walk cannot catch it — `const_idxs_expr` does not
follow `Rec`, and the referent is the constant already under check. Lean
forbids exactly this by checking a definition against an environment that
does not yet contain it, admitting self-reference only for `unsafe` and
`partial`.
Restrict the self slot to `unsafe`/`partial`. Those are where
self-reference is both legitimate and contained:
- legitimate, because the compiler emits a singleton (or fully
collapsed) mutual block as a standalone `Defn` whose body still says
`Rec(0)` -- see the note at Ix/CompileM.lean:2000. Those blocks are
exactly Lean's `unsafe`/`partial` recursive definitions; a `safe`
recursive definition compiles its recursion into `.rec` applications.
Gating on `unsafe` alone breaks `Lean.extractMainModule._unsafe_rec`.
- contained, because safe code may not reference either kind.
That containment was itself missing for `partial`: `is_unsafe_ci`
classified it as safe, so `partial def bad : False := bad` plus a safe
theorem referencing `bad` would have walked straight around the first
half of this fix. Count `Partial` alongside `Unsafe`, matching the
reference's "safe definition references partial definition" rejection
(crates/kernel/src/check.rs:813-820). The predicate serves double duty --
the caller's own leniency and the referent's barrier -- and `Partial`
belongs on the non-safe side of both.
Tests
- `self-referential-theorem-proves-false`: `theorem bad : False := bad`
built directly in Ixon over the real `False` constant, submitted
through the production `verify_claim` entrypoint. Accepted before this
change -- a proof of False -- and rejected after.
- No safe constant in the corpus references a partial definition, and
the `_unsafe_rec` witnesses still check, so neither half of the fix
costs coverage.
- All 66 kernel-check FFT pins and the shard pin move (`Nat.add_comm`
42_967_313 -> 42_965_666); the self slot is no longer built for safe
definitions, so most costs drop slightly.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
The nested branch of the strict-positivity check dropped the parameter
arguments (`list_drop(args, n_ctor_params)`) and descended into the
nested inductive's constructors without substituting them. A nested
constructor's fields mention that inductive's parameters as bound
variables, so the block-occurrence test asked whether the block occurs in
`BVar 0` — it never does, and every parameter position passed vacuously.
The occurrence being hunted lives in the ARGUMENT substituted for the
parameter, which was exactly what got discarded.
So for `Inner α | mk : (α → False) → Inner α`, itself strictly positive,
`Host | mk : Inner Host → Host` was accepted: descending into `Inner.mk`
sees the field `α → False`, which mentions nothing tracked. With `α :=
Host` it reads `(Host → False) → …`, a negative occurrence, and Lean
rejects it. Accepting it admits `False`.
Mirror `check_nested_ctor_fields` (crates/kernel/src/inductive.rs): strip
`n_params` foralls, then simultaneously substitute the parameter
arguments at depth 0. After stripping, `BVar 0` is the innermost (last)
parameter, so the argument list is reversed — `expr_inst_many` maps
`substs[i]` to `BVar(depth + i)`, the same convention as `simul_subst`.
The parameters are substituted away rather than bound, so the field walk
now starts with an empty binder context instead of the peeled types.
Tests
- `nested-inductive-negative-occurrence`: the `Inner`/`Host` pair above,
built directly in Ixon as two single-inductive `Prop` blocks with their
projection wrappers. Accepted before this change, and now rejected by
the intended assert ("strict positivity: block occurs left of an
arrow").
- `control-nested-positive-inductive`: the same construction with
`Inner α | mk : α → Inner α`, which is strictly positive and must keep
checking. The two fixtures differ only in the inner constructor's
field, so the rejection is attributable to the negative occurrence
rather than to nesting.
- 63 FFT pins and the shard pin move, concentrated in inductive-heavy
constants (`IxVMInd.Tree` 767_167 -> 796_717, `IxVMInd.DepthM`
1_312_461 -> 1_347_394) — the nested check now does real work instead
of returning vacuously.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`KLimbs` is canonical in two independent ways — no trailing zero limbs,
and every limb byte in [0, 256) — and `klimbs_eq` is a raw limb compare
that establishes neither. Since it backs `Nat.beq`, `Nat.decEq` and
`literal_eq`, any producer emitting a value-correct but
representation-wrong `KLimbs` makes those answer `false` where Lean
answers `true`. That is an accepted false equation, not an abort. Two
producers did.
`String.utf8ByteSize` built the length as one unchecked byte
(`[u8_from_field_unsafe(n), 0, ...]`) with no normalization. The empty
string therefore reduced to the denormalized zero `[[0;8]]` instead of
the canonical empty limb list, and a string of 256 bytes or more to an
out-of-range digit — `u8_from_field_unsafe` is an erased coercion, so
nothing truncated or rejected. Build it with `klimbs_from_g`, whose byte
split is range-checked and pinned by a recomposition assert, then
normalize.
`klimbs_div_mod` pinned the quotient's VALUE (`q*b + r == a`, `r < b`)
but not its digits. The comment claimed `u8_mul` inside `klimbs_mul`
range-checked them, but `u64_mul` was rewritten to raw field products
plus `#split_carry`, whose u8 checks constrain the split outputs rather
than the input digits — and it re-canonicalizes while multiplying, so a
digit-wrong `q` still produces a canonical `q*b` and satisfies the
equality. Range-check the hint's limbs explicitly, quotient and
remainder both, via a new `klimbs_range_check`.
Tests
- `utf8-size-beq-false-accepted`: `Nat.beq (String.utf8ByteSize "") 0 =
false` proved by `Eq.refl`, over the real `Eq`/`Bool`/`Nat.beq`/
`String.utf8ByteSize` constants. The kernel had the polarity exactly
backwards — this FALSE equation was accepted while the true one was
rejected.
- `control-utf8-size-beq-true`: the same declaration ending in `true`,
which is what Lean computes and must be accepted. The pair differs only
in that operand, so each verdict is attributable to the reduction.
Both now land the right way round.
- The two fixtures share one blob for `""` and `0` — the same empty byte
string read as a string literal and as a nat literal, which is the
Expr-context disambiguation working as intended.
- 10 FFT pins and the shard pin move, confined to the div/mod/gcd/pow/
shift constants (the hint range-checks) and the string-size ones
(`IxVMPrim.str_size_lit` 567_992_937 -> 567_993_099).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT WAS DEMONSTRATED for this one. Unlike the other soundness
fixes in this series it ships without a red-then-green fixture; what
follows is why the rule is wrong as written, and what remains unproven.
`try_eta_struct` accepted `t ≡ S.mk p… a…` after checking only that the
right side is a fully-applied constructor of a struct-like inductive. It
never checked that the two sides have def-eq TYPES. All three references
do, citing lean4lean's `tryEtaStructCore`
(crates/kernel/src/def_eq.rs, Ix/Tc/DefEq.lean).
Without the guard the rule degenerates: for ANY `t` of ANY type,
`compare_struct_fields` builds precisely `Proj(S, i, t)` for each field,
so when the right side's arguments are those same projections the
pointer fast path matches them and the pair is accepted. Those
projections are synthesized inside def-eq and never inferred, so
`k_infer_proj`'s head-address gate — which would reject `Proj S i t` for
`t : Nat` — never sees them.
What is unproven is reachability. At the `k_check(e, expected)` boundary
both sides are types in the same context, and `de_args` compares type
arguments before value arguments, so I could not construct a term that
reaches this rule with genuinely mismatched types. The change is
therefore defense-in-depth against an unjustified rule, not a repair of a
demonstrated break.
The guard runs after the struct-like shape test, so it costs two
`k_infer_only` calls and one `k_is_def_eq` only on pairs that were about
to be accepted. `try_proof_irrel`, earlier in the same fallback chain,
already infers the left side unconditionally, so the added abort exposure
is small; the suite confirms no constant stops checking.
Tests
- No new fixture; the existing 14 adversarial cases and the arena corpus
are unchanged.
- 45 FFT pins and the shard pin move, the largest cost of this series
(`Nat.add_comm` 42_965_759 -> 42_966_970, +0.003%;
`IxVMPrim.str_size_lit` +26_687).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
A `Tag0` large-form header carries a 7-bit `small` field, so a payload
width up to 128 was expressible and unguarded (`Tag2`: 5 bits, up to 32).
A u64 payload is at most 8 bytes, and the three readers disagreed on the
surplus:
- Rust `u64_get_trimmed_le` rejects `len > 8` -- already correct, and
untouched here.
- The in-circuit `get_u64_le` consumed the surplus and DISCARDED it,
keeping only the first eight bytes.
- The Lean host folded it back in: `UInt64.shiftLeft` is taken mod 64,
so byte 8 ORs into bits 0-7, with no length guard.
Same bytes, same content address, three readings. An `Expr.Ref` whose
index is written overlong resolves as `refs[0]` in-circuit and `refs[42]`
on the Lean host, with both consuming the same span so nothing desyncs
afterwards. That also means the trailing-bytes assert does not imply
canonicity: up to 120 arbitrary bytes per `Tag0` site were consumed
inside the structure and dropped.
Bring the other two up to Rust: assert the width is under 9 in both
in-circuit readers, and give `getU64TrimmedLE` the guard it was missing.
No honest serializer is affected -- `putTag0` always emits the minimal
width -- so this narrows the accepted language to the format.
Tests
- `overlong-tag0-accepted`: an axiom `_ : Prop` whose `lvls` Tag0 is
rewritten to the nine-byte form. Accepted before this change, now
rejected by the intended assert.
- `control-canonical-tag0-axiom`: the same axiom with canonical bytes,
which must stay accepted. The pair differs only in that encoding.
- Both need bytes no `Ixon.ser` will emit, so the suite gained
`storeRawAt`, which stores a hand-authored byte string at its own
content address. Every previous fixture was built from a `Constant`
VALUE and was therefore canonical by construction, and structurally
could not reach this class of defect.
- All 66 kernel-check FFT pins and the shard pin move (`Nat.add_comm`
42_966_970 -> 42_980_767, +0.03%); every constant parses Tag0s.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
Four test-only fixes. No kernel code changes, so no FFT pins move.
prim-addrs asserted the literal `true`. It looked up each hardcoded
kernel address in the canonical table and, on a hit, recorded a pass —
but the table is keyed BY ADDRESS, so a permuted table still hits. Had
`nat_add_addr` held `Nat.mul`'s bytes it would have printed
"nat_add_addr = Nat.mul" and passed green, which is precisely the defect
the file's own docstring says it exists to catch: the kernel applying one
primitive's native semantics under another's name. Now the address must
resolve to the constant the function's NAME claims, comparing normalized
keys (`nat_add_addr` <-> `Nat.add`, and the final-component spelling for
`reduce_bool_addr` <-> `Lean.reduceBool`), with dispatch-site suffixes
(`_dec`/`_io`/`_iota`) and `_type`/`_ty` stripped, plus a short exception
list for the `Quot` constructors. Verified by temporarily swapping
`nat_add_addr` to `Nat.mul`'s bytes: red, then reverted. All 76 literals
pass, and the table is confirmed correct today.
The arena negative corpus counted ANY abort as a correct rejection while
its docstring claimed rejections come from an in-kernel `assert_eq!`. An
`invalid IO key` — the witness never supplied the bytes, so execution
died before reaching the logic under test — scored as a pass. That is how
a fixture rots into a green test that checks nothing, and it is the same
failure mode that made two exploit fixtures in this series pass for the
wrong reason. Now `invalid IO key` disqualifies. Unmatched-match aborts
still count: this kernel deliberately relies on an unmatched runtime
value aborting instead of spending width on `_ => assert_eq!(0, 1)`
arms, so those are genuine verdicts. No fixture currently relies on the
disqualified path.
`shardCheckEnvCase` returned `none` for two unrelated reasons and the
runner reported both as "SKIP (target absent)", a passing test. The
second — target present but the partition selected zero owned constants —
means the fixture stopped exercising the Rust witness builder and the
thin-frontier claim convention, and would have left the only test of that
path permanently green. It now throws; `none` keeps its one honest
meaning.
A `--ignored` filter naming no runner ran nothing and returned 0, so
`lake test -- --ignored ixvm2` reported success having executed nothing.
It is now an error. Exit 0 should mean tests ran and passed.
`ixVM`, the pruned production toplevel, carried `verify_const`,
`verify_check` and `verify_check_env` alongside `verify_claim`. Those are
DEBUG entrypoints that trust their dependencies instead of walking them,
so a proof made with one attests far less than a claim does — and
carrying them meant such a proof was verifiable under the SAME verifying
key. The only thing standing between it and acceptance was `ix verify`
pinning the funcIdx to `verify_claim`: a policy, enforced in one place,
with no test asserting it, and no policy at all in the recursive
verifier, which binds the claim bytes but applies no funcIdx check.
Prune to `verify_claim` alone. The guarantee is now structural: under the
production VK a debug-entrypoint proof cannot exist. No CLI path used
them, so nothing in `ix check`/`prove`/`verify` changes.
Callers that genuinely want a subject-only check build `ixVMFull` and are
thereby using a different VK, which is the honest signal — the arena
corpus, and `bench-typecheck`/`bench-recursion-debug` under
`--skip-deps`. That last one also means a `--skip-deps` benchmark is now
visibly measuring a different key than a claim run rather than silently
reporting `verify_const` rows in the claim schema.
`bench-typecheck` also reported success on failure, twice over. A proof
that failed to verify printed to stderr but left `failed` clear, so the
row was emitted as `"status": "ok"` with prove-time and proof-size
present and the run exited 0 — the only signal an absent `verify-time`
key, invisible to anything keying on status. And a constant whose execute
threw was dropped from the results entirely without setting `failed`,
so `execed.any (·.failed)` stayed clear; the `check_const`-missing throw
added earlier landed exactly there. Both now mark the row failed.
Tests
- `lake test -- --ignored ixvm` green with NO pin movement: the pruned
entrypoints were never executed, so removing them changes no cost.
Generated kernel drops from 733 to 731 functions.
- Arena repointed to `ixVMFull`, where `verify_const` still lives.
- `bench-typecheck --execute-only` smoke unchanged (`Nat.add_comm`,
constants=42).
`compare_rules` peeled the binders off the STORED rule and handed their
DOMAINS to `k_is_def_eq` as the local context for the comparison against
the canonical reconstruction. Those domains are prover-authored: nothing
checks them against the reconstruction, and reduction never consults them
(`try_iota` beta-reduces straight through). So the prover chose what the
two sides INFER to.
Retyping every binder to a Prop makes both sides infer to that Prop, so
`try_proof_irrel` fires and accepts ANY body. A recursor whose
computation rules are swapped between constructors was therefore
accepted, and iota then reduces each constructor to the other's branch.
Compare under an empty context instead, discarding the peeled domains. An
honest rule matches the reconstruction structurally, and the structural
path never consults the context — it compares `BVar i` against `BVar i`
directly. A comparison that instead needs to INFER a bound variable's
type now runs out of context and aborts, which is the safe direction.
Every recursor in the corpus still checks, including the mutual, nested
and aux-generated ones.
The reference avoids this differently, comparing whole Lam-wrapped terms
at empty context so `is_def_eq` descends binder-by-binder and compares
the domains themselves (crates/kernel/src/inductive.rs). Reconstructing
canonical domains here would mean re-deriving field domains from each
constructor's telescope and re-lifting their de Bruijn indices into the
rule's context; dropping the untrusted context achieves the same
guarantee without that machinery.
Tests
- `recursor-rule-rhs-doctored-binder-domains`: a real compiled recursor
with its first two computation rules swapped AND every peeled binder
domain replaced by a Prop. Accepted before this change; rejected now.
- `control-swapped-recursor-rules`: the identical swap with the
compiler's own binder domains, which was already rejected. The two
differ only in the domains, so the acceptance was attributable to them
alone.
- Both are built by rewriting a REAL block rather than hand-authoring a
recursor, since a hand-built one must survive `build_rec_type`'s def-eq
and constructor references inside a block are circular with the block's
own address. The fixture selects a recursor whose rules are field-free,
so no minor premise is applied and the retyping cannot abort as a
non-function before the comparison is reached — an earlier attempt did
exactly that and masked the acceptance.
- A guard case fails loudly if no such recursor is found; the first
version of this fixture silently corrupted nothing, and both its cases
"passed" meaninglessly.
- 57 FFT pins and the shard pin move (`Nat.add_comm` 42_980_767 ->
42_980_801).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT DEMONSTRATED. Defense-in-depth, like the struct-eta guard.
`compare_rules` now asserts three invariants that equivalents of already
exist in `check_rec_rules_walk` — which nothing calls. The live path had
none of them.
* `cidx == pos`. `find_rule` selects a rule at reduction time BY LABEL
while `compare_rules` pairs stored against canonical POSITIONALLY, so
permuted labels would validate one constructor's rule and then fire
it for another. It holds today only because ingress assigns `cidx`
positionally (`Convert.lean`) and the Ixon `RecursorRule` record
carries no cidx of its own — an accident of the pipeline rather than
a checked property, which is exactly what an assert is for.
* `cidx < num_ctors`, so a rule cannot name a constructor that does not
exist.
* `nf` equals the constructor's field count. `try_iota` splits the
constructor's arguments with `field_start = ctor_fields_len -
rfields`, so a wrong `nf` hands the minor premise the wrong slice; in
field arithmetic an over-large value wraps.
The reachability argument for the `nf` gap was "combined with the rule
RHS context hole"; that hole is fixed, and with the def-eq comparison
restored a mismatched peel is caught again. So this closes a route that
is currently blocked by another check rather than one that is open —
worth asserting locally rather than depending on a second mechanism.
Not fixed here: iota still selects rules by index rather than by
constructor IDENTITY, so a recursor applied to a foreign constructor
would reduce. That is not reachable through a claim —
`k_infer_app_spine_loop` runs `k_check` on every argument, so the
application fails inference before whnf sees it — and closing it properly
needs the parent inductive available at reduction time, which
`KConstantInfo.Rec` does not carry (its `Addr` is the recursor's own
block; the parent is derived at check time from the type). That is a KCI
layout change, left for its own pass.
Tests
- No new fixture; nothing demonstrates the hole.
- Every recursor in the corpus satisfies all three invariants, including
the mutual, nested and aux-generated ones, so they held by construction
and are now enforced.
- 57 FFT pins and the shard pin move (`Nat.add_comm` 42_980_801 ->
42_981_336) for the per-rule constructor lookup.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT DEMONSTRATED. Defense-in-depth.
`try_iota` selected a computation rule by constructor INDEX alone:
`collect_spine_of_ctor` returned the major's `cidx` and discarded which
inductive it came from, so `find_rule(rules, cidx)` fired whenever any
inductive's constructor happened to carry a matching index. On its face
`Bool.rec motive f t (Nat.succ n)` reduces to `t`. Lean's kernel selects
the rule by the constructor's NAME, which pins the inductive.
Not reachable through a claim: `k_infer_app_spine_loop` runs `k_check` on
every argument, so a recursor applied to a foreign constructor fails
inference long before whnf sees the redex — the ill-typed term cannot be
stated. The Rust reference selects by index too, consistent with that
being why it was tolerated. What the patch removes is the DEPENDENCE on
that: reduction no longer needs inference to have run first to be
correct.
`collect_spine_of_ctor` now also returns the constructor's parent, as the
address of its projection wrapper, and `try_iota` asserts it equals the
recursor's own parent. That parent is read off the recursor's type (the
major premise's head) with the existing `rec_to_parent_addr`, so no
KConstantInfo layout change was needed — its `Addr` field is the
recursor's own block, not the parent's. Aiur memoizes per argument tuple,
so the type walk is paid once per recursor rather than once per
reduction.
Tests
- No new fixture; nothing demonstrates the hole.
- Every recursor in the corpus still reduces, including the mutual,
nested and aux-generated ones, and the Nat/String literal paths that
synthesize a constructor application before iota (`nat_cases_big`,
`nat_pow_big`, `strOfListFold*` all unchanged in behaviour).
- 63 FFT pins and the shard pin move (`Nat.add_comm` 42_981_336 ->
42_986_025, +0.01%) for the parent comparison.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`shardsCover` gates every shard run: "exit 0 has to mean every env const
was checked by some shard". It skipped constants whose bytes fail to
parse (`LazyConstant.get?` discards the error and `.ixe` loading is
lazy, so such a constant sits in `consts` with its key present). A
skipped constant was assigned to no shard, counted as neither owned nor
unowned, and still included in the "covers all N consts" total — so the
gate reported success over constants it had not covered.
It would also enter a referring shard's frontier, since `thin_frontier`
admits an edge on key presence in `env.consts` without parsing, leaving
an assumption no shard discharges.
Count them separately and fail, rather than folding them into `unowned`:
"cannot be assigned to a shard" is a different problem from a partition
gap, and saying so points at the right cause.
`ownedConstsForBlocks` keeps its skip — it runs per shard and would
otherwise need the same error plumbing — but now documents that it is
safe only because this gate rejects such an env first.
Not changed: the composed verdict still has no assumption-cycle check,
and on analysis it does not need one. Assumptions are discharged per
CONSTANT along ref edges, which are acyclic by construction — a
constant's address is blake3 over bytes containing its refs' addresses,
so a reference cycle needs a hash cycle. A cycle in the shard-level graph
still stands for a well-founded per-constant chain, and
`verifyShardComposition` recomputes each digest via `shardClaimDigest`,
so a self-assuming shard claim cannot be substituted.
`compile_univ` memoized `Level -> Univ`, but a `Level::Param` compiles to
`Univ::Var(i)` where `i` is the parameter's POSITION in `univ_params` —
so the result depends on a context the key omitted. Unlike the expression
cache, which is cleared at every constant boundary, `univ_cache` lives
for a whole block, and `compile_definition` compiles each member under
its own `level_params`. `preseed_expr_tables` explicitly feeds
`(expr, univ_params)` pairs with differing param lists into the same
cache.
So two members of one block placing the same parameter NAME at different
positions would give the second member the first's index. Not an abort:
it compiles cleanly and registers a constant denoting a different
universe than the Lean source, under a correct-looking name.
The hazard is already recognized one level up — `collect_expr_tables`
keys `seen_exprs` by `(expr_hash, univ_params_key)` for exactly this
reason. Apply the same key here. The Lean mirror instead drops the cache
in `withUnivCtx`, matching how it already clears the expression cache per
constant.
Unreachable from ordinary Lean sources, since the elaborator gives every
member of a `mutual` block the same `levelParams` — but nothing checked
it, and under the prover threat model the environment is authored.
Also repairs `claim.rs`'s `defn_const`, whose Definition never referenced
its own `refs`. Since the claim walk admits an index only when some Expr
uses it as a constant, that fixture produced no walk edges and
`shard_check_env_claim_is_thin_frontier` has been failing since the walk
moved to positive constant-use. The identical flaw was fixed in
`shard_claim.rs`'s helper at the time and missed here, because only
`ixon` and `ixvm-codegen` were re-tested rather than the whole workspace.
Tests
- `cargo test --release` across all crates: 1195 passed, 0 failed.
- `lake test -- --ignored ixvm` green with NO pin movement and no address
changes, confirming no block in the corpus has members that order a
shared parameter name differently. The fix removes the dependence on
that remaining true.
- clippy `--all-targets` and fmt clean.
@johnchandlerburnham
johnchandlerburnham merged commit 80bb41f into mainAug 3, 2026
10 checks passed
@johnchandlerburnham
johnchandlerburnham deleted the ap/ixvm4 branch August 3, 2026 05:37
samuelburnham added a commit that referenced this pull request Aug 4, 2026
…acking
Port of sb/measured-ingress's planning layer onto the addr-first kernel
(#529), reworked for its lazy fault-in semantics.
Profiling (.ixprof v3): the recording kernel's try_get_const consults are
drained per constant into a touch graph — the measured set a lazy checker
faults in — persisted as a presence-flagged CSR section alongside a new
block-level reference graph and a subst_unique counter (splitmix64-keyed
dedup of substitution work items: the post-memoization volume a memoizing
executor pays). Both graph sections are optional, so older readers'
formats stay decodable by version gate alone.
Planner: partition_for_aiur_ram bin-packs to a host-RAM cap with
faulted-set byte accounting — under #529 the witness always ships the
full closure, but only faulted bytes cost ingress trace rows, so a
shard's byte feature is owned ∪ touched(owned) (full reference closure
as the over-approximating fallback when no touch graph was recorded).
The stub/promotion machinery of the original branch is dropped entirely:
lazy fault-in computes at run time exactly the set the stubs predicted,
with no replay divergence and no repair ladder, and the manifest format
stays byte-identical with main (owned-only).
Cost models: nlogn(x) = x·log2(x+2) feature form matching Aiur's
width·height·log2(height) FFT structure; prove/execute time and RAM
coefficients carried over from the pre-#529 aiur bench-suite fits.
They are placeholders pending a refit against this kernel — the circuit
shrank (deleted positional machinery), the byte feature narrowed to the
faulted set, and the recording counters moved under #442 — with
provenance noted at the definitions. The costs sidecar
(<out>.costs.csv) persists per-shard features incl. subst_unique and
nat_arith so the refit has its corpus.
CLI: ix shard --backend aiur --max-ram G; ix profile --backend/--top
(per-metric block leaderboards with display names) and ix profile sweep
(env-wide closure cost sweep: per-root feature CSV + feasibility,
min-root-per-hot-block, and diversity reports).
samuelburnham added a commit that referenced this pull request Aug 8, 2026
…acking
Port of sb/measured-ingress's planning layer onto the addr-first kernel
(#529), reworked for its lazy fault-in semantics.
Profiling (.ixprof v3): the recording kernel's try_get_const consults are
drained per constant into a touch graph — the measured set a lazy checker
faults in — persisted as a presence-flagged CSR section alongside a new
block-level reference graph and a subst_unique counter (splitmix64-keyed
dedup of substitution work items: the post-memoization volume a memoizing
executor pays). Both graph sections are optional, so older readers'
formats stay decodable by version gate alone.
Planner: partition_for_aiur_ram bin-packs to a host-RAM cap with
faulted-set byte accounting — under #529 the witness always ships the
full closure, but only faulted bytes cost ingress trace rows, so a
shard's byte feature is owned ∪ touched(owned) (full reference closure
as the over-approximating fallback when no touch graph was recorded).
The stub/promotion machinery of the original branch is dropped entirely:
lazy fault-in computes at run time exactly the set the stubs predicted,
with no replay divergence and no repair ladder, and the manifest format
stays byte-identical with main (owned-only).
Cost models: nlogn(x) = x·log2(x+2) feature form matching Aiur's
width·height·log2(height) FFT structure; prove/execute time and RAM
coefficients carried over from the pre-#529 aiur bench-suite fits.
They are placeholders pending a refit against this kernel — the circuit
shrank (deleted positional machinery), the byte feature narrowed to the
faulted set, and the recording counters moved under #442 — with
provenance noted at the definitions. The costs sidecar
(<out>.costs.csv) persists per-shard features incl. subst_unique and
nat_arith so the refit has its corpus.
CLI: ix shard --backend aiur --max-ram G; ix profile --backend/--top
(per-metric block leaderboards with display names) and ix profile sweep
(env-wide closure cost sweep: per-root feature CSV + feasibility,
min-root-per-hot-block, and diversity reports).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@arthurpaulino@johnchandlerburnham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

IxVM: cheaper shard checking via an addr-first kernel, plus soundness hardening - #529

Merged
johnchandlerburnham merged 19 commits into
mainfrom
ap/ixvm4
Aug 3, 2026
Merged

IxVM: cheaper shard checking via an addr-first kernel, plus soundness hardening#529
johnchandlerburnham merged 19 commits into
mainfrom
ap/ixvm4

Conversation

@arthurpaulino

@arthurpaulinoarthurpaulino commented Jul 31, 2026

Copy link
Copy Markdown
Member

Motivation

Make sharded environment checking cheap. A large environment is partitioned across shards, and each shard proves its own constants well-typed against a thin-frontier assumption cutoff, so every constant is checked in exactly one shard and cross-shard assumption trees stay minimal.

Delivering that required migrating the IxVM Aiur kernel from a positional constant table to a content-addressed, lazily-faulted design — and then auditing the result, which turned up five soundness holes with working exploits, including three separate routes to proving False.

Sharded environment checking

A shard proves CheckEnv{root, assumptions}: root is a merkle root over the shard's owned constants, assumptions a merkle root over the thin frontier — the direct out-of-owned walk edges. The kernel's env_walk starts from the owned set, checks each reachable constant, and stops at a frontier member (assumed well-typed by its owning shard: not re-checked, not recursed into).

  • Sub-frontier constants are never re-checked here; they are covered transitively by the frontier members' own shard claims. This is the cost win over checking each constant's full closure: no cross-shard duplication, and the assumption tree is one layer deep instead of the whole closure.
  • Strict membership: a walk-reachable constant in neither owned nor the frontier is a bad partition and hard-errors, so a shard cannot silently drop coverage of its own closure.
  • Byte scope ≠ check scope: delta unfolding reads through the frontier, so a witness ships the full dependency closure of owned even though only the strict-membership set is checked. The witness IOBuffer is built in Rust (aiur_ixvm_witness) rather than Lean, removing the per-byte boxing that dominated shard wall time.

Addr-first kernel

  • KExprNode.Const / Proj key on a 32-byte content Addr, not a numeric position; KConstantInfoInduct/Ctor/Rec carry a (block_addr, ind_idx) parent pair instead of indices. The positional top: List<KCI> table threaded through the kernel is gone.
  • get_ci(addr) is the sole resolver, Aiur-memoized per addr, faulting constant bytes lazily from an IOBuffer. load_verified_constant re-derives blake3 over the bytes and asserts key == addr; the io_get_info idx/len are unconstrained witness and are never branched on.
  • IOBuffer channels: 0 claim, 1 assumption tree, 2 constant bytes, 3 Defn reducibility hint (advisory), 4 blob bytes. The kernel derives blob-vs-constant from Expr context, so no wire discriminator ships. The two sets are not disjoint — one address can be read both ways — so each channel is seeded independently.
  • Kernel/Inductive.lean and Kernel/Primitive.lean deleted (~6.6k lines); inductive validation absorbed into Kernel/Check.lean, primitives split into Kernel/Klimbs.lean (bignum gadgets) and Kernel/NatPrim.lean (Nat/Str addr dispatch). New Kernel/InferOnly.lean breaks the DefEq↔Infer cycle. Ingress.lean collapsed 1700→298 lines.

Recursor validation

  • build_rec_type reconstructs a recursor's canonical type and asserts it def-eq to the declared type, on both the kind-aware and address-only lookup paths. Without it a recursor could declare ... -> False while its rules still reconstructed correctly.
  • spec_params are lowered from the major-premise frame into the recursor-param frame at extraction: the root cause of 17 nested/aux recursor reconstruction failures, and why the canonical check accepted a superset.
  • k_flag and parent-inductive-shape checks hoisted out of the attacker-selectable is_aux gate; the canonical rules comparison, which formerly never ran, now runs for every recursor.

Pre-existing criticals

IDFix
P1blob-vs-constant derived from Expr context instead of a prover-answered io_get_info probe, retiring the ch-2 len=0 sentinel; a prover can no longer mark a walk root or owned leaf "skip me"
P2Nat.div n 0 pins the quotient to zero (was prover-chosen)
P3klimbs_sub/le corrected on operands with trailing zero limbs; div/mod hints normalized before use
P4is_large_eliminator field-index arithmetic parenthesized (Aiur - is right-associative)
P5Reveal recursor rules assert the claimed ruleIdx equals its position

Robustness

  • L5: UTF-8 string-literal decode validates continuation-byte ranges and per-length overlong minimums, so distinct byte sequences (e.g. overlong NUL C0 80 vs 00) no longer decode to the same codepoint and def-eq as equal.
  • L6: run_claim dispatches on the full tag4 size (was the low byte only), matching Rust Claim::get.
  • L8: the K-recursor synth gate checks the major's inferred inductive against the recursor's own, not against itself, so I.rec no longer strips a major of a different inductive.

Aiur assert_eq! messages and circuit cleanup

  • assert_eq!(a, b, "why") gains an optional diagnostic label, threaded Source.Term → Typed → Simple → Concrete → Bytecode.Op → Rust Op::AssertEq(_, _, Option<String>) → ExecError::AssertEqMismatch. The message is diagnostic-only and never enters the circuit; all ~122 IxVM assert sites are labeled.
  • Unreachable _ => assert_eq!(0, 1) match arms removed: an unmatched runtime value already aborts (the kernel's reject), so the extra arm only widened the circuit.
  • Address equality is a full 32-byte content comparison — never a pointer-value compare, which is unsound for distinct-pointer/same-data — packed as five base-256 field limbs behind a byte-0 prefilter.

Witness construction

witness_scope reuses Env::bfs_closure, which already follows refs, Prj → block, and the derived Muts → member/ctor projection edges, so the wrappers get_ci synthesizes ship as forward edges of a single O(closure) walk with no reverse scan of the env. Primitive addresses are seeded as additional walk roots, because def-eq delta-unfolds primitive bodies whose refs are reachable from nowhere else. The Lean seeder computes the same scope by the same rule, so the two cannot drift.

bench-typecheck's constants metric now reads check_const's unique query count — exactly the constants typechecked — instead of the shipped byte scope, which is a superset and inflated throughput.

Soundness audit

Eight parallel read-only audits covered the pipeline end-to-end (compile, Ixon serde, ingress/convert, type-theory core, declaration checking, claim/witness layer, primitives, entry points), under the zkDSL threat model: a prover authors the data and pays for its own trace, so aborts and rejections are not threats and only a wrong answer counts. Thirty findings; every claim relayed here was re-verified against the code before acting on it, and several agent claims were corrected or withdrawn in the process.

Fixed, with a working exploit

Each of these shipped with a fixture that was red before the fix and green after, paired with a minimal-pair control so the verdict is attributable to one variable.

HoleWhat it bought an attacker
Self-referential definitionstheorem bad : False := bad typechecked. A standalone Defn was handed a recur slot naming itself, Rec erases to an ordinary Const, and k_infer reads a Const's declared type without checking it — so the body discharged its own type by citing itself. Now restricted to unsafe/partial, which is where Lean permits self-reference and where safe code cannot reach
Nested-inductive positivityThe nested branch dropped the parameter arguments and descended into the nested inductive's constructors without substituting them, so the occurrence being hunted was never present. Host | mk : Inner Host → Host was accepted for a negative Inner. Admits False
Recursor rules compared under the prover's contextcompare_rules handed the stored rule's own binder domains to k_is_def_eq as the comparison context. Retyping them to a Prop makes both sides infer to that Prop, proof irrelevance fires, and a recursor with its computation rules swapped between constructors was accepted
Non-canonical computed Nat literalsString.utf8ByteSize built a KLimbs as one unchecked byte, and klimbs_div_mod pinned the quotient's value but not its digits. Since klimbs_eq compares limbs rather than values, the kernel had the polarity backwards: Nat.beq (String.utf8ByteSize "") 0 = false was accepted while the true equation was rejected
Tag0/Tag2 overlong payloadsThree readers, three answers on the same bytes: the circuit discarded bytes past the eighth, the Lean host folded them back in (its shift is mod 64), Rust rejected. One address denoted different constants to different readers

Fixed as defense-in-depth

No exploit demonstrated for these; each commit says so in its first line. They remove a dependence on some other check happening to hold.

  • Struct-eta type equality.try_eta_struct omitted the infer(t) ≟ infer(s) side condition all three references carry, degenerating to t ≡ S.mk … (Proj S i t) … for any t of any type.
  • Recursor rule labels.cidx selects a rule at reduction time but was validated only positionally; nf decides how iota splits the constructor's arguments and was never checked against the constructor. Equivalents existed only in dead code.
  • Iota constructor identity. Rules were selected by index alone, so any inductive's constructor with a matching index fired one. Unreachable through a claim — inference k_checks every argument first — but reduction should not depend on that.
  • Universe cache context.compile_univ memoized Level → Univ while a Level::Param compiles to a position in univ_params, and the cache outlives a constant. Now keyed by (level, univ_params_key), matching what collect_expr_tables already does one level up.

Entry points and coverage

  • verify_const / verify_check / verify_check_env are DEBUG entrypoints that trust their dependencies, and they shipped in the production verifying key alongside verify_claim — leaving ix verify's funcIdx check as the only barrier, untested, with no equivalent in the recursive verifier. The production toplevel is now pruned to verify_claim alone, so such a proof cannot exist under that VK. Zero FFT cost.
  • bench-typecheck reported "status": "ok" and exit 0 when verification failed, and dropped thrown constants without marking them failed.
  • shardsCover skipped constants whose bytes do not parse — assigning them to no shard, counting them as neither owned nor unowned, and still including them in "covers all N consts". Now a hard failure.

Test guards that could not fail

Four of these were found by auditing the tests themselves, and they matter because they were the safety net for everything above.

  • prim-addrs asserted the literal true. Keyed by address, so a permuted primitive table passed — precisely the defect its docstring claimed to catch. Now the address must resolve to the constant the function's name claims; verified by temporarily swapping nat_add_addr to Nat.mul's bytes and watching it go red.
  • The arena negative corpus counted any abort as a correct rejection, so a fixture aborting on invalid IO key — never reaching the code under test — scored as a pass. That is the same failure mode that made two exploit fixtures in this series pass for the wrong reason before it was caught.
  • The shard-pipeline SKIP was a passing test conflating "target absent" with "partition selected zero constants".
  • A --ignored filter naming no runner executed nothing and returned 0, so a typo'd suite name reported success having run no tests.

Adversarial Ixon test suite

New Tests/Ix/IxVM/Exploits.lean, 19 cases, wired into the ixvm suite. The kernel-arena corpus attacks the layer above: its bad_* fixtures are raw Lean.ConstantInfo declarations that skip the elaborator, but they are still compiled by rsCompileEnvFFI, so the Ixon they produce is always well-formed. This suite authors Ixon.Constant values directly, and storeRawAt goes one level lower still, storing hand-authored byte strings — needed because every value-level fixture is canonical by construction and so cannot express a malformed encoding.

Each case records the false statement acceptance would buy, so a failure reads as a security statement. Rejection reasons are printed, which earned its keep repeatedly: fixtures that "passed" via invalid IO key without reaching the checker were caught this way, as was a fixture that silently corrupted nothing because non-mutual recursors are standalone constants rather than Muts members.

Testing

  • lake test -- --ignored ixvm green.
  • cargo test --release across all crates: 1195 passed, 0 failed. This also repaired shard_check_env_claim_is_thin_frontier, red since the claim walk moved to positive constant-use because its fixture's Exprs never referenced their own refs.
  • New prim-addrs parity suite; codegen ↔ bytecode parity gate green; arena negative corpus (77 bad_*) all reject, now only via genuine in-kernel refusals.
  • FFT-cost pins re-pinned per change. Cumulative effect is small (Nat.add_comm 42_747_927 → 42_986_025, +0.6% over the whole series); the largest single contributors are the nested-positivity substitution and the struct-eta guard.
  • ix codegen regenerated whenever the Aiur kernel changed, so the committed Rust kernel never lags the source the parity gate compares against.

Known limitations

Recorded rather than silently dropped.

  • Call-site surgery is not covered by the value pin. The compiler rewrites calls to aux-generated constants — permuting arguments, dropping some, replacing the head and its universe list — and the plans driving it live only in ConstantMeta, which prune_to_closure_anon excludes. So a verified claim says the term at address X is well-typed, not that term is the Lean declaration you wrote; the equivalence rests on the rewrite being faithful, checked today only by decompile round-trip tests. Closing it is a claim-format decision, not a patch.
  • Assumption cycles are not rejected, and a conditional Check claim naming its own target is accepted. That is the specified semantics — some root means "conditional on every leaf", so the claim asserts A → A, vacuous rather than false — and the obligation sits with whoever discharges assumptions. Sharded CheckEnv is not exposed, since shardClaimDigest recomputes the claim from (env, blocks). Pinned as ACCEPT so the boundary cannot move silently.
  • Alpha-collapse comparators ignore safety, so two members agreeing on kind/lvls/typ/value and disagreeing on safety collapse. Deliberately not fixed: a prover authors the environment directly and never goes through the compiler, so hardening it stops no attacker, while adding safety to the ordering would move every address and refusing the collapse would put a hard error in the compiler on the strength of an unverified assumption.
  • klimbs_le / klimbs_eq are in the logical trusted base, not just the performance path: Nat.decLe mints Decidable.isTrue with a proof term whose correctness is asserted rather than checked, so one wrong verdict yields False.
  • check_inductive_shape_ctors omits validate_expr_well_scoped / k_ensure_sort / assert_safety that check_const's Ctor arm performs; only an entirely unreferenced constructor escapes those.

@arthurpaulino
arthurpaulinoforce-pushed the ap/ixvm4 branch 4 times, most recently from 06e0489 to 559ced9CompareAugust 1, 2026 00:53
… hardening
Make sharded environment checking cheap. A large environment is partitioned
across shards, and each shard proves its own constants well-typed against a
thin-frontier assumption cutoff, so every constant is checked in exactly one
shard and cross-shard assumption trees stay minimal. Delivering that required
migrating the IxVM Aiur kernel from a positional constant table to a
content-addressed, lazily-faulted design -- and closing the soundness
regressions that migration introduced along with several pre-existing holes.
Sharded environment checking
- A shard proves `CheckEnv{root, assumptions}`: root is a merkle over the
shard's owned constants, assumptions a merkle over the THIN FRONTIER --
the direct out-of-owned walk edges. The kernel's env_walk starts from the
owned set, checks each reachable constant, and stops at a frontier member
(assumed well-typed by its owning shard, not re-checked, not recursed).
- Sub-frontier constants are never re-checked here; they are covered
transitively by the frontier members' own shard claims. This is the cost
win over checking each constant's full closure: no cross-shard duplication,
and the assumption tree is one layer deep instead of the whole closure.
- Strict membership: a walk-reachable constant in neither owned nor the
frontier is a bad partition and hard-errors, so a shard cannot silently
drop coverage of its own closure.
- Byte scope != check scope: delta unfolding reads through the frontier, so
a witness ships the full walk closure of owned even though only the
strict-membership set is checked. The witness IOBuffer is built in Rust
(aiur_ixvm_witness) rather than Lean, removing the per-byte boxing that
dominated shard wall time.
Addr-first kernel
- KExprNode.Const/Proj key on a 32-byte content Addr, not a numeric
position; KConstantInfo Induct/Ctor/Rec carry a (block_addr, ind_idx)
parent pair instead of indices. The positional `top: List<KCI>` table
threaded through the kernel is gone.
- get_ci(addr) is the sole resolver, Aiur-memoized per addr, faulting
constant bytes lazily from an IOBuffer. load_verified_constant re-derives
blake3 over the bytes and asserts key == addr; the io_get_info idx/len
are unconstrained witness and are never branched on.
- IOBuffer channels: 0 claim, 1 assumption tree, 2 constant bytes, 3 Defn
reducibility hint (advisory), 4 blob bytes. An address is on ch2 iff a
constant and ch4 iff a blob; the kernel derives which from Expr context
(Str/Nat -> blob, Ref/Prj -> constant), so no wire discriminator ships.
- Kernel/Inductive.lean and Kernel/Primitive.lean deleted (~6.6k lines);
inductive validation absorbed into Kernel/Check.lean, primitives split
into Kernel/Klimbs.lean (bignum gadgets) and Kernel/NatPrim.lean (Nat/Str
addr dispatch). New Kernel/InferOnly.lean breaks the DefEq<->Infer cycle.
Ingress.lean collapsed 1700->298 lines. V1/V2 naming removed throughout.
Recursor validation (R1, R2, R3)
- build_rec_type reconstructs a recursor's canonical type and asserts it
def-eq to the declared type, on both the kind-aware and address-only
lookup paths. Without it a recursor could declare `... -> False` while
its rules still reconstructed correctly.
- spec_params are lowered from the major-premise frame into the
recursor-param frame at extraction: the root cause of 17 nested/aux
recursor reconstruction failures, and why the canonical check accepted a
superset.
- k_flag and parent-inductive-shape checks hoisted out of the
attacker-selectable `is_aux` gate; the canonical rules comparison, which
formerly never ran, now runs for every recursor.
Pre-existing criticals (P1-P5)
- P1: blob-vs-constant is derived from Expr context instead of a prover-
answered io_get_info probe, retiring the ch-2 len=0 sentinel; a prover
can no longer mark a walk root or owned leaf "skip me".
- P2: Nat.div n 0 pins the quotient to zero (was prover-chosen).
- P3: klimbs_sub/le corrected on operands with trailing zero limbs; div/mod
hints normalized before use.
- P4: is_large_eliminator field-index arithmetic parenthesized (Aiur `-` is
right-associative).
- P5: Reveal recursor rules assert the claimed ruleIdx equals its position.
Robustness (L5, L6, L8)
- L5: UTF-8 string-literal decode validates continuation-byte ranges and
per-length overlong minimums, so distinct byte sequences (e.g. overlong
NUL C0 80 vs 00) no longer decode to the same codepoint and def-eq as
equal. Roughly zero added FFT cost.
- L6: run_claim dispatches on the full tag4 size (was the low byte only),
matching Rust Claim::get.
- L8: the K-recursor synth gate checks the major's inferred inductive
against the recursor's own (via se_parent_addr), not against itself, so
I.rec no longer strips a major of a different inductive.
Aiur assert_eq! messages + circuit cleanup
- assert_eq!(a, b, "why") gains an optional diagnostic label, threaded
Source.Term -> Typed -> Simple -> Concrete -> Bytecode.Op -> Rust
Op::AssertEq(_, _, Option<String>) -> ExecError::AssertEqMismatch. The
message is diagnostic-only and never enters the circuit; all ~122 IxVM
assert sites are labeled.
- Unreachable `_ => assert_eq!(0, 1)` match arms removed: an unmatched
runtime value already aborts (the kernel's reject), so the extra arm only
widened the circuit.
- address equality is a full 32-byte content comparison (never a pointer-
value compare, which is unsound for distinct-pointer/same-data), packed
as five base-256 field limbs (7+7+7+7+4) behind a byte-0 prefilter and
short-circuiting on the first mismatched group.
Crate reorganization
- PrimAddrs / reserved_marker_name moved verbatim from
crates/kernel/src/primitive.rs to crates/common/src/prim_addrs.rs (kernel
re-exports), so witness builders reach the primitive address table
without an ix-kernel -> ixon -> ix-kernel cycle.
- witness_scope reuses Env::bfs_closure -- the env's own dependency-closure
pass, which already follows refs, Prj->block, and the derived
Muts->member/ctor projection edges -- so the wrappers the kernel's get_ci
synthesizes ship as forward edges of a single O(closure) walk, with no
reverse scan of the whole env. Primitive-address seeding is unioned on
top: primitives are fabricated during reduction and need not appear in a
target's ref closure, so a closure-only scope aborts with `invalid IO
key`.
Tests
- New prim-addrs parity suite (Lean<->Aiur primitive address table).
- Codegen<->bytecode parity gate restored, green (198/198).
- FFT-cost pins re-pinned to current behavior; arena negative-fixture
corpus (77 bad_* constants) all reject.
…onstants
closureFromBase now walks the derived Muts->member/ctor projection
wrapper addresses as forward edges (projWrapperAddr: blake3 over the
serialized proj-only Constant, compile's construction), so closureFrom
and buildShardCheckEnvWitness drop their O(env) reverse scans that
parsed every env constant per witness build. Primitive addresses are
seeded bytes-only after the walk instead of being pushed onto the
worklist, so their transitive closures no longer inflate the byte scope
(a delta-unfold into an unshipped primitive body aborts with invalid IO
key rather than proving anything wrong).
bench-typecheck's constants metric now reads check_const's unique query
count (memoized per (ci, addr) = exactly the constants typechecked)
instead of closureFrom.size, which counted the shipped byte scope.
check_const is resolved up front and both lookups fail with IO.userError
instead of panicking.
Experiment: lake exe bench-typecheck --ixe seed.ixe --consts
Nat.add_comm --execute-only (seed.ixe = ix compile Tests.lean --consts
Nat.add_comm) reports constants=42, matching check_const's height in
lake exe ix check Nat.add_comm; fft-cost unchanged at 42747927.
The claim walk decided which of a constant's `refs` to recurse into by
SKIPPING every index the constant's Exprs used as a blob payload
(`Str(i)` / `Nat(i)`). That rule is unsound: the Exprs are authored by
the prover, and `refs` is one index space shared by `Ref`, `Prj`, `Str`
and `Nat`, so marking an index suppresses the walk of that index used as
a constant everywhere else in the same constant.
The marking need not even be live. `blob_idxs_of` traversed `sharing` in
full, but conversion reaches a `sharing` entry only through an
`Expr.Share` node, and nothing validates the array. A dead `Nat(i)`
entry is therefore free: it is never converted, never typechecked, and
needs no blob on channel 4. It just hides `refs[i]` from the checker.
`k_infer` then reads that dependency's DECLARED type without ever
checking it, so the kernel accepts a constant whose transitive closure
is ill-typed -- a wrong answer, not an abort. Both claim variants were
affected; for CheckEnv the skip happens before `env_walk` is entered, so
the strict owned-membership assert never fires either.
Replace the family with `const_idxs_*`: collect the indices `Ref(i, ..)`
and `Prj(i, ..)` use, and walk `refs[i]` iff `i` is among them. `Rec`
resolves through `recur_addrs` and a projection's block comes from the
ConstantInfo payload, both walked separately, so the positive set is
complete. Same single traversal as before, so no added cost. The failure
direction is now safe: over-approximating walks an address that faults
an unseeded key and aborts, which only costs a dishonest prover.
Adversarial Ixon test infrastructure
- New `Tests/Ix/IxVM/Exploits.lean`, wired into the ixvm suite. The
kernel-arena corpus attacks the layer above this one: its `bad_*`
fixtures are raw `Lean.ConstantInfo` declarations that skip the
elaborator, but they are still compiled by `rsCompileEnvFFI`, so the
Ixon they produce is always well-formed. This suite authors
`Ixon.Constant` values directly, which is what a prover can do.
- Each `ExploitCase` records the false statement acceptance would buy,
so a failure reads as a security statement. Rejection reasons are
printed: a fixture that aborts before reaching the mechanism under
attack is indistinguishable from a kernel that correctly refuses it,
and the first run of this fixture passed for exactly that wrong reason
(`invalid IO key` from an unseeded channel-3 hint, now seeded by
`storeAt`).
- The exploit ships with two controls that differ from it minimally: one
checking the smuggled dependency alone, one checking the same target
without the dead `sharing` entry. Both reject via the intended
in-kernel assert, so the exploit's acceptance was attributable to the
`sharing` entry alone.
- Two further cases probe what the fix does NOT cover. An index used as
both a `Nat` literal and a `Ref` — reachable because blob and constant
addresses share one un-domain-separated blake3 space and
`load_verified_blob` parses nothing — is now walked and rejected,
though by channel fault rather than type error, which is the safe
direction. A conditional Check claim naming its own target as its own
assumption IS accepted, and is pinned that way: `some root` means
"conditional on every leaf being well-typed", so the claim asserts
`A -> A` and is vacuous rather than false. The obligation sits with
whoever discharges assumptions, which must reject such cycles; sharded
CheckEnv is not exposed, since `shardClaimDigest` recomputes the claim
from (env, blocks).
Primitive closures are walked again
- The previous commit seeded primitive addresses as bytes only, without
walking their closures. That is too aggressive: def-eq delta-unfolds
primitive BODIES, and a body's refs are reachable from nowhere else in
the scope, so the `strAppend*` arena fixtures aborted with `invalid IO
key`. Both seeders now pass the primitives as walk ROOTS
(`closureFromRoots` in Lean, `bfs_closure` over target+primitives in
Rust); the Rust side had the same gap latently, since only the Lean
path had arena coverage.
- The removal of the O(env) reverse wrapper scan, which was that
commit's actual win, is unaffected.
Tests
- `lake test -- --ignored ixvm` green. FFT pins re-pinned across the 66
kernel-check entries and the shard pipeline; costs move in both
directions (e.g. `Eq.rec` 1_249_383 -> 1_249_094, `Vector.append`
1_994_697_378 -> 2_010_055_088), the increases coming from the
restored primitive closures.
- `ix codegen` regenerated: the Aiur kernel changed, so the committed
Rust kernel would otherwise be stale and the codegen/bytecode parity
gate would compare against it.
`Expr.Str` conversion wrapped any hash-valid blob in `KLiteral.Str`, and
`k_infer_lit` types a `KLiteral.Str` as `String` without inspecting the
bytes. Nothing on the ordinary checking path ever decodes a literal, so a
prover-authored definition carrying malformed bytes typechecked as
`x : String` even though no Lean `String` corresponds to those bytes. The
reference kernel rejects this at ingress via `String::from_utf8`
(crates/kernel/src/ingress.rs).
The UTF-8 validation added earlier lives in the string DECODER, which is
reached only during string reductions, so it did not cover this path.
`utf8_decode_one` already rejects out-of-range continuation bytes, stray
continuations used as leaders, and overlong forms. Walk the whole blob
with it at conversion (`utf8_validate`), which is the same rejection the
reference kernel performs.
Tests
- `invalid-utf8-accepted-as-string` in the adversarial Ixon suite: a
definition declaring `String` over the blob `[0xFF]`, which no
well-formed UTF-8 can contain. Accepted before this change, now
rejected.
- Paired with `control-valid-utf8-string-literal`, the same shape over
`[0x68, 0x69]`, which must still be accepted — so the verdict is
attributable to the bytes rather than to the fixture's shape. The
fixtures build on a base env supplying the real `String` constant, so
the declared type is the genuine one.
- Six FFT pins move, all string-touching, by between +17 and +66_530
(e.g. `IxVMPrim.str_size_lit` 567_999_042 -> 568_000_142,
`Vector.append` 2_010_055_088 -> 2_010_088_729, +0.0017%). Constants
with no string literals are unchanged, `Nat.add_comm` included.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`add_entries_parallel` returned early after a constant hit, so an address
present in both `env.consts` and `env.blobs` was seeded on channel 2 but
never on channel 4. The Lean seeder walks `consts` and `blobs` in
separate passes and seeds both, so the two builders disagreed, and the
Rust-native check/prove paths aborted with `invalid IO key` on such an
environment.
The two tables are not disjoint. Constant and blob addresses share a hash
codomain, and it is the Expr context — `Ref`/`Prj` versus `Str`/`Nat` —
that says which table a reference means, so the same bytes can legitimately
be read both ways. The kernel faults the two channels separately, so the
witness has to seed them separately. Reaching an address through both
kinds of use became expressible when the claim walk moved to selecting
indices by positive constant-use.
Corrects the module header and the `ChannelEntries` doc, which both
asserted the disjointness this relies on not having.
Tests
- `dual_use_address_is_seeded_on_both_channels`: an address stored as a
constant and as a blob must appear under both `(ch 2, key)` and
`(ch 4, key)` in the built IOBuffer. Fails on the previous code with
the ch-4 entry missing.
- `lake test -- --ignored ixvm` green; clippy `--all-targets` and fmt
clean.
The kernel walks a constant's `refs` by positive constant-use — index `i`
is followed only when some Expr names it via `Ref(i, ..)` or `Prj(i, ..)`
— but both host claim builders still treated every `refs` entry as a walk
edge. The host and the kernel therefore disagreed about the edge
relation.
A `refs` entry that is a real constant but that no Expr uses as one is
never walked in-circuit, so admitting it host-side puts a phantom member
in the thin frontier: a cross-shard assumption the kernel never makes,
and a spurious dependency edge that can close a cycle once assumptions
are discharged. `walk_check_set` likewise stopped computing the set the
verifier actually checks.
Mirror `const_idxs_of` on both hosts and emit an edge only for
positively-used indices. `Env::bfs_closure` is deliberately left alone:
it computes the witness BYTE scope, where over-approximating is correct
(shipping unused bytes is free, missing bytes aborts).
Honest constants are unaffected, since a compiler only puts an address in
`refs` when an Expr references it — the shard FFT pin is unchanged. The
divergence is reachable only through prover-authored `refs`.
Tests
- `thin_frontier_excludes_unused_ref_entries`: `refs` carries two
constants but only index 0 is named by a `Ref`; the frontier must
contain the used one and not the dead one. Verified red against the
previous edge relation ("unused refs entry became a phantom frontier
member").
- `const_with_refs` now names each of its `refs` entries via a `Ref`
node. The old helper left them unreferenced, which under the walk rule
would have produced no edges at all and quietly hollowed out the
existing frontier tests.
- `lake test -- --ignored ixvm` green; clippy `--all-targets` and fmt
clean.
`idx_to_u64` packed a synthesized member/constructor index into the
projection wrapper's `u64` field by populating only the low byte behind a
`u8_range_check`, so a mutual block with more than 256 members — or an
inductive with more than 256 constructors, which is the likelier case —
was rejected outright with `value 256 out of u8 range`.
Field-to-bytes is not free: the bytes have to be supplied as advice and
then pinned. Take them from `unconstrained_g_to_bytes` and discharge the
three obligations its contract names:
- range: four `u8_range_check` calls cover the eight bytes, the byte chip
taking a pair per lookup row;
- recomposition and canonicality: `flatten_u64(bytes) == idx`, since
`flatten_u64` asserts the top byte is zero and sums the low seven.
Canonicality is what makes the decomposition unique, and it is enforced
by RESTRICTING THE DOMAIN rather than by comparing against `p`. Goldilocks
is 2^64 - 2^32 + 1, so an arbitrary eight-byte string is not injective
into the field: any value and that value plus `p` recompose to the same
element, e.g. `[00]*8` and `[01,00,00,00,FF,FF,FF,FF]` both give 0.
Forcing the top byte to zero bounds the value by 2^56 - 1 < `p`, so the
sum cannot wrap and exactly one string encodes each index. That
uniqueness is the soundness property: the projection address is blake3
over these serialized bytes, so a prover free to choose a second string
for one index would obtain a second address for a member, or collide two
members onto one address. The pre-existing `u8_from_field_unsafe`
version did collide members 0 and 256, which the range check had already
fixed by rejecting instead.
Both halves are load-bearing. `flatten_u64` does NOT range-check its
input — `[U8; 8]` is nominal typing and `u8_from_field_unsafe` mints a
`U8` holding any field element — so without the range checks the sum is
satisfiable many ways (`b1 = k`, `b0 = idx - 256k`). Its doc comment now
records that its injectivity is conditional on the caller, and names this
call site.
Indices at or beyond 2^56 still fail the assert rather than truncating.
That is a liveness bound only, 2^48 times the previous cap and far past
any real block or constructor count; buying the remaining `[2^56, p)`
would cost a multi-byte comparison against `p` for no practical gain.
Tests
- `muts-block-past-single-byte-index`: a 257-member block of well-typed
definitions with a projection wrapper for each, which now checks.
Rejected before this change with `value 256 out of u8 range`.
- `control-small-muts-block`: the same construction at two members,
already accepted — so the big-block result is attributable to the index
width alone and not to a malformed fixture.
- `storeAt` also registers a reducibility hint for `DPrj` wrappers, which
`get_ci_dprj` faults under the wrapper's own address; without it these
fixtures aborted with `invalid IO key` before reaching the index logic.
- All 66 kernel-check FFT pins and the shard pin move by +17 to +485
(`Nat.add_comm` 42_967_279 -> 42_967_313, +0.00008%), the cost of the
hint plus four range checks per synthesized index.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`get_ci` handed every standalone `Defn` a one-element `recur_addrs` whose
only entry was its own address. `Rec` converts to a plain `Const`, and
`k_infer` discharges a `Const` against its DECLARED type without checking
it, so a definition could prove its own type by citing itself:
theorem bad : False := bad
typechecked. The ref-walk cannot catch it — `const_idxs_expr` does not
follow `Rec`, and the referent is the constant already under check. Lean
forbids exactly this by checking a definition against an environment that
does not yet contain it, admitting self-reference only for `unsafe` and
`partial`.
Restrict the self slot to `unsafe`/`partial`. Those are where
self-reference is both legitimate and contained:
- legitimate, because the compiler emits a singleton (or fully
collapsed) mutual block as a standalone `Defn` whose body still says
`Rec(0)` -- see the note at Ix/CompileM.lean:2000. Those blocks are
exactly Lean's `unsafe`/`partial` recursive definitions; a `safe`
recursive definition compiles its recursion into `.rec` applications.
Gating on `unsafe` alone breaks `Lean.extractMainModule._unsafe_rec`.
- contained, because safe code may not reference either kind.
That containment was itself missing for `partial`: `is_unsafe_ci`
classified it as safe, so `partial def bad : False := bad` plus a safe
theorem referencing `bad` would have walked straight around the first
half of this fix. Count `Partial` alongside `Unsafe`, matching the
reference's "safe definition references partial definition" rejection
(crates/kernel/src/check.rs:813-820). The predicate serves double duty --
the caller's own leniency and the referent's barrier -- and `Partial`
belongs on the non-safe side of both.
Tests
- `self-referential-theorem-proves-false`: `theorem bad : False := bad`
built directly in Ixon over the real `False` constant, submitted
through the production `verify_claim` entrypoint. Accepted before this
change -- a proof of False -- and rejected after.
- No safe constant in the corpus references a partial definition, and
the `_unsafe_rec` witnesses still check, so neither half of the fix
costs coverage.
- All 66 kernel-check FFT pins and the shard pin move (`Nat.add_comm`
42_967_313 -> 42_965_666); the self slot is no longer built for safe
definitions, so most costs drop slightly.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
The nested branch of the strict-positivity check dropped the parameter
arguments (`list_drop(args, n_ctor_params)`) and descended into the
nested inductive's constructors without substituting them. A nested
constructor's fields mention that inductive's parameters as bound
variables, so the block-occurrence test asked whether the block occurs in
`BVar 0` — it never does, and every parameter position passed vacuously.
The occurrence being hunted lives in the ARGUMENT substituted for the
parameter, which was exactly what got discarded.
So for `Inner α | mk : (α → False) → Inner α`, itself strictly positive,
`Host | mk : Inner Host → Host` was accepted: descending into `Inner.mk`
sees the field `α → False`, which mentions nothing tracked. With `α :=
Host` it reads `(Host → False) → …`, a negative occurrence, and Lean
rejects it. Accepting it admits `False`.
Mirror `check_nested_ctor_fields` (crates/kernel/src/inductive.rs): strip
`n_params` foralls, then simultaneously substitute the parameter
arguments at depth 0. After stripping, `BVar 0` is the innermost (last)
parameter, so the argument list is reversed — `expr_inst_many` maps
`substs[i]` to `BVar(depth + i)`, the same convention as `simul_subst`.
The parameters are substituted away rather than bound, so the field walk
now starts with an empty binder context instead of the peeled types.
Tests
- `nested-inductive-negative-occurrence`: the `Inner`/`Host` pair above,
built directly in Ixon as two single-inductive `Prop` blocks with their
projection wrappers. Accepted before this change, and now rejected by
the intended assert ("strict positivity: block occurs left of an
arrow").
- `control-nested-positive-inductive`: the same construction with
`Inner α | mk : α → Inner α`, which is strictly positive and must keep
checking. The two fixtures differ only in the inner constructor's
field, so the rejection is attributable to the negative occurrence
rather than to nesting.
- 63 FFT pins and the shard pin move, concentrated in inductive-heavy
constants (`IxVMInd.Tree` 767_167 -> 796_717, `IxVMInd.DepthM`
1_312_461 -> 1_347_394) — the nested check now does real work instead
of returning vacuously.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`KLimbs` is canonical in two independent ways — no trailing zero limbs,
and every limb byte in [0, 256) — and `klimbs_eq` is a raw limb compare
that establishes neither. Since it backs `Nat.beq`, `Nat.decEq` and
`literal_eq`, any producer emitting a value-correct but
representation-wrong `KLimbs` makes those answer `false` where Lean
answers `true`. That is an accepted false equation, not an abort. Two
producers did.
`String.utf8ByteSize` built the length as one unchecked byte
(`[u8_from_field_unsafe(n), 0, ...]`) with no normalization. The empty
string therefore reduced to the denormalized zero `[[0;8]]` instead of
the canonical empty limb list, and a string of 256 bytes or more to an
out-of-range digit — `u8_from_field_unsafe` is an erased coercion, so
nothing truncated or rejected. Build it with `klimbs_from_g`, whose byte
split is range-checked and pinned by a recomposition assert, then
normalize.
`klimbs_div_mod` pinned the quotient's VALUE (`q*b + r == a`, `r < b`)
but not its digits. The comment claimed `u8_mul` inside `klimbs_mul`
range-checked them, but `u64_mul` was rewritten to raw field products
plus `#split_carry`, whose u8 checks constrain the split outputs rather
than the input digits — and it re-canonicalizes while multiplying, so a
digit-wrong `q` still produces a canonical `q*b` and satisfies the
equality. Range-check the hint's limbs explicitly, quotient and
remainder both, via a new `klimbs_range_check`.
Tests
- `utf8-size-beq-false-accepted`: `Nat.beq (String.utf8ByteSize "") 0 =
false` proved by `Eq.refl`, over the real `Eq`/`Bool`/`Nat.beq`/
`String.utf8ByteSize` constants. The kernel had the polarity exactly
backwards — this FALSE equation was accepted while the true one was
rejected.
- `control-utf8-size-beq-true`: the same declaration ending in `true`,
which is what Lean computes and must be accepted. The pair differs only
in that operand, so each verdict is attributable to the reduction.
Both now land the right way round.
- The two fixtures share one blob for `""` and `0` — the same empty byte
string read as a string literal and as a nat literal, which is the
Expr-context disambiguation working as intended.
- 10 FFT pins and the shard pin move, confined to the div/mod/gcd/pow/
shift constants (the hint range-checks) and the string-size ones
(`IxVMPrim.str_size_lit` 567_992_937 -> 567_993_099).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT WAS DEMONSTRATED for this one. Unlike the other soundness
fixes in this series it ships without a red-then-green fixture; what
follows is why the rule is wrong as written, and what remains unproven.
`try_eta_struct` accepted `t ≡ S.mk p… a…` after checking only that the
right side is a fully-applied constructor of a struct-like inductive. It
never checked that the two sides have def-eq TYPES. All three references
do, citing lean4lean's `tryEtaStructCore`
(crates/kernel/src/def_eq.rs, Ix/Tc/DefEq.lean).
Without the guard the rule degenerates: for ANY `t` of ANY type,
`compare_struct_fields` builds precisely `Proj(S, i, t)` for each field,
so when the right side's arguments are those same projections the
pointer fast path matches them and the pair is accepted. Those
projections are synthesized inside def-eq and never inferred, so
`k_infer_proj`'s head-address gate — which would reject `Proj S i t` for
`t : Nat` — never sees them.
What is unproven is reachability. At the `k_check(e, expected)` boundary
both sides are types in the same context, and `de_args` compares type
arguments before value arguments, so I could not construct a term that
reaches this rule with genuinely mismatched types. The change is
therefore defense-in-depth against an unjustified rule, not a repair of a
demonstrated break.
The guard runs after the struct-like shape test, so it costs two
`k_infer_only` calls and one `k_is_def_eq` only on pairs that were about
to be accepted. `try_proof_irrel`, earlier in the same fallback chain,
already infers the left side unconditionally, so the added abort exposure
is small; the suite confirms no constant stops checking.
Tests
- No new fixture; the existing 14 adversarial cases and the arena corpus
are unchanged.
- 45 FFT pins and the shard pin move, the largest cost of this series
(`Nat.add_comm` 42_965_759 -> 42_966_970, +0.003%;
`IxVMPrim.str_size_lit` +26_687).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
A `Tag0` large-form header carries a 7-bit `small` field, so a payload
width up to 128 was expressible and unguarded (`Tag2`: 5 bits, up to 32).
A u64 payload is at most 8 bytes, and the three readers disagreed on the
surplus:
- Rust `u64_get_trimmed_le` rejects `len > 8` -- already correct, and
untouched here.
- The in-circuit `get_u64_le` consumed the surplus and DISCARDED it,
keeping only the first eight bytes.
- The Lean host folded it back in: `UInt64.shiftLeft` is taken mod 64,
so byte 8 ORs into bits 0-7, with no length guard.
Same bytes, same content address, three readings. An `Expr.Ref` whose
index is written overlong resolves as `refs[0]` in-circuit and `refs[42]`
on the Lean host, with both consuming the same span so nothing desyncs
afterwards. That also means the trailing-bytes assert does not imply
canonicity: up to 120 arbitrary bytes per `Tag0` site were consumed
inside the structure and dropped.
Bring the other two up to Rust: assert the width is under 9 in both
in-circuit readers, and give `getU64TrimmedLE` the guard it was missing.
No honest serializer is affected -- `putTag0` always emits the minimal
width -- so this narrows the accepted language to the format.
Tests
- `overlong-tag0-accepted`: an axiom `_ : Prop` whose `lvls` Tag0 is
rewritten to the nine-byte form. Accepted before this change, now
rejected by the intended assert.
- `control-canonical-tag0-axiom`: the same axiom with canonical bytes,
which must stay accepted. The pair differs only in that encoding.
- Both need bytes no `Ixon.ser` will emit, so the suite gained
`storeRawAt`, which stores a hand-authored byte string at its own
content address. Every previous fixture was built from a `Constant`
VALUE and was therefore canonical by construction, and structurally
could not reach this class of defect.
- All 66 kernel-check FFT pins and the shard pin move (`Nat.add_comm`
42_966_970 -> 42_980_767, +0.03%); every constant parses Tag0s.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
Four test-only fixes. No kernel code changes, so no FFT pins move.
prim-addrs asserted the literal `true`. It looked up each hardcoded
kernel address in the canonical table and, on a hit, recorded a pass —
but the table is keyed BY ADDRESS, so a permuted table still hits. Had
`nat_add_addr` held `Nat.mul`'s bytes it would have printed
"nat_add_addr = Nat.mul" and passed green, which is precisely the defect
the file's own docstring says it exists to catch: the kernel applying one
primitive's native semantics under another's name. Now the address must
resolve to the constant the function's NAME claims, comparing normalized
keys (`nat_add_addr` <-> `Nat.add`, and the final-component spelling for
`reduce_bool_addr` <-> `Lean.reduceBool`), with dispatch-site suffixes
(`_dec`/`_io`/`_iota`) and `_type`/`_ty` stripped, plus a short exception
list for the `Quot` constructors. Verified by temporarily swapping
`nat_add_addr` to `Nat.mul`'s bytes: red, then reverted. All 76 literals
pass, and the table is confirmed correct today.
The arena negative corpus counted ANY abort as a correct rejection while
its docstring claimed rejections come from an in-kernel `assert_eq!`. An
`invalid IO key` — the witness never supplied the bytes, so execution
died before reaching the logic under test — scored as a pass. That is how
a fixture rots into a green test that checks nothing, and it is the same
failure mode that made two exploit fixtures in this series pass for the
wrong reason. Now `invalid IO key` disqualifies. Unmatched-match aborts
still count: this kernel deliberately relies on an unmatched runtime
value aborting instead of spending width on `_ => assert_eq!(0, 1)`
arms, so those are genuine verdicts. No fixture currently relies on the
disqualified path.
`shardCheckEnvCase` returned `none` for two unrelated reasons and the
runner reported both as "SKIP (target absent)", a passing test. The
second — target present but the partition selected zero owned constants —
means the fixture stopped exercising the Rust witness builder and the
thin-frontier claim convention, and would have left the only test of that
path permanently green. It now throws; `none` keeps its one honest
meaning.
A `--ignored` filter naming no runner ran nothing and returned 0, so
`lake test -- --ignored ixvm2` reported success having executed nothing.
It is now an error. Exit 0 should mean tests ran and passed.
`ixVM`, the pruned production toplevel, carried `verify_const`,
`verify_check` and `verify_check_env` alongside `verify_claim`. Those are
DEBUG entrypoints that trust their dependencies instead of walking them,
so a proof made with one attests far less than a claim does — and
carrying them meant such a proof was verifiable under the SAME verifying
key. The only thing standing between it and acceptance was `ix verify`
pinning the funcIdx to `verify_claim`: a policy, enforced in one place,
with no test asserting it, and no policy at all in the recursive
verifier, which binds the claim bytes but applies no funcIdx check.
Prune to `verify_claim` alone. The guarantee is now structural: under the
production VK a debug-entrypoint proof cannot exist. No CLI path used
them, so nothing in `ix check`/`prove`/`verify` changes.
Callers that genuinely want a subject-only check build `ixVMFull` and are
thereby using a different VK, which is the honest signal — the arena
corpus, and `bench-typecheck`/`bench-recursion-debug` under
`--skip-deps`. That last one also means a `--skip-deps` benchmark is now
visibly measuring a different key than a claim run rather than silently
reporting `verify_const` rows in the claim schema.
`bench-typecheck` also reported success on failure, twice over. A proof
that failed to verify printed to stderr but left `failed` clear, so the
row was emitted as `"status": "ok"` with prove-time and proof-size
present and the run exited 0 — the only signal an absent `verify-time`
key, invisible to anything keying on status. And a constant whose execute
threw was dropped from the results entirely without setting `failed`,
so `execed.any (·.failed)` stayed clear; the `check_const`-missing throw
added earlier landed exactly there. Both now mark the row failed.
Tests
- `lake test -- --ignored ixvm` green with NO pin movement: the pruned
entrypoints were never executed, so removing them changes no cost.
Generated kernel drops from 733 to 731 functions.
- Arena repointed to `ixVMFull`, where `verify_const` still lives.
- `bench-typecheck --execute-only` smoke unchanged (`Nat.add_comm`,
constants=42).
`compare_rules` peeled the binders off the STORED rule and handed their
DOMAINS to `k_is_def_eq` as the local context for the comparison against
the canonical reconstruction. Those domains are prover-authored: nothing
checks them against the reconstruction, and reduction never consults them
(`try_iota` beta-reduces straight through). So the prover chose what the
two sides INFER to.
Retyping every binder to a Prop makes both sides infer to that Prop, so
`try_proof_irrel` fires and accepts ANY body. A recursor whose
computation rules are swapped between constructors was therefore
accepted, and iota then reduces each constructor to the other's branch.
Compare under an empty context instead, discarding the peeled domains. An
honest rule matches the reconstruction structurally, and the structural
path never consults the context — it compares `BVar i` against `BVar i`
directly. A comparison that instead needs to INFER a bound variable's
type now runs out of context and aborts, which is the safe direction.
Every recursor in the corpus still checks, including the mutual, nested
and aux-generated ones.
The reference avoids this differently, comparing whole Lam-wrapped terms
at empty context so `is_def_eq` descends binder-by-binder and compares
the domains themselves (crates/kernel/src/inductive.rs). Reconstructing
canonical domains here would mean re-deriving field domains from each
constructor's telescope and re-lifting their de Bruijn indices into the
rule's context; dropping the untrusted context achieves the same
guarantee without that machinery.
Tests
- `recursor-rule-rhs-doctored-binder-domains`: a real compiled recursor
with its first two computation rules swapped AND every peeled binder
domain replaced by a Prop. Accepted before this change; rejected now.
- `control-swapped-recursor-rules`: the identical swap with the
compiler's own binder domains, which was already rejected. The two
differ only in the domains, so the acceptance was attributable to them
alone.
- Both are built by rewriting a REAL block rather than hand-authoring a
recursor, since a hand-built one must survive `build_rec_type`'s def-eq
and constructor references inside a block are circular with the block's
own address. The fixture selects a recursor whose rules are field-free,
so no minor premise is applied and the retyping cannot abort as a
non-function before the comparison is reached — an earlier attempt did
exactly that and masked the acceptance.
- A guard case fails loudly if no such recursor is found; the first
version of this fixture silently corrupted nothing, and both its cases
"passed" meaninglessly.
- 57 FFT pins and the shard pin move (`Nat.add_comm` 42_980_767 ->
42_980_801).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT DEMONSTRATED. Defense-in-depth, like the struct-eta guard.
`compare_rules` now asserts three invariants that equivalents of already
exist in `check_rec_rules_walk` — which nothing calls. The live path had
none of them.
* `cidx == pos`. `find_rule` selects a rule at reduction time BY LABEL
while `compare_rules` pairs stored against canonical POSITIONALLY, so
permuted labels would validate one constructor's rule and then fire
it for another. It holds today only because ingress assigns `cidx`
positionally (`Convert.lean`) and the Ixon `RecursorRule` record
carries no cidx of its own — an accident of the pipeline rather than
a checked property, which is exactly what an assert is for.
* `cidx < num_ctors`, so a rule cannot name a constructor that does not
exist.
* `nf` equals the constructor's field count. `try_iota` splits the
constructor's arguments with `field_start = ctor_fields_len -
rfields`, so a wrong `nf` hands the minor premise the wrong slice; in
field arithmetic an over-large value wraps.
The reachability argument for the `nf` gap was "combined with the rule
RHS context hole"; that hole is fixed, and with the def-eq comparison
restored a mismatched peel is caught again. So this closes a route that
is currently blocked by another check rather than one that is open —
worth asserting locally rather than depending on a second mechanism.
Not fixed here: iota still selects rules by index rather than by
constructor IDENTITY, so a recursor applied to a foreign constructor
would reduce. That is not reachable through a claim —
`k_infer_app_spine_loop` runs `k_check` on every argument, so the
application fails inference before whnf sees it — and closing it properly
needs the parent inductive available at reduction time, which
`KConstantInfo.Rec` does not carry (its `Addr` is the recursor's own
block; the parent is derived at check time from the type). That is a KCI
layout change, left for its own pass.
Tests
- No new fixture; nothing demonstrates the hole.
- Every recursor in the corpus satisfies all three invariants, including
the mutual, nested and aux-generated ones, so they held by construction
and are now enforced.
- 57 FFT pins and the shard pin move (`Nat.add_comm` 42_980_801 ->
42_981_336) for the per-rule constructor lookup.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT DEMONSTRATED. Defense-in-depth.
`try_iota` selected a computation rule by constructor INDEX alone:
`collect_spine_of_ctor` returned the major's `cidx` and discarded which
inductive it came from, so `find_rule(rules, cidx)` fired whenever any
inductive's constructor happened to carry a matching index. On its face
`Bool.rec motive f t (Nat.succ n)` reduces to `t`. Lean's kernel selects
the rule by the constructor's NAME, which pins the inductive.
Not reachable through a claim: `k_infer_app_spine_loop` runs `k_check` on
every argument, so a recursor applied to a foreign constructor fails
inference long before whnf sees the redex — the ill-typed term cannot be
stated. The Rust reference selects by index too, consistent with that
being why it was tolerated. What the patch removes is the DEPENDENCE on
that: reduction no longer needs inference to have run first to be
correct.
`collect_spine_of_ctor` now also returns the constructor's parent, as the
address of its projection wrapper, and `try_iota` asserts it equals the
recursor's own parent. That parent is read off the recursor's type (the
major premise's head) with the existing `rec_to_parent_addr`, so no
KConstantInfo layout change was needed — its `Addr` field is the
recursor's own block, not the parent's. Aiur memoizes per argument tuple,
so the type walk is paid once per recursor rather than once per
reduction.
Tests
- No new fixture; nothing demonstrates the hole.
- Every recursor in the corpus still reduces, including the mutual,
nested and aux-generated ones, and the Nat/String literal paths that
synthesize a constructor application before iota (`nat_cases_big`,
`nat_pow_big`, `strOfListFold*` all unchanged in behaviour).
- 63 FFT pins and the shard pin move (`Nat.add_comm` 42_981_336 ->
42_986_025, +0.01%) for the parent comparison.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`shardsCover` gates every shard run: "exit 0 has to mean every env const
was checked by some shard". It skipped constants whose bytes fail to
parse (`LazyConstant.get?` discards the error and `.ixe` loading is
lazy, so such a constant sits in `consts` with its key present). A
skipped constant was assigned to no shard, counted as neither owned nor
unowned, and still included in the "covers all N consts" total — so the
gate reported success over constants it had not covered.
It would also enter a referring shard's frontier, since `thin_frontier`
admits an edge on key presence in `env.consts` without parsing, leaving
an assumption no shard discharges.
Count them separately and fail, rather than folding them into `unowned`:
"cannot be assigned to a shard" is a different problem from a partition
gap, and saying so points at the right cause.
`ownedConstsForBlocks` keeps its skip — it runs per shard and would
otherwise need the same error plumbing — but now documents that it is
safe only because this gate rejects such an env first.
Not changed: the composed verdict still has no assumption-cycle check,
and on analysis it does not need one. Assumptions are discharged per
CONSTANT along ref edges, which are acyclic by construction — a
constant's address is blake3 over bytes containing its refs' addresses,
so a reference cycle needs a hash cycle. A cycle in the shard-level graph
still stands for a well-founded per-constant chain, and
`verifyShardComposition` recomputes each digest via `shardClaimDigest`,
so a self-assuming shard claim cannot be substituted.
`compile_univ` memoized `Level -> Univ`, but a `Level::Param` compiles to
`Univ::Var(i)` where `i` is the parameter's POSITION in `univ_params` —
so the result depends on a context the key omitted. Unlike the expression
cache, which is cleared at every constant boundary, `univ_cache` lives
for a whole block, and `compile_definition` compiles each member under
its own `level_params`. `preseed_expr_tables` explicitly feeds
`(expr, univ_params)` pairs with differing param lists into the same
cache.
So two members of one block placing the same parameter NAME at different
positions would give the second member the first's index. Not an abort:
it compiles cleanly and registers a constant denoting a different
universe than the Lean source, under a correct-looking name.
The hazard is already recognized one level up — `collect_expr_tables`
keys `seen_exprs` by `(expr_hash, univ_params_key)` for exactly this
reason. Apply the same key here. The Lean mirror instead drops the cache
in `withUnivCtx`, matching how it already clears the expression cache per
constant.
Unreachable from ordinary Lean sources, since the elaborator gives every
member of a `mutual` block the same `levelParams` — but nothing checked
it, and under the prover threat model the environment is authored.
Also repairs `claim.rs`'s `defn_const`, whose Definition never referenced
its own `refs`. Since the claim walk admits an index only when some Expr
uses it as a constant, that fixture produced no walk edges and
`shard_check_env_claim_is_thin_frontier` has been failing since the walk
moved to positive constant-use. The identical flaw was fixed in
`shard_claim.rs`'s helper at the time and missed here, because only
`ixon` and `ixvm-codegen` were re-tested rather than the whole workspace.
Tests
- `cargo test --release` across all crates: 1195 passed, 0 failed.
- `lake test -- --ignored ixvm` green with NO pin movement and no address
changes, confirming no block in the corpus has members that order a
shared parameter name differently. The fix removes the dependence on
that remaining true.
- clippy `--all-targets` and fmt clean.
@johnchandlerburnham
johnchandlerburnham merged commit 80bb41f into mainAug 3, 2026
10 checks passed
@johnchandlerburnham
johnchandlerburnham deleted the ap/ixvm4 branch August 3, 2026 05:37
samuelburnham added a commit that referenced this pull request Aug 4, 2026
…acking
Port of sb/measured-ingress's planning layer onto the addr-first kernel
(#529), reworked for its lazy fault-in semantics.
Profiling (.ixprof v3): the recording kernel's try_get_const consults are
drained per constant into a touch graph — the measured set a lazy checker
faults in — persisted as a presence-flagged CSR section alongside a new
block-level reference graph and a subst_unique counter (splitmix64-keyed
dedup of substitution work items: the post-memoization volume a memoizing
executor pays). Both graph sections are optional, so older readers'
formats stay decodable by version gate alone.
Planner: partition_for_aiur_ram bin-packs to a host-RAM cap with
faulted-set byte accounting — under #529 the witness always ships the
full closure, but only faulted bytes cost ingress trace rows, so a
shard's byte feature is owned ∪ touched(owned) (full reference closure
as the over-approximating fallback when no touch graph was recorded).
The stub/promotion machinery of the original branch is dropped entirely:
lazy fault-in computes at run time exactly the set the stubs predicted,
with no replay divergence and no repair ladder, and the manifest format
stays byte-identical with main (owned-only).
Cost models: nlogn(x) = x·log2(x+2) feature form matching Aiur's
width·height·log2(height) FFT structure; prove/execute time and RAM
coefficients carried over from the pre-#529 aiur bench-suite fits.
They are placeholders pending a refit against this kernel — the circuit
shrank (deleted positional machinery), the byte feature narrowed to the
faulted set, and the recording counters moved under #442 — with
provenance noted at the definitions. The costs sidecar
(<out>.costs.csv) persists per-shard features incl. subst_unique and
nat_arith so the refit has its corpus.
CLI: ix shard --backend aiur --max-ram G; ix profile --backend/--top
(per-metric block leaderboards with display names) and ix profile sweep
(env-wide closure cost sweep: per-root feature CSV + feasibility,
min-root-per-hot-block, and diversity reports).
samuelburnham added a commit that referenced this pull request Aug 8, 2026
…acking
Port of sb/measured-ingress's planning layer onto the addr-first kernel
(#529), reworked for its lazy fault-in semantics.
Profiling (.ixprof v3): the recording kernel's try_get_const consults are
drained per constant into a touch graph — the measured set a lazy checker
faults in — persisted as a presence-flagged CSR section alongside a new
block-level reference graph and a subst_unique counter (splitmix64-keyed
dedup of substitution work items: the post-memoization volume a memoizing
executor pays). Both graph sections are optional, so older readers'
formats stay decodable by version gate alone.
Planner: partition_for_aiur_ram bin-packs to a host-RAM cap with
faulted-set byte accounting — under #529 the witness always ships the
full closure, but only faulted bytes cost ingress trace rows, so a
shard's byte feature is owned ∪ touched(owned) (full reference closure
as the over-approximating fallback when no touch graph was recorded).
The stub/promotion machinery of the original branch is dropped entirely:
lazy fault-in computes at run time exactly the set the stubs predicted,
with no replay divergence and no repair ladder, and the manifest format
stays byte-identical with main (owned-only).
Cost models: nlogn(x) = x·log2(x+2) feature form matching Aiur's
width·height·log2(height) FFT structure; prove/execute time and RAM
coefficients carried over from the pre-#529 aiur bench-suite fits.
They are placeholders pending a refit against this kernel — the circuit
shrank (deleted positional machinery), the byte feature narrowed to the
faulted set, and the recording counters moved under #442 — with
provenance noted at the definitions. The costs sidecar
(<out>.costs.csv) persists per-shard features incl. subst_unique and
nat_arith so the refit has its corpus.
CLI: ix shard --backend aiur --max-ram G; ix profile --backend/--top
(per-metric block leaderboards with display names) and ix profile sweep
(env-wide closure cost sweep: per-root feature CSV + feasibility,
min-root-per-hot-block, and diversity reports).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@arthurpaulino@johnchandlerburnham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

IxVM: cheaper shard checking via an addr-first kernel, plus soundness hardening - #529

Merged
johnchandlerburnham merged 19 commits into
mainfrom
ap/ixvm4
Aug 3, 2026
Merged

IxVM: cheaper shard checking via an addr-first kernel, plus soundness hardening#529
johnchandlerburnham merged 19 commits into
mainfrom
ap/ixvm4

Conversation

@arthurpaulino

@arthurpaulinoarthurpaulino commented Jul 31, 2026

Copy link
Copy Markdown
Member

Motivation

Make sharded environment checking cheap. A large environment is partitioned across shards, and each shard proves its own constants well-typed against a thin-frontier assumption cutoff, so every constant is checked in exactly one shard and cross-shard assumption trees stay minimal.

Delivering that required migrating the IxVM Aiur kernel from a positional constant table to a content-addressed, lazily-faulted design — and then auditing the result, which turned up five soundness holes with working exploits, including three separate routes to proving False.

Sharded environment checking

A shard proves CheckEnv{root, assumptions}: root is a merkle root over the shard's owned constants, assumptions a merkle root over the thin frontier — the direct out-of-owned walk edges. The kernel's env_walk starts from the owned set, checks each reachable constant, and stops at a frontier member (assumed well-typed by its owning shard: not re-checked, not recursed into).

  • Sub-frontier constants are never re-checked here; they are covered transitively by the frontier members' own shard claims. This is the cost win over checking each constant's full closure: no cross-shard duplication, and the assumption tree is one layer deep instead of the whole closure.
  • Strict membership: a walk-reachable constant in neither owned nor the frontier is a bad partition and hard-errors, so a shard cannot silently drop coverage of its own closure.
  • Byte scope ≠ check scope: delta unfolding reads through the frontier, so a witness ships the full dependency closure of owned even though only the strict-membership set is checked. The witness IOBuffer is built in Rust (aiur_ixvm_witness) rather than Lean, removing the per-byte boxing that dominated shard wall time.

Addr-first kernel

  • KExprNode.Const / Proj key on a 32-byte content Addr, not a numeric position; KConstantInfoInduct/Ctor/Rec carry a (block_addr, ind_idx) parent pair instead of indices. The positional top: List<KCI> table threaded through the kernel is gone.
  • get_ci(addr) is the sole resolver, Aiur-memoized per addr, faulting constant bytes lazily from an IOBuffer. load_verified_constant re-derives blake3 over the bytes and asserts key == addr; the io_get_info idx/len are unconstrained witness and are never branched on.
  • IOBuffer channels: 0 claim, 1 assumption tree, 2 constant bytes, 3 Defn reducibility hint (advisory), 4 blob bytes. The kernel derives blob-vs-constant from Expr context, so no wire discriminator ships. The two sets are not disjoint — one address can be read both ways — so each channel is seeded independently.
  • Kernel/Inductive.lean and Kernel/Primitive.lean deleted (~6.6k lines); inductive validation absorbed into Kernel/Check.lean, primitives split into Kernel/Klimbs.lean (bignum gadgets) and Kernel/NatPrim.lean (Nat/Str addr dispatch). New Kernel/InferOnly.lean breaks the DefEq↔Infer cycle. Ingress.lean collapsed 1700→298 lines.

Recursor validation

  • build_rec_type reconstructs a recursor's canonical type and asserts it def-eq to the declared type, on both the kind-aware and address-only lookup paths. Without it a recursor could declare ... -> False while its rules still reconstructed correctly.
  • spec_params are lowered from the major-premise frame into the recursor-param frame at extraction: the root cause of 17 nested/aux recursor reconstruction failures, and why the canonical check accepted a superset.
  • k_flag and parent-inductive-shape checks hoisted out of the attacker-selectable is_aux gate; the canonical rules comparison, which formerly never ran, now runs for every recursor.

Pre-existing criticals

IDFix
P1blob-vs-constant derived from Expr context instead of a prover-answered io_get_info probe, retiring the ch-2 len=0 sentinel; a prover can no longer mark a walk root or owned leaf "skip me"
P2Nat.div n 0 pins the quotient to zero (was prover-chosen)
P3klimbs_sub/le corrected on operands with trailing zero limbs; div/mod hints normalized before use
P4is_large_eliminator field-index arithmetic parenthesized (Aiur - is right-associative)
P5Reveal recursor rules assert the claimed ruleIdx equals its position

Robustness

  • L5: UTF-8 string-literal decode validates continuation-byte ranges and per-length overlong minimums, so distinct byte sequences (e.g. overlong NUL C0 80 vs 00) no longer decode to the same codepoint and def-eq as equal.
  • L6: run_claim dispatches on the full tag4 size (was the low byte only), matching Rust Claim::get.
  • L8: the K-recursor synth gate checks the major's inferred inductive against the recursor's own, not against itself, so I.rec no longer strips a major of a different inductive.

Aiur assert_eq! messages and circuit cleanup

  • assert_eq!(a, b, "why") gains an optional diagnostic label, threaded Source.Term → Typed → Simple → Concrete → Bytecode.Op → Rust Op::AssertEq(_, _, Option<String>) → ExecError::AssertEqMismatch. The message is diagnostic-only and never enters the circuit; all ~122 IxVM assert sites are labeled.
  • Unreachable _ => assert_eq!(0, 1) match arms removed: an unmatched runtime value already aborts (the kernel's reject), so the extra arm only widened the circuit.
  • Address equality is a full 32-byte content comparison — never a pointer-value compare, which is unsound for distinct-pointer/same-data — packed as five base-256 field limbs behind a byte-0 prefilter.

Witness construction

witness_scope reuses Env::bfs_closure, which already follows refs, Prj → block, and the derived Muts → member/ctor projection edges, so the wrappers get_ci synthesizes ship as forward edges of a single O(closure) walk with no reverse scan of the env. Primitive addresses are seeded as additional walk roots, because def-eq delta-unfolds primitive bodies whose refs are reachable from nowhere else. The Lean seeder computes the same scope by the same rule, so the two cannot drift.

bench-typecheck's constants metric now reads check_const's unique query count — exactly the constants typechecked — instead of the shipped byte scope, which is a superset and inflated throughput.

Soundness audit

Eight parallel read-only audits covered the pipeline end-to-end (compile, Ixon serde, ingress/convert, type-theory core, declaration checking, claim/witness layer, primitives, entry points), under the zkDSL threat model: a prover authors the data and pays for its own trace, so aborts and rejections are not threats and only a wrong answer counts. Thirty findings; every claim relayed here was re-verified against the code before acting on it, and several agent claims were corrected or withdrawn in the process.

Fixed, with a working exploit

Each of these shipped with a fixture that was red before the fix and green after, paired with a minimal-pair control so the verdict is attributable to one variable.

HoleWhat it bought an attacker
Self-referential definitionstheorem bad : False := bad typechecked. A standalone Defn was handed a recur slot naming itself, Rec erases to an ordinary Const, and k_infer reads a Const's declared type without checking it — so the body discharged its own type by citing itself. Now restricted to unsafe/partial, which is where Lean permits self-reference and where safe code cannot reach
Nested-inductive positivityThe nested branch dropped the parameter arguments and descended into the nested inductive's constructors without substituting them, so the occurrence being hunted was never present. Host | mk : Inner Host → Host was accepted for a negative Inner. Admits False
Recursor rules compared under the prover's contextcompare_rules handed the stored rule's own binder domains to k_is_def_eq as the comparison context. Retyping them to a Prop makes both sides infer to that Prop, proof irrelevance fires, and a recursor with its computation rules swapped between constructors was accepted
Non-canonical computed Nat literalsString.utf8ByteSize built a KLimbs as one unchecked byte, and klimbs_div_mod pinned the quotient's value but not its digits. Since klimbs_eq compares limbs rather than values, the kernel had the polarity backwards: Nat.beq (String.utf8ByteSize "") 0 = false was accepted while the true equation was rejected
Tag0/Tag2 overlong payloadsThree readers, three answers on the same bytes: the circuit discarded bytes past the eighth, the Lean host folded them back in (its shift is mod 64), Rust rejected. One address denoted different constants to different readers

Fixed as defense-in-depth

No exploit demonstrated for these; each commit says so in its first line. They remove a dependence on some other check happening to hold.

  • Struct-eta type equality.try_eta_struct omitted the infer(t) ≟ infer(s) side condition all three references carry, degenerating to t ≡ S.mk … (Proj S i t) … for any t of any type.
  • Recursor rule labels.cidx selects a rule at reduction time but was validated only positionally; nf decides how iota splits the constructor's arguments and was never checked against the constructor. Equivalents existed only in dead code.
  • Iota constructor identity. Rules were selected by index alone, so any inductive's constructor with a matching index fired one. Unreachable through a claim — inference k_checks every argument first — but reduction should not depend on that.
  • Universe cache context.compile_univ memoized Level → Univ while a Level::Param compiles to a position in univ_params, and the cache outlives a constant. Now keyed by (level, univ_params_key), matching what collect_expr_tables already does one level up.

Entry points and coverage

  • verify_const / verify_check / verify_check_env are DEBUG entrypoints that trust their dependencies, and they shipped in the production verifying key alongside verify_claim — leaving ix verify's funcIdx check as the only barrier, untested, with no equivalent in the recursive verifier. The production toplevel is now pruned to verify_claim alone, so such a proof cannot exist under that VK. Zero FFT cost.
  • bench-typecheck reported "status": "ok" and exit 0 when verification failed, and dropped thrown constants without marking them failed.
  • shardsCover skipped constants whose bytes do not parse — assigning them to no shard, counting them as neither owned nor unowned, and still including them in "covers all N consts". Now a hard failure.

Test guards that could not fail

Four of these were found by auditing the tests themselves, and they matter because they were the safety net for everything above.

  • prim-addrs asserted the literal true. Keyed by address, so a permuted primitive table passed — precisely the defect its docstring claimed to catch. Now the address must resolve to the constant the function's name claims; verified by temporarily swapping nat_add_addr to Nat.mul's bytes and watching it go red.
  • The arena negative corpus counted any abort as a correct rejection, so a fixture aborting on invalid IO key — never reaching the code under test — scored as a pass. That is the same failure mode that made two exploit fixtures in this series pass for the wrong reason before it was caught.
  • The shard-pipeline SKIP was a passing test conflating "target absent" with "partition selected zero constants".
  • A --ignored filter naming no runner executed nothing and returned 0, so a typo'd suite name reported success having run no tests.

Adversarial Ixon test suite

New Tests/Ix/IxVM/Exploits.lean, 19 cases, wired into the ixvm suite. The kernel-arena corpus attacks the layer above: its bad_* fixtures are raw Lean.ConstantInfo declarations that skip the elaborator, but they are still compiled by rsCompileEnvFFI, so the Ixon they produce is always well-formed. This suite authors Ixon.Constant values directly, and storeRawAt goes one level lower still, storing hand-authored byte strings — needed because every value-level fixture is canonical by construction and so cannot express a malformed encoding.

Each case records the false statement acceptance would buy, so a failure reads as a security statement. Rejection reasons are printed, which earned its keep repeatedly: fixtures that "passed" via invalid IO key without reaching the checker were caught this way, as was a fixture that silently corrupted nothing because non-mutual recursors are standalone constants rather than Muts members.

Testing

  • lake test -- --ignored ixvm green.
  • cargo test --release across all crates: 1195 passed, 0 failed. This also repaired shard_check_env_claim_is_thin_frontier, red since the claim walk moved to positive constant-use because its fixture's Exprs never referenced their own refs.
  • New prim-addrs parity suite; codegen ↔ bytecode parity gate green; arena negative corpus (77 bad_*) all reject, now only via genuine in-kernel refusals.
  • FFT-cost pins re-pinned per change. Cumulative effect is small (Nat.add_comm 42_747_927 → 42_986_025, +0.6% over the whole series); the largest single contributors are the nested-positivity substitution and the struct-eta guard.
  • ix codegen regenerated whenever the Aiur kernel changed, so the committed Rust kernel never lags the source the parity gate compares against.

Known limitations

Recorded rather than silently dropped.

  • Call-site surgery is not covered by the value pin. The compiler rewrites calls to aux-generated constants — permuting arguments, dropping some, replacing the head and its universe list — and the plans driving it live only in ConstantMeta, which prune_to_closure_anon excludes. So a verified claim says the term at address X is well-typed, not that term is the Lean declaration you wrote; the equivalence rests on the rewrite being faithful, checked today only by decompile round-trip tests. Closing it is a claim-format decision, not a patch.
  • Assumption cycles are not rejected, and a conditional Check claim naming its own target is accepted. That is the specified semantics — some root means "conditional on every leaf", so the claim asserts A → A, vacuous rather than false — and the obligation sits with whoever discharges assumptions. Sharded CheckEnv is not exposed, since shardClaimDigest recomputes the claim from (env, blocks). Pinned as ACCEPT so the boundary cannot move silently.
  • Alpha-collapse comparators ignore safety, so two members agreeing on kind/lvls/typ/value and disagreeing on safety collapse. Deliberately not fixed: a prover authors the environment directly and never goes through the compiler, so hardening it stops no attacker, while adding safety to the ordering would move every address and refusing the collapse would put a hard error in the compiler on the strength of an unverified assumption.
  • klimbs_le / klimbs_eq are in the logical trusted base, not just the performance path: Nat.decLe mints Decidable.isTrue with a proof term whose correctness is asserted rather than checked, so one wrong verdict yields False.
  • check_inductive_shape_ctors omits validate_expr_well_scoped / k_ensure_sort / assert_safety that check_const's Ctor arm performs; only an entirely unreferenced constructor escapes those.

@arthurpaulino
arthurpaulinoforce-pushed the ap/ixvm4 branch 4 times, most recently from 06e0489 to 559ced9CompareAugust 1, 2026 00:53
… hardening
Make sharded environment checking cheap. A large environment is partitioned
across shards, and each shard proves its own constants well-typed against a
thin-frontier assumption cutoff, so every constant is checked in exactly one
shard and cross-shard assumption trees stay minimal. Delivering that required
migrating the IxVM Aiur kernel from a positional constant table to a
content-addressed, lazily-faulted design -- and closing the soundness
regressions that migration introduced along with several pre-existing holes.
Sharded environment checking
- A shard proves `CheckEnv{root, assumptions}`: root is a merkle over the
shard's owned constants, assumptions a merkle over the THIN FRONTIER --
the direct out-of-owned walk edges. The kernel's env_walk starts from the
owned set, checks each reachable constant, and stops at a frontier member
(assumed well-typed by its owning shard, not re-checked, not recursed).
- Sub-frontier constants are never re-checked here; they are covered
transitively by the frontier members' own shard claims. This is the cost
win over checking each constant's full closure: no cross-shard duplication,
and the assumption tree is one layer deep instead of the whole closure.
- Strict membership: a walk-reachable constant in neither owned nor the
frontier is a bad partition and hard-errors, so a shard cannot silently
drop coverage of its own closure.
- Byte scope != check scope: delta unfolding reads through the frontier, so
a witness ships the full walk closure of owned even though only the
strict-membership set is checked. The witness IOBuffer is built in Rust
(aiur_ixvm_witness) rather than Lean, removing the per-byte boxing that
dominated shard wall time.
Addr-first kernel
- KExprNode.Const/Proj key on a 32-byte content Addr, not a numeric
position; KConstantInfo Induct/Ctor/Rec carry a (block_addr, ind_idx)
parent pair instead of indices. The positional `top: List<KCI>` table
threaded through the kernel is gone.
- get_ci(addr) is the sole resolver, Aiur-memoized per addr, faulting
constant bytes lazily from an IOBuffer. load_verified_constant re-derives
blake3 over the bytes and asserts key == addr; the io_get_info idx/len
are unconstrained witness and are never branched on.
- IOBuffer channels: 0 claim, 1 assumption tree, 2 constant bytes, 3 Defn
reducibility hint (advisory), 4 blob bytes. An address is on ch2 iff a
constant and ch4 iff a blob; the kernel derives which from Expr context
(Str/Nat -> blob, Ref/Prj -> constant), so no wire discriminator ships.
- Kernel/Inductive.lean and Kernel/Primitive.lean deleted (~6.6k lines);
inductive validation absorbed into Kernel/Check.lean, primitives split
into Kernel/Klimbs.lean (bignum gadgets) and Kernel/NatPrim.lean (Nat/Str
addr dispatch). New Kernel/InferOnly.lean breaks the DefEq<->Infer cycle.
Ingress.lean collapsed 1700->298 lines. V1/V2 naming removed throughout.
Recursor validation (R1, R2, R3)
- build_rec_type reconstructs a recursor's canonical type and asserts it
def-eq to the declared type, on both the kind-aware and address-only
lookup paths. Without it a recursor could declare `... -> False` while
its rules still reconstructed correctly.
- spec_params are lowered from the major-premise frame into the
recursor-param frame at extraction: the root cause of 17 nested/aux
recursor reconstruction failures, and why the canonical check accepted a
superset.
- k_flag and parent-inductive-shape checks hoisted out of the
attacker-selectable `is_aux` gate; the canonical rules comparison, which
formerly never ran, now runs for every recursor.
Pre-existing criticals (P1-P5)
- P1: blob-vs-constant is derived from Expr context instead of a prover-
answered io_get_info probe, retiring the ch-2 len=0 sentinel; a prover
can no longer mark a walk root or owned leaf "skip me".
- P2: Nat.div n 0 pins the quotient to zero (was prover-chosen).
- P3: klimbs_sub/le corrected on operands with trailing zero limbs; div/mod
hints normalized before use.
- P4: is_large_eliminator field-index arithmetic parenthesized (Aiur `-` is
right-associative).
- P5: Reveal recursor rules assert the claimed ruleIdx equals its position.
Robustness (L5, L6, L8)
- L5: UTF-8 string-literal decode validates continuation-byte ranges and
per-length overlong minimums, so distinct byte sequences (e.g. overlong
NUL C0 80 vs 00) no longer decode to the same codepoint and def-eq as
equal. Roughly zero added FFT cost.
- L6: run_claim dispatches on the full tag4 size (was the low byte only),
matching Rust Claim::get.
- L8: the K-recursor synth gate checks the major's inferred inductive
against the recursor's own (via se_parent_addr), not against itself, so
I.rec no longer strips a major of a different inductive.
Aiur assert_eq! messages + circuit cleanup
- assert_eq!(a, b, "why") gains an optional diagnostic label, threaded
Source.Term -> Typed -> Simple -> Concrete -> Bytecode.Op -> Rust
Op::AssertEq(_, _, Option<String>) -> ExecError::AssertEqMismatch. The
message is diagnostic-only and never enters the circuit; all ~122 IxVM
assert sites are labeled.
- Unreachable `_ => assert_eq!(0, 1)` match arms removed: an unmatched
runtime value already aborts (the kernel's reject), so the extra arm only
widened the circuit.
- address equality is a full 32-byte content comparison (never a pointer-
value compare, which is unsound for distinct-pointer/same-data), packed
as five base-256 field limbs (7+7+7+7+4) behind a byte-0 prefilter and
short-circuiting on the first mismatched group.
Crate reorganization
- PrimAddrs / reserved_marker_name moved verbatim from
crates/kernel/src/primitive.rs to crates/common/src/prim_addrs.rs (kernel
re-exports), so witness builders reach the primitive address table
without an ix-kernel -> ixon -> ix-kernel cycle.
- witness_scope reuses Env::bfs_closure -- the env's own dependency-closure
pass, which already follows refs, Prj->block, and the derived
Muts->member/ctor projection edges -- so the wrappers the kernel's get_ci
synthesizes ship as forward edges of a single O(closure) walk, with no
reverse scan of the whole env. Primitive-address seeding is unioned on
top: primitives are fabricated during reduction and need not appear in a
target's ref closure, so a closure-only scope aborts with `invalid IO
key`.
Tests
- New prim-addrs parity suite (Lean<->Aiur primitive address table).
- Codegen<->bytecode parity gate restored, green (198/198).
- FFT-cost pins re-pinned to current behavior; arena negative-fixture
corpus (77 bad_* constants) all reject.
…onstants
closureFromBase now walks the derived Muts->member/ctor projection
wrapper addresses as forward edges (projWrapperAddr: blake3 over the
serialized proj-only Constant, compile's construction), so closureFrom
and buildShardCheckEnvWitness drop their O(env) reverse scans that
parsed every env constant per witness build. Primitive addresses are
seeded bytes-only after the walk instead of being pushed onto the
worklist, so their transitive closures no longer inflate the byte scope
(a delta-unfold into an unshipped primitive body aborts with invalid IO
key rather than proving anything wrong).
bench-typecheck's constants metric now reads check_const's unique query
count (memoized per (ci, addr) = exactly the constants typechecked)
instead of closureFrom.size, which counted the shipped byte scope.
check_const is resolved up front and both lookups fail with IO.userError
instead of panicking.
Experiment: lake exe bench-typecheck --ixe seed.ixe --consts
Nat.add_comm --execute-only (seed.ixe = ix compile Tests.lean --consts
Nat.add_comm) reports constants=42, matching check_const's height in
lake exe ix check Nat.add_comm; fft-cost unchanged at 42747927.
The claim walk decided which of a constant's `refs` to recurse into by
SKIPPING every index the constant's Exprs used as a blob payload
(`Str(i)` / `Nat(i)`). That rule is unsound: the Exprs are authored by
the prover, and `refs` is one index space shared by `Ref`, `Prj`, `Str`
and `Nat`, so marking an index suppresses the walk of that index used as
a constant everywhere else in the same constant.
The marking need not even be live. `blob_idxs_of` traversed `sharing` in
full, but conversion reaches a `sharing` entry only through an
`Expr.Share` node, and nothing validates the array. A dead `Nat(i)`
entry is therefore free: it is never converted, never typechecked, and
needs no blob on channel 4. It just hides `refs[i]` from the checker.
`k_infer` then reads that dependency's DECLARED type without ever
checking it, so the kernel accepts a constant whose transitive closure
is ill-typed -- a wrong answer, not an abort. Both claim variants were
affected; for CheckEnv the skip happens before `env_walk` is entered, so
the strict owned-membership assert never fires either.
Replace the family with `const_idxs_*`: collect the indices `Ref(i, ..)`
and `Prj(i, ..)` use, and walk `refs[i]` iff `i` is among them. `Rec`
resolves through `recur_addrs` and a projection's block comes from the
ConstantInfo payload, both walked separately, so the positive set is
complete. Same single traversal as before, so no added cost. The failure
direction is now safe: over-approximating walks an address that faults
an unseeded key and aborts, which only costs a dishonest prover.
Adversarial Ixon test infrastructure
- New `Tests/Ix/IxVM/Exploits.lean`, wired into the ixvm suite. The
kernel-arena corpus attacks the layer above this one: its `bad_*`
fixtures are raw `Lean.ConstantInfo` declarations that skip the
elaborator, but they are still compiled by `rsCompileEnvFFI`, so the
Ixon they produce is always well-formed. This suite authors
`Ixon.Constant` values directly, which is what a prover can do.
- Each `ExploitCase` records the false statement acceptance would buy,
so a failure reads as a security statement. Rejection reasons are
printed: a fixture that aborts before reaching the mechanism under
attack is indistinguishable from a kernel that correctly refuses it,
and the first run of this fixture passed for exactly that wrong reason
(`invalid IO key` from an unseeded channel-3 hint, now seeded by
`storeAt`).
- The exploit ships with two controls that differ from it minimally: one
checking the smuggled dependency alone, one checking the same target
without the dead `sharing` entry. Both reject via the intended
in-kernel assert, so the exploit's acceptance was attributable to the
`sharing` entry alone.
- Two further cases probe what the fix does NOT cover. An index used as
both a `Nat` literal and a `Ref` — reachable because blob and constant
addresses share one un-domain-separated blake3 space and
`load_verified_blob` parses nothing — is now walked and rejected,
though by channel fault rather than type error, which is the safe
direction. A conditional Check claim naming its own target as its own
assumption IS accepted, and is pinned that way: `some root` means
"conditional on every leaf being well-typed", so the claim asserts
`A -> A` and is vacuous rather than false. The obligation sits with
whoever discharges assumptions, which must reject such cycles; sharded
CheckEnv is not exposed, since `shardClaimDigest` recomputes the claim
from (env, blocks).
Primitive closures are walked again
- The previous commit seeded primitive addresses as bytes only, without
walking their closures. That is too aggressive: def-eq delta-unfolds
primitive BODIES, and a body's refs are reachable from nowhere else in
the scope, so the `strAppend*` arena fixtures aborted with `invalid IO
key`. Both seeders now pass the primitives as walk ROOTS
(`closureFromRoots` in Lean, `bfs_closure` over target+primitives in
Rust); the Rust side had the same gap latently, since only the Lean
path had arena coverage.
- The removal of the O(env) reverse wrapper scan, which was that
commit's actual win, is unaffected.
Tests
- `lake test -- --ignored ixvm` green. FFT pins re-pinned across the 66
kernel-check entries and the shard pipeline; costs move in both
directions (e.g. `Eq.rec` 1_249_383 -> 1_249_094, `Vector.append`
1_994_697_378 -> 2_010_055_088), the increases coming from the
restored primitive closures.
- `ix codegen` regenerated: the Aiur kernel changed, so the committed
Rust kernel would otherwise be stale and the codegen/bytecode parity
gate would compare against it.
`Expr.Str` conversion wrapped any hash-valid blob in `KLiteral.Str`, and
`k_infer_lit` types a `KLiteral.Str` as `String` without inspecting the
bytes. Nothing on the ordinary checking path ever decodes a literal, so a
prover-authored definition carrying malformed bytes typechecked as
`x : String` even though no Lean `String` corresponds to those bytes. The
reference kernel rejects this at ingress via `String::from_utf8`
(crates/kernel/src/ingress.rs).
The UTF-8 validation added earlier lives in the string DECODER, which is
reached only during string reductions, so it did not cover this path.
`utf8_decode_one` already rejects out-of-range continuation bytes, stray
continuations used as leaders, and overlong forms. Walk the whole blob
with it at conversion (`utf8_validate`), which is the same rejection the
reference kernel performs.
Tests
- `invalid-utf8-accepted-as-string` in the adversarial Ixon suite: a
definition declaring `String` over the blob `[0xFF]`, which no
well-formed UTF-8 can contain. Accepted before this change, now
rejected.
- Paired with `control-valid-utf8-string-literal`, the same shape over
`[0x68, 0x69]`, which must still be accepted — so the verdict is
attributable to the bytes rather than to the fixture's shape. The
fixtures build on a base env supplying the real `String` constant, so
the declared type is the genuine one.
- Six FFT pins move, all string-touching, by between +17 and +66_530
(e.g. `IxVMPrim.str_size_lit` 567_999_042 -> 568_000_142,
`Vector.append` 2_010_055_088 -> 2_010_088_729, +0.0017%). Constants
with no string literals are unchanged, `Nat.add_comm` included.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`add_entries_parallel` returned early after a constant hit, so an address
present in both `env.consts` and `env.blobs` was seeded on channel 2 but
never on channel 4. The Lean seeder walks `consts` and `blobs` in
separate passes and seeds both, so the two builders disagreed, and the
Rust-native check/prove paths aborted with `invalid IO key` on such an
environment.
The two tables are not disjoint. Constant and blob addresses share a hash
codomain, and it is the Expr context — `Ref`/`Prj` versus `Str`/`Nat` —
that says which table a reference means, so the same bytes can legitimately
be read both ways. The kernel faults the two channels separately, so the
witness has to seed them separately. Reaching an address through both
kinds of use became expressible when the claim walk moved to selecting
indices by positive constant-use.
Corrects the module header and the `ChannelEntries` doc, which both
asserted the disjointness this relies on not having.
Tests
- `dual_use_address_is_seeded_on_both_channels`: an address stored as a
constant and as a blob must appear under both `(ch 2, key)` and
`(ch 4, key)` in the built IOBuffer. Fails on the previous code with
the ch-4 entry missing.
- `lake test -- --ignored ixvm` green; clippy `--all-targets` and fmt
clean.
The kernel walks a constant's `refs` by positive constant-use — index `i`
is followed only when some Expr names it via `Ref(i, ..)` or `Prj(i, ..)`
— but both host claim builders still treated every `refs` entry as a walk
edge. The host and the kernel therefore disagreed about the edge
relation.
A `refs` entry that is a real constant but that no Expr uses as one is
never walked in-circuit, so admitting it host-side puts a phantom member
in the thin frontier: a cross-shard assumption the kernel never makes,
and a spurious dependency edge that can close a cycle once assumptions
are discharged. `walk_check_set` likewise stopped computing the set the
verifier actually checks.
Mirror `const_idxs_of` on both hosts and emit an edge only for
positively-used indices. `Env::bfs_closure` is deliberately left alone:
it computes the witness BYTE scope, where over-approximating is correct
(shipping unused bytes is free, missing bytes aborts).
Honest constants are unaffected, since a compiler only puts an address in
`refs` when an Expr references it — the shard FFT pin is unchanged. The
divergence is reachable only through prover-authored `refs`.
Tests
- `thin_frontier_excludes_unused_ref_entries`: `refs` carries two
constants but only index 0 is named by a `Ref`; the frontier must
contain the used one and not the dead one. Verified red against the
previous edge relation ("unused refs entry became a phantom frontier
member").
- `const_with_refs` now names each of its `refs` entries via a `Ref`
node. The old helper left them unreferenced, which under the walk rule
would have produced no edges at all and quietly hollowed out the
existing frontier tests.
- `lake test -- --ignored ixvm` green; clippy `--all-targets` and fmt
clean.
`idx_to_u64` packed a synthesized member/constructor index into the
projection wrapper's `u64` field by populating only the low byte behind a
`u8_range_check`, so a mutual block with more than 256 members — or an
inductive with more than 256 constructors, which is the likelier case —
was rejected outright with `value 256 out of u8 range`.
Field-to-bytes is not free: the bytes have to be supplied as advice and
then pinned. Take them from `unconstrained_g_to_bytes` and discharge the
three obligations its contract names:
- range: four `u8_range_check` calls cover the eight bytes, the byte chip
taking a pair per lookup row;
- recomposition and canonicality: `flatten_u64(bytes) == idx`, since
`flatten_u64` asserts the top byte is zero and sums the low seven.
Canonicality is what makes the decomposition unique, and it is enforced
by RESTRICTING THE DOMAIN rather than by comparing against `p`. Goldilocks
is 2^64 - 2^32 + 1, so an arbitrary eight-byte string is not injective
into the field: any value and that value plus `p` recompose to the same
element, e.g. `[00]*8` and `[01,00,00,00,FF,FF,FF,FF]` both give 0.
Forcing the top byte to zero bounds the value by 2^56 - 1 < `p`, so the
sum cannot wrap and exactly one string encodes each index. That
uniqueness is the soundness property: the projection address is blake3
over these serialized bytes, so a prover free to choose a second string
for one index would obtain a second address for a member, or collide two
members onto one address. The pre-existing `u8_from_field_unsafe`
version did collide members 0 and 256, which the range check had already
fixed by rejecting instead.
Both halves are load-bearing. `flatten_u64` does NOT range-check its
input — `[U8; 8]` is nominal typing and `u8_from_field_unsafe` mints a
`U8` holding any field element — so without the range checks the sum is
satisfiable many ways (`b1 = k`, `b0 = idx - 256k`). Its doc comment now
records that its injectivity is conditional on the caller, and names this
call site.
Indices at or beyond 2^56 still fail the assert rather than truncating.
That is a liveness bound only, 2^48 times the previous cap and far past
any real block or constructor count; buying the remaining `[2^56, p)`
would cost a multi-byte comparison against `p` for no practical gain.
Tests
- `muts-block-past-single-byte-index`: a 257-member block of well-typed
definitions with a projection wrapper for each, which now checks.
Rejected before this change with `value 256 out of u8 range`.
- `control-small-muts-block`: the same construction at two members,
already accepted — so the big-block result is attributable to the index
width alone and not to a malformed fixture.
- `storeAt` also registers a reducibility hint for `DPrj` wrappers, which
`get_ci_dprj` faults under the wrapper's own address; without it these
fixtures aborted with `invalid IO key` before reaching the index logic.
- All 66 kernel-check FFT pins and the shard pin move by +17 to +485
(`Nat.add_comm` 42_967_279 -> 42_967_313, +0.00008%), the cost of the
hint plus four range checks per synthesized index.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`get_ci` handed every standalone `Defn` a one-element `recur_addrs` whose
only entry was its own address. `Rec` converts to a plain `Const`, and
`k_infer` discharges a `Const` against its DECLARED type without checking
it, so a definition could prove its own type by citing itself:
theorem bad : False := bad
typechecked. The ref-walk cannot catch it — `const_idxs_expr` does not
follow `Rec`, and the referent is the constant already under check. Lean
forbids exactly this by checking a definition against an environment that
does not yet contain it, admitting self-reference only for `unsafe` and
`partial`.
Restrict the self slot to `unsafe`/`partial`. Those are where
self-reference is both legitimate and contained:
- legitimate, because the compiler emits a singleton (or fully
collapsed) mutual block as a standalone `Defn` whose body still says
`Rec(0)` -- see the note at Ix/CompileM.lean:2000. Those blocks are
exactly Lean's `unsafe`/`partial` recursive definitions; a `safe`
recursive definition compiles its recursion into `.rec` applications.
Gating on `unsafe` alone breaks `Lean.extractMainModule._unsafe_rec`.
- contained, because safe code may not reference either kind.
That containment was itself missing for `partial`: `is_unsafe_ci`
classified it as safe, so `partial def bad : False := bad` plus a safe
theorem referencing `bad` would have walked straight around the first
half of this fix. Count `Partial` alongside `Unsafe`, matching the
reference's "safe definition references partial definition" rejection
(crates/kernel/src/check.rs:813-820). The predicate serves double duty --
the caller's own leniency and the referent's barrier -- and `Partial`
belongs on the non-safe side of both.
Tests
- `self-referential-theorem-proves-false`: `theorem bad : False := bad`
built directly in Ixon over the real `False` constant, submitted
through the production `verify_claim` entrypoint. Accepted before this
change -- a proof of False -- and rejected after.
- No safe constant in the corpus references a partial definition, and
the `_unsafe_rec` witnesses still check, so neither half of the fix
costs coverage.
- All 66 kernel-check FFT pins and the shard pin move (`Nat.add_comm`
42_967_313 -> 42_965_666); the self slot is no longer built for safe
definitions, so most costs drop slightly.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
The nested branch of the strict-positivity check dropped the parameter
arguments (`list_drop(args, n_ctor_params)`) and descended into the
nested inductive's constructors without substituting them. A nested
constructor's fields mention that inductive's parameters as bound
variables, so the block-occurrence test asked whether the block occurs in
`BVar 0` — it never does, and every parameter position passed vacuously.
The occurrence being hunted lives in the ARGUMENT substituted for the
parameter, which was exactly what got discarded.
So for `Inner α | mk : (α → False) → Inner α`, itself strictly positive,
`Host | mk : Inner Host → Host` was accepted: descending into `Inner.mk`
sees the field `α → False`, which mentions nothing tracked. With `α :=
Host` it reads `(Host → False) → …`, a negative occurrence, and Lean
rejects it. Accepting it admits `False`.
Mirror `check_nested_ctor_fields` (crates/kernel/src/inductive.rs): strip
`n_params` foralls, then simultaneously substitute the parameter
arguments at depth 0. After stripping, `BVar 0` is the innermost (last)
parameter, so the argument list is reversed — `expr_inst_many` maps
`substs[i]` to `BVar(depth + i)`, the same convention as `simul_subst`.
The parameters are substituted away rather than bound, so the field walk
now starts with an empty binder context instead of the peeled types.
Tests
- `nested-inductive-negative-occurrence`: the `Inner`/`Host` pair above,
built directly in Ixon as two single-inductive `Prop` blocks with their
projection wrappers. Accepted before this change, and now rejected by
the intended assert ("strict positivity: block occurs left of an
arrow").
- `control-nested-positive-inductive`: the same construction with
`Inner α | mk : α → Inner α`, which is strictly positive and must keep
checking. The two fixtures differ only in the inner constructor's
field, so the rejection is attributable to the negative occurrence
rather than to nesting.
- 63 FFT pins and the shard pin move, concentrated in inductive-heavy
constants (`IxVMInd.Tree` 767_167 -> 796_717, `IxVMInd.DepthM`
1_312_461 -> 1_347_394) — the nested check now does real work instead
of returning vacuously.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`KLimbs` is canonical in two independent ways — no trailing zero limbs,
and every limb byte in [0, 256) — and `klimbs_eq` is a raw limb compare
that establishes neither. Since it backs `Nat.beq`, `Nat.decEq` and
`literal_eq`, any producer emitting a value-correct but
representation-wrong `KLimbs` makes those answer `false` where Lean
answers `true`. That is an accepted false equation, not an abort. Two
producers did.
`String.utf8ByteSize` built the length as one unchecked byte
(`[u8_from_field_unsafe(n), 0, ...]`) with no normalization. The empty
string therefore reduced to the denormalized zero `[[0;8]]` instead of
the canonical empty limb list, and a string of 256 bytes or more to an
out-of-range digit — `u8_from_field_unsafe` is an erased coercion, so
nothing truncated or rejected. Build it with `klimbs_from_g`, whose byte
split is range-checked and pinned by a recomposition assert, then
normalize.
`klimbs_div_mod` pinned the quotient's VALUE (`q*b + r == a`, `r < b`)
but not its digits. The comment claimed `u8_mul` inside `klimbs_mul`
range-checked them, but `u64_mul` was rewritten to raw field products
plus `#split_carry`, whose u8 checks constrain the split outputs rather
than the input digits — and it re-canonicalizes while multiplying, so a
digit-wrong `q` still produces a canonical `q*b` and satisfies the
equality. Range-check the hint's limbs explicitly, quotient and
remainder both, via a new `klimbs_range_check`.
Tests
- `utf8-size-beq-false-accepted`: `Nat.beq (String.utf8ByteSize "") 0 =
false` proved by `Eq.refl`, over the real `Eq`/`Bool`/`Nat.beq`/
`String.utf8ByteSize` constants. The kernel had the polarity exactly
backwards — this FALSE equation was accepted while the true one was
rejected.
- `control-utf8-size-beq-true`: the same declaration ending in `true`,
which is what Lean computes and must be accepted. The pair differs only
in that operand, so each verdict is attributable to the reduction.
Both now land the right way round.
- The two fixtures share one blob for `""` and `0` — the same empty byte
string read as a string literal and as a nat literal, which is the
Expr-context disambiguation working as intended.
- 10 FFT pins and the shard pin move, confined to the div/mod/gcd/pow/
shift constants (the hint range-checks) and the string-size ones
(`IxVMPrim.str_size_lit` 567_992_937 -> 567_993_099).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT WAS DEMONSTRATED for this one. Unlike the other soundness
fixes in this series it ships without a red-then-green fixture; what
follows is why the rule is wrong as written, and what remains unproven.
`try_eta_struct` accepted `t ≡ S.mk p… a…` after checking only that the
right side is a fully-applied constructor of a struct-like inductive. It
never checked that the two sides have def-eq TYPES. All three references
do, citing lean4lean's `tryEtaStructCore`
(crates/kernel/src/def_eq.rs, Ix/Tc/DefEq.lean).
Without the guard the rule degenerates: for ANY `t` of ANY type,
`compare_struct_fields` builds precisely `Proj(S, i, t)` for each field,
so when the right side's arguments are those same projections the
pointer fast path matches them and the pair is accepted. Those
projections are synthesized inside def-eq and never inferred, so
`k_infer_proj`'s head-address gate — which would reject `Proj S i t` for
`t : Nat` — never sees them.
What is unproven is reachability. At the `k_check(e, expected)` boundary
both sides are types in the same context, and `de_args` compares type
arguments before value arguments, so I could not construct a term that
reaches this rule with genuinely mismatched types. The change is
therefore defense-in-depth against an unjustified rule, not a repair of a
demonstrated break.
The guard runs after the struct-like shape test, so it costs two
`k_infer_only` calls and one `k_is_def_eq` only on pairs that were about
to be accepted. `try_proof_irrel`, earlier in the same fallback chain,
already infers the left side unconditionally, so the added abort exposure
is small; the suite confirms no constant stops checking.
Tests
- No new fixture; the existing 14 adversarial cases and the arena corpus
are unchanged.
- 45 FFT pins and the shard pin move, the largest cost of this series
(`Nat.add_comm` 42_965_759 -> 42_966_970, +0.003%;
`IxVMPrim.str_size_lit` +26_687).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
A `Tag0` large-form header carries a 7-bit `small` field, so a payload
width up to 128 was expressible and unguarded (`Tag2`: 5 bits, up to 32).
A u64 payload is at most 8 bytes, and the three readers disagreed on the
surplus:
- Rust `u64_get_trimmed_le` rejects `len > 8` -- already correct, and
untouched here.
- The in-circuit `get_u64_le` consumed the surplus and DISCARDED it,
keeping only the first eight bytes.
- The Lean host folded it back in: `UInt64.shiftLeft` is taken mod 64,
so byte 8 ORs into bits 0-7, with no length guard.
Same bytes, same content address, three readings. An `Expr.Ref` whose
index is written overlong resolves as `refs[0]` in-circuit and `refs[42]`
on the Lean host, with both consuming the same span so nothing desyncs
afterwards. That also means the trailing-bytes assert does not imply
canonicity: up to 120 arbitrary bytes per `Tag0` site were consumed
inside the structure and dropped.
Bring the other two up to Rust: assert the width is under 9 in both
in-circuit readers, and give `getU64TrimmedLE` the guard it was missing.
No honest serializer is affected -- `putTag0` always emits the minimal
width -- so this narrows the accepted language to the format.
Tests
- `overlong-tag0-accepted`: an axiom `_ : Prop` whose `lvls` Tag0 is
rewritten to the nine-byte form. Accepted before this change, now
rejected by the intended assert.
- `control-canonical-tag0-axiom`: the same axiom with canonical bytes,
which must stay accepted. The pair differs only in that encoding.
- Both need bytes no `Ixon.ser` will emit, so the suite gained
`storeRawAt`, which stores a hand-authored byte string at its own
content address. Every previous fixture was built from a `Constant`
VALUE and was therefore canonical by construction, and structurally
could not reach this class of defect.
- All 66 kernel-check FFT pins and the shard pin move (`Nat.add_comm`
42_966_970 -> 42_980_767, +0.03%); every constant parses Tag0s.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
Four test-only fixes. No kernel code changes, so no FFT pins move.
prim-addrs asserted the literal `true`. It looked up each hardcoded
kernel address in the canonical table and, on a hit, recorded a pass —
but the table is keyed BY ADDRESS, so a permuted table still hits. Had
`nat_add_addr` held `Nat.mul`'s bytes it would have printed
"nat_add_addr = Nat.mul" and passed green, which is precisely the defect
the file's own docstring says it exists to catch: the kernel applying one
primitive's native semantics under another's name. Now the address must
resolve to the constant the function's NAME claims, comparing normalized
keys (`nat_add_addr` <-> `Nat.add`, and the final-component spelling for
`reduce_bool_addr` <-> `Lean.reduceBool`), with dispatch-site suffixes
(`_dec`/`_io`/`_iota`) and `_type`/`_ty` stripped, plus a short exception
list for the `Quot` constructors. Verified by temporarily swapping
`nat_add_addr` to `Nat.mul`'s bytes: red, then reverted. All 76 literals
pass, and the table is confirmed correct today.
The arena negative corpus counted ANY abort as a correct rejection while
its docstring claimed rejections come from an in-kernel `assert_eq!`. An
`invalid IO key` — the witness never supplied the bytes, so execution
died before reaching the logic under test — scored as a pass. That is how
a fixture rots into a green test that checks nothing, and it is the same
failure mode that made two exploit fixtures in this series pass for the
wrong reason. Now `invalid IO key` disqualifies. Unmatched-match aborts
still count: this kernel deliberately relies on an unmatched runtime
value aborting instead of spending width on `_ => assert_eq!(0, 1)`
arms, so those are genuine verdicts. No fixture currently relies on the
disqualified path.
`shardCheckEnvCase` returned `none` for two unrelated reasons and the
runner reported both as "SKIP (target absent)", a passing test. The
second — target present but the partition selected zero owned constants —
means the fixture stopped exercising the Rust witness builder and the
thin-frontier claim convention, and would have left the only test of that
path permanently green. It now throws; `none` keeps its one honest
meaning.
A `--ignored` filter naming no runner ran nothing and returned 0, so
`lake test -- --ignored ixvm2` reported success having executed nothing.
It is now an error. Exit 0 should mean tests ran and passed.
`ixVM`, the pruned production toplevel, carried `verify_const`,
`verify_check` and `verify_check_env` alongside `verify_claim`. Those are
DEBUG entrypoints that trust their dependencies instead of walking them,
so a proof made with one attests far less than a claim does — and
carrying them meant such a proof was verifiable under the SAME verifying
key. The only thing standing between it and acceptance was `ix verify`
pinning the funcIdx to `verify_claim`: a policy, enforced in one place,
with no test asserting it, and no policy at all in the recursive
verifier, which binds the claim bytes but applies no funcIdx check.
Prune to `verify_claim` alone. The guarantee is now structural: under the
production VK a debug-entrypoint proof cannot exist. No CLI path used
them, so nothing in `ix check`/`prove`/`verify` changes.
Callers that genuinely want a subject-only check build `ixVMFull` and are
thereby using a different VK, which is the honest signal — the arena
corpus, and `bench-typecheck`/`bench-recursion-debug` under
`--skip-deps`. That last one also means a `--skip-deps` benchmark is now
visibly measuring a different key than a claim run rather than silently
reporting `verify_const` rows in the claim schema.
`bench-typecheck` also reported success on failure, twice over. A proof
that failed to verify printed to stderr but left `failed` clear, so the
row was emitted as `"status": "ok"` with prove-time and proof-size
present and the run exited 0 — the only signal an absent `verify-time`
key, invisible to anything keying on status. And a constant whose execute
threw was dropped from the results entirely without setting `failed`,
so `execed.any (·.failed)` stayed clear; the `check_const`-missing throw
added earlier landed exactly there. Both now mark the row failed.
Tests
- `lake test -- --ignored ixvm` green with NO pin movement: the pruned
entrypoints were never executed, so removing them changes no cost.
Generated kernel drops from 733 to 731 functions.
- Arena repointed to `ixVMFull`, where `verify_const` still lives.
- `bench-typecheck --execute-only` smoke unchanged (`Nat.add_comm`,
constants=42).
`compare_rules` peeled the binders off the STORED rule and handed their
DOMAINS to `k_is_def_eq` as the local context for the comparison against
the canonical reconstruction. Those domains are prover-authored: nothing
checks them against the reconstruction, and reduction never consults them
(`try_iota` beta-reduces straight through). So the prover chose what the
two sides INFER to.
Retyping every binder to a Prop makes both sides infer to that Prop, so
`try_proof_irrel` fires and accepts ANY body. A recursor whose
computation rules are swapped between constructors was therefore
accepted, and iota then reduces each constructor to the other's branch.
Compare under an empty context instead, discarding the peeled domains. An
honest rule matches the reconstruction structurally, and the structural
path never consults the context — it compares `BVar i` against `BVar i`
directly. A comparison that instead needs to INFER a bound variable's
type now runs out of context and aborts, which is the safe direction.
Every recursor in the corpus still checks, including the mutual, nested
and aux-generated ones.
The reference avoids this differently, comparing whole Lam-wrapped terms
at empty context so `is_def_eq` descends binder-by-binder and compares
the domains themselves (crates/kernel/src/inductive.rs). Reconstructing
canonical domains here would mean re-deriving field domains from each
constructor's telescope and re-lifting their de Bruijn indices into the
rule's context; dropping the untrusted context achieves the same
guarantee without that machinery.
Tests
- `recursor-rule-rhs-doctored-binder-domains`: a real compiled recursor
with its first two computation rules swapped AND every peeled binder
domain replaced by a Prop. Accepted before this change; rejected now.
- `control-swapped-recursor-rules`: the identical swap with the
compiler's own binder domains, which was already rejected. The two
differ only in the domains, so the acceptance was attributable to them
alone.
- Both are built by rewriting a REAL block rather than hand-authoring a
recursor, since a hand-built one must survive `build_rec_type`'s def-eq
and constructor references inside a block are circular with the block's
own address. The fixture selects a recursor whose rules are field-free,
so no minor premise is applied and the retyping cannot abort as a
non-function before the comparison is reached — an earlier attempt did
exactly that and masked the acceptance.
- A guard case fails loudly if no such recursor is found; the first
version of this fixture silently corrupted nothing, and both its cases
"passed" meaninglessly.
- 57 FFT pins and the shard pin move (`Nat.add_comm` 42_980_767 ->
42_980_801).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT DEMONSTRATED. Defense-in-depth, like the struct-eta guard.
`compare_rules` now asserts three invariants that equivalents of already
exist in `check_rec_rules_walk` — which nothing calls. The live path had
none of them.
* `cidx == pos`. `find_rule` selects a rule at reduction time BY LABEL
while `compare_rules` pairs stored against canonical POSITIONALLY, so
permuted labels would validate one constructor's rule and then fire
it for another. It holds today only because ingress assigns `cidx`
positionally (`Convert.lean`) and the Ixon `RecursorRule` record
carries no cidx of its own — an accident of the pipeline rather than
a checked property, which is exactly what an assert is for.
* `cidx < num_ctors`, so a rule cannot name a constructor that does not
exist.
* `nf` equals the constructor's field count. `try_iota` splits the
constructor's arguments with `field_start = ctor_fields_len -
rfields`, so a wrong `nf` hands the minor premise the wrong slice; in
field arithmetic an over-large value wraps.
The reachability argument for the `nf` gap was "combined with the rule
RHS context hole"; that hole is fixed, and with the def-eq comparison
restored a mismatched peel is caught again. So this closes a route that
is currently blocked by another check rather than one that is open —
worth asserting locally rather than depending on a second mechanism.
Not fixed here: iota still selects rules by index rather than by
constructor IDENTITY, so a recursor applied to a foreign constructor
would reduce. That is not reachable through a claim —
`k_infer_app_spine_loop` runs `k_check` on every argument, so the
application fails inference before whnf sees it — and closing it properly
needs the parent inductive available at reduction time, which
`KConstantInfo.Rec` does not carry (its `Addr` is the recursor's own
block; the parent is derived at check time from the type). That is a KCI
layout change, left for its own pass.
Tests
- No new fixture; nothing demonstrates the hole.
- Every recursor in the corpus satisfies all three invariants, including
the mutual, nested and aux-generated ones, so they held by construction
and are now enforced.
- 57 FFT pins and the shard pin move (`Nat.add_comm` 42_980_801 ->
42_981_336) for the per-rule constructor lookup.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT DEMONSTRATED. Defense-in-depth.
`try_iota` selected a computation rule by constructor INDEX alone:
`collect_spine_of_ctor` returned the major's `cidx` and discarded which
inductive it came from, so `find_rule(rules, cidx)` fired whenever any
inductive's constructor happened to carry a matching index. On its face
`Bool.rec motive f t (Nat.succ n)` reduces to `t`. Lean's kernel selects
the rule by the constructor's NAME, which pins the inductive.
Not reachable through a claim: `k_infer_app_spine_loop` runs `k_check` on
every argument, so a recursor applied to a foreign constructor fails
inference long before whnf sees the redex — the ill-typed term cannot be
stated. The Rust reference selects by index too, consistent with that
being why it was tolerated. What the patch removes is the DEPENDENCE on
that: reduction no longer needs inference to have run first to be
correct.
`collect_spine_of_ctor` now also returns the constructor's parent, as the
address of its projection wrapper, and `try_iota` asserts it equals the
recursor's own parent. That parent is read off the recursor's type (the
major premise's head) with the existing `rec_to_parent_addr`, so no
KConstantInfo layout change was needed — its `Addr` field is the
recursor's own block, not the parent's. Aiur memoizes per argument tuple,
so the type walk is paid once per recursor rather than once per
reduction.
Tests
- No new fixture; nothing demonstrates the hole.
- Every recursor in the corpus still reduces, including the mutual,
nested and aux-generated ones, and the Nat/String literal paths that
synthesize a constructor application before iota (`nat_cases_big`,
`nat_pow_big`, `strOfListFold*` all unchanged in behaviour).
- 63 FFT pins and the shard pin move (`Nat.add_comm` 42_981_336 ->
42_986_025, +0.01%) for the parent comparison.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`shardsCover` gates every shard run: "exit 0 has to mean every env const
was checked by some shard". It skipped constants whose bytes fail to
parse (`LazyConstant.get?` discards the error and `.ixe` loading is
lazy, so such a constant sits in `consts` with its key present). A
skipped constant was assigned to no shard, counted as neither owned nor
unowned, and still included in the "covers all N consts" total — so the
gate reported success over constants it had not covered.
It would also enter a referring shard's frontier, since `thin_frontier`
admits an edge on key presence in `env.consts` without parsing, leaving
an assumption no shard discharges.
Count them separately and fail, rather than folding them into `unowned`:
"cannot be assigned to a shard" is a different problem from a partition
gap, and saying so points at the right cause.
`ownedConstsForBlocks` keeps its skip — it runs per shard and would
otherwise need the same error plumbing — but now documents that it is
safe only because this gate rejects such an env first.
Not changed: the composed verdict still has no assumption-cycle check,
and on analysis it does not need one. Assumptions are discharged per
CONSTANT along ref edges, which are acyclic by construction — a
constant's address is blake3 over bytes containing its refs' addresses,
so a reference cycle needs a hash cycle. A cycle in the shard-level graph
still stands for a well-founded per-constant chain, and
`verifyShardComposition` recomputes each digest via `shardClaimDigest`,
so a self-assuming shard claim cannot be substituted.
`compile_univ` memoized `Level -> Univ`, but a `Level::Param` compiles to
`Univ::Var(i)` where `i` is the parameter's POSITION in `univ_params` —
so the result depends on a context the key omitted. Unlike the expression
cache, which is cleared at every constant boundary, `univ_cache` lives
for a whole block, and `compile_definition` compiles each member under
its own `level_params`. `preseed_expr_tables` explicitly feeds
`(expr, univ_params)` pairs with differing param lists into the same
cache.
So two members of one block placing the same parameter NAME at different
positions would give the second member the first's index. Not an abort:
it compiles cleanly and registers a constant denoting a different
universe than the Lean source, under a correct-looking name.
The hazard is already recognized one level up — `collect_expr_tables`
keys `seen_exprs` by `(expr_hash, univ_params_key)` for exactly this
reason. Apply the same key here. The Lean mirror instead drops the cache
in `withUnivCtx`, matching how it already clears the expression cache per
constant.
Unreachable from ordinary Lean sources, since the elaborator gives every
member of a `mutual` block the same `levelParams` — but nothing checked
it, and under the prover threat model the environment is authored.
Also repairs `claim.rs`'s `defn_const`, whose Definition never referenced
its own `refs`. Since the claim walk admits an index only when some Expr
uses it as a constant, that fixture produced no walk edges and
`shard_check_env_claim_is_thin_frontier` has been failing since the walk
moved to positive constant-use. The identical flaw was fixed in
`shard_claim.rs`'s helper at the time and missed here, because only
`ixon` and `ixvm-codegen` were re-tested rather than the whole workspace.
Tests
- `cargo test --release` across all crates: 1195 passed, 0 failed.
- `lake test -- --ignored ixvm` green with NO pin movement and no address
changes, confirming no block in the corpus has members that order a
shared parameter name differently. The fix removes the dependence on
that remaining true.
- clippy `--all-targets` and fmt clean.
@johnchandlerburnham
johnchandlerburnham merged commit 80bb41f into mainAug 3, 2026
10 checks passed
@johnchandlerburnham
johnchandlerburnham deleted the ap/ixvm4 branch August 3, 2026 05:37
samuelburnham added a commit that referenced this pull request Aug 4, 2026
…acking
Port of sb/measured-ingress's planning layer onto the addr-first kernel
(#529), reworked for its lazy fault-in semantics.
Profiling (.ixprof v3): the recording kernel's try_get_const consults are
drained per constant into a touch graph — the measured set a lazy checker
faults in — persisted as a presence-flagged CSR section alongside a new
block-level reference graph and a subst_unique counter (splitmix64-keyed
dedup of substitution work items: the post-memoization volume a memoizing
executor pays). Both graph sections are optional, so older readers'
formats stay decodable by version gate alone.
Planner: partition_for_aiur_ram bin-packs to a host-RAM cap with
faulted-set byte accounting — under #529 the witness always ships the
full closure, but only faulted bytes cost ingress trace rows, so a
shard's byte feature is owned ∪ touched(owned) (full reference closure
as the over-approximating fallback when no touch graph was recorded).
The stub/promotion machinery of the original branch is dropped entirely:
lazy fault-in computes at run time exactly the set the stubs predicted,
with no replay divergence and no repair ladder, and the manifest format
stays byte-identical with main (owned-only).
Cost models: nlogn(x) = x·log2(x+2) feature form matching Aiur's
width·height·log2(height) FFT structure; prove/execute time and RAM
coefficients carried over from the pre-#529 aiur bench-suite fits.
They are placeholders pending a refit against this kernel — the circuit
shrank (deleted positional machinery), the byte feature narrowed to the
faulted set, and the recording counters moved under #442 — with
provenance noted at the definitions. The costs sidecar
(<out>.costs.csv) persists per-shard features incl. subst_unique and
nat_arith so the refit has its corpus.
CLI: ix shard --backend aiur --max-ram G; ix profile --backend/--top
(per-metric block leaderboards with display names) and ix profile sweep
(env-wide closure cost sweep: per-root feature CSV + feasibility,
min-root-per-hot-block, and diversity reports).
samuelburnham added a commit that referenced this pull request Aug 8, 2026
…acking
Port of sb/measured-ingress's planning layer onto the addr-first kernel
(#529), reworked for its lazy fault-in semantics.
Profiling (.ixprof v3): the recording kernel's try_get_const consults are
drained per constant into a touch graph — the measured set a lazy checker
faults in — persisted as a presence-flagged CSR section alongside a new
block-level reference graph and a subst_unique counter (splitmix64-keyed
dedup of substitution work items: the post-memoization volume a memoizing
executor pays). Both graph sections are optional, so older readers'
formats stay decodable by version gate alone.
Planner: partition_for_aiur_ram bin-packs to a host-RAM cap with
faulted-set byte accounting — under #529 the witness always ships the
full closure, but only faulted bytes cost ingress trace rows, so a
shard's byte feature is owned ∪ touched(owned) (full reference closure
as the over-approximating fallback when no touch graph was recorded).
The stub/promotion machinery of the original branch is dropped entirely:
lazy fault-in computes at run time exactly the set the stubs predicted,
with no replay divergence and no repair ladder, and the manifest format
stays byte-identical with main (owned-only).
Cost models: nlogn(x) = x·log2(x+2) feature form matching Aiur's
width·height·log2(height) FFT structure; prove/execute time and RAM
coefficients carried over from the pre-#529 aiur bench-suite fits.
They are placeholders pending a refit against this kernel — the circuit
shrank (deleted positional machinery), the byte feature narrowed to the
faulted set, and the recording counters moved under #442 — with
provenance noted at the definitions. The costs sidecar
(<out>.costs.csv) persists per-shard features incl. subst_unique and
nat_arith so the refit has its corpus.
CLI: ix shard --backend aiur --max-ram G; ix profile --backend/--top
(per-metric block leaderboards with display names) and ix profile sweep
(env-wide closure cost sweep: per-root feature CSV + feasibility,
min-root-per-hot-block, and diversity reports).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@arthurpaulino@johnchandlerburnham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

IxVM: cheaper shard checking via an addr-first kernel, plus soundness hardening - #529

Merged
johnchandlerburnham merged 19 commits into
mainfrom
ap/ixvm4
Aug 3, 2026
Merged

IxVM: cheaper shard checking via an addr-first kernel, plus soundness hardening#529
johnchandlerburnham merged 19 commits into
mainfrom
ap/ixvm4

Conversation

@arthurpaulino

@arthurpaulinoarthurpaulino commented Jul 31, 2026

Copy link
Copy Markdown
Member

Motivation

Make sharded environment checking cheap. A large environment is partitioned across shards, and each shard proves its own constants well-typed against a thin-frontier assumption cutoff, so every constant is checked in exactly one shard and cross-shard assumption trees stay minimal.

Delivering that required migrating the IxVM Aiur kernel from a positional constant table to a content-addressed, lazily-faulted design — and then auditing the result, which turned up five soundness holes with working exploits, including three separate routes to proving False.

Sharded environment checking

A shard proves CheckEnv{root, assumptions}: root is a merkle root over the shard's owned constants, assumptions a merkle root over the thin frontier — the direct out-of-owned walk edges. The kernel's env_walk starts from the owned set, checks each reachable constant, and stops at a frontier member (assumed well-typed by its owning shard: not re-checked, not recursed into).

  • Sub-frontier constants are never re-checked here; they are covered transitively by the frontier members' own shard claims. This is the cost win over checking each constant's full closure: no cross-shard duplication, and the assumption tree is one layer deep instead of the whole closure.
  • Strict membership: a walk-reachable constant in neither owned nor the frontier is a bad partition and hard-errors, so a shard cannot silently drop coverage of its own closure.
  • Byte scope ≠ check scope: delta unfolding reads through the frontier, so a witness ships the full dependency closure of owned even though only the strict-membership set is checked. The witness IOBuffer is built in Rust (aiur_ixvm_witness) rather than Lean, removing the per-byte boxing that dominated shard wall time.

Addr-first kernel

  • KExprNode.Const / Proj key on a 32-byte content Addr, not a numeric position; KConstantInfoInduct/Ctor/Rec carry a (block_addr, ind_idx) parent pair instead of indices. The positional top: List<KCI> table threaded through the kernel is gone.
  • get_ci(addr) is the sole resolver, Aiur-memoized per addr, faulting constant bytes lazily from an IOBuffer. load_verified_constant re-derives blake3 over the bytes and asserts key == addr; the io_get_info idx/len are unconstrained witness and are never branched on.
  • IOBuffer channels: 0 claim, 1 assumption tree, 2 constant bytes, 3 Defn reducibility hint (advisory), 4 blob bytes. The kernel derives blob-vs-constant from Expr context, so no wire discriminator ships. The two sets are not disjoint — one address can be read both ways — so each channel is seeded independently.
  • Kernel/Inductive.lean and Kernel/Primitive.lean deleted (~6.6k lines); inductive validation absorbed into Kernel/Check.lean, primitives split into Kernel/Klimbs.lean (bignum gadgets) and Kernel/NatPrim.lean (Nat/Str addr dispatch). New Kernel/InferOnly.lean breaks the DefEq↔Infer cycle. Ingress.lean collapsed 1700→298 lines.

Recursor validation

  • build_rec_type reconstructs a recursor's canonical type and asserts it def-eq to the declared type, on both the kind-aware and address-only lookup paths. Without it a recursor could declare ... -> False while its rules still reconstructed correctly.
  • spec_params are lowered from the major-premise frame into the recursor-param frame at extraction: the root cause of 17 nested/aux recursor reconstruction failures, and why the canonical check accepted a superset.
  • k_flag and parent-inductive-shape checks hoisted out of the attacker-selectable is_aux gate; the canonical rules comparison, which formerly never ran, now runs for every recursor.

Pre-existing criticals

IDFix
P1blob-vs-constant derived from Expr context instead of a prover-answered io_get_info probe, retiring the ch-2 len=0 sentinel; a prover can no longer mark a walk root or owned leaf "skip me"
P2Nat.div n 0 pins the quotient to zero (was prover-chosen)
P3klimbs_sub/le corrected on operands with trailing zero limbs; div/mod hints normalized before use
P4is_large_eliminator field-index arithmetic parenthesized (Aiur - is right-associative)
P5Reveal recursor rules assert the claimed ruleIdx equals its position

Robustness

  • L5: UTF-8 string-literal decode validates continuation-byte ranges and per-length overlong minimums, so distinct byte sequences (e.g. overlong NUL C0 80 vs 00) no longer decode to the same codepoint and def-eq as equal.
  • L6: run_claim dispatches on the full tag4 size (was the low byte only), matching Rust Claim::get.
  • L8: the K-recursor synth gate checks the major's inferred inductive against the recursor's own, not against itself, so I.rec no longer strips a major of a different inductive.

Aiur assert_eq! messages and circuit cleanup

  • assert_eq!(a, b, "why") gains an optional diagnostic label, threaded Source.Term → Typed → Simple → Concrete → Bytecode.Op → Rust Op::AssertEq(_, _, Option<String>) → ExecError::AssertEqMismatch. The message is diagnostic-only and never enters the circuit; all ~122 IxVM assert sites are labeled.
  • Unreachable _ => assert_eq!(0, 1) match arms removed: an unmatched runtime value already aborts (the kernel's reject), so the extra arm only widened the circuit.
  • Address equality is a full 32-byte content comparison — never a pointer-value compare, which is unsound for distinct-pointer/same-data — packed as five base-256 field limbs behind a byte-0 prefilter.

Witness construction

witness_scope reuses Env::bfs_closure, which already follows refs, Prj → block, and the derived Muts → member/ctor projection edges, so the wrappers get_ci synthesizes ship as forward edges of a single O(closure) walk with no reverse scan of the env. Primitive addresses are seeded as additional walk roots, because def-eq delta-unfolds primitive bodies whose refs are reachable from nowhere else. The Lean seeder computes the same scope by the same rule, so the two cannot drift.

bench-typecheck's constants metric now reads check_const's unique query count — exactly the constants typechecked — instead of the shipped byte scope, which is a superset and inflated throughput.

Soundness audit

Eight parallel read-only audits covered the pipeline end-to-end (compile, Ixon serde, ingress/convert, type-theory core, declaration checking, claim/witness layer, primitives, entry points), under the zkDSL threat model: a prover authors the data and pays for its own trace, so aborts and rejections are not threats and only a wrong answer counts. Thirty findings; every claim relayed here was re-verified against the code before acting on it, and several agent claims were corrected or withdrawn in the process.

Fixed, with a working exploit

Each of these shipped with a fixture that was red before the fix and green after, paired with a minimal-pair control so the verdict is attributable to one variable.

HoleWhat it bought an attacker
Self-referential definitionstheorem bad : False := bad typechecked. A standalone Defn was handed a recur slot naming itself, Rec erases to an ordinary Const, and k_infer reads a Const's declared type without checking it — so the body discharged its own type by citing itself. Now restricted to unsafe/partial, which is where Lean permits self-reference and where safe code cannot reach
Nested-inductive positivityThe nested branch dropped the parameter arguments and descended into the nested inductive's constructors without substituting them, so the occurrence being hunted was never present. Host | mk : Inner Host → Host was accepted for a negative Inner. Admits False
Recursor rules compared under the prover's contextcompare_rules handed the stored rule's own binder domains to k_is_def_eq as the comparison context. Retyping them to a Prop makes both sides infer to that Prop, proof irrelevance fires, and a recursor with its computation rules swapped between constructors was accepted
Non-canonical computed Nat literalsString.utf8ByteSize built a KLimbs as one unchecked byte, and klimbs_div_mod pinned the quotient's value but not its digits. Since klimbs_eq compares limbs rather than values, the kernel had the polarity backwards: Nat.beq (String.utf8ByteSize "") 0 = false was accepted while the true equation was rejected
Tag0/Tag2 overlong payloadsThree readers, three answers on the same bytes: the circuit discarded bytes past the eighth, the Lean host folded them back in (its shift is mod 64), Rust rejected. One address denoted different constants to different readers

Fixed as defense-in-depth

No exploit demonstrated for these; each commit says so in its first line. They remove a dependence on some other check happening to hold.

  • Struct-eta type equality.try_eta_struct omitted the infer(t) ≟ infer(s) side condition all three references carry, degenerating to t ≡ S.mk … (Proj S i t) … for any t of any type.
  • Recursor rule labels.cidx selects a rule at reduction time but was validated only positionally; nf decides how iota splits the constructor's arguments and was never checked against the constructor. Equivalents existed only in dead code.
  • Iota constructor identity. Rules were selected by index alone, so any inductive's constructor with a matching index fired one. Unreachable through a claim — inference k_checks every argument first — but reduction should not depend on that.
  • Universe cache context.compile_univ memoized Level → Univ while a Level::Param compiles to a position in univ_params, and the cache outlives a constant. Now keyed by (level, univ_params_key), matching what collect_expr_tables already does one level up.

Entry points and coverage

  • verify_const / verify_check / verify_check_env are DEBUG entrypoints that trust their dependencies, and they shipped in the production verifying key alongside verify_claim — leaving ix verify's funcIdx check as the only barrier, untested, with no equivalent in the recursive verifier. The production toplevel is now pruned to verify_claim alone, so such a proof cannot exist under that VK. Zero FFT cost.
  • bench-typecheck reported "status": "ok" and exit 0 when verification failed, and dropped thrown constants without marking them failed.
  • shardsCover skipped constants whose bytes do not parse — assigning them to no shard, counting them as neither owned nor unowned, and still including them in "covers all N consts". Now a hard failure.

Test guards that could not fail

Four of these were found by auditing the tests themselves, and they matter because they were the safety net for everything above.

  • prim-addrs asserted the literal true. Keyed by address, so a permuted primitive table passed — precisely the defect its docstring claimed to catch. Now the address must resolve to the constant the function's name claims; verified by temporarily swapping nat_add_addr to Nat.mul's bytes and watching it go red.
  • The arena negative corpus counted any abort as a correct rejection, so a fixture aborting on invalid IO key — never reaching the code under test — scored as a pass. That is the same failure mode that made two exploit fixtures in this series pass for the wrong reason before it was caught.
  • The shard-pipeline SKIP was a passing test conflating "target absent" with "partition selected zero constants".
  • A --ignored filter naming no runner executed nothing and returned 0, so a typo'd suite name reported success having run no tests.

Adversarial Ixon test suite

New Tests/Ix/IxVM/Exploits.lean, 19 cases, wired into the ixvm suite. The kernel-arena corpus attacks the layer above: its bad_* fixtures are raw Lean.ConstantInfo declarations that skip the elaborator, but they are still compiled by rsCompileEnvFFI, so the Ixon they produce is always well-formed. This suite authors Ixon.Constant values directly, and storeRawAt goes one level lower still, storing hand-authored byte strings — needed because every value-level fixture is canonical by construction and so cannot express a malformed encoding.

Each case records the false statement acceptance would buy, so a failure reads as a security statement. Rejection reasons are printed, which earned its keep repeatedly: fixtures that "passed" via invalid IO key without reaching the checker were caught this way, as was a fixture that silently corrupted nothing because non-mutual recursors are standalone constants rather than Muts members.

Testing

  • lake test -- --ignored ixvm green.
  • cargo test --release across all crates: 1195 passed, 0 failed. This also repaired shard_check_env_claim_is_thin_frontier, red since the claim walk moved to positive constant-use because its fixture's Exprs never referenced their own refs.
  • New prim-addrs parity suite; codegen ↔ bytecode parity gate green; arena negative corpus (77 bad_*) all reject, now only via genuine in-kernel refusals.
  • FFT-cost pins re-pinned per change. Cumulative effect is small (Nat.add_comm 42_747_927 → 42_986_025, +0.6% over the whole series); the largest single contributors are the nested-positivity substitution and the struct-eta guard.
  • ix codegen regenerated whenever the Aiur kernel changed, so the committed Rust kernel never lags the source the parity gate compares against.

Known limitations

Recorded rather than silently dropped.

  • Call-site surgery is not covered by the value pin. The compiler rewrites calls to aux-generated constants — permuting arguments, dropping some, replacing the head and its universe list — and the plans driving it live only in ConstantMeta, which prune_to_closure_anon excludes. So a verified claim says the term at address X is well-typed, not that term is the Lean declaration you wrote; the equivalence rests on the rewrite being faithful, checked today only by decompile round-trip tests. Closing it is a claim-format decision, not a patch.
  • Assumption cycles are not rejected, and a conditional Check claim naming its own target is accepted. That is the specified semantics — some root means "conditional on every leaf", so the claim asserts A → A, vacuous rather than false — and the obligation sits with whoever discharges assumptions. Sharded CheckEnv is not exposed, since shardClaimDigest recomputes the claim from (env, blocks). Pinned as ACCEPT so the boundary cannot move silently.
  • Alpha-collapse comparators ignore safety, so two members agreeing on kind/lvls/typ/value and disagreeing on safety collapse. Deliberately not fixed: a prover authors the environment directly and never goes through the compiler, so hardening it stops no attacker, while adding safety to the ordering would move every address and refusing the collapse would put a hard error in the compiler on the strength of an unverified assumption.
  • klimbs_le / klimbs_eq are in the logical trusted base, not just the performance path: Nat.decLe mints Decidable.isTrue with a proof term whose correctness is asserted rather than checked, so one wrong verdict yields False.
  • check_inductive_shape_ctors omits validate_expr_well_scoped / k_ensure_sort / assert_safety that check_const's Ctor arm performs; only an entirely unreferenced constructor escapes those.

@arthurpaulino
arthurpaulinoforce-pushed the ap/ixvm4 branch 4 times, most recently from 06e0489 to 559ced9CompareAugust 1, 2026 00:53
… hardening
Make sharded environment checking cheap. A large environment is partitioned
across shards, and each shard proves its own constants well-typed against a
thin-frontier assumption cutoff, so every constant is checked in exactly one
shard and cross-shard assumption trees stay minimal. Delivering that required
migrating the IxVM Aiur kernel from a positional constant table to a
content-addressed, lazily-faulted design -- and closing the soundness
regressions that migration introduced along with several pre-existing holes.
Sharded environment checking
- A shard proves `CheckEnv{root, assumptions}`: root is a merkle over the
shard's owned constants, assumptions a merkle over the THIN FRONTIER --
the direct out-of-owned walk edges. The kernel's env_walk starts from the
owned set, checks each reachable constant, and stops at a frontier member
(assumed well-typed by its owning shard, not re-checked, not recursed).
- Sub-frontier constants are never re-checked here; they are covered
transitively by the frontier members' own shard claims. This is the cost
win over checking each constant's full closure: no cross-shard duplication,
and the assumption tree is one layer deep instead of the whole closure.
- Strict membership: a walk-reachable constant in neither owned nor the
frontier is a bad partition and hard-errors, so a shard cannot silently
drop coverage of its own closure.
- Byte scope != check scope: delta unfolding reads through the frontier, so
a witness ships the full walk closure of owned even though only the
strict-membership set is checked. The witness IOBuffer is built in Rust
(aiur_ixvm_witness) rather than Lean, removing the per-byte boxing that
dominated shard wall time.
Addr-first kernel
- KExprNode.Const/Proj key on a 32-byte content Addr, not a numeric
position; KConstantInfo Induct/Ctor/Rec carry a (block_addr, ind_idx)
parent pair instead of indices. The positional `top: List<KCI>` table
threaded through the kernel is gone.
- get_ci(addr) is the sole resolver, Aiur-memoized per addr, faulting
constant bytes lazily from an IOBuffer. load_verified_constant re-derives
blake3 over the bytes and asserts key == addr; the io_get_info idx/len
are unconstrained witness and are never branched on.
- IOBuffer channels: 0 claim, 1 assumption tree, 2 constant bytes, 3 Defn
reducibility hint (advisory), 4 blob bytes. An address is on ch2 iff a
constant and ch4 iff a blob; the kernel derives which from Expr context
(Str/Nat -> blob, Ref/Prj -> constant), so no wire discriminator ships.
- Kernel/Inductive.lean and Kernel/Primitive.lean deleted (~6.6k lines);
inductive validation absorbed into Kernel/Check.lean, primitives split
into Kernel/Klimbs.lean (bignum gadgets) and Kernel/NatPrim.lean (Nat/Str
addr dispatch). New Kernel/InferOnly.lean breaks the DefEq<->Infer cycle.
Ingress.lean collapsed 1700->298 lines. V1/V2 naming removed throughout.
Recursor validation (R1, R2, R3)
- build_rec_type reconstructs a recursor's canonical type and asserts it
def-eq to the declared type, on both the kind-aware and address-only
lookup paths. Without it a recursor could declare `... -> False` while
its rules still reconstructed correctly.
- spec_params are lowered from the major-premise frame into the
recursor-param frame at extraction: the root cause of 17 nested/aux
recursor reconstruction failures, and why the canonical check accepted a
superset.
- k_flag and parent-inductive-shape checks hoisted out of the
attacker-selectable `is_aux` gate; the canonical rules comparison, which
formerly never ran, now runs for every recursor.
Pre-existing criticals (P1-P5)
- P1: blob-vs-constant is derived from Expr context instead of a prover-
answered io_get_info probe, retiring the ch-2 len=0 sentinel; a prover
can no longer mark a walk root or owned leaf "skip me".
- P2: Nat.div n 0 pins the quotient to zero (was prover-chosen).
- P3: klimbs_sub/le corrected on operands with trailing zero limbs; div/mod
hints normalized before use.
- P4: is_large_eliminator field-index arithmetic parenthesized (Aiur `-` is
right-associative).
- P5: Reveal recursor rules assert the claimed ruleIdx equals its position.
Robustness (L5, L6, L8)
- L5: UTF-8 string-literal decode validates continuation-byte ranges and
per-length overlong minimums, so distinct byte sequences (e.g. overlong
NUL C0 80 vs 00) no longer decode to the same codepoint and def-eq as
equal. Roughly zero added FFT cost.
- L6: run_claim dispatches on the full tag4 size (was the low byte only),
matching Rust Claim::get.
- L8: the K-recursor synth gate checks the major's inferred inductive
against the recursor's own (via se_parent_addr), not against itself, so
I.rec no longer strips a major of a different inductive.
Aiur assert_eq! messages + circuit cleanup
- assert_eq!(a, b, "why") gains an optional diagnostic label, threaded
Source.Term -> Typed -> Simple -> Concrete -> Bytecode.Op -> Rust
Op::AssertEq(_, _, Option<String>) -> ExecError::AssertEqMismatch. The
message is diagnostic-only and never enters the circuit; all ~122 IxVM
assert sites are labeled.
- Unreachable `_ => assert_eq!(0, 1)` match arms removed: an unmatched
runtime value already aborts (the kernel's reject), so the extra arm only
widened the circuit.
- address equality is a full 32-byte content comparison (never a pointer-
value compare, which is unsound for distinct-pointer/same-data), packed
as five base-256 field limbs (7+7+7+7+4) behind a byte-0 prefilter and
short-circuiting on the first mismatched group.
Crate reorganization
- PrimAddrs / reserved_marker_name moved verbatim from
crates/kernel/src/primitive.rs to crates/common/src/prim_addrs.rs (kernel
re-exports), so witness builders reach the primitive address table
without an ix-kernel -> ixon -> ix-kernel cycle.
- witness_scope reuses Env::bfs_closure -- the env's own dependency-closure
pass, which already follows refs, Prj->block, and the derived
Muts->member/ctor projection edges -- so the wrappers the kernel's get_ci
synthesizes ship as forward edges of a single O(closure) walk, with no
reverse scan of the whole env. Primitive-address seeding is unioned on
top: primitives are fabricated during reduction and need not appear in a
target's ref closure, so a closure-only scope aborts with `invalid IO
key`.
Tests
- New prim-addrs parity suite (Lean<->Aiur primitive address table).
- Codegen<->bytecode parity gate restored, green (198/198).
- FFT-cost pins re-pinned to current behavior; arena negative-fixture
corpus (77 bad_* constants) all reject.
…onstants
closureFromBase now walks the derived Muts->member/ctor projection
wrapper addresses as forward edges (projWrapperAddr: blake3 over the
serialized proj-only Constant, compile's construction), so closureFrom
and buildShardCheckEnvWitness drop their O(env) reverse scans that
parsed every env constant per witness build. Primitive addresses are
seeded bytes-only after the walk instead of being pushed onto the
worklist, so their transitive closures no longer inflate the byte scope
(a delta-unfold into an unshipped primitive body aborts with invalid IO
key rather than proving anything wrong).
bench-typecheck's constants metric now reads check_const's unique query
count (memoized per (ci, addr) = exactly the constants typechecked)
instead of closureFrom.size, which counted the shipped byte scope.
check_const is resolved up front and both lookups fail with IO.userError
instead of panicking.
Experiment: lake exe bench-typecheck --ixe seed.ixe --consts
Nat.add_comm --execute-only (seed.ixe = ix compile Tests.lean --consts
Nat.add_comm) reports constants=42, matching check_const's height in
lake exe ix check Nat.add_comm; fft-cost unchanged at 42747927.
The claim walk decided which of a constant's `refs` to recurse into by
SKIPPING every index the constant's Exprs used as a blob payload
(`Str(i)` / `Nat(i)`). That rule is unsound: the Exprs are authored by
the prover, and `refs` is one index space shared by `Ref`, `Prj`, `Str`
and `Nat`, so marking an index suppresses the walk of that index used as
a constant everywhere else in the same constant.
The marking need not even be live. `blob_idxs_of` traversed `sharing` in
full, but conversion reaches a `sharing` entry only through an
`Expr.Share` node, and nothing validates the array. A dead `Nat(i)`
entry is therefore free: it is never converted, never typechecked, and
needs no blob on channel 4. It just hides `refs[i]` from the checker.
`k_infer` then reads that dependency's DECLARED type without ever
checking it, so the kernel accepts a constant whose transitive closure
is ill-typed -- a wrong answer, not an abort. Both claim variants were
affected; for CheckEnv the skip happens before `env_walk` is entered, so
the strict owned-membership assert never fires either.
Replace the family with `const_idxs_*`: collect the indices `Ref(i, ..)`
and `Prj(i, ..)` use, and walk `refs[i]` iff `i` is among them. `Rec`
resolves through `recur_addrs` and a projection's block comes from the
ConstantInfo payload, both walked separately, so the positive set is
complete. Same single traversal as before, so no added cost. The failure
direction is now safe: over-approximating walks an address that faults
an unseeded key and aborts, which only costs a dishonest prover.
Adversarial Ixon test infrastructure
- New `Tests/Ix/IxVM/Exploits.lean`, wired into the ixvm suite. The
kernel-arena corpus attacks the layer above this one: its `bad_*`
fixtures are raw `Lean.ConstantInfo` declarations that skip the
elaborator, but they are still compiled by `rsCompileEnvFFI`, so the
Ixon they produce is always well-formed. This suite authors
`Ixon.Constant` values directly, which is what a prover can do.
- Each `ExploitCase` records the false statement acceptance would buy,
so a failure reads as a security statement. Rejection reasons are
printed: a fixture that aborts before reaching the mechanism under
attack is indistinguishable from a kernel that correctly refuses it,
and the first run of this fixture passed for exactly that wrong reason
(`invalid IO key` from an unseeded channel-3 hint, now seeded by
`storeAt`).
- The exploit ships with two controls that differ from it minimally: one
checking the smuggled dependency alone, one checking the same target
without the dead `sharing` entry. Both reject via the intended
in-kernel assert, so the exploit's acceptance was attributable to the
`sharing` entry alone.
- Two further cases probe what the fix does NOT cover. An index used as
both a `Nat` literal and a `Ref` — reachable because blob and constant
addresses share one un-domain-separated blake3 space and
`load_verified_blob` parses nothing — is now walked and rejected,
though by channel fault rather than type error, which is the safe
direction. A conditional Check claim naming its own target as its own
assumption IS accepted, and is pinned that way: `some root` means
"conditional on every leaf being well-typed", so the claim asserts
`A -> A` and is vacuous rather than false. The obligation sits with
whoever discharges assumptions, which must reject such cycles; sharded
CheckEnv is not exposed, since `shardClaimDigest` recomputes the claim
from (env, blocks).
Primitive closures are walked again
- The previous commit seeded primitive addresses as bytes only, without
walking their closures. That is too aggressive: def-eq delta-unfolds
primitive BODIES, and a body's refs are reachable from nowhere else in
the scope, so the `strAppend*` arena fixtures aborted with `invalid IO
key`. Both seeders now pass the primitives as walk ROOTS
(`closureFromRoots` in Lean, `bfs_closure` over target+primitives in
Rust); the Rust side had the same gap latently, since only the Lean
path had arena coverage.
- The removal of the O(env) reverse wrapper scan, which was that
commit's actual win, is unaffected.
Tests
- `lake test -- --ignored ixvm` green. FFT pins re-pinned across the 66
kernel-check entries and the shard pipeline; costs move in both
directions (e.g. `Eq.rec` 1_249_383 -> 1_249_094, `Vector.append`
1_994_697_378 -> 2_010_055_088), the increases coming from the
restored primitive closures.
- `ix codegen` regenerated: the Aiur kernel changed, so the committed
Rust kernel would otherwise be stale and the codegen/bytecode parity
gate would compare against it.
`Expr.Str` conversion wrapped any hash-valid blob in `KLiteral.Str`, and
`k_infer_lit` types a `KLiteral.Str` as `String` without inspecting the
bytes. Nothing on the ordinary checking path ever decodes a literal, so a
prover-authored definition carrying malformed bytes typechecked as
`x : String` even though no Lean `String` corresponds to those bytes. The
reference kernel rejects this at ingress via `String::from_utf8`
(crates/kernel/src/ingress.rs).
The UTF-8 validation added earlier lives in the string DECODER, which is
reached only during string reductions, so it did not cover this path.
`utf8_decode_one` already rejects out-of-range continuation bytes, stray
continuations used as leaders, and overlong forms. Walk the whole blob
with it at conversion (`utf8_validate`), which is the same rejection the
reference kernel performs.
Tests
- `invalid-utf8-accepted-as-string` in the adversarial Ixon suite: a
definition declaring `String` over the blob `[0xFF]`, which no
well-formed UTF-8 can contain. Accepted before this change, now
rejected.
- Paired with `control-valid-utf8-string-literal`, the same shape over
`[0x68, 0x69]`, which must still be accepted — so the verdict is
attributable to the bytes rather than to the fixture's shape. The
fixtures build on a base env supplying the real `String` constant, so
the declared type is the genuine one.
- Six FFT pins move, all string-touching, by between +17 and +66_530
(e.g. `IxVMPrim.str_size_lit` 567_999_042 -> 568_000_142,
`Vector.append` 2_010_055_088 -> 2_010_088_729, +0.0017%). Constants
with no string literals are unchanged, `Nat.add_comm` included.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`add_entries_parallel` returned early after a constant hit, so an address
present in both `env.consts` and `env.blobs` was seeded on channel 2 but
never on channel 4. The Lean seeder walks `consts` and `blobs` in
separate passes and seeds both, so the two builders disagreed, and the
Rust-native check/prove paths aborted with `invalid IO key` on such an
environment.
The two tables are not disjoint. Constant and blob addresses share a hash
codomain, and it is the Expr context — `Ref`/`Prj` versus `Str`/`Nat` —
that says which table a reference means, so the same bytes can legitimately
be read both ways. The kernel faults the two channels separately, so the
witness has to seed them separately. Reaching an address through both
kinds of use became expressible when the claim walk moved to selecting
indices by positive constant-use.
Corrects the module header and the `ChannelEntries` doc, which both
asserted the disjointness this relies on not having.
Tests
- `dual_use_address_is_seeded_on_both_channels`: an address stored as a
constant and as a blob must appear under both `(ch 2, key)` and
`(ch 4, key)` in the built IOBuffer. Fails on the previous code with
the ch-4 entry missing.
- `lake test -- --ignored ixvm` green; clippy `--all-targets` and fmt
clean.
The kernel walks a constant's `refs` by positive constant-use — index `i`
is followed only when some Expr names it via `Ref(i, ..)` or `Prj(i, ..)`
— but both host claim builders still treated every `refs` entry as a walk
edge. The host and the kernel therefore disagreed about the edge
relation.
A `refs` entry that is a real constant but that no Expr uses as one is
never walked in-circuit, so admitting it host-side puts a phantom member
in the thin frontier: a cross-shard assumption the kernel never makes,
and a spurious dependency edge that can close a cycle once assumptions
are discharged. `walk_check_set` likewise stopped computing the set the
verifier actually checks.
Mirror `const_idxs_of` on both hosts and emit an edge only for
positively-used indices. `Env::bfs_closure` is deliberately left alone:
it computes the witness BYTE scope, where over-approximating is correct
(shipping unused bytes is free, missing bytes aborts).
Honest constants are unaffected, since a compiler only puts an address in
`refs` when an Expr references it — the shard FFT pin is unchanged. The
divergence is reachable only through prover-authored `refs`.
Tests
- `thin_frontier_excludes_unused_ref_entries`: `refs` carries two
constants but only index 0 is named by a `Ref`; the frontier must
contain the used one and not the dead one. Verified red against the
previous edge relation ("unused refs entry became a phantom frontier
member").
- `const_with_refs` now names each of its `refs` entries via a `Ref`
node. The old helper left them unreferenced, which under the walk rule
would have produced no edges at all and quietly hollowed out the
existing frontier tests.
- `lake test -- --ignored ixvm` green; clippy `--all-targets` and fmt
clean.
`idx_to_u64` packed a synthesized member/constructor index into the
projection wrapper's `u64` field by populating only the low byte behind a
`u8_range_check`, so a mutual block with more than 256 members — or an
inductive with more than 256 constructors, which is the likelier case —
was rejected outright with `value 256 out of u8 range`.
Field-to-bytes is not free: the bytes have to be supplied as advice and
then pinned. Take them from `unconstrained_g_to_bytes` and discharge the
three obligations its contract names:
- range: four `u8_range_check` calls cover the eight bytes, the byte chip
taking a pair per lookup row;
- recomposition and canonicality: `flatten_u64(bytes) == idx`, since
`flatten_u64` asserts the top byte is zero and sums the low seven.
Canonicality is what makes the decomposition unique, and it is enforced
by RESTRICTING THE DOMAIN rather than by comparing against `p`. Goldilocks
is 2^64 - 2^32 + 1, so an arbitrary eight-byte string is not injective
into the field: any value and that value plus `p` recompose to the same
element, e.g. `[00]*8` and `[01,00,00,00,FF,FF,FF,FF]` both give 0.
Forcing the top byte to zero bounds the value by 2^56 - 1 < `p`, so the
sum cannot wrap and exactly one string encodes each index. That
uniqueness is the soundness property: the projection address is blake3
over these serialized bytes, so a prover free to choose a second string
for one index would obtain a second address for a member, or collide two
members onto one address. The pre-existing `u8_from_field_unsafe`
version did collide members 0 and 256, which the range check had already
fixed by rejecting instead.
Both halves are load-bearing. `flatten_u64` does NOT range-check its
input — `[U8; 8]` is nominal typing and `u8_from_field_unsafe` mints a
`U8` holding any field element — so without the range checks the sum is
satisfiable many ways (`b1 = k`, `b0 = idx - 256k`). Its doc comment now
records that its injectivity is conditional on the caller, and names this
call site.
Indices at or beyond 2^56 still fail the assert rather than truncating.
That is a liveness bound only, 2^48 times the previous cap and far past
any real block or constructor count; buying the remaining `[2^56, p)`
would cost a multi-byte comparison against `p` for no practical gain.
Tests
- `muts-block-past-single-byte-index`: a 257-member block of well-typed
definitions with a projection wrapper for each, which now checks.
Rejected before this change with `value 256 out of u8 range`.
- `control-small-muts-block`: the same construction at two members,
already accepted — so the big-block result is attributable to the index
width alone and not to a malformed fixture.
- `storeAt` also registers a reducibility hint for `DPrj` wrappers, which
`get_ci_dprj` faults under the wrapper's own address; without it these
fixtures aborted with `invalid IO key` before reaching the index logic.
- All 66 kernel-check FFT pins and the shard pin move by +17 to +485
(`Nat.add_comm` 42_967_279 -> 42_967_313, +0.00008%), the cost of the
hint plus four range checks per synthesized index.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`get_ci` handed every standalone `Defn` a one-element `recur_addrs` whose
only entry was its own address. `Rec` converts to a plain `Const`, and
`k_infer` discharges a `Const` against its DECLARED type without checking
it, so a definition could prove its own type by citing itself:
theorem bad : False := bad
typechecked. The ref-walk cannot catch it — `const_idxs_expr` does not
follow `Rec`, and the referent is the constant already under check. Lean
forbids exactly this by checking a definition against an environment that
does not yet contain it, admitting self-reference only for `unsafe` and
`partial`.
Restrict the self slot to `unsafe`/`partial`. Those are where
self-reference is both legitimate and contained:
- legitimate, because the compiler emits a singleton (or fully
collapsed) mutual block as a standalone `Defn` whose body still says
`Rec(0)` -- see the note at Ix/CompileM.lean:2000. Those blocks are
exactly Lean's `unsafe`/`partial` recursive definitions; a `safe`
recursive definition compiles its recursion into `.rec` applications.
Gating on `unsafe` alone breaks `Lean.extractMainModule._unsafe_rec`.
- contained, because safe code may not reference either kind.
That containment was itself missing for `partial`: `is_unsafe_ci`
classified it as safe, so `partial def bad : False := bad` plus a safe
theorem referencing `bad` would have walked straight around the first
half of this fix. Count `Partial` alongside `Unsafe`, matching the
reference's "safe definition references partial definition" rejection
(crates/kernel/src/check.rs:813-820). The predicate serves double duty --
the caller's own leniency and the referent's barrier -- and `Partial`
belongs on the non-safe side of both.
Tests
- `self-referential-theorem-proves-false`: `theorem bad : False := bad`
built directly in Ixon over the real `False` constant, submitted
through the production `verify_claim` entrypoint. Accepted before this
change -- a proof of False -- and rejected after.
- No safe constant in the corpus references a partial definition, and
the `_unsafe_rec` witnesses still check, so neither half of the fix
costs coverage.
- All 66 kernel-check FFT pins and the shard pin move (`Nat.add_comm`
42_967_313 -> 42_965_666); the self slot is no longer built for safe
definitions, so most costs drop slightly.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
The nested branch of the strict-positivity check dropped the parameter
arguments (`list_drop(args, n_ctor_params)`) and descended into the
nested inductive's constructors without substituting them. A nested
constructor's fields mention that inductive's parameters as bound
variables, so the block-occurrence test asked whether the block occurs in
`BVar 0` — it never does, and every parameter position passed vacuously.
The occurrence being hunted lives in the ARGUMENT substituted for the
parameter, which was exactly what got discarded.
So for `Inner α | mk : (α → False) → Inner α`, itself strictly positive,
`Host | mk : Inner Host → Host` was accepted: descending into `Inner.mk`
sees the field `α → False`, which mentions nothing tracked. With `α :=
Host` it reads `(Host → False) → …`, a negative occurrence, and Lean
rejects it. Accepting it admits `False`.
Mirror `check_nested_ctor_fields` (crates/kernel/src/inductive.rs): strip
`n_params` foralls, then simultaneously substitute the parameter
arguments at depth 0. After stripping, `BVar 0` is the innermost (last)
parameter, so the argument list is reversed — `expr_inst_many` maps
`substs[i]` to `BVar(depth + i)`, the same convention as `simul_subst`.
The parameters are substituted away rather than bound, so the field walk
now starts with an empty binder context instead of the peeled types.
Tests
- `nested-inductive-negative-occurrence`: the `Inner`/`Host` pair above,
built directly in Ixon as two single-inductive `Prop` blocks with their
projection wrappers. Accepted before this change, and now rejected by
the intended assert ("strict positivity: block occurs left of an
arrow").
- `control-nested-positive-inductive`: the same construction with
`Inner α | mk : α → Inner α`, which is strictly positive and must keep
checking. The two fixtures differ only in the inner constructor's
field, so the rejection is attributable to the negative occurrence
rather than to nesting.
- 63 FFT pins and the shard pin move, concentrated in inductive-heavy
constants (`IxVMInd.Tree` 767_167 -> 796_717, `IxVMInd.DepthM`
1_312_461 -> 1_347_394) — the nested check now does real work instead
of returning vacuously.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`KLimbs` is canonical in two independent ways — no trailing zero limbs,
and every limb byte in [0, 256) — and `klimbs_eq` is a raw limb compare
that establishes neither. Since it backs `Nat.beq`, `Nat.decEq` and
`literal_eq`, any producer emitting a value-correct but
representation-wrong `KLimbs` makes those answer `false` where Lean
answers `true`. That is an accepted false equation, not an abort. Two
producers did.
`String.utf8ByteSize` built the length as one unchecked byte
(`[u8_from_field_unsafe(n), 0, ...]`) with no normalization. The empty
string therefore reduced to the denormalized zero `[[0;8]]` instead of
the canonical empty limb list, and a string of 256 bytes or more to an
out-of-range digit — `u8_from_field_unsafe` is an erased coercion, so
nothing truncated or rejected. Build it with `klimbs_from_g`, whose byte
split is range-checked and pinned by a recomposition assert, then
normalize.
`klimbs_div_mod` pinned the quotient's VALUE (`q*b + r == a`, `r < b`)
but not its digits. The comment claimed `u8_mul` inside `klimbs_mul`
range-checked them, but `u64_mul` was rewritten to raw field products
plus `#split_carry`, whose u8 checks constrain the split outputs rather
than the input digits — and it re-canonicalizes while multiplying, so a
digit-wrong `q` still produces a canonical `q*b` and satisfies the
equality. Range-check the hint's limbs explicitly, quotient and
remainder both, via a new `klimbs_range_check`.
Tests
- `utf8-size-beq-false-accepted`: `Nat.beq (String.utf8ByteSize "") 0 =
false` proved by `Eq.refl`, over the real `Eq`/`Bool`/`Nat.beq`/
`String.utf8ByteSize` constants. The kernel had the polarity exactly
backwards — this FALSE equation was accepted while the true one was
rejected.
- `control-utf8-size-beq-true`: the same declaration ending in `true`,
which is what Lean computes and must be accepted. The pair differs only
in that operand, so each verdict is attributable to the reduction.
Both now land the right way round.
- The two fixtures share one blob for `""` and `0` — the same empty byte
string read as a string literal and as a nat literal, which is the
Expr-context disambiguation working as intended.
- 10 FFT pins and the shard pin move, confined to the div/mod/gcd/pow/
shift constants (the hint range-checks) and the string-size ones
(`IxVMPrim.str_size_lit` 567_992_937 -> 567_993_099).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT WAS DEMONSTRATED for this one. Unlike the other soundness
fixes in this series it ships without a red-then-green fixture; what
follows is why the rule is wrong as written, and what remains unproven.
`try_eta_struct` accepted `t ≡ S.mk p… a…` after checking only that the
right side is a fully-applied constructor of a struct-like inductive. It
never checked that the two sides have def-eq TYPES. All three references
do, citing lean4lean's `tryEtaStructCore`
(crates/kernel/src/def_eq.rs, Ix/Tc/DefEq.lean).
Without the guard the rule degenerates: for ANY `t` of ANY type,
`compare_struct_fields` builds precisely `Proj(S, i, t)` for each field,
so when the right side's arguments are those same projections the
pointer fast path matches them and the pair is accepted. Those
projections are synthesized inside def-eq and never inferred, so
`k_infer_proj`'s head-address gate — which would reject `Proj S i t` for
`t : Nat` — never sees them.
What is unproven is reachability. At the `k_check(e, expected)` boundary
both sides are types in the same context, and `de_args` compares type
arguments before value arguments, so I could not construct a term that
reaches this rule with genuinely mismatched types. The change is
therefore defense-in-depth against an unjustified rule, not a repair of a
demonstrated break.
The guard runs after the struct-like shape test, so it costs two
`k_infer_only` calls and one `k_is_def_eq` only on pairs that were about
to be accepted. `try_proof_irrel`, earlier in the same fallback chain,
already infers the left side unconditionally, so the added abort exposure
is small; the suite confirms no constant stops checking.
Tests
- No new fixture; the existing 14 adversarial cases and the arena corpus
are unchanged.
- 45 FFT pins and the shard pin move, the largest cost of this series
(`Nat.add_comm` 42_965_759 -> 42_966_970, +0.003%;
`IxVMPrim.str_size_lit` +26_687).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
A `Tag0` large-form header carries a 7-bit `small` field, so a payload
width up to 128 was expressible and unguarded (`Tag2`: 5 bits, up to 32).
A u64 payload is at most 8 bytes, and the three readers disagreed on the
surplus:
- Rust `u64_get_trimmed_le` rejects `len > 8` -- already correct, and
untouched here.
- The in-circuit `get_u64_le` consumed the surplus and DISCARDED it,
keeping only the first eight bytes.
- The Lean host folded it back in: `UInt64.shiftLeft` is taken mod 64,
so byte 8 ORs into bits 0-7, with no length guard.
Same bytes, same content address, three readings. An `Expr.Ref` whose
index is written overlong resolves as `refs[0]` in-circuit and `refs[42]`
on the Lean host, with both consuming the same span so nothing desyncs
afterwards. That also means the trailing-bytes assert does not imply
canonicity: up to 120 arbitrary bytes per `Tag0` site were consumed
inside the structure and dropped.
Bring the other two up to Rust: assert the width is under 9 in both
in-circuit readers, and give `getU64TrimmedLE` the guard it was missing.
No honest serializer is affected -- `putTag0` always emits the minimal
width -- so this narrows the accepted language to the format.
Tests
- `overlong-tag0-accepted`: an axiom `_ : Prop` whose `lvls` Tag0 is
rewritten to the nine-byte form. Accepted before this change, now
rejected by the intended assert.
- `control-canonical-tag0-axiom`: the same axiom with canonical bytes,
which must stay accepted. The pair differs only in that encoding.
- Both need bytes no `Ixon.ser` will emit, so the suite gained
`storeRawAt`, which stores a hand-authored byte string at its own
content address. Every previous fixture was built from a `Constant`
VALUE and was therefore canonical by construction, and structurally
could not reach this class of defect.
- All 66 kernel-check FFT pins and the shard pin move (`Nat.add_comm`
42_966_970 -> 42_980_767, +0.03%); every constant parses Tag0s.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
Four test-only fixes. No kernel code changes, so no FFT pins move.
prim-addrs asserted the literal `true`. It looked up each hardcoded
kernel address in the canonical table and, on a hit, recorded a pass —
but the table is keyed BY ADDRESS, so a permuted table still hits. Had
`nat_add_addr` held `Nat.mul`'s bytes it would have printed
"nat_add_addr = Nat.mul" and passed green, which is precisely the defect
the file's own docstring says it exists to catch: the kernel applying one
primitive's native semantics under another's name. Now the address must
resolve to the constant the function's NAME claims, comparing normalized
keys (`nat_add_addr` <-> `Nat.add`, and the final-component spelling for
`reduce_bool_addr` <-> `Lean.reduceBool`), with dispatch-site suffixes
(`_dec`/`_io`/`_iota`) and `_type`/`_ty` stripped, plus a short exception
list for the `Quot` constructors. Verified by temporarily swapping
`nat_add_addr` to `Nat.mul`'s bytes: red, then reverted. All 76 literals
pass, and the table is confirmed correct today.
The arena negative corpus counted ANY abort as a correct rejection while
its docstring claimed rejections come from an in-kernel `assert_eq!`. An
`invalid IO key` — the witness never supplied the bytes, so execution
died before reaching the logic under test — scored as a pass. That is how
a fixture rots into a green test that checks nothing, and it is the same
failure mode that made two exploit fixtures in this series pass for the
wrong reason. Now `invalid IO key` disqualifies. Unmatched-match aborts
still count: this kernel deliberately relies on an unmatched runtime
value aborting instead of spending width on `_ => assert_eq!(0, 1)`
arms, so those are genuine verdicts. No fixture currently relies on the
disqualified path.
`shardCheckEnvCase` returned `none` for two unrelated reasons and the
runner reported both as "SKIP (target absent)", a passing test. The
second — target present but the partition selected zero owned constants —
means the fixture stopped exercising the Rust witness builder and the
thin-frontier claim convention, and would have left the only test of that
path permanently green. It now throws; `none` keeps its one honest
meaning.
A `--ignored` filter naming no runner ran nothing and returned 0, so
`lake test -- --ignored ixvm2` reported success having executed nothing.
It is now an error. Exit 0 should mean tests ran and passed.
`ixVM`, the pruned production toplevel, carried `verify_const`,
`verify_check` and `verify_check_env` alongside `verify_claim`. Those are
DEBUG entrypoints that trust their dependencies instead of walking them,
so a proof made with one attests far less than a claim does — and
carrying them meant such a proof was verifiable under the SAME verifying
key. The only thing standing between it and acceptance was `ix verify`
pinning the funcIdx to `verify_claim`: a policy, enforced in one place,
with no test asserting it, and no policy at all in the recursive
verifier, which binds the claim bytes but applies no funcIdx check.
Prune to `verify_claim` alone. The guarantee is now structural: under the
production VK a debug-entrypoint proof cannot exist. No CLI path used
them, so nothing in `ix check`/`prove`/`verify` changes.
Callers that genuinely want a subject-only check build `ixVMFull` and are
thereby using a different VK, which is the honest signal — the arena
corpus, and `bench-typecheck`/`bench-recursion-debug` under
`--skip-deps`. That last one also means a `--skip-deps` benchmark is now
visibly measuring a different key than a claim run rather than silently
reporting `verify_const` rows in the claim schema.
`bench-typecheck` also reported success on failure, twice over. A proof
that failed to verify printed to stderr but left `failed` clear, so the
row was emitted as `"status": "ok"` with prove-time and proof-size
present and the run exited 0 — the only signal an absent `verify-time`
key, invisible to anything keying on status. And a constant whose execute
threw was dropped from the results entirely without setting `failed`,
so `execed.any (·.failed)` stayed clear; the `check_const`-missing throw
added earlier landed exactly there. Both now mark the row failed.
Tests
- `lake test -- --ignored ixvm` green with NO pin movement: the pruned
entrypoints were never executed, so removing them changes no cost.
Generated kernel drops from 733 to 731 functions.
- Arena repointed to `ixVMFull`, where `verify_const` still lives.
- `bench-typecheck --execute-only` smoke unchanged (`Nat.add_comm`,
constants=42).
`compare_rules` peeled the binders off the STORED rule and handed their
DOMAINS to `k_is_def_eq` as the local context for the comparison against
the canonical reconstruction. Those domains are prover-authored: nothing
checks them against the reconstruction, and reduction never consults them
(`try_iota` beta-reduces straight through). So the prover chose what the
two sides INFER to.
Retyping every binder to a Prop makes both sides infer to that Prop, so
`try_proof_irrel` fires and accepts ANY body. A recursor whose
computation rules are swapped between constructors was therefore
accepted, and iota then reduces each constructor to the other's branch.
Compare under an empty context instead, discarding the peeled domains. An
honest rule matches the reconstruction structurally, and the structural
path never consults the context — it compares `BVar i` against `BVar i`
directly. A comparison that instead needs to INFER a bound variable's
type now runs out of context and aborts, which is the safe direction.
Every recursor in the corpus still checks, including the mutual, nested
and aux-generated ones.
The reference avoids this differently, comparing whole Lam-wrapped terms
at empty context so `is_def_eq` descends binder-by-binder and compares
the domains themselves (crates/kernel/src/inductive.rs). Reconstructing
canonical domains here would mean re-deriving field domains from each
constructor's telescope and re-lifting their de Bruijn indices into the
rule's context; dropping the untrusted context achieves the same
guarantee without that machinery.
Tests
- `recursor-rule-rhs-doctored-binder-domains`: a real compiled recursor
with its first two computation rules swapped AND every peeled binder
domain replaced by a Prop. Accepted before this change; rejected now.
- `control-swapped-recursor-rules`: the identical swap with the
compiler's own binder domains, which was already rejected. The two
differ only in the domains, so the acceptance was attributable to them
alone.
- Both are built by rewriting a REAL block rather than hand-authoring a
recursor, since a hand-built one must survive `build_rec_type`'s def-eq
and constructor references inside a block are circular with the block's
own address. The fixture selects a recursor whose rules are field-free,
so no minor premise is applied and the retyping cannot abort as a
non-function before the comparison is reached — an earlier attempt did
exactly that and masked the acceptance.
- A guard case fails loudly if no such recursor is found; the first
version of this fixture silently corrupted nothing, and both its cases
"passed" meaninglessly.
- 57 FFT pins and the shard pin move (`Nat.add_comm` 42_980_767 ->
42_980_801).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT DEMONSTRATED. Defense-in-depth, like the struct-eta guard.
`compare_rules` now asserts three invariants that equivalents of already
exist in `check_rec_rules_walk` — which nothing calls. The live path had
none of them.
* `cidx == pos`. `find_rule` selects a rule at reduction time BY LABEL
while `compare_rules` pairs stored against canonical POSITIONALLY, so
permuted labels would validate one constructor's rule and then fire
it for another. It holds today only because ingress assigns `cidx`
positionally (`Convert.lean`) and the Ixon `RecursorRule` record
carries no cidx of its own — an accident of the pipeline rather than
a checked property, which is exactly what an assert is for.
* `cidx < num_ctors`, so a rule cannot name a constructor that does not
exist.
* `nf` equals the constructor's field count. `try_iota` splits the
constructor's arguments with `field_start = ctor_fields_len -
rfields`, so a wrong `nf` hands the minor premise the wrong slice; in
field arithmetic an over-large value wraps.
The reachability argument for the `nf` gap was "combined with the rule
RHS context hole"; that hole is fixed, and with the def-eq comparison
restored a mismatched peel is caught again. So this closes a route that
is currently blocked by another check rather than one that is open —
worth asserting locally rather than depending on a second mechanism.
Not fixed here: iota still selects rules by index rather than by
constructor IDENTITY, so a recursor applied to a foreign constructor
would reduce. That is not reachable through a claim —
`k_infer_app_spine_loop` runs `k_check` on every argument, so the
application fails inference before whnf sees it — and closing it properly
needs the parent inductive available at reduction time, which
`KConstantInfo.Rec` does not carry (its `Addr` is the recursor's own
block; the parent is derived at check time from the type). That is a KCI
layout change, left for its own pass.
Tests
- No new fixture; nothing demonstrates the hole.
- Every recursor in the corpus satisfies all three invariants, including
the mutual, nested and aux-generated ones, so they held by construction
and are now enforced.
- 57 FFT pins and the shard pin move (`Nat.add_comm` 42_980_801 ->
42_981_336) for the per-rule constructor lookup.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT DEMONSTRATED. Defense-in-depth.
`try_iota` selected a computation rule by constructor INDEX alone:
`collect_spine_of_ctor` returned the major's `cidx` and discarded which
inductive it came from, so `find_rule(rules, cidx)` fired whenever any
inductive's constructor happened to carry a matching index. On its face
`Bool.rec motive f t (Nat.succ n)` reduces to `t`. Lean's kernel selects
the rule by the constructor's NAME, which pins the inductive.
Not reachable through a claim: `k_infer_app_spine_loop` runs `k_check` on
every argument, so a recursor applied to a foreign constructor fails
inference long before whnf sees the redex — the ill-typed term cannot be
stated. The Rust reference selects by index too, consistent with that
being why it was tolerated. What the patch removes is the DEPENDENCE on
that: reduction no longer needs inference to have run first to be
correct.
`collect_spine_of_ctor` now also returns the constructor's parent, as the
address of its projection wrapper, and `try_iota` asserts it equals the
recursor's own parent. That parent is read off the recursor's type (the
major premise's head) with the existing `rec_to_parent_addr`, so no
KConstantInfo layout change was needed — its `Addr` field is the
recursor's own block, not the parent's. Aiur memoizes per argument tuple,
so the type walk is paid once per recursor rather than once per
reduction.
Tests
- No new fixture; nothing demonstrates the hole.
- Every recursor in the corpus still reduces, including the mutual,
nested and aux-generated ones, and the Nat/String literal paths that
synthesize a constructor application before iota (`nat_cases_big`,
`nat_pow_big`, `strOfListFold*` all unchanged in behaviour).
- 63 FFT pins and the shard pin move (`Nat.add_comm` 42_981_336 ->
42_986_025, +0.01%) for the parent comparison.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`shardsCover` gates every shard run: "exit 0 has to mean every env const
was checked by some shard". It skipped constants whose bytes fail to
parse (`LazyConstant.get?` discards the error and `.ixe` loading is
lazy, so such a constant sits in `consts` with its key present). A
skipped constant was assigned to no shard, counted as neither owned nor
unowned, and still included in the "covers all N consts" total — so the
gate reported success over constants it had not covered.
It would also enter a referring shard's frontier, since `thin_frontier`
admits an edge on key presence in `env.consts` without parsing, leaving
an assumption no shard discharges.
Count them separately and fail, rather than folding them into `unowned`:
"cannot be assigned to a shard" is a different problem from a partition
gap, and saying so points at the right cause.
`ownedConstsForBlocks` keeps its skip — it runs per shard and would
otherwise need the same error plumbing — but now documents that it is
safe only because this gate rejects such an env first.
Not changed: the composed verdict still has no assumption-cycle check,
and on analysis it does not need one. Assumptions are discharged per
CONSTANT along ref edges, which are acyclic by construction — a
constant's address is blake3 over bytes containing its refs' addresses,
so a reference cycle needs a hash cycle. A cycle in the shard-level graph
still stands for a well-founded per-constant chain, and
`verifyShardComposition` recomputes each digest via `shardClaimDigest`,
so a self-assuming shard claim cannot be substituted.
`compile_univ` memoized `Level -> Univ`, but a `Level::Param` compiles to
`Univ::Var(i)` where `i` is the parameter's POSITION in `univ_params` —
so the result depends on a context the key omitted. Unlike the expression
cache, which is cleared at every constant boundary, `univ_cache` lives
for a whole block, and `compile_definition` compiles each member under
its own `level_params`. `preseed_expr_tables` explicitly feeds
`(expr, univ_params)` pairs with differing param lists into the same
cache.
So two members of one block placing the same parameter NAME at different
positions would give the second member the first's index. Not an abort:
it compiles cleanly and registers a constant denoting a different
universe than the Lean source, under a correct-looking name.
The hazard is already recognized one level up — `collect_expr_tables`
keys `seen_exprs` by `(expr_hash, univ_params_key)` for exactly this
reason. Apply the same key here. The Lean mirror instead drops the cache
in `withUnivCtx`, matching how it already clears the expression cache per
constant.
Unreachable from ordinary Lean sources, since the elaborator gives every
member of a `mutual` block the same `levelParams` — but nothing checked
it, and under the prover threat model the environment is authored.
Also repairs `claim.rs`'s `defn_const`, whose Definition never referenced
its own `refs`. Since the claim walk admits an index only when some Expr
uses it as a constant, that fixture produced no walk edges and
`shard_check_env_claim_is_thin_frontier` has been failing since the walk
moved to positive constant-use. The identical flaw was fixed in
`shard_claim.rs`'s helper at the time and missed here, because only
`ixon` and `ixvm-codegen` were re-tested rather than the whole workspace.
Tests
- `cargo test --release` across all crates: 1195 passed, 0 failed.
- `lake test -- --ignored ixvm` green with NO pin movement and no address
changes, confirming no block in the corpus has members that order a
shared parameter name differently. The fix removes the dependence on
that remaining true.
- clippy `--all-targets` and fmt clean.
@johnchandlerburnham
johnchandlerburnham merged commit 80bb41f into mainAug 3, 2026
10 checks passed
@johnchandlerburnham
johnchandlerburnham deleted the ap/ixvm4 branch August 3, 2026 05:37
samuelburnham added a commit that referenced this pull request Aug 4, 2026
…acking
Port of sb/measured-ingress's planning layer onto the addr-first kernel
(#529), reworked for its lazy fault-in semantics.
Profiling (.ixprof v3): the recording kernel's try_get_const consults are
drained per constant into a touch graph — the measured set a lazy checker
faults in — persisted as a presence-flagged CSR section alongside a new
block-level reference graph and a subst_unique counter (splitmix64-keyed
dedup of substitution work items: the post-memoization volume a memoizing
executor pays). Both graph sections are optional, so older readers'
formats stay decodable by version gate alone.
Planner: partition_for_aiur_ram bin-packs to a host-RAM cap with
faulted-set byte accounting — under #529 the witness always ships the
full closure, but only faulted bytes cost ingress trace rows, so a
shard's byte feature is owned ∪ touched(owned) (full reference closure
as the over-approximating fallback when no touch graph was recorded).
The stub/promotion machinery of the original branch is dropped entirely:
lazy fault-in computes at run time exactly the set the stubs predicted,
with no replay divergence and no repair ladder, and the manifest format
stays byte-identical with main (owned-only).
Cost models: nlogn(x) = x·log2(x+2) feature form matching Aiur's
width·height·log2(height) FFT structure; prove/execute time and RAM
coefficients carried over from the pre-#529 aiur bench-suite fits.
They are placeholders pending a refit against this kernel — the circuit
shrank (deleted positional machinery), the byte feature narrowed to the
faulted set, and the recording counters moved under #442 — with
provenance noted at the definitions. The costs sidecar
(<out>.costs.csv) persists per-shard features incl. subst_unique and
nat_arith so the refit has its corpus.
CLI: ix shard --backend aiur --max-ram G; ix profile --backend/--top
(per-metric block leaderboards with display names) and ix profile sweep
(env-wide closure cost sweep: per-root feature CSV + feasibility,
min-root-per-hot-block, and diversity reports).
samuelburnham added a commit that referenced this pull request Aug 8, 2026
…acking
Port of sb/measured-ingress's planning layer onto the addr-first kernel
(#529), reworked for its lazy fault-in semantics.
Profiling (.ixprof v3): the recording kernel's try_get_const consults are
drained per constant into a touch graph — the measured set a lazy checker
faults in — persisted as a presence-flagged CSR section alongside a new
block-level reference graph and a subst_unique counter (splitmix64-keyed
dedup of substitution work items: the post-memoization volume a memoizing
executor pays). Both graph sections are optional, so older readers'
formats stay decodable by version gate alone.
Planner: partition_for_aiur_ram bin-packs to a host-RAM cap with
faulted-set byte accounting — under #529 the witness always ships the
full closure, but only faulted bytes cost ingress trace rows, so a
shard's byte feature is owned ∪ touched(owned) (full reference closure
as the over-approximating fallback when no touch graph was recorded).
The stub/promotion machinery of the original branch is dropped entirely:
lazy fault-in computes at run time exactly the set the stubs predicted,
with no replay divergence and no repair ladder, and the manifest format
stays byte-identical with main (owned-only).
Cost models: nlogn(x) = x·log2(x+2) feature form matching Aiur's
width·height·log2(height) FFT structure; prove/execute time and RAM
coefficients carried over from the pre-#529 aiur bench-suite fits.
They are placeholders pending a refit against this kernel — the circuit
shrank (deleted positional machinery), the byte feature narrowed to the
faulted set, and the recording counters moved under #442 — with
provenance noted at the definitions. The costs sidecar
(<out>.costs.csv) persists per-shard features incl. subst_unique and
nat_arith so the refit has its corpus.
CLI: ix shard --backend aiur --max-ram G; ix profile --backend/--top
(per-metric block leaderboards with display names) and ix profile sweep
(env-wide closure cost sweep: per-root feature CSV + feasibility,
min-root-per-hot-block, and diversity reports).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@arthurpaulino@johnchandlerburnham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

IxVM: cheaper shard checking via an addr-first kernel, plus soundness hardening - #529

Merged
johnchandlerburnham merged 19 commits into
mainfrom
ap/ixvm4
Aug 3, 2026
Merged

IxVM: cheaper shard checking via an addr-first kernel, plus soundness hardening#529
johnchandlerburnham merged 19 commits into
mainfrom
ap/ixvm4

Conversation

@arthurpaulino

@arthurpaulinoarthurpaulino commented Jul 31, 2026

Copy link
Copy Markdown
Member

Motivation

Make sharded environment checking cheap. A large environment is partitioned across shards, and each shard proves its own constants well-typed against a thin-frontier assumption cutoff, so every constant is checked in exactly one shard and cross-shard assumption trees stay minimal.

Delivering that required migrating the IxVM Aiur kernel from a positional constant table to a content-addressed, lazily-faulted design — and then auditing the result, which turned up five soundness holes with working exploits, including three separate routes to proving False.

Sharded environment checking

A shard proves CheckEnv{root, assumptions}: root is a merkle root over the shard's owned constants, assumptions a merkle root over the thin frontier — the direct out-of-owned walk edges. The kernel's env_walk starts from the owned set, checks each reachable constant, and stops at a frontier member (assumed well-typed by its owning shard: not re-checked, not recursed into).

  • Sub-frontier constants are never re-checked here; they are covered transitively by the frontier members' own shard claims. This is the cost win over checking each constant's full closure: no cross-shard duplication, and the assumption tree is one layer deep instead of the whole closure.
  • Strict membership: a walk-reachable constant in neither owned nor the frontier is a bad partition and hard-errors, so a shard cannot silently drop coverage of its own closure.
  • Byte scope ≠ check scope: delta unfolding reads through the frontier, so a witness ships the full dependency closure of owned even though only the strict-membership set is checked. The witness IOBuffer is built in Rust (aiur_ixvm_witness) rather than Lean, removing the per-byte boxing that dominated shard wall time.

Addr-first kernel

  • KExprNode.Const / Proj key on a 32-byte content Addr, not a numeric position; KConstantInfoInduct/Ctor/Rec carry a (block_addr, ind_idx) parent pair instead of indices. The positional top: List<KCI> table threaded through the kernel is gone.
  • get_ci(addr) is the sole resolver, Aiur-memoized per addr, faulting constant bytes lazily from an IOBuffer. load_verified_constant re-derives blake3 over the bytes and asserts key == addr; the io_get_info idx/len are unconstrained witness and are never branched on.
  • IOBuffer channels: 0 claim, 1 assumption tree, 2 constant bytes, 3 Defn reducibility hint (advisory), 4 blob bytes. The kernel derives blob-vs-constant from Expr context, so no wire discriminator ships. The two sets are not disjoint — one address can be read both ways — so each channel is seeded independently.
  • Kernel/Inductive.lean and Kernel/Primitive.lean deleted (~6.6k lines); inductive validation absorbed into Kernel/Check.lean, primitives split into Kernel/Klimbs.lean (bignum gadgets) and Kernel/NatPrim.lean (Nat/Str addr dispatch). New Kernel/InferOnly.lean breaks the DefEq↔Infer cycle. Ingress.lean collapsed 1700→298 lines.

Recursor validation

  • build_rec_type reconstructs a recursor's canonical type and asserts it def-eq to the declared type, on both the kind-aware and address-only lookup paths. Without it a recursor could declare ... -> False while its rules still reconstructed correctly.
  • spec_params are lowered from the major-premise frame into the recursor-param frame at extraction: the root cause of 17 nested/aux recursor reconstruction failures, and why the canonical check accepted a superset.
  • k_flag and parent-inductive-shape checks hoisted out of the attacker-selectable is_aux gate; the canonical rules comparison, which formerly never ran, now runs for every recursor.

Pre-existing criticals

IDFix
P1blob-vs-constant derived from Expr context instead of a prover-answered io_get_info probe, retiring the ch-2 len=0 sentinel; a prover can no longer mark a walk root or owned leaf "skip me"
P2Nat.div n 0 pins the quotient to zero (was prover-chosen)
P3klimbs_sub/le corrected on operands with trailing zero limbs; div/mod hints normalized before use
P4is_large_eliminator field-index arithmetic parenthesized (Aiur - is right-associative)
P5Reveal recursor rules assert the claimed ruleIdx equals its position

Robustness

  • L5: UTF-8 string-literal decode validates continuation-byte ranges and per-length overlong minimums, so distinct byte sequences (e.g. overlong NUL C0 80 vs 00) no longer decode to the same codepoint and def-eq as equal.
  • L6: run_claim dispatches on the full tag4 size (was the low byte only), matching Rust Claim::get.
  • L8: the K-recursor synth gate checks the major's inferred inductive against the recursor's own, not against itself, so I.rec no longer strips a major of a different inductive.

Aiur assert_eq! messages and circuit cleanup

  • assert_eq!(a, b, "why") gains an optional diagnostic label, threaded Source.Term → Typed → Simple → Concrete → Bytecode.Op → Rust Op::AssertEq(_, _, Option<String>) → ExecError::AssertEqMismatch. The message is diagnostic-only and never enters the circuit; all ~122 IxVM assert sites are labeled.
  • Unreachable _ => assert_eq!(0, 1) match arms removed: an unmatched runtime value already aborts (the kernel's reject), so the extra arm only widened the circuit.
  • Address equality is a full 32-byte content comparison — never a pointer-value compare, which is unsound for distinct-pointer/same-data — packed as five base-256 field limbs behind a byte-0 prefilter.

Witness construction

witness_scope reuses Env::bfs_closure, which already follows refs, Prj → block, and the derived Muts → member/ctor projection edges, so the wrappers get_ci synthesizes ship as forward edges of a single O(closure) walk with no reverse scan of the env. Primitive addresses are seeded as additional walk roots, because def-eq delta-unfolds primitive bodies whose refs are reachable from nowhere else. The Lean seeder computes the same scope by the same rule, so the two cannot drift.

bench-typecheck's constants metric now reads check_const's unique query count — exactly the constants typechecked — instead of the shipped byte scope, which is a superset and inflated throughput.

Soundness audit

Eight parallel read-only audits covered the pipeline end-to-end (compile, Ixon serde, ingress/convert, type-theory core, declaration checking, claim/witness layer, primitives, entry points), under the zkDSL threat model: a prover authors the data and pays for its own trace, so aborts and rejections are not threats and only a wrong answer counts. Thirty findings; every claim relayed here was re-verified against the code before acting on it, and several agent claims were corrected or withdrawn in the process.

Fixed, with a working exploit

Each of these shipped with a fixture that was red before the fix and green after, paired with a minimal-pair control so the verdict is attributable to one variable.

HoleWhat it bought an attacker
Self-referential definitionstheorem bad : False := bad typechecked. A standalone Defn was handed a recur slot naming itself, Rec erases to an ordinary Const, and k_infer reads a Const's declared type without checking it — so the body discharged its own type by citing itself. Now restricted to unsafe/partial, which is where Lean permits self-reference and where safe code cannot reach
Nested-inductive positivityThe nested branch dropped the parameter arguments and descended into the nested inductive's constructors without substituting them, so the occurrence being hunted was never present. Host | mk : Inner Host → Host was accepted for a negative Inner. Admits False
Recursor rules compared under the prover's contextcompare_rules handed the stored rule's own binder domains to k_is_def_eq as the comparison context. Retyping them to a Prop makes both sides infer to that Prop, proof irrelevance fires, and a recursor with its computation rules swapped between constructors was accepted
Non-canonical computed Nat literalsString.utf8ByteSize built a KLimbs as one unchecked byte, and klimbs_div_mod pinned the quotient's value but not its digits. Since klimbs_eq compares limbs rather than values, the kernel had the polarity backwards: Nat.beq (String.utf8ByteSize "") 0 = false was accepted while the true equation was rejected
Tag0/Tag2 overlong payloadsThree readers, three answers on the same bytes: the circuit discarded bytes past the eighth, the Lean host folded them back in (its shift is mod 64), Rust rejected. One address denoted different constants to different readers

Fixed as defense-in-depth

No exploit demonstrated for these; each commit says so in its first line. They remove a dependence on some other check happening to hold.

  • Struct-eta type equality.try_eta_struct omitted the infer(t) ≟ infer(s) side condition all three references carry, degenerating to t ≡ S.mk … (Proj S i t) … for any t of any type.
  • Recursor rule labels.cidx selects a rule at reduction time but was validated only positionally; nf decides how iota splits the constructor's arguments and was never checked against the constructor. Equivalents existed only in dead code.
  • Iota constructor identity. Rules were selected by index alone, so any inductive's constructor with a matching index fired one. Unreachable through a claim — inference k_checks every argument first — but reduction should not depend on that.
  • Universe cache context.compile_univ memoized Level → Univ while a Level::Param compiles to a position in univ_params, and the cache outlives a constant. Now keyed by (level, univ_params_key), matching what collect_expr_tables already does one level up.

Entry points and coverage

  • verify_const / verify_check / verify_check_env are DEBUG entrypoints that trust their dependencies, and they shipped in the production verifying key alongside verify_claim — leaving ix verify's funcIdx check as the only barrier, untested, with no equivalent in the recursive verifier. The production toplevel is now pruned to verify_claim alone, so such a proof cannot exist under that VK. Zero FFT cost.
  • bench-typecheck reported "status": "ok" and exit 0 when verification failed, and dropped thrown constants without marking them failed.
  • shardsCover skipped constants whose bytes do not parse — assigning them to no shard, counting them as neither owned nor unowned, and still including them in "covers all N consts". Now a hard failure.

Test guards that could not fail

Four of these were found by auditing the tests themselves, and they matter because they were the safety net for everything above.

  • prim-addrs asserted the literal true. Keyed by address, so a permuted primitive table passed — precisely the defect its docstring claimed to catch. Now the address must resolve to the constant the function's name claims; verified by temporarily swapping nat_add_addr to Nat.mul's bytes and watching it go red.
  • The arena negative corpus counted any abort as a correct rejection, so a fixture aborting on invalid IO key — never reaching the code under test — scored as a pass. That is the same failure mode that made two exploit fixtures in this series pass for the wrong reason before it was caught.
  • The shard-pipeline SKIP was a passing test conflating "target absent" with "partition selected zero constants".
  • A --ignored filter naming no runner executed nothing and returned 0, so a typo'd suite name reported success having run no tests.

Adversarial Ixon test suite

New Tests/Ix/IxVM/Exploits.lean, 19 cases, wired into the ixvm suite. The kernel-arena corpus attacks the layer above: its bad_* fixtures are raw Lean.ConstantInfo declarations that skip the elaborator, but they are still compiled by rsCompileEnvFFI, so the Ixon they produce is always well-formed. This suite authors Ixon.Constant values directly, and storeRawAt goes one level lower still, storing hand-authored byte strings — needed because every value-level fixture is canonical by construction and so cannot express a malformed encoding.

Each case records the false statement acceptance would buy, so a failure reads as a security statement. Rejection reasons are printed, which earned its keep repeatedly: fixtures that "passed" via invalid IO key without reaching the checker were caught this way, as was a fixture that silently corrupted nothing because non-mutual recursors are standalone constants rather than Muts members.

Testing

  • lake test -- --ignored ixvm green.
  • cargo test --release across all crates: 1195 passed, 0 failed. This also repaired shard_check_env_claim_is_thin_frontier, red since the claim walk moved to positive constant-use because its fixture's Exprs never referenced their own refs.
  • New prim-addrs parity suite; codegen ↔ bytecode parity gate green; arena negative corpus (77 bad_*) all reject, now only via genuine in-kernel refusals.
  • FFT-cost pins re-pinned per change. Cumulative effect is small (Nat.add_comm 42_747_927 → 42_986_025, +0.6% over the whole series); the largest single contributors are the nested-positivity substitution and the struct-eta guard.
  • ix codegen regenerated whenever the Aiur kernel changed, so the committed Rust kernel never lags the source the parity gate compares against.

Known limitations

Recorded rather than silently dropped.

  • Call-site surgery is not covered by the value pin. The compiler rewrites calls to aux-generated constants — permuting arguments, dropping some, replacing the head and its universe list — and the plans driving it live only in ConstantMeta, which prune_to_closure_anon excludes. So a verified claim says the term at address X is well-typed, not that term is the Lean declaration you wrote; the equivalence rests on the rewrite being faithful, checked today only by decompile round-trip tests. Closing it is a claim-format decision, not a patch.
  • Assumption cycles are not rejected, and a conditional Check claim naming its own target is accepted. That is the specified semantics — some root means "conditional on every leaf", so the claim asserts A → A, vacuous rather than false — and the obligation sits with whoever discharges assumptions. Sharded CheckEnv is not exposed, since shardClaimDigest recomputes the claim from (env, blocks). Pinned as ACCEPT so the boundary cannot move silently.
  • Alpha-collapse comparators ignore safety, so two members agreeing on kind/lvls/typ/value and disagreeing on safety collapse. Deliberately not fixed: a prover authors the environment directly and never goes through the compiler, so hardening it stops no attacker, while adding safety to the ordering would move every address and refusing the collapse would put a hard error in the compiler on the strength of an unverified assumption.
  • klimbs_le / klimbs_eq are in the logical trusted base, not just the performance path: Nat.decLe mints Decidable.isTrue with a proof term whose correctness is asserted rather than checked, so one wrong verdict yields False.
  • check_inductive_shape_ctors omits validate_expr_well_scoped / k_ensure_sort / assert_safety that check_const's Ctor arm performs; only an entirely unreferenced constructor escapes those.

@arthurpaulino
arthurpaulinoforce-pushed the ap/ixvm4 branch 4 times, most recently from 06e0489 to 559ced9CompareAugust 1, 2026 00:53
… hardening
Make sharded environment checking cheap. A large environment is partitioned
across shards, and each shard proves its own constants well-typed against a
thin-frontier assumption cutoff, so every constant is checked in exactly one
shard and cross-shard assumption trees stay minimal. Delivering that required
migrating the IxVM Aiur kernel from a positional constant table to a
content-addressed, lazily-faulted design -- and closing the soundness
regressions that migration introduced along with several pre-existing holes.
Sharded environment checking
- A shard proves `CheckEnv{root, assumptions}`: root is a merkle over the
shard's owned constants, assumptions a merkle over the THIN FRONTIER --
the direct out-of-owned walk edges. The kernel's env_walk starts from the
owned set, checks each reachable constant, and stops at a frontier member
(assumed well-typed by its owning shard, not re-checked, not recursed).
- Sub-frontier constants are never re-checked here; they are covered
transitively by the frontier members' own shard claims. This is the cost
win over checking each constant's full closure: no cross-shard duplication,
and the assumption tree is one layer deep instead of the whole closure.
- Strict membership: a walk-reachable constant in neither owned nor the
frontier is a bad partition and hard-errors, so a shard cannot silently
drop coverage of its own closure.
- Byte scope != check scope: delta unfolding reads through the frontier, so
a witness ships the full walk closure of owned even though only the
strict-membership set is checked. The witness IOBuffer is built in Rust
(aiur_ixvm_witness) rather than Lean, removing the per-byte boxing that
dominated shard wall time.
Addr-first kernel
- KExprNode.Const/Proj key on a 32-byte content Addr, not a numeric
position; KConstantInfo Induct/Ctor/Rec carry a (block_addr, ind_idx)
parent pair instead of indices. The positional `top: List<KCI>` table
threaded through the kernel is gone.
- get_ci(addr) is the sole resolver, Aiur-memoized per addr, faulting
constant bytes lazily from an IOBuffer. load_verified_constant re-derives
blake3 over the bytes and asserts key == addr; the io_get_info idx/len
are unconstrained witness and are never branched on.
- IOBuffer channels: 0 claim, 1 assumption tree, 2 constant bytes, 3 Defn
reducibility hint (advisory), 4 blob bytes. An address is on ch2 iff a
constant and ch4 iff a blob; the kernel derives which from Expr context
(Str/Nat -> blob, Ref/Prj -> constant), so no wire discriminator ships.
- Kernel/Inductive.lean and Kernel/Primitive.lean deleted (~6.6k lines);
inductive validation absorbed into Kernel/Check.lean, primitives split
into Kernel/Klimbs.lean (bignum gadgets) and Kernel/NatPrim.lean (Nat/Str
addr dispatch). New Kernel/InferOnly.lean breaks the DefEq<->Infer cycle.
Ingress.lean collapsed 1700->298 lines. V1/V2 naming removed throughout.
Recursor validation (R1, R2, R3)
- build_rec_type reconstructs a recursor's canonical type and asserts it
def-eq to the declared type, on both the kind-aware and address-only
lookup paths. Without it a recursor could declare `... -> False` while
its rules still reconstructed correctly.
- spec_params are lowered from the major-premise frame into the
recursor-param frame at extraction: the root cause of 17 nested/aux
recursor reconstruction failures, and why the canonical check accepted a
superset.
- k_flag and parent-inductive-shape checks hoisted out of the
attacker-selectable `is_aux` gate; the canonical rules comparison, which
formerly never ran, now runs for every recursor.
Pre-existing criticals (P1-P5)
- P1: blob-vs-constant is derived from Expr context instead of a prover-
answered io_get_info probe, retiring the ch-2 len=0 sentinel; a prover
can no longer mark a walk root or owned leaf "skip me".
- P2: Nat.div n 0 pins the quotient to zero (was prover-chosen).
- P3: klimbs_sub/le corrected on operands with trailing zero limbs; div/mod
hints normalized before use.
- P4: is_large_eliminator field-index arithmetic parenthesized (Aiur `-` is
right-associative).
- P5: Reveal recursor rules assert the claimed ruleIdx equals its position.
Robustness (L5, L6, L8)
- L5: UTF-8 string-literal decode validates continuation-byte ranges and
per-length overlong minimums, so distinct byte sequences (e.g. overlong
NUL C0 80 vs 00) no longer decode to the same codepoint and def-eq as
equal. Roughly zero added FFT cost.
- L6: run_claim dispatches on the full tag4 size (was the low byte only),
matching Rust Claim::get.
- L8: the K-recursor synth gate checks the major's inferred inductive
against the recursor's own (via se_parent_addr), not against itself, so
I.rec no longer strips a major of a different inductive.
Aiur assert_eq! messages + circuit cleanup
- assert_eq!(a, b, "why") gains an optional diagnostic label, threaded
Source.Term -> Typed -> Simple -> Concrete -> Bytecode.Op -> Rust
Op::AssertEq(_, _, Option<String>) -> ExecError::AssertEqMismatch. The
message is diagnostic-only and never enters the circuit; all ~122 IxVM
assert sites are labeled.
- Unreachable `_ => assert_eq!(0, 1)` match arms removed: an unmatched
runtime value already aborts (the kernel's reject), so the extra arm only
widened the circuit.
- address equality is a full 32-byte content comparison (never a pointer-
value compare, which is unsound for distinct-pointer/same-data), packed
as five base-256 field limbs (7+7+7+7+4) behind a byte-0 prefilter and
short-circuiting on the first mismatched group.
Crate reorganization
- PrimAddrs / reserved_marker_name moved verbatim from
crates/kernel/src/primitive.rs to crates/common/src/prim_addrs.rs (kernel
re-exports), so witness builders reach the primitive address table
without an ix-kernel -> ixon -> ix-kernel cycle.
- witness_scope reuses Env::bfs_closure -- the env's own dependency-closure
pass, which already follows refs, Prj->block, and the derived
Muts->member/ctor projection edges -- so the wrappers the kernel's get_ci
synthesizes ship as forward edges of a single O(closure) walk, with no
reverse scan of the whole env. Primitive-address seeding is unioned on
top: primitives are fabricated during reduction and need not appear in a
target's ref closure, so a closure-only scope aborts with `invalid IO
key`.
Tests
- New prim-addrs parity suite (Lean<->Aiur primitive address table).
- Codegen<->bytecode parity gate restored, green (198/198).
- FFT-cost pins re-pinned to current behavior; arena negative-fixture
corpus (77 bad_* constants) all reject.
…onstants
closureFromBase now walks the derived Muts->member/ctor projection
wrapper addresses as forward edges (projWrapperAddr: blake3 over the
serialized proj-only Constant, compile's construction), so closureFrom
and buildShardCheckEnvWitness drop their O(env) reverse scans that
parsed every env constant per witness build. Primitive addresses are
seeded bytes-only after the walk instead of being pushed onto the
worklist, so their transitive closures no longer inflate the byte scope
(a delta-unfold into an unshipped primitive body aborts with invalid IO
key rather than proving anything wrong).
bench-typecheck's constants metric now reads check_const's unique query
count (memoized per (ci, addr) = exactly the constants typechecked)
instead of closureFrom.size, which counted the shipped byte scope.
check_const is resolved up front and both lookups fail with IO.userError
instead of panicking.
Experiment: lake exe bench-typecheck --ixe seed.ixe --consts
Nat.add_comm --execute-only (seed.ixe = ix compile Tests.lean --consts
Nat.add_comm) reports constants=42, matching check_const's height in
lake exe ix check Nat.add_comm; fft-cost unchanged at 42747927.
The claim walk decided which of a constant's `refs` to recurse into by
SKIPPING every index the constant's Exprs used as a blob payload
(`Str(i)` / `Nat(i)`). That rule is unsound: the Exprs are authored by
the prover, and `refs` is one index space shared by `Ref`, `Prj`, `Str`
and `Nat`, so marking an index suppresses the walk of that index used as
a constant everywhere else in the same constant.
The marking need not even be live. `blob_idxs_of` traversed `sharing` in
full, but conversion reaches a `sharing` entry only through an
`Expr.Share` node, and nothing validates the array. A dead `Nat(i)`
entry is therefore free: it is never converted, never typechecked, and
needs no blob on channel 4. It just hides `refs[i]` from the checker.
`k_infer` then reads that dependency's DECLARED type without ever
checking it, so the kernel accepts a constant whose transitive closure
is ill-typed -- a wrong answer, not an abort. Both claim variants were
affected; for CheckEnv the skip happens before `env_walk` is entered, so
the strict owned-membership assert never fires either.
Replace the family with `const_idxs_*`: collect the indices `Ref(i, ..)`
and `Prj(i, ..)` use, and walk `refs[i]` iff `i` is among them. `Rec`
resolves through `recur_addrs` and a projection's block comes from the
ConstantInfo payload, both walked separately, so the positive set is
complete. Same single traversal as before, so no added cost. The failure
direction is now safe: over-approximating walks an address that faults
an unseeded key and aborts, which only costs a dishonest prover.
Adversarial Ixon test infrastructure
- New `Tests/Ix/IxVM/Exploits.lean`, wired into the ixvm suite. The
kernel-arena corpus attacks the layer above this one: its `bad_*`
fixtures are raw `Lean.ConstantInfo` declarations that skip the
elaborator, but they are still compiled by `rsCompileEnvFFI`, so the
Ixon they produce is always well-formed. This suite authors
`Ixon.Constant` values directly, which is what a prover can do.
- Each `ExploitCase` records the false statement acceptance would buy,
so a failure reads as a security statement. Rejection reasons are
printed: a fixture that aborts before reaching the mechanism under
attack is indistinguishable from a kernel that correctly refuses it,
and the first run of this fixture passed for exactly that wrong reason
(`invalid IO key` from an unseeded channel-3 hint, now seeded by
`storeAt`).
- The exploit ships with two controls that differ from it minimally: one
checking the smuggled dependency alone, one checking the same target
without the dead `sharing` entry. Both reject via the intended
in-kernel assert, so the exploit's acceptance was attributable to the
`sharing` entry alone.
- Two further cases probe what the fix does NOT cover. An index used as
both a `Nat` literal and a `Ref` — reachable because blob and constant
addresses share one un-domain-separated blake3 space and
`load_verified_blob` parses nothing — is now walked and rejected,
though by channel fault rather than type error, which is the safe
direction. A conditional Check claim naming its own target as its own
assumption IS accepted, and is pinned that way: `some root` means
"conditional on every leaf being well-typed", so the claim asserts
`A -> A` and is vacuous rather than false. The obligation sits with
whoever discharges assumptions, which must reject such cycles; sharded
CheckEnv is not exposed, since `shardClaimDigest` recomputes the claim
from (env, blocks).
Primitive closures are walked again
- The previous commit seeded primitive addresses as bytes only, without
walking their closures. That is too aggressive: def-eq delta-unfolds
primitive BODIES, and a body's refs are reachable from nowhere else in
the scope, so the `strAppend*` arena fixtures aborted with `invalid IO
key`. Both seeders now pass the primitives as walk ROOTS
(`closureFromRoots` in Lean, `bfs_closure` over target+primitives in
Rust); the Rust side had the same gap latently, since only the Lean
path had arena coverage.
- The removal of the O(env) reverse wrapper scan, which was that
commit's actual win, is unaffected.
Tests
- `lake test -- --ignored ixvm` green. FFT pins re-pinned across the 66
kernel-check entries and the shard pipeline; costs move in both
directions (e.g. `Eq.rec` 1_249_383 -> 1_249_094, `Vector.append`
1_994_697_378 -> 2_010_055_088), the increases coming from the
restored primitive closures.
- `ix codegen` regenerated: the Aiur kernel changed, so the committed
Rust kernel would otherwise be stale and the codegen/bytecode parity
gate would compare against it.
`Expr.Str` conversion wrapped any hash-valid blob in `KLiteral.Str`, and
`k_infer_lit` types a `KLiteral.Str` as `String` without inspecting the
bytes. Nothing on the ordinary checking path ever decodes a literal, so a
prover-authored definition carrying malformed bytes typechecked as
`x : String` even though no Lean `String` corresponds to those bytes. The
reference kernel rejects this at ingress via `String::from_utf8`
(crates/kernel/src/ingress.rs).
The UTF-8 validation added earlier lives in the string DECODER, which is
reached only during string reductions, so it did not cover this path.
`utf8_decode_one` already rejects out-of-range continuation bytes, stray
continuations used as leaders, and overlong forms. Walk the whole blob
with it at conversion (`utf8_validate`), which is the same rejection the
reference kernel performs.
Tests
- `invalid-utf8-accepted-as-string` in the adversarial Ixon suite: a
definition declaring `String` over the blob `[0xFF]`, which no
well-formed UTF-8 can contain. Accepted before this change, now
rejected.
- Paired with `control-valid-utf8-string-literal`, the same shape over
`[0x68, 0x69]`, which must still be accepted — so the verdict is
attributable to the bytes rather than to the fixture's shape. The
fixtures build on a base env supplying the real `String` constant, so
the declared type is the genuine one.
- Six FFT pins move, all string-touching, by between +17 and +66_530
(e.g. `IxVMPrim.str_size_lit` 567_999_042 -> 568_000_142,
`Vector.append` 2_010_055_088 -> 2_010_088_729, +0.0017%). Constants
with no string literals are unchanged, `Nat.add_comm` included.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`add_entries_parallel` returned early after a constant hit, so an address
present in both `env.consts` and `env.blobs` was seeded on channel 2 but
never on channel 4. The Lean seeder walks `consts` and `blobs` in
separate passes and seeds both, so the two builders disagreed, and the
Rust-native check/prove paths aborted with `invalid IO key` on such an
environment.
The two tables are not disjoint. Constant and blob addresses share a hash
codomain, and it is the Expr context — `Ref`/`Prj` versus `Str`/`Nat` —
that says which table a reference means, so the same bytes can legitimately
be read both ways. The kernel faults the two channels separately, so the
witness has to seed them separately. Reaching an address through both
kinds of use became expressible when the claim walk moved to selecting
indices by positive constant-use.
Corrects the module header and the `ChannelEntries` doc, which both
asserted the disjointness this relies on not having.
Tests
- `dual_use_address_is_seeded_on_both_channels`: an address stored as a
constant and as a blob must appear under both `(ch 2, key)` and
`(ch 4, key)` in the built IOBuffer. Fails on the previous code with
the ch-4 entry missing.
- `lake test -- --ignored ixvm` green; clippy `--all-targets` and fmt
clean.
The kernel walks a constant's `refs` by positive constant-use — index `i`
is followed only when some Expr names it via `Ref(i, ..)` or `Prj(i, ..)`
— but both host claim builders still treated every `refs` entry as a walk
edge. The host and the kernel therefore disagreed about the edge
relation.
A `refs` entry that is a real constant but that no Expr uses as one is
never walked in-circuit, so admitting it host-side puts a phantom member
in the thin frontier: a cross-shard assumption the kernel never makes,
and a spurious dependency edge that can close a cycle once assumptions
are discharged. `walk_check_set` likewise stopped computing the set the
verifier actually checks.
Mirror `const_idxs_of` on both hosts and emit an edge only for
positively-used indices. `Env::bfs_closure` is deliberately left alone:
it computes the witness BYTE scope, where over-approximating is correct
(shipping unused bytes is free, missing bytes aborts).
Honest constants are unaffected, since a compiler only puts an address in
`refs` when an Expr references it — the shard FFT pin is unchanged. The
divergence is reachable only through prover-authored `refs`.
Tests
- `thin_frontier_excludes_unused_ref_entries`: `refs` carries two
constants but only index 0 is named by a `Ref`; the frontier must
contain the used one and not the dead one. Verified red against the
previous edge relation ("unused refs entry became a phantom frontier
member").
- `const_with_refs` now names each of its `refs` entries via a `Ref`
node. The old helper left them unreferenced, which under the walk rule
would have produced no edges at all and quietly hollowed out the
existing frontier tests.
- `lake test -- --ignored ixvm` green; clippy `--all-targets` and fmt
clean.
`idx_to_u64` packed a synthesized member/constructor index into the
projection wrapper's `u64` field by populating only the low byte behind a
`u8_range_check`, so a mutual block with more than 256 members — or an
inductive with more than 256 constructors, which is the likelier case —
was rejected outright with `value 256 out of u8 range`.
Field-to-bytes is not free: the bytes have to be supplied as advice and
then pinned. Take them from `unconstrained_g_to_bytes` and discharge the
three obligations its contract names:
- range: four `u8_range_check` calls cover the eight bytes, the byte chip
taking a pair per lookup row;
- recomposition and canonicality: `flatten_u64(bytes) == idx`, since
`flatten_u64` asserts the top byte is zero and sums the low seven.
Canonicality is what makes the decomposition unique, and it is enforced
by RESTRICTING THE DOMAIN rather than by comparing against `p`. Goldilocks
is 2^64 - 2^32 + 1, so an arbitrary eight-byte string is not injective
into the field: any value and that value plus `p` recompose to the same
element, e.g. `[00]*8` and `[01,00,00,00,FF,FF,FF,FF]` both give 0.
Forcing the top byte to zero bounds the value by 2^56 - 1 < `p`, so the
sum cannot wrap and exactly one string encodes each index. That
uniqueness is the soundness property: the projection address is blake3
over these serialized bytes, so a prover free to choose a second string
for one index would obtain a second address for a member, or collide two
members onto one address. The pre-existing `u8_from_field_unsafe`
version did collide members 0 and 256, which the range check had already
fixed by rejecting instead.
Both halves are load-bearing. `flatten_u64` does NOT range-check its
input — `[U8; 8]` is nominal typing and `u8_from_field_unsafe` mints a
`U8` holding any field element — so without the range checks the sum is
satisfiable many ways (`b1 = k`, `b0 = idx - 256k`). Its doc comment now
records that its injectivity is conditional on the caller, and names this
call site.
Indices at or beyond 2^56 still fail the assert rather than truncating.
That is a liveness bound only, 2^48 times the previous cap and far past
any real block or constructor count; buying the remaining `[2^56, p)`
would cost a multi-byte comparison against `p` for no practical gain.
Tests
- `muts-block-past-single-byte-index`: a 257-member block of well-typed
definitions with a projection wrapper for each, which now checks.
Rejected before this change with `value 256 out of u8 range`.
- `control-small-muts-block`: the same construction at two members,
already accepted — so the big-block result is attributable to the index
width alone and not to a malformed fixture.
- `storeAt` also registers a reducibility hint for `DPrj` wrappers, which
`get_ci_dprj` faults under the wrapper's own address; without it these
fixtures aborted with `invalid IO key` before reaching the index logic.
- All 66 kernel-check FFT pins and the shard pin move by +17 to +485
(`Nat.add_comm` 42_967_279 -> 42_967_313, +0.00008%), the cost of the
hint plus four range checks per synthesized index.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`get_ci` handed every standalone `Defn` a one-element `recur_addrs` whose
only entry was its own address. `Rec` converts to a plain `Const`, and
`k_infer` discharges a `Const` against its DECLARED type without checking
it, so a definition could prove its own type by citing itself:
theorem bad : False := bad
typechecked. The ref-walk cannot catch it — `const_idxs_expr` does not
follow `Rec`, and the referent is the constant already under check. Lean
forbids exactly this by checking a definition against an environment that
does not yet contain it, admitting self-reference only for `unsafe` and
`partial`.
Restrict the self slot to `unsafe`/`partial`. Those are where
self-reference is both legitimate and contained:
- legitimate, because the compiler emits a singleton (or fully
collapsed) mutual block as a standalone `Defn` whose body still says
`Rec(0)` -- see the note at Ix/CompileM.lean:2000. Those blocks are
exactly Lean's `unsafe`/`partial` recursive definitions; a `safe`
recursive definition compiles its recursion into `.rec` applications.
Gating on `unsafe` alone breaks `Lean.extractMainModule._unsafe_rec`.
- contained, because safe code may not reference either kind.
That containment was itself missing for `partial`: `is_unsafe_ci`
classified it as safe, so `partial def bad : False := bad` plus a safe
theorem referencing `bad` would have walked straight around the first
half of this fix. Count `Partial` alongside `Unsafe`, matching the
reference's "safe definition references partial definition" rejection
(crates/kernel/src/check.rs:813-820). The predicate serves double duty --
the caller's own leniency and the referent's barrier -- and `Partial`
belongs on the non-safe side of both.
Tests
- `self-referential-theorem-proves-false`: `theorem bad : False := bad`
built directly in Ixon over the real `False` constant, submitted
through the production `verify_claim` entrypoint. Accepted before this
change -- a proof of False -- and rejected after.
- No safe constant in the corpus references a partial definition, and
the `_unsafe_rec` witnesses still check, so neither half of the fix
costs coverage.
- All 66 kernel-check FFT pins and the shard pin move (`Nat.add_comm`
42_967_313 -> 42_965_666); the self slot is no longer built for safe
definitions, so most costs drop slightly.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
The nested branch of the strict-positivity check dropped the parameter
arguments (`list_drop(args, n_ctor_params)`) and descended into the
nested inductive's constructors without substituting them. A nested
constructor's fields mention that inductive's parameters as bound
variables, so the block-occurrence test asked whether the block occurs in
`BVar 0` — it never does, and every parameter position passed vacuously.
The occurrence being hunted lives in the ARGUMENT substituted for the
parameter, which was exactly what got discarded.
So for `Inner α | mk : (α → False) → Inner α`, itself strictly positive,
`Host | mk : Inner Host → Host` was accepted: descending into `Inner.mk`
sees the field `α → False`, which mentions nothing tracked. With `α :=
Host` it reads `(Host → False) → …`, a negative occurrence, and Lean
rejects it. Accepting it admits `False`.
Mirror `check_nested_ctor_fields` (crates/kernel/src/inductive.rs): strip
`n_params` foralls, then simultaneously substitute the parameter
arguments at depth 0. After stripping, `BVar 0` is the innermost (last)
parameter, so the argument list is reversed — `expr_inst_many` maps
`substs[i]` to `BVar(depth + i)`, the same convention as `simul_subst`.
The parameters are substituted away rather than bound, so the field walk
now starts with an empty binder context instead of the peeled types.
Tests
- `nested-inductive-negative-occurrence`: the `Inner`/`Host` pair above,
built directly in Ixon as two single-inductive `Prop` blocks with their
projection wrappers. Accepted before this change, and now rejected by
the intended assert ("strict positivity: block occurs left of an
arrow").
- `control-nested-positive-inductive`: the same construction with
`Inner α | mk : α → Inner α`, which is strictly positive and must keep
checking. The two fixtures differ only in the inner constructor's
field, so the rejection is attributable to the negative occurrence
rather than to nesting.
- 63 FFT pins and the shard pin move, concentrated in inductive-heavy
constants (`IxVMInd.Tree` 767_167 -> 796_717, `IxVMInd.DepthM`
1_312_461 -> 1_347_394) — the nested check now does real work instead
of returning vacuously.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`KLimbs` is canonical in two independent ways — no trailing zero limbs,
and every limb byte in [0, 256) — and `klimbs_eq` is a raw limb compare
that establishes neither. Since it backs `Nat.beq`, `Nat.decEq` and
`literal_eq`, any producer emitting a value-correct but
representation-wrong `KLimbs` makes those answer `false` where Lean
answers `true`. That is an accepted false equation, not an abort. Two
producers did.
`String.utf8ByteSize` built the length as one unchecked byte
(`[u8_from_field_unsafe(n), 0, ...]`) with no normalization. The empty
string therefore reduced to the denormalized zero `[[0;8]]` instead of
the canonical empty limb list, and a string of 256 bytes or more to an
out-of-range digit — `u8_from_field_unsafe` is an erased coercion, so
nothing truncated or rejected. Build it with `klimbs_from_g`, whose byte
split is range-checked and pinned by a recomposition assert, then
normalize.
`klimbs_div_mod` pinned the quotient's VALUE (`q*b + r == a`, `r < b`)
but not its digits. The comment claimed `u8_mul` inside `klimbs_mul`
range-checked them, but `u64_mul` was rewritten to raw field products
plus `#split_carry`, whose u8 checks constrain the split outputs rather
than the input digits — and it re-canonicalizes while multiplying, so a
digit-wrong `q` still produces a canonical `q*b` and satisfies the
equality. Range-check the hint's limbs explicitly, quotient and
remainder both, via a new `klimbs_range_check`.
Tests
- `utf8-size-beq-false-accepted`: `Nat.beq (String.utf8ByteSize "") 0 =
false` proved by `Eq.refl`, over the real `Eq`/`Bool`/`Nat.beq`/
`String.utf8ByteSize` constants. The kernel had the polarity exactly
backwards — this FALSE equation was accepted while the true one was
rejected.
- `control-utf8-size-beq-true`: the same declaration ending in `true`,
which is what Lean computes and must be accepted. The pair differs only
in that operand, so each verdict is attributable to the reduction.
Both now land the right way round.
- The two fixtures share one blob for `""` and `0` — the same empty byte
string read as a string literal and as a nat literal, which is the
Expr-context disambiguation working as intended.
- 10 FFT pins and the shard pin move, confined to the div/mod/gcd/pow/
shift constants (the hint range-checks) and the string-size ones
(`IxVMPrim.str_size_lit` 567_992_937 -> 567_993_099).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT WAS DEMONSTRATED for this one. Unlike the other soundness
fixes in this series it ships without a red-then-green fixture; what
follows is why the rule is wrong as written, and what remains unproven.
`try_eta_struct` accepted `t ≡ S.mk p… a…` after checking only that the
right side is a fully-applied constructor of a struct-like inductive. It
never checked that the two sides have def-eq TYPES. All three references
do, citing lean4lean's `tryEtaStructCore`
(crates/kernel/src/def_eq.rs, Ix/Tc/DefEq.lean).
Without the guard the rule degenerates: for ANY `t` of ANY type,
`compare_struct_fields` builds precisely `Proj(S, i, t)` for each field,
so when the right side's arguments are those same projections the
pointer fast path matches them and the pair is accepted. Those
projections are synthesized inside def-eq and never inferred, so
`k_infer_proj`'s head-address gate — which would reject `Proj S i t` for
`t : Nat` — never sees them.
What is unproven is reachability. At the `k_check(e, expected)` boundary
both sides are types in the same context, and `de_args` compares type
arguments before value arguments, so I could not construct a term that
reaches this rule with genuinely mismatched types. The change is
therefore defense-in-depth against an unjustified rule, not a repair of a
demonstrated break.
The guard runs after the struct-like shape test, so it costs two
`k_infer_only` calls and one `k_is_def_eq` only on pairs that were about
to be accepted. `try_proof_irrel`, earlier in the same fallback chain,
already infers the left side unconditionally, so the added abort exposure
is small; the suite confirms no constant stops checking.
Tests
- No new fixture; the existing 14 adversarial cases and the arena corpus
are unchanged.
- 45 FFT pins and the shard pin move, the largest cost of this series
(`Nat.add_comm` 42_965_759 -> 42_966_970, +0.003%;
`IxVMPrim.str_size_lit` +26_687).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
A `Tag0` large-form header carries a 7-bit `small` field, so a payload
width up to 128 was expressible and unguarded (`Tag2`: 5 bits, up to 32).
A u64 payload is at most 8 bytes, and the three readers disagreed on the
surplus:
- Rust `u64_get_trimmed_le` rejects `len > 8` -- already correct, and
untouched here.
- The in-circuit `get_u64_le` consumed the surplus and DISCARDED it,
keeping only the first eight bytes.
- The Lean host folded it back in: `UInt64.shiftLeft` is taken mod 64,
so byte 8 ORs into bits 0-7, with no length guard.
Same bytes, same content address, three readings. An `Expr.Ref` whose
index is written overlong resolves as `refs[0]` in-circuit and `refs[42]`
on the Lean host, with both consuming the same span so nothing desyncs
afterwards. That also means the trailing-bytes assert does not imply
canonicity: up to 120 arbitrary bytes per `Tag0` site were consumed
inside the structure and dropped.
Bring the other two up to Rust: assert the width is under 9 in both
in-circuit readers, and give `getU64TrimmedLE` the guard it was missing.
No honest serializer is affected -- `putTag0` always emits the minimal
width -- so this narrows the accepted language to the format.
Tests
- `overlong-tag0-accepted`: an axiom `_ : Prop` whose `lvls` Tag0 is
rewritten to the nine-byte form. Accepted before this change, now
rejected by the intended assert.
- `control-canonical-tag0-axiom`: the same axiom with canonical bytes,
which must stay accepted. The pair differs only in that encoding.
- Both need bytes no `Ixon.ser` will emit, so the suite gained
`storeRawAt`, which stores a hand-authored byte string at its own
content address. Every previous fixture was built from a `Constant`
VALUE and was therefore canonical by construction, and structurally
could not reach this class of defect.
- All 66 kernel-check FFT pins and the shard pin move (`Nat.add_comm`
42_966_970 -> 42_980_767, +0.03%); every constant parses Tag0s.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
Four test-only fixes. No kernel code changes, so no FFT pins move.
prim-addrs asserted the literal `true`. It looked up each hardcoded
kernel address in the canonical table and, on a hit, recorded a pass —
but the table is keyed BY ADDRESS, so a permuted table still hits. Had
`nat_add_addr` held `Nat.mul`'s bytes it would have printed
"nat_add_addr = Nat.mul" and passed green, which is precisely the defect
the file's own docstring says it exists to catch: the kernel applying one
primitive's native semantics under another's name. Now the address must
resolve to the constant the function's NAME claims, comparing normalized
keys (`nat_add_addr` <-> `Nat.add`, and the final-component spelling for
`reduce_bool_addr` <-> `Lean.reduceBool`), with dispatch-site suffixes
(`_dec`/`_io`/`_iota`) and `_type`/`_ty` stripped, plus a short exception
list for the `Quot` constructors. Verified by temporarily swapping
`nat_add_addr` to `Nat.mul`'s bytes: red, then reverted. All 76 literals
pass, and the table is confirmed correct today.
The arena negative corpus counted ANY abort as a correct rejection while
its docstring claimed rejections come from an in-kernel `assert_eq!`. An
`invalid IO key` — the witness never supplied the bytes, so execution
died before reaching the logic under test — scored as a pass. That is how
a fixture rots into a green test that checks nothing, and it is the same
failure mode that made two exploit fixtures in this series pass for the
wrong reason. Now `invalid IO key` disqualifies. Unmatched-match aborts
still count: this kernel deliberately relies on an unmatched runtime
value aborting instead of spending width on `_ => assert_eq!(0, 1)`
arms, so those are genuine verdicts. No fixture currently relies on the
disqualified path.
`shardCheckEnvCase` returned `none` for two unrelated reasons and the
runner reported both as "SKIP (target absent)", a passing test. The
second — target present but the partition selected zero owned constants —
means the fixture stopped exercising the Rust witness builder and the
thin-frontier claim convention, and would have left the only test of that
path permanently green. It now throws; `none` keeps its one honest
meaning.
A `--ignored` filter naming no runner ran nothing and returned 0, so
`lake test -- --ignored ixvm2` reported success having executed nothing.
It is now an error. Exit 0 should mean tests ran and passed.
`ixVM`, the pruned production toplevel, carried `verify_const`,
`verify_check` and `verify_check_env` alongside `verify_claim`. Those are
DEBUG entrypoints that trust their dependencies instead of walking them,
so a proof made with one attests far less than a claim does — and
carrying them meant such a proof was verifiable under the SAME verifying
key. The only thing standing between it and acceptance was `ix verify`
pinning the funcIdx to `verify_claim`: a policy, enforced in one place,
with no test asserting it, and no policy at all in the recursive
verifier, which binds the claim bytes but applies no funcIdx check.
Prune to `verify_claim` alone. The guarantee is now structural: under the
production VK a debug-entrypoint proof cannot exist. No CLI path used
them, so nothing in `ix check`/`prove`/`verify` changes.
Callers that genuinely want a subject-only check build `ixVMFull` and are
thereby using a different VK, which is the honest signal — the arena
corpus, and `bench-typecheck`/`bench-recursion-debug` under
`--skip-deps`. That last one also means a `--skip-deps` benchmark is now
visibly measuring a different key than a claim run rather than silently
reporting `verify_const` rows in the claim schema.
`bench-typecheck` also reported success on failure, twice over. A proof
that failed to verify printed to stderr but left `failed` clear, so the
row was emitted as `"status": "ok"` with prove-time and proof-size
present and the run exited 0 — the only signal an absent `verify-time`
key, invisible to anything keying on status. And a constant whose execute
threw was dropped from the results entirely without setting `failed`,
so `execed.any (·.failed)` stayed clear; the `check_const`-missing throw
added earlier landed exactly there. Both now mark the row failed.
Tests
- `lake test -- --ignored ixvm` green with NO pin movement: the pruned
entrypoints were never executed, so removing them changes no cost.
Generated kernel drops from 733 to 731 functions.
- Arena repointed to `ixVMFull`, where `verify_const` still lives.
- `bench-typecheck --execute-only` smoke unchanged (`Nat.add_comm`,
constants=42).
`compare_rules` peeled the binders off the STORED rule and handed their
DOMAINS to `k_is_def_eq` as the local context for the comparison against
the canonical reconstruction. Those domains are prover-authored: nothing
checks them against the reconstruction, and reduction never consults them
(`try_iota` beta-reduces straight through). So the prover chose what the
two sides INFER to.
Retyping every binder to a Prop makes both sides infer to that Prop, so
`try_proof_irrel` fires and accepts ANY body. A recursor whose
computation rules are swapped between constructors was therefore
accepted, and iota then reduces each constructor to the other's branch.
Compare under an empty context instead, discarding the peeled domains. An
honest rule matches the reconstruction structurally, and the structural
path never consults the context — it compares `BVar i` against `BVar i`
directly. A comparison that instead needs to INFER a bound variable's
type now runs out of context and aborts, which is the safe direction.
Every recursor in the corpus still checks, including the mutual, nested
and aux-generated ones.
The reference avoids this differently, comparing whole Lam-wrapped terms
at empty context so `is_def_eq` descends binder-by-binder and compares
the domains themselves (crates/kernel/src/inductive.rs). Reconstructing
canonical domains here would mean re-deriving field domains from each
constructor's telescope and re-lifting their de Bruijn indices into the
rule's context; dropping the untrusted context achieves the same
guarantee without that machinery.
Tests
- `recursor-rule-rhs-doctored-binder-domains`: a real compiled recursor
with its first two computation rules swapped AND every peeled binder
domain replaced by a Prop. Accepted before this change; rejected now.
- `control-swapped-recursor-rules`: the identical swap with the
compiler's own binder domains, which was already rejected. The two
differ only in the domains, so the acceptance was attributable to them
alone.
- Both are built by rewriting a REAL block rather than hand-authoring a
recursor, since a hand-built one must survive `build_rec_type`'s def-eq
and constructor references inside a block are circular with the block's
own address. The fixture selects a recursor whose rules are field-free,
so no minor premise is applied and the retyping cannot abort as a
non-function before the comparison is reached — an earlier attempt did
exactly that and masked the acceptance.
- A guard case fails loudly if no such recursor is found; the first
version of this fixture silently corrupted nothing, and both its cases
"passed" meaninglessly.
- 57 FFT pins and the shard pin move (`Nat.add_comm` 42_980_767 ->
42_980_801).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT DEMONSTRATED. Defense-in-depth, like the struct-eta guard.
`compare_rules` now asserts three invariants that equivalents of already
exist in `check_rec_rules_walk` — which nothing calls. The live path had
none of them.
* `cidx == pos`. `find_rule` selects a rule at reduction time BY LABEL
while `compare_rules` pairs stored against canonical POSITIONALLY, so
permuted labels would validate one constructor's rule and then fire
it for another. It holds today only because ingress assigns `cidx`
positionally (`Convert.lean`) and the Ixon `RecursorRule` record
carries no cidx of its own — an accident of the pipeline rather than
a checked property, which is exactly what an assert is for.
* `cidx < num_ctors`, so a rule cannot name a constructor that does not
exist.
* `nf` equals the constructor's field count. `try_iota` splits the
constructor's arguments with `field_start = ctor_fields_len -
rfields`, so a wrong `nf` hands the minor premise the wrong slice; in
field arithmetic an over-large value wraps.
The reachability argument for the `nf` gap was "combined with the rule
RHS context hole"; that hole is fixed, and with the def-eq comparison
restored a mismatched peel is caught again. So this closes a route that
is currently blocked by another check rather than one that is open —
worth asserting locally rather than depending on a second mechanism.
Not fixed here: iota still selects rules by index rather than by
constructor IDENTITY, so a recursor applied to a foreign constructor
would reduce. That is not reachable through a claim —
`k_infer_app_spine_loop` runs `k_check` on every argument, so the
application fails inference before whnf sees it — and closing it properly
needs the parent inductive available at reduction time, which
`KConstantInfo.Rec` does not carry (its `Addr` is the recursor's own
block; the parent is derived at check time from the type). That is a KCI
layout change, left for its own pass.
Tests
- No new fixture; nothing demonstrates the hole.
- Every recursor in the corpus satisfies all three invariants, including
the mutual, nested and aux-generated ones, so they held by construction
and are now enforced.
- 57 FFT pins and the shard pin move (`Nat.add_comm` 42_980_801 ->
42_981_336) for the per-rule constructor lookup.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT DEMONSTRATED. Defense-in-depth.
`try_iota` selected a computation rule by constructor INDEX alone:
`collect_spine_of_ctor` returned the major's `cidx` and discarded which
inductive it came from, so `find_rule(rules, cidx)` fired whenever any
inductive's constructor happened to carry a matching index. On its face
`Bool.rec motive f t (Nat.succ n)` reduces to `t`. Lean's kernel selects
the rule by the constructor's NAME, which pins the inductive.
Not reachable through a claim: `k_infer_app_spine_loop` runs `k_check` on
every argument, so a recursor applied to a foreign constructor fails
inference long before whnf sees the redex — the ill-typed term cannot be
stated. The Rust reference selects by index too, consistent with that
being why it was tolerated. What the patch removes is the DEPENDENCE on
that: reduction no longer needs inference to have run first to be
correct.
`collect_spine_of_ctor` now also returns the constructor's parent, as the
address of its projection wrapper, and `try_iota` asserts it equals the
recursor's own parent. That parent is read off the recursor's type (the
major premise's head) with the existing `rec_to_parent_addr`, so no
KConstantInfo layout change was needed — its `Addr` field is the
recursor's own block, not the parent's. Aiur memoizes per argument tuple,
so the type walk is paid once per recursor rather than once per
reduction.
Tests
- No new fixture; nothing demonstrates the hole.
- Every recursor in the corpus still reduces, including the mutual,
nested and aux-generated ones, and the Nat/String literal paths that
synthesize a constructor application before iota (`nat_cases_big`,
`nat_pow_big`, `strOfListFold*` all unchanged in behaviour).
- 63 FFT pins and the shard pin move (`Nat.add_comm` 42_981_336 ->
42_986_025, +0.01%) for the parent comparison.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`shardsCover` gates every shard run: "exit 0 has to mean every env const
was checked by some shard". It skipped constants whose bytes fail to
parse (`LazyConstant.get?` discards the error and `.ixe` loading is
lazy, so such a constant sits in `consts` with its key present). A
skipped constant was assigned to no shard, counted as neither owned nor
unowned, and still included in the "covers all N consts" total — so the
gate reported success over constants it had not covered.
It would also enter a referring shard's frontier, since `thin_frontier`
admits an edge on key presence in `env.consts` without parsing, leaving
an assumption no shard discharges.
Count them separately and fail, rather than folding them into `unowned`:
"cannot be assigned to a shard" is a different problem from a partition
gap, and saying so points at the right cause.
`ownedConstsForBlocks` keeps its skip — it runs per shard and would
otherwise need the same error plumbing — but now documents that it is
safe only because this gate rejects such an env first.
Not changed: the composed verdict still has no assumption-cycle check,
and on analysis it does not need one. Assumptions are discharged per
CONSTANT along ref edges, which are acyclic by construction — a
constant's address is blake3 over bytes containing its refs' addresses,
so a reference cycle needs a hash cycle. A cycle in the shard-level graph
still stands for a well-founded per-constant chain, and
`verifyShardComposition` recomputes each digest via `shardClaimDigest`,
so a self-assuming shard claim cannot be substituted.
`compile_univ` memoized `Level -> Univ`, but a `Level::Param` compiles to
`Univ::Var(i)` where `i` is the parameter's POSITION in `univ_params` —
so the result depends on a context the key omitted. Unlike the expression
cache, which is cleared at every constant boundary, `univ_cache` lives
for a whole block, and `compile_definition` compiles each member under
its own `level_params`. `preseed_expr_tables` explicitly feeds
`(expr, univ_params)` pairs with differing param lists into the same
cache.
So two members of one block placing the same parameter NAME at different
positions would give the second member the first's index. Not an abort:
it compiles cleanly and registers a constant denoting a different
universe than the Lean source, under a correct-looking name.
The hazard is already recognized one level up — `collect_expr_tables`
keys `seen_exprs` by `(expr_hash, univ_params_key)` for exactly this
reason. Apply the same key here. The Lean mirror instead drops the cache
in `withUnivCtx`, matching how it already clears the expression cache per
constant.
Unreachable from ordinary Lean sources, since the elaborator gives every
member of a `mutual` block the same `levelParams` — but nothing checked
it, and under the prover threat model the environment is authored.
Also repairs `claim.rs`'s `defn_const`, whose Definition never referenced
its own `refs`. Since the claim walk admits an index only when some Expr
uses it as a constant, that fixture produced no walk edges and
`shard_check_env_claim_is_thin_frontier` has been failing since the walk
moved to positive constant-use. The identical flaw was fixed in
`shard_claim.rs`'s helper at the time and missed here, because only
`ixon` and `ixvm-codegen` were re-tested rather than the whole workspace.
Tests
- `cargo test --release` across all crates: 1195 passed, 0 failed.
- `lake test -- --ignored ixvm` green with NO pin movement and no address
changes, confirming no block in the corpus has members that order a
shared parameter name differently. The fix removes the dependence on
that remaining true.
- clippy `--all-targets` and fmt clean.
@johnchandlerburnham
johnchandlerburnham merged commit 80bb41f into mainAug 3, 2026
10 checks passed
@johnchandlerburnham
johnchandlerburnham deleted the ap/ixvm4 branch August 3, 2026 05:37
samuelburnham added a commit that referenced this pull request Aug 4, 2026
…acking
Port of sb/measured-ingress's planning layer onto the addr-first kernel
(#529), reworked for its lazy fault-in semantics.
Profiling (.ixprof v3): the recording kernel's try_get_const consults are
drained per constant into a touch graph — the measured set a lazy checker
faults in — persisted as a presence-flagged CSR section alongside a new
block-level reference graph and a subst_unique counter (splitmix64-keyed
dedup of substitution work items: the post-memoization volume a memoizing
executor pays). Both graph sections are optional, so older readers'
formats stay decodable by version gate alone.
Planner: partition_for_aiur_ram bin-packs to a host-RAM cap with
faulted-set byte accounting — under #529 the witness always ships the
full closure, but only faulted bytes cost ingress trace rows, so a
shard's byte feature is owned ∪ touched(owned) (full reference closure
as the over-approximating fallback when no touch graph was recorded).
The stub/promotion machinery of the original branch is dropped entirely:
lazy fault-in computes at run time exactly the set the stubs predicted,
with no replay divergence and no repair ladder, and the manifest format
stays byte-identical with main (owned-only).
Cost models: nlogn(x) = x·log2(x+2) feature form matching Aiur's
width·height·log2(height) FFT structure; prove/execute time and RAM
coefficients carried over from the pre-#529 aiur bench-suite fits.
They are placeholders pending a refit against this kernel — the circuit
shrank (deleted positional machinery), the byte feature narrowed to the
faulted set, and the recording counters moved under #442 — with
provenance noted at the definitions. The costs sidecar
(<out>.costs.csv) persists per-shard features incl. subst_unique and
nat_arith so the refit has its corpus.
CLI: ix shard --backend aiur --max-ram G; ix profile --backend/--top
(per-metric block leaderboards with display names) and ix profile sweep
(env-wide closure cost sweep: per-root feature CSV + feasibility,
min-root-per-hot-block, and diversity reports).
samuelburnham added a commit that referenced this pull request Aug 8, 2026
…acking
Port of sb/measured-ingress's planning layer onto the addr-first kernel
(#529), reworked for its lazy fault-in semantics.
Profiling (.ixprof v3): the recording kernel's try_get_const consults are
drained per constant into a touch graph — the measured set a lazy checker
faults in — persisted as a presence-flagged CSR section alongside a new
block-level reference graph and a subst_unique counter (splitmix64-keyed
dedup of substitution work items: the post-memoization volume a memoizing
executor pays). Both graph sections are optional, so older readers'
formats stay decodable by version gate alone.
Planner: partition_for_aiur_ram bin-packs to a host-RAM cap with
faulted-set byte accounting — under #529 the witness always ships the
full closure, but only faulted bytes cost ingress trace rows, so a
shard's byte feature is owned ∪ touched(owned) (full reference closure
as the over-approximating fallback when no touch graph was recorded).
The stub/promotion machinery of the original branch is dropped entirely:
lazy fault-in computes at run time exactly the set the stubs predicted,
with no replay divergence and no repair ladder, and the manifest format
stays byte-identical with main (owned-only).
Cost models: nlogn(x) = x·log2(x+2) feature form matching Aiur's
width·height·log2(height) FFT structure; prove/execute time and RAM
coefficients carried over from the pre-#529 aiur bench-suite fits.
They are placeholders pending a refit against this kernel — the circuit
shrank (deleted positional machinery), the byte feature narrowed to the
faulted set, and the recording counters moved under #442 — with
provenance noted at the definitions. The costs sidecar
(<out>.costs.csv) persists per-shard features incl. subst_unique and
nat_arith so the refit has its corpus.
CLI: ix shard --backend aiur --max-ram G; ix profile --backend/--top
(per-metric block leaderboards with display names) and ix profile sweep
(env-wide closure cost sweep: per-root feature CSV + feasibility,
min-root-per-hot-block, and diversity reports).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@arthurpaulino@johnchandlerburnham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

IxVM: cheaper shard checking via an addr-first kernel, plus soundness hardening - #529

Merged
johnchandlerburnham merged 19 commits into
mainfrom
ap/ixvm4
Aug 3, 2026
Merged

IxVM: cheaper shard checking via an addr-first kernel, plus soundness hardening#529
johnchandlerburnham merged 19 commits into
mainfrom
ap/ixvm4

Conversation

@arthurpaulino

@arthurpaulinoarthurpaulino commented Jul 31, 2026

Copy link
Copy Markdown
Member

Motivation

Make sharded environment checking cheap. A large environment is partitioned across shards, and each shard proves its own constants well-typed against a thin-frontier assumption cutoff, so every constant is checked in exactly one shard and cross-shard assumption trees stay minimal.

Delivering that required migrating the IxVM Aiur kernel from a positional constant table to a content-addressed, lazily-faulted design — and then auditing the result, which turned up five soundness holes with working exploits, including three separate routes to proving False.

Sharded environment checking

A shard proves CheckEnv{root, assumptions}: root is a merkle root over the shard's owned constants, assumptions a merkle root over the thin frontier — the direct out-of-owned walk edges. The kernel's env_walk starts from the owned set, checks each reachable constant, and stops at a frontier member (assumed well-typed by its owning shard: not re-checked, not recursed into).

  • Sub-frontier constants are never re-checked here; they are covered transitively by the frontier members' own shard claims. This is the cost win over checking each constant's full closure: no cross-shard duplication, and the assumption tree is one layer deep instead of the whole closure.
  • Strict membership: a walk-reachable constant in neither owned nor the frontier is a bad partition and hard-errors, so a shard cannot silently drop coverage of its own closure.
  • Byte scope ≠ check scope: delta unfolding reads through the frontier, so a witness ships the full dependency closure of owned even though only the strict-membership set is checked. The witness IOBuffer is built in Rust (aiur_ixvm_witness) rather than Lean, removing the per-byte boxing that dominated shard wall time.

Addr-first kernel

  • KExprNode.Const / Proj key on a 32-byte content Addr, not a numeric position; KConstantInfoInduct/Ctor/Rec carry a (block_addr, ind_idx) parent pair instead of indices. The positional top: List<KCI> table threaded through the kernel is gone.
  • get_ci(addr) is the sole resolver, Aiur-memoized per addr, faulting constant bytes lazily from an IOBuffer. load_verified_constant re-derives blake3 over the bytes and asserts key == addr; the io_get_info idx/len are unconstrained witness and are never branched on.
  • IOBuffer channels: 0 claim, 1 assumption tree, 2 constant bytes, 3 Defn reducibility hint (advisory), 4 blob bytes. The kernel derives blob-vs-constant from Expr context, so no wire discriminator ships. The two sets are not disjoint — one address can be read both ways — so each channel is seeded independently.
  • Kernel/Inductive.lean and Kernel/Primitive.lean deleted (~6.6k lines); inductive validation absorbed into Kernel/Check.lean, primitives split into Kernel/Klimbs.lean (bignum gadgets) and Kernel/NatPrim.lean (Nat/Str addr dispatch). New Kernel/InferOnly.lean breaks the DefEq↔Infer cycle. Ingress.lean collapsed 1700→298 lines.

Recursor validation

  • build_rec_type reconstructs a recursor's canonical type and asserts it def-eq to the declared type, on both the kind-aware and address-only lookup paths. Without it a recursor could declare ... -> False while its rules still reconstructed correctly.
  • spec_params are lowered from the major-premise frame into the recursor-param frame at extraction: the root cause of 17 nested/aux recursor reconstruction failures, and why the canonical check accepted a superset.
  • k_flag and parent-inductive-shape checks hoisted out of the attacker-selectable is_aux gate; the canonical rules comparison, which formerly never ran, now runs for every recursor.

Pre-existing criticals

IDFix
P1blob-vs-constant derived from Expr context instead of a prover-answered io_get_info probe, retiring the ch-2 len=0 sentinel; a prover can no longer mark a walk root or owned leaf "skip me"
P2Nat.div n 0 pins the quotient to zero (was prover-chosen)
P3klimbs_sub/le corrected on operands with trailing zero limbs; div/mod hints normalized before use
P4is_large_eliminator field-index arithmetic parenthesized (Aiur - is right-associative)
P5Reveal recursor rules assert the claimed ruleIdx equals its position

Robustness

  • L5: UTF-8 string-literal decode validates continuation-byte ranges and per-length overlong minimums, so distinct byte sequences (e.g. overlong NUL C0 80 vs 00) no longer decode to the same codepoint and def-eq as equal.
  • L6: run_claim dispatches on the full tag4 size (was the low byte only), matching Rust Claim::get.
  • L8: the K-recursor synth gate checks the major's inferred inductive against the recursor's own, not against itself, so I.rec no longer strips a major of a different inductive.

Aiur assert_eq! messages and circuit cleanup

  • assert_eq!(a, b, "why") gains an optional diagnostic label, threaded Source.Term → Typed → Simple → Concrete → Bytecode.Op → Rust Op::AssertEq(_, _, Option<String>) → ExecError::AssertEqMismatch. The message is diagnostic-only and never enters the circuit; all ~122 IxVM assert sites are labeled.
  • Unreachable _ => assert_eq!(0, 1) match arms removed: an unmatched runtime value already aborts (the kernel's reject), so the extra arm only widened the circuit.
  • Address equality is a full 32-byte content comparison — never a pointer-value compare, which is unsound for distinct-pointer/same-data — packed as five base-256 field limbs behind a byte-0 prefilter.

Witness construction

witness_scope reuses Env::bfs_closure, which already follows refs, Prj → block, and the derived Muts → member/ctor projection edges, so the wrappers get_ci synthesizes ship as forward edges of a single O(closure) walk with no reverse scan of the env. Primitive addresses are seeded as additional walk roots, because def-eq delta-unfolds primitive bodies whose refs are reachable from nowhere else. The Lean seeder computes the same scope by the same rule, so the two cannot drift.

bench-typecheck's constants metric now reads check_const's unique query count — exactly the constants typechecked — instead of the shipped byte scope, which is a superset and inflated throughput.

Soundness audit

Eight parallel read-only audits covered the pipeline end-to-end (compile, Ixon serde, ingress/convert, type-theory core, declaration checking, claim/witness layer, primitives, entry points), under the zkDSL threat model: a prover authors the data and pays for its own trace, so aborts and rejections are not threats and only a wrong answer counts. Thirty findings; every claim relayed here was re-verified against the code before acting on it, and several agent claims were corrected or withdrawn in the process.

Fixed, with a working exploit

Each of these shipped with a fixture that was red before the fix and green after, paired with a minimal-pair control so the verdict is attributable to one variable.

HoleWhat it bought an attacker
Self-referential definitionstheorem bad : False := bad typechecked. A standalone Defn was handed a recur slot naming itself, Rec erases to an ordinary Const, and k_infer reads a Const's declared type without checking it — so the body discharged its own type by citing itself. Now restricted to unsafe/partial, which is where Lean permits self-reference and where safe code cannot reach
Nested-inductive positivityThe nested branch dropped the parameter arguments and descended into the nested inductive's constructors without substituting them, so the occurrence being hunted was never present. Host | mk : Inner Host → Host was accepted for a negative Inner. Admits False
Recursor rules compared under the prover's contextcompare_rules handed the stored rule's own binder domains to k_is_def_eq as the comparison context. Retyping them to a Prop makes both sides infer to that Prop, proof irrelevance fires, and a recursor with its computation rules swapped between constructors was accepted
Non-canonical computed Nat literalsString.utf8ByteSize built a KLimbs as one unchecked byte, and klimbs_div_mod pinned the quotient's value but not its digits. Since klimbs_eq compares limbs rather than values, the kernel had the polarity backwards: Nat.beq (String.utf8ByteSize "") 0 = false was accepted while the true equation was rejected
Tag0/Tag2 overlong payloadsThree readers, three answers on the same bytes: the circuit discarded bytes past the eighth, the Lean host folded them back in (its shift is mod 64), Rust rejected. One address denoted different constants to different readers

Fixed as defense-in-depth

No exploit demonstrated for these; each commit says so in its first line. They remove a dependence on some other check happening to hold.

  • Struct-eta type equality.try_eta_struct omitted the infer(t) ≟ infer(s) side condition all three references carry, degenerating to t ≡ S.mk … (Proj S i t) … for any t of any type.
  • Recursor rule labels.cidx selects a rule at reduction time but was validated only positionally; nf decides how iota splits the constructor's arguments and was never checked against the constructor. Equivalents existed only in dead code.
  • Iota constructor identity. Rules were selected by index alone, so any inductive's constructor with a matching index fired one. Unreachable through a claim — inference k_checks every argument first — but reduction should not depend on that.
  • Universe cache context.compile_univ memoized Level → Univ while a Level::Param compiles to a position in univ_params, and the cache outlives a constant. Now keyed by (level, univ_params_key), matching what collect_expr_tables already does one level up.

Entry points and coverage

  • verify_const / verify_check / verify_check_env are DEBUG entrypoints that trust their dependencies, and they shipped in the production verifying key alongside verify_claim — leaving ix verify's funcIdx check as the only barrier, untested, with no equivalent in the recursive verifier. The production toplevel is now pruned to verify_claim alone, so such a proof cannot exist under that VK. Zero FFT cost.
  • bench-typecheck reported "status": "ok" and exit 0 when verification failed, and dropped thrown constants without marking them failed.
  • shardsCover skipped constants whose bytes do not parse — assigning them to no shard, counting them as neither owned nor unowned, and still including them in "covers all N consts". Now a hard failure.

Test guards that could not fail

Four of these were found by auditing the tests themselves, and they matter because they were the safety net for everything above.

  • prim-addrs asserted the literal true. Keyed by address, so a permuted primitive table passed — precisely the defect its docstring claimed to catch. Now the address must resolve to the constant the function's name claims; verified by temporarily swapping nat_add_addr to Nat.mul's bytes and watching it go red.
  • The arena negative corpus counted any abort as a correct rejection, so a fixture aborting on invalid IO key — never reaching the code under test — scored as a pass. That is the same failure mode that made two exploit fixtures in this series pass for the wrong reason before it was caught.
  • The shard-pipeline SKIP was a passing test conflating "target absent" with "partition selected zero constants".
  • A --ignored filter naming no runner executed nothing and returned 0, so a typo'd suite name reported success having run no tests.

Adversarial Ixon test suite

New Tests/Ix/IxVM/Exploits.lean, 19 cases, wired into the ixvm suite. The kernel-arena corpus attacks the layer above: its bad_* fixtures are raw Lean.ConstantInfo declarations that skip the elaborator, but they are still compiled by rsCompileEnvFFI, so the Ixon they produce is always well-formed. This suite authors Ixon.Constant values directly, and storeRawAt goes one level lower still, storing hand-authored byte strings — needed because every value-level fixture is canonical by construction and so cannot express a malformed encoding.

Each case records the false statement acceptance would buy, so a failure reads as a security statement. Rejection reasons are printed, which earned its keep repeatedly: fixtures that "passed" via invalid IO key without reaching the checker were caught this way, as was a fixture that silently corrupted nothing because non-mutual recursors are standalone constants rather than Muts members.

Testing

  • lake test -- --ignored ixvm green.
  • cargo test --release across all crates: 1195 passed, 0 failed. This also repaired shard_check_env_claim_is_thin_frontier, red since the claim walk moved to positive constant-use because its fixture's Exprs never referenced their own refs.
  • New prim-addrs parity suite; codegen ↔ bytecode parity gate green; arena negative corpus (77 bad_*) all reject, now only via genuine in-kernel refusals.
  • FFT-cost pins re-pinned per change. Cumulative effect is small (Nat.add_comm 42_747_927 → 42_986_025, +0.6% over the whole series); the largest single contributors are the nested-positivity substitution and the struct-eta guard.
  • ix codegen regenerated whenever the Aiur kernel changed, so the committed Rust kernel never lags the source the parity gate compares against.

Known limitations

Recorded rather than silently dropped.

  • Call-site surgery is not covered by the value pin. The compiler rewrites calls to aux-generated constants — permuting arguments, dropping some, replacing the head and its universe list — and the plans driving it live only in ConstantMeta, which prune_to_closure_anon excludes. So a verified claim says the term at address X is well-typed, not that term is the Lean declaration you wrote; the equivalence rests on the rewrite being faithful, checked today only by decompile round-trip tests. Closing it is a claim-format decision, not a patch.
  • Assumption cycles are not rejected, and a conditional Check claim naming its own target is accepted. That is the specified semantics — some root means "conditional on every leaf", so the claim asserts A → A, vacuous rather than false — and the obligation sits with whoever discharges assumptions. Sharded CheckEnv is not exposed, since shardClaimDigest recomputes the claim from (env, blocks). Pinned as ACCEPT so the boundary cannot move silently.
  • Alpha-collapse comparators ignore safety, so two members agreeing on kind/lvls/typ/value and disagreeing on safety collapse. Deliberately not fixed: a prover authors the environment directly and never goes through the compiler, so hardening it stops no attacker, while adding safety to the ordering would move every address and refusing the collapse would put a hard error in the compiler on the strength of an unverified assumption.
  • klimbs_le / klimbs_eq are in the logical trusted base, not just the performance path: Nat.decLe mints Decidable.isTrue with a proof term whose correctness is asserted rather than checked, so one wrong verdict yields False.
  • check_inductive_shape_ctors omits validate_expr_well_scoped / k_ensure_sort / assert_safety that check_const's Ctor arm performs; only an entirely unreferenced constructor escapes those.

@arthurpaulino
arthurpaulinoforce-pushed the ap/ixvm4 branch 4 times, most recently from 06e0489 to 559ced9CompareAugust 1, 2026 00:53
… hardening
Make sharded environment checking cheap. A large environment is partitioned
across shards, and each shard proves its own constants well-typed against a
thin-frontier assumption cutoff, so every constant is checked in exactly one
shard and cross-shard assumption trees stay minimal. Delivering that required
migrating the IxVM Aiur kernel from a positional constant table to a
content-addressed, lazily-faulted design -- and closing the soundness
regressions that migration introduced along with several pre-existing holes.
Sharded environment checking
- A shard proves `CheckEnv{root, assumptions}`: root is a merkle over the
shard's owned constants, assumptions a merkle over the THIN FRONTIER --
the direct out-of-owned walk edges. The kernel's env_walk starts from the
owned set, checks each reachable constant, and stops at a frontier member
(assumed well-typed by its owning shard, not re-checked, not recursed).
- Sub-frontier constants are never re-checked here; they are covered
transitively by the frontier members' own shard claims. This is the cost
win over checking each constant's full closure: no cross-shard duplication,
and the assumption tree is one layer deep instead of the whole closure.
- Strict membership: a walk-reachable constant in neither owned nor the
frontier is a bad partition and hard-errors, so a shard cannot silently
drop coverage of its own closure.
- Byte scope != check scope: delta unfolding reads through the frontier, so
a witness ships the full walk closure of owned even though only the
strict-membership set is checked. The witness IOBuffer is built in Rust
(aiur_ixvm_witness) rather than Lean, removing the per-byte boxing that
dominated shard wall time.
Addr-first kernel
- KExprNode.Const/Proj key on a 32-byte content Addr, not a numeric
position; KConstantInfo Induct/Ctor/Rec carry a (block_addr, ind_idx)
parent pair instead of indices. The positional `top: List<KCI>` table
threaded through the kernel is gone.
- get_ci(addr) is the sole resolver, Aiur-memoized per addr, faulting
constant bytes lazily from an IOBuffer. load_verified_constant re-derives
blake3 over the bytes and asserts key == addr; the io_get_info idx/len
are unconstrained witness and are never branched on.
- IOBuffer channels: 0 claim, 1 assumption tree, 2 constant bytes, 3 Defn
reducibility hint (advisory), 4 blob bytes. An address is on ch2 iff a
constant and ch4 iff a blob; the kernel derives which from Expr context
(Str/Nat -> blob, Ref/Prj -> constant), so no wire discriminator ships.
- Kernel/Inductive.lean and Kernel/Primitive.lean deleted (~6.6k lines);
inductive validation absorbed into Kernel/Check.lean, primitives split
into Kernel/Klimbs.lean (bignum gadgets) and Kernel/NatPrim.lean (Nat/Str
addr dispatch). New Kernel/InferOnly.lean breaks the DefEq<->Infer cycle.
Ingress.lean collapsed 1700->298 lines. V1/V2 naming removed throughout.
Recursor validation (R1, R2, R3)
- build_rec_type reconstructs a recursor's canonical type and asserts it
def-eq to the declared type, on both the kind-aware and address-only
lookup paths. Without it a recursor could declare `... -> False` while
its rules still reconstructed correctly.
- spec_params are lowered from the major-premise frame into the
recursor-param frame at extraction: the root cause of 17 nested/aux
recursor reconstruction failures, and why the canonical check accepted a
superset.
- k_flag and parent-inductive-shape checks hoisted out of the
attacker-selectable `is_aux` gate; the canonical rules comparison, which
formerly never ran, now runs for every recursor.
Pre-existing criticals (P1-P5)
- P1: blob-vs-constant is derived from Expr context instead of a prover-
answered io_get_info probe, retiring the ch-2 len=0 sentinel; a prover
can no longer mark a walk root or owned leaf "skip me".
- P2: Nat.div n 0 pins the quotient to zero (was prover-chosen).
- P3: klimbs_sub/le corrected on operands with trailing zero limbs; div/mod
hints normalized before use.
- P4: is_large_eliminator field-index arithmetic parenthesized (Aiur `-` is
right-associative).
- P5: Reveal recursor rules assert the claimed ruleIdx equals its position.
Robustness (L5, L6, L8)
- L5: UTF-8 string-literal decode validates continuation-byte ranges and
per-length overlong minimums, so distinct byte sequences (e.g. overlong
NUL C0 80 vs 00) no longer decode to the same codepoint and def-eq as
equal. Roughly zero added FFT cost.
- L6: run_claim dispatches on the full tag4 size (was the low byte only),
matching Rust Claim::get.
- L8: the K-recursor synth gate checks the major's inferred inductive
against the recursor's own (via se_parent_addr), not against itself, so
I.rec no longer strips a major of a different inductive.
Aiur assert_eq! messages + circuit cleanup
- assert_eq!(a, b, "why") gains an optional diagnostic label, threaded
Source.Term -> Typed -> Simple -> Concrete -> Bytecode.Op -> Rust
Op::AssertEq(_, _, Option<String>) -> ExecError::AssertEqMismatch. The
message is diagnostic-only and never enters the circuit; all ~122 IxVM
assert sites are labeled.
- Unreachable `_ => assert_eq!(0, 1)` match arms removed: an unmatched
runtime value already aborts (the kernel's reject), so the extra arm only
widened the circuit.
- address equality is a full 32-byte content comparison (never a pointer-
value compare, which is unsound for distinct-pointer/same-data), packed
as five base-256 field limbs (7+7+7+7+4) behind a byte-0 prefilter and
short-circuiting on the first mismatched group.
Crate reorganization
- PrimAddrs / reserved_marker_name moved verbatim from
crates/kernel/src/primitive.rs to crates/common/src/prim_addrs.rs (kernel
re-exports), so witness builders reach the primitive address table
without an ix-kernel -> ixon -> ix-kernel cycle.
- witness_scope reuses Env::bfs_closure -- the env's own dependency-closure
pass, which already follows refs, Prj->block, and the derived
Muts->member/ctor projection edges -- so the wrappers the kernel's get_ci
synthesizes ship as forward edges of a single O(closure) walk, with no
reverse scan of the whole env. Primitive-address seeding is unioned on
top: primitives are fabricated during reduction and need not appear in a
target's ref closure, so a closure-only scope aborts with `invalid IO
key`.
Tests
- New prim-addrs parity suite (Lean<->Aiur primitive address table).
- Codegen<->bytecode parity gate restored, green (198/198).
- FFT-cost pins re-pinned to current behavior; arena negative-fixture
corpus (77 bad_* constants) all reject.
…onstants
closureFromBase now walks the derived Muts->member/ctor projection
wrapper addresses as forward edges (projWrapperAddr: blake3 over the
serialized proj-only Constant, compile's construction), so closureFrom
and buildShardCheckEnvWitness drop their O(env) reverse scans that
parsed every env constant per witness build. Primitive addresses are
seeded bytes-only after the walk instead of being pushed onto the
worklist, so their transitive closures no longer inflate the byte scope
(a delta-unfold into an unshipped primitive body aborts with invalid IO
key rather than proving anything wrong).
bench-typecheck's constants metric now reads check_const's unique query
count (memoized per (ci, addr) = exactly the constants typechecked)
instead of closureFrom.size, which counted the shipped byte scope.
check_const is resolved up front and both lookups fail with IO.userError
instead of panicking.
Experiment: lake exe bench-typecheck --ixe seed.ixe --consts
Nat.add_comm --execute-only (seed.ixe = ix compile Tests.lean --consts
Nat.add_comm) reports constants=42, matching check_const's height in
lake exe ix check Nat.add_comm; fft-cost unchanged at 42747927.
The claim walk decided which of a constant's `refs` to recurse into by
SKIPPING every index the constant's Exprs used as a blob payload
(`Str(i)` / `Nat(i)`). That rule is unsound: the Exprs are authored by
the prover, and `refs` is one index space shared by `Ref`, `Prj`, `Str`
and `Nat`, so marking an index suppresses the walk of that index used as
a constant everywhere else in the same constant.
The marking need not even be live. `blob_idxs_of` traversed `sharing` in
full, but conversion reaches a `sharing` entry only through an
`Expr.Share` node, and nothing validates the array. A dead `Nat(i)`
entry is therefore free: it is never converted, never typechecked, and
needs no blob on channel 4. It just hides `refs[i]` from the checker.
`k_infer` then reads that dependency's DECLARED type without ever
checking it, so the kernel accepts a constant whose transitive closure
is ill-typed -- a wrong answer, not an abort. Both claim variants were
affected; for CheckEnv the skip happens before `env_walk` is entered, so
the strict owned-membership assert never fires either.
Replace the family with `const_idxs_*`: collect the indices `Ref(i, ..)`
and `Prj(i, ..)` use, and walk `refs[i]` iff `i` is among them. `Rec`
resolves through `recur_addrs` and a projection's block comes from the
ConstantInfo payload, both walked separately, so the positive set is
complete. Same single traversal as before, so no added cost. The failure
direction is now safe: over-approximating walks an address that faults
an unseeded key and aborts, which only costs a dishonest prover.
Adversarial Ixon test infrastructure
- New `Tests/Ix/IxVM/Exploits.lean`, wired into the ixvm suite. The
kernel-arena corpus attacks the layer above this one: its `bad_*`
fixtures are raw `Lean.ConstantInfo` declarations that skip the
elaborator, but they are still compiled by `rsCompileEnvFFI`, so the
Ixon they produce is always well-formed. This suite authors
`Ixon.Constant` values directly, which is what a prover can do.
- Each `ExploitCase` records the false statement acceptance would buy,
so a failure reads as a security statement. Rejection reasons are
printed: a fixture that aborts before reaching the mechanism under
attack is indistinguishable from a kernel that correctly refuses it,
and the first run of this fixture passed for exactly that wrong reason
(`invalid IO key` from an unseeded channel-3 hint, now seeded by
`storeAt`).
- The exploit ships with two controls that differ from it minimally: one
checking the smuggled dependency alone, one checking the same target
without the dead `sharing` entry. Both reject via the intended
in-kernel assert, so the exploit's acceptance was attributable to the
`sharing` entry alone.
- Two further cases probe what the fix does NOT cover. An index used as
both a `Nat` literal and a `Ref` — reachable because blob and constant
addresses share one un-domain-separated blake3 space and
`load_verified_blob` parses nothing — is now walked and rejected,
though by channel fault rather than type error, which is the safe
direction. A conditional Check claim naming its own target as its own
assumption IS accepted, and is pinned that way: `some root` means
"conditional on every leaf being well-typed", so the claim asserts
`A -> A` and is vacuous rather than false. The obligation sits with
whoever discharges assumptions, which must reject such cycles; sharded
CheckEnv is not exposed, since `shardClaimDigest` recomputes the claim
from (env, blocks).
Primitive closures are walked again
- The previous commit seeded primitive addresses as bytes only, without
walking their closures. That is too aggressive: def-eq delta-unfolds
primitive BODIES, and a body's refs are reachable from nowhere else in
the scope, so the `strAppend*` arena fixtures aborted with `invalid IO
key`. Both seeders now pass the primitives as walk ROOTS
(`closureFromRoots` in Lean, `bfs_closure` over target+primitives in
Rust); the Rust side had the same gap latently, since only the Lean
path had arena coverage.
- The removal of the O(env) reverse wrapper scan, which was that
commit's actual win, is unaffected.
Tests
- `lake test -- --ignored ixvm` green. FFT pins re-pinned across the 66
kernel-check entries and the shard pipeline; costs move in both
directions (e.g. `Eq.rec` 1_249_383 -> 1_249_094, `Vector.append`
1_994_697_378 -> 2_010_055_088), the increases coming from the
restored primitive closures.
- `ix codegen` regenerated: the Aiur kernel changed, so the committed
Rust kernel would otherwise be stale and the codegen/bytecode parity
gate would compare against it.
`Expr.Str` conversion wrapped any hash-valid blob in `KLiteral.Str`, and
`k_infer_lit` types a `KLiteral.Str` as `String` without inspecting the
bytes. Nothing on the ordinary checking path ever decodes a literal, so a
prover-authored definition carrying malformed bytes typechecked as
`x : String` even though no Lean `String` corresponds to those bytes. The
reference kernel rejects this at ingress via `String::from_utf8`
(crates/kernel/src/ingress.rs).
The UTF-8 validation added earlier lives in the string DECODER, which is
reached only during string reductions, so it did not cover this path.
`utf8_decode_one` already rejects out-of-range continuation bytes, stray
continuations used as leaders, and overlong forms. Walk the whole blob
with it at conversion (`utf8_validate`), which is the same rejection the
reference kernel performs.
Tests
- `invalid-utf8-accepted-as-string` in the adversarial Ixon suite: a
definition declaring `String` over the blob `[0xFF]`, which no
well-formed UTF-8 can contain. Accepted before this change, now
rejected.
- Paired with `control-valid-utf8-string-literal`, the same shape over
`[0x68, 0x69]`, which must still be accepted — so the verdict is
attributable to the bytes rather than to the fixture's shape. The
fixtures build on a base env supplying the real `String` constant, so
the declared type is the genuine one.
- Six FFT pins move, all string-touching, by between +17 and +66_530
(e.g. `IxVMPrim.str_size_lit` 567_999_042 -> 568_000_142,
`Vector.append` 2_010_055_088 -> 2_010_088_729, +0.0017%). Constants
with no string literals are unchanged, `Nat.add_comm` included.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`add_entries_parallel` returned early after a constant hit, so an address
present in both `env.consts` and `env.blobs` was seeded on channel 2 but
never on channel 4. The Lean seeder walks `consts` and `blobs` in
separate passes and seeds both, so the two builders disagreed, and the
Rust-native check/prove paths aborted with `invalid IO key` on such an
environment.
The two tables are not disjoint. Constant and blob addresses share a hash
codomain, and it is the Expr context — `Ref`/`Prj` versus `Str`/`Nat` —
that says which table a reference means, so the same bytes can legitimately
be read both ways. The kernel faults the two channels separately, so the
witness has to seed them separately. Reaching an address through both
kinds of use became expressible when the claim walk moved to selecting
indices by positive constant-use.
Corrects the module header and the `ChannelEntries` doc, which both
asserted the disjointness this relies on not having.
Tests
- `dual_use_address_is_seeded_on_both_channels`: an address stored as a
constant and as a blob must appear under both `(ch 2, key)` and
`(ch 4, key)` in the built IOBuffer. Fails on the previous code with
the ch-4 entry missing.
- `lake test -- --ignored ixvm` green; clippy `--all-targets` and fmt
clean.
The kernel walks a constant's `refs` by positive constant-use — index `i`
is followed only when some Expr names it via `Ref(i, ..)` or `Prj(i, ..)`
— but both host claim builders still treated every `refs` entry as a walk
edge. The host and the kernel therefore disagreed about the edge
relation.
A `refs` entry that is a real constant but that no Expr uses as one is
never walked in-circuit, so admitting it host-side puts a phantom member
in the thin frontier: a cross-shard assumption the kernel never makes,
and a spurious dependency edge that can close a cycle once assumptions
are discharged. `walk_check_set` likewise stopped computing the set the
verifier actually checks.
Mirror `const_idxs_of` on both hosts and emit an edge only for
positively-used indices. `Env::bfs_closure` is deliberately left alone:
it computes the witness BYTE scope, where over-approximating is correct
(shipping unused bytes is free, missing bytes aborts).
Honest constants are unaffected, since a compiler only puts an address in
`refs` when an Expr references it — the shard FFT pin is unchanged. The
divergence is reachable only through prover-authored `refs`.
Tests
- `thin_frontier_excludes_unused_ref_entries`: `refs` carries two
constants but only index 0 is named by a `Ref`; the frontier must
contain the used one and not the dead one. Verified red against the
previous edge relation ("unused refs entry became a phantom frontier
member").
- `const_with_refs` now names each of its `refs` entries via a `Ref`
node. The old helper left them unreferenced, which under the walk rule
would have produced no edges at all and quietly hollowed out the
existing frontier tests.
- `lake test -- --ignored ixvm` green; clippy `--all-targets` and fmt
clean.
`idx_to_u64` packed a synthesized member/constructor index into the
projection wrapper's `u64` field by populating only the low byte behind a
`u8_range_check`, so a mutual block with more than 256 members — or an
inductive with more than 256 constructors, which is the likelier case —
was rejected outright with `value 256 out of u8 range`.
Field-to-bytes is not free: the bytes have to be supplied as advice and
then pinned. Take them from `unconstrained_g_to_bytes` and discharge the
three obligations its contract names:
- range: four `u8_range_check` calls cover the eight bytes, the byte chip
taking a pair per lookup row;
- recomposition and canonicality: `flatten_u64(bytes) == idx`, since
`flatten_u64` asserts the top byte is zero and sums the low seven.
Canonicality is what makes the decomposition unique, and it is enforced
by RESTRICTING THE DOMAIN rather than by comparing against `p`. Goldilocks
is 2^64 - 2^32 + 1, so an arbitrary eight-byte string is not injective
into the field: any value and that value plus `p` recompose to the same
element, e.g. `[00]*8` and `[01,00,00,00,FF,FF,FF,FF]` both give 0.
Forcing the top byte to zero bounds the value by 2^56 - 1 < `p`, so the
sum cannot wrap and exactly one string encodes each index. That
uniqueness is the soundness property: the projection address is blake3
over these serialized bytes, so a prover free to choose a second string
for one index would obtain a second address for a member, or collide two
members onto one address. The pre-existing `u8_from_field_unsafe`
version did collide members 0 and 256, which the range check had already
fixed by rejecting instead.
Both halves are load-bearing. `flatten_u64` does NOT range-check its
input — `[U8; 8]` is nominal typing and `u8_from_field_unsafe` mints a
`U8` holding any field element — so without the range checks the sum is
satisfiable many ways (`b1 = k`, `b0 = idx - 256k`). Its doc comment now
records that its injectivity is conditional on the caller, and names this
call site.
Indices at or beyond 2^56 still fail the assert rather than truncating.
That is a liveness bound only, 2^48 times the previous cap and far past
any real block or constructor count; buying the remaining `[2^56, p)`
would cost a multi-byte comparison against `p` for no practical gain.
Tests
- `muts-block-past-single-byte-index`: a 257-member block of well-typed
definitions with a projection wrapper for each, which now checks.
Rejected before this change with `value 256 out of u8 range`.
- `control-small-muts-block`: the same construction at two members,
already accepted — so the big-block result is attributable to the index
width alone and not to a malformed fixture.
- `storeAt` also registers a reducibility hint for `DPrj` wrappers, which
`get_ci_dprj` faults under the wrapper's own address; without it these
fixtures aborted with `invalid IO key` before reaching the index logic.
- All 66 kernel-check FFT pins and the shard pin move by +17 to +485
(`Nat.add_comm` 42_967_279 -> 42_967_313, +0.00008%), the cost of the
hint plus four range checks per synthesized index.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`get_ci` handed every standalone `Defn` a one-element `recur_addrs` whose
only entry was its own address. `Rec` converts to a plain `Const`, and
`k_infer` discharges a `Const` against its DECLARED type without checking
it, so a definition could prove its own type by citing itself:
theorem bad : False := bad
typechecked. The ref-walk cannot catch it — `const_idxs_expr` does not
follow `Rec`, and the referent is the constant already under check. Lean
forbids exactly this by checking a definition against an environment that
does not yet contain it, admitting self-reference only for `unsafe` and
`partial`.
Restrict the self slot to `unsafe`/`partial`. Those are where
self-reference is both legitimate and contained:
- legitimate, because the compiler emits a singleton (or fully
collapsed) mutual block as a standalone `Defn` whose body still says
`Rec(0)` -- see the note at Ix/CompileM.lean:2000. Those blocks are
exactly Lean's `unsafe`/`partial` recursive definitions; a `safe`
recursive definition compiles its recursion into `.rec` applications.
Gating on `unsafe` alone breaks `Lean.extractMainModule._unsafe_rec`.
- contained, because safe code may not reference either kind.
That containment was itself missing for `partial`: `is_unsafe_ci`
classified it as safe, so `partial def bad : False := bad` plus a safe
theorem referencing `bad` would have walked straight around the first
half of this fix. Count `Partial` alongside `Unsafe`, matching the
reference's "safe definition references partial definition" rejection
(crates/kernel/src/check.rs:813-820). The predicate serves double duty --
the caller's own leniency and the referent's barrier -- and `Partial`
belongs on the non-safe side of both.
Tests
- `self-referential-theorem-proves-false`: `theorem bad : False := bad`
built directly in Ixon over the real `False` constant, submitted
through the production `verify_claim` entrypoint. Accepted before this
change -- a proof of False -- and rejected after.
- No safe constant in the corpus references a partial definition, and
the `_unsafe_rec` witnesses still check, so neither half of the fix
costs coverage.
- All 66 kernel-check FFT pins and the shard pin move (`Nat.add_comm`
42_967_313 -> 42_965_666); the self slot is no longer built for safe
definitions, so most costs drop slightly.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
The nested branch of the strict-positivity check dropped the parameter
arguments (`list_drop(args, n_ctor_params)`) and descended into the
nested inductive's constructors without substituting them. A nested
constructor's fields mention that inductive's parameters as bound
variables, so the block-occurrence test asked whether the block occurs in
`BVar 0` — it never does, and every parameter position passed vacuously.
The occurrence being hunted lives in the ARGUMENT substituted for the
parameter, which was exactly what got discarded.
So for `Inner α | mk : (α → False) → Inner α`, itself strictly positive,
`Host | mk : Inner Host → Host` was accepted: descending into `Inner.mk`
sees the field `α → False`, which mentions nothing tracked. With `α :=
Host` it reads `(Host → False) → …`, a negative occurrence, and Lean
rejects it. Accepting it admits `False`.
Mirror `check_nested_ctor_fields` (crates/kernel/src/inductive.rs): strip
`n_params` foralls, then simultaneously substitute the parameter
arguments at depth 0. After stripping, `BVar 0` is the innermost (last)
parameter, so the argument list is reversed — `expr_inst_many` maps
`substs[i]` to `BVar(depth + i)`, the same convention as `simul_subst`.
The parameters are substituted away rather than bound, so the field walk
now starts with an empty binder context instead of the peeled types.
Tests
- `nested-inductive-negative-occurrence`: the `Inner`/`Host` pair above,
built directly in Ixon as two single-inductive `Prop` blocks with their
projection wrappers. Accepted before this change, and now rejected by
the intended assert ("strict positivity: block occurs left of an
arrow").
- `control-nested-positive-inductive`: the same construction with
`Inner α | mk : α → Inner α`, which is strictly positive and must keep
checking. The two fixtures differ only in the inner constructor's
field, so the rejection is attributable to the negative occurrence
rather than to nesting.
- 63 FFT pins and the shard pin move, concentrated in inductive-heavy
constants (`IxVMInd.Tree` 767_167 -> 796_717, `IxVMInd.DepthM`
1_312_461 -> 1_347_394) — the nested check now does real work instead
of returning vacuously.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`KLimbs` is canonical in two independent ways — no trailing zero limbs,
and every limb byte in [0, 256) — and `klimbs_eq` is a raw limb compare
that establishes neither. Since it backs `Nat.beq`, `Nat.decEq` and
`literal_eq`, any producer emitting a value-correct but
representation-wrong `KLimbs` makes those answer `false` where Lean
answers `true`. That is an accepted false equation, not an abort. Two
producers did.
`String.utf8ByteSize` built the length as one unchecked byte
(`[u8_from_field_unsafe(n), 0, ...]`) with no normalization. The empty
string therefore reduced to the denormalized zero `[[0;8]]` instead of
the canonical empty limb list, and a string of 256 bytes or more to an
out-of-range digit — `u8_from_field_unsafe` is an erased coercion, so
nothing truncated or rejected. Build it with `klimbs_from_g`, whose byte
split is range-checked and pinned by a recomposition assert, then
normalize.
`klimbs_div_mod` pinned the quotient's VALUE (`q*b + r == a`, `r < b`)
but not its digits. The comment claimed `u8_mul` inside `klimbs_mul`
range-checked them, but `u64_mul` was rewritten to raw field products
plus `#split_carry`, whose u8 checks constrain the split outputs rather
than the input digits — and it re-canonicalizes while multiplying, so a
digit-wrong `q` still produces a canonical `q*b` and satisfies the
equality. Range-check the hint's limbs explicitly, quotient and
remainder both, via a new `klimbs_range_check`.
Tests
- `utf8-size-beq-false-accepted`: `Nat.beq (String.utf8ByteSize "") 0 =
false` proved by `Eq.refl`, over the real `Eq`/`Bool`/`Nat.beq`/
`String.utf8ByteSize` constants. The kernel had the polarity exactly
backwards — this FALSE equation was accepted while the true one was
rejected.
- `control-utf8-size-beq-true`: the same declaration ending in `true`,
which is what Lean computes and must be accepted. The pair differs only
in that operand, so each verdict is attributable to the reduction.
Both now land the right way round.
- The two fixtures share one blob for `""` and `0` — the same empty byte
string read as a string literal and as a nat literal, which is the
Expr-context disambiguation working as intended.
- 10 FFT pins and the shard pin move, confined to the div/mod/gcd/pow/
shift constants (the hint range-checks) and the string-size ones
(`IxVMPrim.str_size_lit` 567_992_937 -> 567_993_099).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT WAS DEMONSTRATED for this one. Unlike the other soundness
fixes in this series it ships without a red-then-green fixture; what
follows is why the rule is wrong as written, and what remains unproven.
`try_eta_struct` accepted `t ≡ S.mk p… a…` after checking only that the
right side is a fully-applied constructor of a struct-like inductive. It
never checked that the two sides have def-eq TYPES. All three references
do, citing lean4lean's `tryEtaStructCore`
(crates/kernel/src/def_eq.rs, Ix/Tc/DefEq.lean).
Without the guard the rule degenerates: for ANY `t` of ANY type,
`compare_struct_fields` builds precisely `Proj(S, i, t)` for each field,
so when the right side's arguments are those same projections the
pointer fast path matches them and the pair is accepted. Those
projections are synthesized inside def-eq and never inferred, so
`k_infer_proj`'s head-address gate — which would reject `Proj S i t` for
`t : Nat` — never sees them.
What is unproven is reachability. At the `k_check(e, expected)` boundary
both sides are types in the same context, and `de_args` compares type
arguments before value arguments, so I could not construct a term that
reaches this rule with genuinely mismatched types. The change is
therefore defense-in-depth against an unjustified rule, not a repair of a
demonstrated break.
The guard runs after the struct-like shape test, so it costs two
`k_infer_only` calls and one `k_is_def_eq` only on pairs that were about
to be accepted. `try_proof_irrel`, earlier in the same fallback chain,
already infers the left side unconditionally, so the added abort exposure
is small; the suite confirms no constant stops checking.
Tests
- No new fixture; the existing 14 adversarial cases and the arena corpus
are unchanged.
- 45 FFT pins and the shard pin move, the largest cost of this series
(`Nat.add_comm` 42_965_759 -> 42_966_970, +0.003%;
`IxVMPrim.str_size_lit` +26_687).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
A `Tag0` large-form header carries a 7-bit `small` field, so a payload
width up to 128 was expressible and unguarded (`Tag2`: 5 bits, up to 32).
A u64 payload is at most 8 bytes, and the three readers disagreed on the
surplus:
- Rust `u64_get_trimmed_le` rejects `len > 8` -- already correct, and
untouched here.
- The in-circuit `get_u64_le` consumed the surplus and DISCARDED it,
keeping only the first eight bytes.
- The Lean host folded it back in: `UInt64.shiftLeft` is taken mod 64,
so byte 8 ORs into bits 0-7, with no length guard.
Same bytes, same content address, three readings. An `Expr.Ref` whose
index is written overlong resolves as `refs[0]` in-circuit and `refs[42]`
on the Lean host, with both consuming the same span so nothing desyncs
afterwards. That also means the trailing-bytes assert does not imply
canonicity: up to 120 arbitrary bytes per `Tag0` site were consumed
inside the structure and dropped.
Bring the other two up to Rust: assert the width is under 9 in both
in-circuit readers, and give `getU64TrimmedLE` the guard it was missing.
No honest serializer is affected -- `putTag0` always emits the minimal
width -- so this narrows the accepted language to the format.
Tests
- `overlong-tag0-accepted`: an axiom `_ : Prop` whose `lvls` Tag0 is
rewritten to the nine-byte form. Accepted before this change, now
rejected by the intended assert.
- `control-canonical-tag0-axiom`: the same axiom with canonical bytes,
which must stay accepted. The pair differs only in that encoding.
- Both need bytes no `Ixon.ser` will emit, so the suite gained
`storeRawAt`, which stores a hand-authored byte string at its own
content address. Every previous fixture was built from a `Constant`
VALUE and was therefore canonical by construction, and structurally
could not reach this class of defect.
- All 66 kernel-check FFT pins and the shard pin move (`Nat.add_comm`
42_966_970 -> 42_980_767, +0.03%); every constant parses Tag0s.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
Four test-only fixes. No kernel code changes, so no FFT pins move.
prim-addrs asserted the literal `true`. It looked up each hardcoded
kernel address in the canonical table and, on a hit, recorded a pass —
but the table is keyed BY ADDRESS, so a permuted table still hits. Had
`nat_add_addr` held `Nat.mul`'s bytes it would have printed
"nat_add_addr = Nat.mul" and passed green, which is precisely the defect
the file's own docstring says it exists to catch: the kernel applying one
primitive's native semantics under another's name. Now the address must
resolve to the constant the function's NAME claims, comparing normalized
keys (`nat_add_addr` <-> `Nat.add`, and the final-component spelling for
`reduce_bool_addr` <-> `Lean.reduceBool`), with dispatch-site suffixes
(`_dec`/`_io`/`_iota`) and `_type`/`_ty` stripped, plus a short exception
list for the `Quot` constructors. Verified by temporarily swapping
`nat_add_addr` to `Nat.mul`'s bytes: red, then reverted. All 76 literals
pass, and the table is confirmed correct today.
The arena negative corpus counted ANY abort as a correct rejection while
its docstring claimed rejections come from an in-kernel `assert_eq!`. An
`invalid IO key` — the witness never supplied the bytes, so execution
died before reaching the logic under test — scored as a pass. That is how
a fixture rots into a green test that checks nothing, and it is the same
failure mode that made two exploit fixtures in this series pass for the
wrong reason. Now `invalid IO key` disqualifies. Unmatched-match aborts
still count: this kernel deliberately relies on an unmatched runtime
value aborting instead of spending width on `_ => assert_eq!(0, 1)`
arms, so those are genuine verdicts. No fixture currently relies on the
disqualified path.
`shardCheckEnvCase` returned `none` for two unrelated reasons and the
runner reported both as "SKIP (target absent)", a passing test. The
second — target present but the partition selected zero owned constants —
means the fixture stopped exercising the Rust witness builder and the
thin-frontier claim convention, and would have left the only test of that
path permanently green. It now throws; `none` keeps its one honest
meaning.
A `--ignored` filter naming no runner ran nothing and returned 0, so
`lake test -- --ignored ixvm2` reported success having executed nothing.
It is now an error. Exit 0 should mean tests ran and passed.
`ixVM`, the pruned production toplevel, carried `verify_const`,
`verify_check` and `verify_check_env` alongside `verify_claim`. Those are
DEBUG entrypoints that trust their dependencies instead of walking them,
so a proof made with one attests far less than a claim does — and
carrying them meant such a proof was verifiable under the SAME verifying
key. The only thing standing between it and acceptance was `ix verify`
pinning the funcIdx to `verify_claim`: a policy, enforced in one place,
with no test asserting it, and no policy at all in the recursive
verifier, which binds the claim bytes but applies no funcIdx check.
Prune to `verify_claim` alone. The guarantee is now structural: under the
production VK a debug-entrypoint proof cannot exist. No CLI path used
them, so nothing in `ix check`/`prove`/`verify` changes.
Callers that genuinely want a subject-only check build `ixVMFull` and are
thereby using a different VK, which is the honest signal — the arena
corpus, and `bench-typecheck`/`bench-recursion-debug` under
`--skip-deps`. That last one also means a `--skip-deps` benchmark is now
visibly measuring a different key than a claim run rather than silently
reporting `verify_const` rows in the claim schema.
`bench-typecheck` also reported success on failure, twice over. A proof
that failed to verify printed to stderr but left `failed` clear, so the
row was emitted as `"status": "ok"` with prove-time and proof-size
present and the run exited 0 — the only signal an absent `verify-time`
key, invisible to anything keying on status. And a constant whose execute
threw was dropped from the results entirely without setting `failed`,
so `execed.any (·.failed)` stayed clear; the `check_const`-missing throw
added earlier landed exactly there. Both now mark the row failed.
Tests
- `lake test -- --ignored ixvm` green with NO pin movement: the pruned
entrypoints were never executed, so removing them changes no cost.
Generated kernel drops from 733 to 731 functions.
- Arena repointed to `ixVMFull`, where `verify_const` still lives.
- `bench-typecheck --execute-only` smoke unchanged (`Nat.add_comm`,
constants=42).
`compare_rules` peeled the binders off the STORED rule and handed their
DOMAINS to `k_is_def_eq` as the local context for the comparison against
the canonical reconstruction. Those domains are prover-authored: nothing
checks them against the reconstruction, and reduction never consults them
(`try_iota` beta-reduces straight through). So the prover chose what the
two sides INFER to.
Retyping every binder to a Prop makes both sides infer to that Prop, so
`try_proof_irrel` fires and accepts ANY body. A recursor whose
computation rules are swapped between constructors was therefore
accepted, and iota then reduces each constructor to the other's branch.
Compare under an empty context instead, discarding the peeled domains. An
honest rule matches the reconstruction structurally, and the structural
path never consults the context — it compares `BVar i` against `BVar i`
directly. A comparison that instead needs to INFER a bound variable's
type now runs out of context and aborts, which is the safe direction.
Every recursor in the corpus still checks, including the mutual, nested
and aux-generated ones.
The reference avoids this differently, comparing whole Lam-wrapped terms
at empty context so `is_def_eq` descends binder-by-binder and compares
the domains themselves (crates/kernel/src/inductive.rs). Reconstructing
canonical domains here would mean re-deriving field domains from each
constructor's telescope and re-lifting their de Bruijn indices into the
rule's context; dropping the untrusted context achieves the same
guarantee without that machinery.
Tests
- `recursor-rule-rhs-doctored-binder-domains`: a real compiled recursor
with its first two computation rules swapped AND every peeled binder
domain replaced by a Prop. Accepted before this change; rejected now.
- `control-swapped-recursor-rules`: the identical swap with the
compiler's own binder domains, which was already rejected. The two
differ only in the domains, so the acceptance was attributable to them
alone.
- Both are built by rewriting a REAL block rather than hand-authoring a
recursor, since a hand-built one must survive `build_rec_type`'s def-eq
and constructor references inside a block are circular with the block's
own address. The fixture selects a recursor whose rules are field-free,
so no minor premise is applied and the retyping cannot abort as a
non-function before the comparison is reached — an earlier attempt did
exactly that and masked the acceptance.
- A guard case fails loudly if no such recursor is found; the first
version of this fixture silently corrupted nothing, and both its cases
"passed" meaninglessly.
- 57 FFT pins and the shard pin move (`Nat.add_comm` 42_980_767 ->
42_980_801).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT DEMONSTRATED. Defense-in-depth, like the struct-eta guard.
`compare_rules` now asserts three invariants that equivalents of already
exist in `check_rec_rules_walk` — which nothing calls. The live path had
none of them.
* `cidx == pos`. `find_rule` selects a rule at reduction time BY LABEL
while `compare_rules` pairs stored against canonical POSITIONALLY, so
permuted labels would validate one constructor's rule and then fire
it for another. It holds today only because ingress assigns `cidx`
positionally (`Convert.lean`) and the Ixon `RecursorRule` record
carries no cidx of its own — an accident of the pipeline rather than
a checked property, which is exactly what an assert is for.
* `cidx < num_ctors`, so a rule cannot name a constructor that does not
exist.
* `nf` equals the constructor's field count. `try_iota` splits the
constructor's arguments with `field_start = ctor_fields_len -
rfields`, so a wrong `nf` hands the minor premise the wrong slice; in
field arithmetic an over-large value wraps.
The reachability argument for the `nf` gap was "combined with the rule
RHS context hole"; that hole is fixed, and with the def-eq comparison
restored a mismatched peel is caught again. So this closes a route that
is currently blocked by another check rather than one that is open —
worth asserting locally rather than depending on a second mechanism.
Not fixed here: iota still selects rules by index rather than by
constructor IDENTITY, so a recursor applied to a foreign constructor
would reduce. That is not reachable through a claim —
`k_infer_app_spine_loop` runs `k_check` on every argument, so the
application fails inference before whnf sees it — and closing it properly
needs the parent inductive available at reduction time, which
`KConstantInfo.Rec` does not carry (its `Addr` is the recursor's own
block; the parent is derived at check time from the type). That is a KCI
layout change, left for its own pass.
Tests
- No new fixture; nothing demonstrates the hole.
- Every recursor in the corpus satisfies all three invariants, including
the mutual, nested and aux-generated ones, so they held by construction
and are now enforced.
- 57 FFT pins and the shard pin move (`Nat.add_comm` 42_980_801 ->
42_981_336) for the per-rule constructor lookup.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT DEMONSTRATED. Defense-in-depth.
`try_iota` selected a computation rule by constructor INDEX alone:
`collect_spine_of_ctor` returned the major's `cidx` and discarded which
inductive it came from, so `find_rule(rules, cidx)` fired whenever any
inductive's constructor happened to carry a matching index. On its face
`Bool.rec motive f t (Nat.succ n)` reduces to `t`. Lean's kernel selects
the rule by the constructor's NAME, which pins the inductive.
Not reachable through a claim: `k_infer_app_spine_loop` runs `k_check` on
every argument, so a recursor applied to a foreign constructor fails
inference long before whnf sees the redex — the ill-typed term cannot be
stated. The Rust reference selects by index too, consistent with that
being why it was tolerated. What the patch removes is the DEPENDENCE on
that: reduction no longer needs inference to have run first to be
correct.
`collect_spine_of_ctor` now also returns the constructor's parent, as the
address of its projection wrapper, and `try_iota` asserts it equals the
recursor's own parent. That parent is read off the recursor's type (the
major premise's head) with the existing `rec_to_parent_addr`, so no
KConstantInfo layout change was needed — its `Addr` field is the
recursor's own block, not the parent's. Aiur memoizes per argument tuple,
so the type walk is paid once per recursor rather than once per
reduction.
Tests
- No new fixture; nothing demonstrates the hole.
- Every recursor in the corpus still reduces, including the mutual,
nested and aux-generated ones, and the Nat/String literal paths that
synthesize a constructor application before iota (`nat_cases_big`,
`nat_pow_big`, `strOfListFold*` all unchanged in behaviour).
- 63 FFT pins and the shard pin move (`Nat.add_comm` 42_981_336 ->
42_986_025, +0.01%) for the parent comparison.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`shardsCover` gates every shard run: "exit 0 has to mean every env const
was checked by some shard". It skipped constants whose bytes fail to
parse (`LazyConstant.get?` discards the error and `.ixe` loading is
lazy, so such a constant sits in `consts` with its key present). A
skipped constant was assigned to no shard, counted as neither owned nor
unowned, and still included in the "covers all N consts" total — so the
gate reported success over constants it had not covered.
It would also enter a referring shard's frontier, since `thin_frontier`
admits an edge on key presence in `env.consts` without parsing, leaving
an assumption no shard discharges.
Count them separately and fail, rather than folding them into `unowned`:
"cannot be assigned to a shard" is a different problem from a partition
gap, and saying so points at the right cause.
`ownedConstsForBlocks` keeps its skip — it runs per shard and would
otherwise need the same error plumbing — but now documents that it is
safe only because this gate rejects such an env first.
Not changed: the composed verdict still has no assumption-cycle check,
and on analysis it does not need one. Assumptions are discharged per
CONSTANT along ref edges, which are acyclic by construction — a
constant's address is blake3 over bytes containing its refs' addresses,
so a reference cycle needs a hash cycle. A cycle in the shard-level graph
still stands for a well-founded per-constant chain, and
`verifyShardComposition` recomputes each digest via `shardClaimDigest`,
so a self-assuming shard claim cannot be substituted.
`compile_univ` memoized `Level -> Univ`, but a `Level::Param` compiles to
`Univ::Var(i)` where `i` is the parameter's POSITION in `univ_params` —
so the result depends on a context the key omitted. Unlike the expression
cache, which is cleared at every constant boundary, `univ_cache` lives
for a whole block, and `compile_definition` compiles each member under
its own `level_params`. `preseed_expr_tables` explicitly feeds
`(expr, univ_params)` pairs with differing param lists into the same
cache.
So two members of one block placing the same parameter NAME at different
positions would give the second member the first's index. Not an abort:
it compiles cleanly and registers a constant denoting a different
universe than the Lean source, under a correct-looking name.
The hazard is already recognized one level up — `collect_expr_tables`
keys `seen_exprs` by `(expr_hash, univ_params_key)` for exactly this
reason. Apply the same key here. The Lean mirror instead drops the cache
in `withUnivCtx`, matching how it already clears the expression cache per
constant.
Unreachable from ordinary Lean sources, since the elaborator gives every
member of a `mutual` block the same `levelParams` — but nothing checked
it, and under the prover threat model the environment is authored.
Also repairs `claim.rs`'s `defn_const`, whose Definition never referenced
its own `refs`. Since the claim walk admits an index only when some Expr
uses it as a constant, that fixture produced no walk edges and
`shard_check_env_claim_is_thin_frontier` has been failing since the walk
moved to positive constant-use. The identical flaw was fixed in
`shard_claim.rs`'s helper at the time and missed here, because only
`ixon` and `ixvm-codegen` were re-tested rather than the whole workspace.
Tests
- `cargo test --release` across all crates: 1195 passed, 0 failed.
- `lake test -- --ignored ixvm` green with NO pin movement and no address
changes, confirming no block in the corpus has members that order a
shared parameter name differently. The fix removes the dependence on
that remaining true.
- clippy `--all-targets` and fmt clean.
@johnchandlerburnham
johnchandlerburnham merged commit 80bb41f into mainAug 3, 2026
10 checks passed
@johnchandlerburnham
johnchandlerburnham deleted the ap/ixvm4 branch August 3, 2026 05:37
samuelburnham added a commit that referenced this pull request Aug 4, 2026
…acking
Port of sb/measured-ingress's planning layer onto the addr-first kernel
(#529), reworked for its lazy fault-in semantics.
Profiling (.ixprof v3): the recording kernel's try_get_const consults are
drained per constant into a touch graph — the measured set a lazy checker
faults in — persisted as a presence-flagged CSR section alongside a new
block-level reference graph and a subst_unique counter (splitmix64-keyed
dedup of substitution work items: the post-memoization volume a memoizing
executor pays). Both graph sections are optional, so older readers'
formats stay decodable by version gate alone.
Planner: partition_for_aiur_ram bin-packs to a host-RAM cap with
faulted-set byte accounting — under #529 the witness always ships the
full closure, but only faulted bytes cost ingress trace rows, so a
shard's byte feature is owned ∪ touched(owned) (full reference closure
as the over-approximating fallback when no touch graph was recorded).
The stub/promotion machinery of the original branch is dropped entirely:
lazy fault-in computes at run time exactly the set the stubs predicted,
with no replay divergence and no repair ladder, and the manifest format
stays byte-identical with main (owned-only).
Cost models: nlogn(x) = x·log2(x+2) feature form matching Aiur's
width·height·log2(height) FFT structure; prove/execute time and RAM
coefficients carried over from the pre-#529 aiur bench-suite fits.
They are placeholders pending a refit against this kernel — the circuit
shrank (deleted positional machinery), the byte feature narrowed to the
faulted set, and the recording counters moved under #442 — with
provenance noted at the definitions. The costs sidecar
(<out>.costs.csv) persists per-shard features incl. subst_unique and
nat_arith so the refit has its corpus.
CLI: ix shard --backend aiur --max-ram G; ix profile --backend/--top
(per-metric block leaderboards with display names) and ix profile sweep
(env-wide closure cost sweep: per-root feature CSV + feasibility,
min-root-per-hot-block, and diversity reports).
samuelburnham added a commit that referenced this pull request Aug 8, 2026
…acking
Port of sb/measured-ingress's planning layer onto the addr-first kernel
(#529), reworked for its lazy fault-in semantics.
Profiling (.ixprof v3): the recording kernel's try_get_const consults are
drained per constant into a touch graph — the measured set a lazy checker
faults in — persisted as a presence-flagged CSR section alongside a new
block-level reference graph and a subst_unique counter (splitmix64-keyed
dedup of substitution work items: the post-memoization volume a memoizing
executor pays). Both graph sections are optional, so older readers'
formats stay decodable by version gate alone.
Planner: partition_for_aiur_ram bin-packs to a host-RAM cap with
faulted-set byte accounting — under #529 the witness always ships the
full closure, but only faulted bytes cost ingress trace rows, so a
shard's byte feature is owned ∪ touched(owned) (full reference closure
as the over-approximating fallback when no touch graph was recorded).
The stub/promotion machinery of the original branch is dropped entirely:
lazy fault-in computes at run time exactly the set the stubs predicted,
with no replay divergence and no repair ladder, and the manifest format
stays byte-identical with main (owned-only).
Cost models: nlogn(x) = x·log2(x+2) feature form matching Aiur's
width·height·log2(height) FFT structure; prove/execute time and RAM
coefficients carried over from the pre-#529 aiur bench-suite fits.
They are placeholders pending a refit against this kernel — the circuit
shrank (deleted positional machinery), the byte feature narrowed to the
faulted set, and the recording counters moved under #442 — with
provenance noted at the definitions. The costs sidecar
(<out>.costs.csv) persists per-shard features incl. subst_unique and
nat_arith so the refit has its corpus.
CLI: ix shard --backend aiur --max-ram G; ix profile --backend/--top
(per-metric block leaderboards with display names) and ix profile sweep
(env-wide closure cost sweep: per-root feature CSV + feasibility,
min-root-per-hot-block, and diversity reports).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@arthurpaulino@johnchandlerburnham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

IxVM: cheaper shard checking via an addr-first kernel, plus soundness hardening - #529

Merged
johnchandlerburnham merged 19 commits into
mainfrom
ap/ixvm4
Aug 3, 2026
Merged

IxVM: cheaper shard checking via an addr-first kernel, plus soundness hardening#529
johnchandlerburnham merged 19 commits into
mainfrom
ap/ixvm4

Conversation

@arthurpaulino

@arthurpaulinoarthurpaulino commented Jul 31, 2026

Copy link
Copy Markdown
Member

Motivation

Make sharded environment checking cheap. A large environment is partitioned across shards, and each shard proves its own constants well-typed against a thin-frontier assumption cutoff, so every constant is checked in exactly one shard and cross-shard assumption trees stay minimal.

Delivering that required migrating the IxVM Aiur kernel from a positional constant table to a content-addressed, lazily-faulted design — and then auditing the result, which turned up five soundness holes with working exploits, including three separate routes to proving False.

Sharded environment checking

A shard proves CheckEnv{root, assumptions}: root is a merkle root over the shard's owned constants, assumptions a merkle root over the thin frontier — the direct out-of-owned walk edges. The kernel's env_walk starts from the owned set, checks each reachable constant, and stops at a frontier member (assumed well-typed by its owning shard: not re-checked, not recursed into).

  • Sub-frontier constants are never re-checked here; they are covered transitively by the frontier members' own shard claims. This is the cost win over checking each constant's full closure: no cross-shard duplication, and the assumption tree is one layer deep instead of the whole closure.
  • Strict membership: a walk-reachable constant in neither owned nor the frontier is a bad partition and hard-errors, so a shard cannot silently drop coverage of its own closure.
  • Byte scope ≠ check scope: delta unfolding reads through the frontier, so a witness ships the full dependency closure of owned even though only the strict-membership set is checked. The witness IOBuffer is built in Rust (aiur_ixvm_witness) rather than Lean, removing the per-byte boxing that dominated shard wall time.

Addr-first kernel

  • KExprNode.Const / Proj key on a 32-byte content Addr, not a numeric position; KConstantInfoInduct/Ctor/Rec carry a (block_addr, ind_idx) parent pair instead of indices. The positional top: List<KCI> table threaded through the kernel is gone.
  • get_ci(addr) is the sole resolver, Aiur-memoized per addr, faulting constant bytes lazily from an IOBuffer. load_verified_constant re-derives blake3 over the bytes and asserts key == addr; the io_get_info idx/len are unconstrained witness and are never branched on.
  • IOBuffer channels: 0 claim, 1 assumption tree, 2 constant bytes, 3 Defn reducibility hint (advisory), 4 blob bytes. The kernel derives blob-vs-constant from Expr context, so no wire discriminator ships. The two sets are not disjoint — one address can be read both ways — so each channel is seeded independently.
  • Kernel/Inductive.lean and Kernel/Primitive.lean deleted (~6.6k lines); inductive validation absorbed into Kernel/Check.lean, primitives split into Kernel/Klimbs.lean (bignum gadgets) and Kernel/NatPrim.lean (Nat/Str addr dispatch). New Kernel/InferOnly.lean breaks the DefEq↔Infer cycle. Ingress.lean collapsed 1700→298 lines.

Recursor validation

  • build_rec_type reconstructs a recursor's canonical type and asserts it def-eq to the declared type, on both the kind-aware and address-only lookup paths. Without it a recursor could declare ... -> False while its rules still reconstructed correctly.
  • spec_params are lowered from the major-premise frame into the recursor-param frame at extraction: the root cause of 17 nested/aux recursor reconstruction failures, and why the canonical check accepted a superset.
  • k_flag and parent-inductive-shape checks hoisted out of the attacker-selectable is_aux gate; the canonical rules comparison, which formerly never ran, now runs for every recursor.

Pre-existing criticals

IDFix
P1blob-vs-constant derived from Expr context instead of a prover-answered io_get_info probe, retiring the ch-2 len=0 sentinel; a prover can no longer mark a walk root or owned leaf "skip me"
P2Nat.div n 0 pins the quotient to zero (was prover-chosen)
P3klimbs_sub/le corrected on operands with trailing zero limbs; div/mod hints normalized before use
P4is_large_eliminator field-index arithmetic parenthesized (Aiur - is right-associative)
P5Reveal recursor rules assert the claimed ruleIdx equals its position

Robustness

  • L5: UTF-8 string-literal decode validates continuation-byte ranges and per-length overlong minimums, so distinct byte sequences (e.g. overlong NUL C0 80 vs 00) no longer decode to the same codepoint and def-eq as equal.
  • L6: run_claim dispatches on the full tag4 size (was the low byte only), matching Rust Claim::get.
  • L8: the K-recursor synth gate checks the major's inferred inductive against the recursor's own, not against itself, so I.rec no longer strips a major of a different inductive.

Aiur assert_eq! messages and circuit cleanup

  • assert_eq!(a, b, "why") gains an optional diagnostic label, threaded Source.Term → Typed → Simple → Concrete → Bytecode.Op → Rust Op::AssertEq(_, _, Option<String>) → ExecError::AssertEqMismatch. The message is diagnostic-only and never enters the circuit; all ~122 IxVM assert sites are labeled.
  • Unreachable _ => assert_eq!(0, 1) match arms removed: an unmatched runtime value already aborts (the kernel's reject), so the extra arm only widened the circuit.
  • Address equality is a full 32-byte content comparison — never a pointer-value compare, which is unsound for distinct-pointer/same-data — packed as five base-256 field limbs behind a byte-0 prefilter.

Witness construction

witness_scope reuses Env::bfs_closure, which already follows refs, Prj → block, and the derived Muts → member/ctor projection edges, so the wrappers get_ci synthesizes ship as forward edges of a single O(closure) walk with no reverse scan of the env. Primitive addresses are seeded as additional walk roots, because def-eq delta-unfolds primitive bodies whose refs are reachable from nowhere else. The Lean seeder computes the same scope by the same rule, so the two cannot drift.

bench-typecheck's constants metric now reads check_const's unique query count — exactly the constants typechecked — instead of the shipped byte scope, which is a superset and inflated throughput.

Soundness audit

Eight parallel read-only audits covered the pipeline end-to-end (compile, Ixon serde, ingress/convert, type-theory core, declaration checking, claim/witness layer, primitives, entry points), under the zkDSL threat model: a prover authors the data and pays for its own trace, so aborts and rejections are not threats and only a wrong answer counts. Thirty findings; every claim relayed here was re-verified against the code before acting on it, and several agent claims were corrected or withdrawn in the process.

Fixed, with a working exploit

Each of these shipped with a fixture that was red before the fix and green after, paired with a minimal-pair control so the verdict is attributable to one variable.

HoleWhat it bought an attacker
Self-referential definitionstheorem bad : False := bad typechecked. A standalone Defn was handed a recur slot naming itself, Rec erases to an ordinary Const, and k_infer reads a Const's declared type without checking it — so the body discharged its own type by citing itself. Now restricted to unsafe/partial, which is where Lean permits self-reference and where safe code cannot reach
Nested-inductive positivityThe nested branch dropped the parameter arguments and descended into the nested inductive's constructors without substituting them, so the occurrence being hunted was never present. Host | mk : Inner Host → Host was accepted for a negative Inner. Admits False
Recursor rules compared under the prover's contextcompare_rules handed the stored rule's own binder domains to k_is_def_eq as the comparison context. Retyping them to a Prop makes both sides infer to that Prop, proof irrelevance fires, and a recursor with its computation rules swapped between constructors was accepted
Non-canonical computed Nat literalsString.utf8ByteSize built a KLimbs as one unchecked byte, and klimbs_div_mod pinned the quotient's value but not its digits. Since klimbs_eq compares limbs rather than values, the kernel had the polarity backwards: Nat.beq (String.utf8ByteSize "") 0 = false was accepted while the true equation was rejected
Tag0/Tag2 overlong payloadsThree readers, three answers on the same bytes: the circuit discarded bytes past the eighth, the Lean host folded them back in (its shift is mod 64), Rust rejected. One address denoted different constants to different readers

Fixed as defense-in-depth

No exploit demonstrated for these; each commit says so in its first line. They remove a dependence on some other check happening to hold.

  • Struct-eta type equality.try_eta_struct omitted the infer(t) ≟ infer(s) side condition all three references carry, degenerating to t ≡ S.mk … (Proj S i t) … for any t of any type.
  • Recursor rule labels.cidx selects a rule at reduction time but was validated only positionally; nf decides how iota splits the constructor's arguments and was never checked against the constructor. Equivalents existed only in dead code.
  • Iota constructor identity. Rules were selected by index alone, so any inductive's constructor with a matching index fired one. Unreachable through a claim — inference k_checks every argument first — but reduction should not depend on that.
  • Universe cache context.compile_univ memoized Level → Univ while a Level::Param compiles to a position in univ_params, and the cache outlives a constant. Now keyed by (level, univ_params_key), matching what collect_expr_tables already does one level up.

Entry points and coverage

  • verify_const / verify_check / verify_check_env are DEBUG entrypoints that trust their dependencies, and they shipped in the production verifying key alongside verify_claim — leaving ix verify's funcIdx check as the only barrier, untested, with no equivalent in the recursive verifier. The production toplevel is now pruned to verify_claim alone, so such a proof cannot exist under that VK. Zero FFT cost.
  • bench-typecheck reported "status": "ok" and exit 0 when verification failed, and dropped thrown constants without marking them failed.
  • shardsCover skipped constants whose bytes do not parse — assigning them to no shard, counting them as neither owned nor unowned, and still including them in "covers all N consts". Now a hard failure.

Test guards that could not fail

Four of these were found by auditing the tests themselves, and they matter because they were the safety net for everything above.

  • prim-addrs asserted the literal true. Keyed by address, so a permuted primitive table passed — precisely the defect its docstring claimed to catch. Now the address must resolve to the constant the function's name claims; verified by temporarily swapping nat_add_addr to Nat.mul's bytes and watching it go red.
  • The arena negative corpus counted any abort as a correct rejection, so a fixture aborting on invalid IO key — never reaching the code under test — scored as a pass. That is the same failure mode that made two exploit fixtures in this series pass for the wrong reason before it was caught.
  • The shard-pipeline SKIP was a passing test conflating "target absent" with "partition selected zero constants".
  • A --ignored filter naming no runner executed nothing and returned 0, so a typo'd suite name reported success having run no tests.

Adversarial Ixon test suite

New Tests/Ix/IxVM/Exploits.lean, 19 cases, wired into the ixvm suite. The kernel-arena corpus attacks the layer above: its bad_* fixtures are raw Lean.ConstantInfo declarations that skip the elaborator, but they are still compiled by rsCompileEnvFFI, so the Ixon they produce is always well-formed. This suite authors Ixon.Constant values directly, and storeRawAt goes one level lower still, storing hand-authored byte strings — needed because every value-level fixture is canonical by construction and so cannot express a malformed encoding.

Each case records the false statement acceptance would buy, so a failure reads as a security statement. Rejection reasons are printed, which earned its keep repeatedly: fixtures that "passed" via invalid IO key without reaching the checker were caught this way, as was a fixture that silently corrupted nothing because non-mutual recursors are standalone constants rather than Muts members.

Testing

  • lake test -- --ignored ixvm green.
  • cargo test --release across all crates: 1195 passed, 0 failed. This also repaired shard_check_env_claim_is_thin_frontier, red since the claim walk moved to positive constant-use because its fixture's Exprs never referenced their own refs.
  • New prim-addrs parity suite; codegen ↔ bytecode parity gate green; arena negative corpus (77 bad_*) all reject, now only via genuine in-kernel refusals.
  • FFT-cost pins re-pinned per change. Cumulative effect is small (Nat.add_comm 42_747_927 → 42_986_025, +0.6% over the whole series); the largest single contributors are the nested-positivity substitution and the struct-eta guard.
  • ix codegen regenerated whenever the Aiur kernel changed, so the committed Rust kernel never lags the source the parity gate compares against.

Known limitations

Recorded rather than silently dropped.

  • Call-site surgery is not covered by the value pin. The compiler rewrites calls to aux-generated constants — permuting arguments, dropping some, replacing the head and its universe list — and the plans driving it live only in ConstantMeta, which prune_to_closure_anon excludes. So a verified claim says the term at address X is well-typed, not that term is the Lean declaration you wrote; the equivalence rests on the rewrite being faithful, checked today only by decompile round-trip tests. Closing it is a claim-format decision, not a patch.
  • Assumption cycles are not rejected, and a conditional Check claim naming its own target is accepted. That is the specified semantics — some root means "conditional on every leaf", so the claim asserts A → A, vacuous rather than false — and the obligation sits with whoever discharges assumptions. Sharded CheckEnv is not exposed, since shardClaimDigest recomputes the claim from (env, blocks). Pinned as ACCEPT so the boundary cannot move silently.
  • Alpha-collapse comparators ignore safety, so two members agreeing on kind/lvls/typ/value and disagreeing on safety collapse. Deliberately not fixed: a prover authors the environment directly and never goes through the compiler, so hardening it stops no attacker, while adding safety to the ordering would move every address and refusing the collapse would put a hard error in the compiler on the strength of an unverified assumption.
  • klimbs_le / klimbs_eq are in the logical trusted base, not just the performance path: Nat.decLe mints Decidable.isTrue with a proof term whose correctness is asserted rather than checked, so one wrong verdict yields False.
  • check_inductive_shape_ctors omits validate_expr_well_scoped / k_ensure_sort / assert_safety that check_const's Ctor arm performs; only an entirely unreferenced constructor escapes those.

@arthurpaulino
arthurpaulinoforce-pushed the ap/ixvm4 branch 4 times, most recently from 06e0489 to 559ced9CompareAugust 1, 2026 00:53
… hardening
Make sharded environment checking cheap. A large environment is partitioned
across shards, and each shard proves its own constants well-typed against a
thin-frontier assumption cutoff, so every constant is checked in exactly one
shard and cross-shard assumption trees stay minimal. Delivering that required
migrating the IxVM Aiur kernel from a positional constant table to a
content-addressed, lazily-faulted design -- and closing the soundness
regressions that migration introduced along with several pre-existing holes.
Sharded environment checking
- A shard proves `CheckEnv{root, assumptions}`: root is a merkle over the
shard's owned constants, assumptions a merkle over the THIN FRONTIER --
the direct out-of-owned walk edges. The kernel's env_walk starts from the
owned set, checks each reachable constant, and stops at a frontier member
(assumed well-typed by its owning shard, not re-checked, not recursed).
- Sub-frontier constants are never re-checked here; they are covered
transitively by the frontier members' own shard claims. This is the cost
win over checking each constant's full closure: no cross-shard duplication,
and the assumption tree is one layer deep instead of the whole closure.
- Strict membership: a walk-reachable constant in neither owned nor the
frontier is a bad partition and hard-errors, so a shard cannot silently
drop coverage of its own closure.
- Byte scope != check scope: delta unfolding reads through the frontier, so
a witness ships the full walk closure of owned even though only the
strict-membership set is checked. The witness IOBuffer is built in Rust
(aiur_ixvm_witness) rather than Lean, removing the per-byte boxing that
dominated shard wall time.
Addr-first kernel
- KExprNode.Const/Proj key on a 32-byte content Addr, not a numeric
position; KConstantInfo Induct/Ctor/Rec carry a (block_addr, ind_idx)
parent pair instead of indices. The positional `top: List<KCI>` table
threaded through the kernel is gone.
- get_ci(addr) is the sole resolver, Aiur-memoized per addr, faulting
constant bytes lazily from an IOBuffer. load_verified_constant re-derives
blake3 over the bytes and asserts key == addr; the io_get_info idx/len
are unconstrained witness and are never branched on.
- IOBuffer channels: 0 claim, 1 assumption tree, 2 constant bytes, 3 Defn
reducibility hint (advisory), 4 blob bytes. An address is on ch2 iff a
constant and ch4 iff a blob; the kernel derives which from Expr context
(Str/Nat -> blob, Ref/Prj -> constant), so no wire discriminator ships.
- Kernel/Inductive.lean and Kernel/Primitive.lean deleted (~6.6k lines);
inductive validation absorbed into Kernel/Check.lean, primitives split
into Kernel/Klimbs.lean (bignum gadgets) and Kernel/NatPrim.lean (Nat/Str
addr dispatch). New Kernel/InferOnly.lean breaks the DefEq<->Infer cycle.
Ingress.lean collapsed 1700->298 lines. V1/V2 naming removed throughout.
Recursor validation (R1, R2, R3)
- build_rec_type reconstructs a recursor's canonical type and asserts it
def-eq to the declared type, on both the kind-aware and address-only
lookup paths. Without it a recursor could declare `... -> False` while
its rules still reconstructed correctly.
- spec_params are lowered from the major-premise frame into the
recursor-param frame at extraction: the root cause of 17 nested/aux
recursor reconstruction failures, and why the canonical check accepted a
superset.
- k_flag and parent-inductive-shape checks hoisted out of the
attacker-selectable `is_aux` gate; the canonical rules comparison, which
formerly never ran, now runs for every recursor.
Pre-existing criticals (P1-P5)
- P1: blob-vs-constant is derived from Expr context instead of a prover-
answered io_get_info probe, retiring the ch-2 len=0 sentinel; a prover
can no longer mark a walk root or owned leaf "skip me".
- P2: Nat.div n 0 pins the quotient to zero (was prover-chosen).
- P3: klimbs_sub/le corrected on operands with trailing zero limbs; div/mod
hints normalized before use.
- P4: is_large_eliminator field-index arithmetic parenthesized (Aiur `-` is
right-associative).
- P5: Reveal recursor rules assert the claimed ruleIdx equals its position.
Robustness (L5, L6, L8)
- L5: UTF-8 string-literal decode validates continuation-byte ranges and
per-length overlong minimums, so distinct byte sequences (e.g. overlong
NUL C0 80 vs 00) no longer decode to the same codepoint and def-eq as
equal. Roughly zero added FFT cost.
- L6: run_claim dispatches on the full tag4 size (was the low byte only),
matching Rust Claim::get.
- L8: the K-recursor synth gate checks the major's inferred inductive
against the recursor's own (via se_parent_addr), not against itself, so
I.rec no longer strips a major of a different inductive.
Aiur assert_eq! messages + circuit cleanup
- assert_eq!(a, b, "why") gains an optional diagnostic label, threaded
Source.Term -> Typed -> Simple -> Concrete -> Bytecode.Op -> Rust
Op::AssertEq(_, _, Option<String>) -> ExecError::AssertEqMismatch. The
message is diagnostic-only and never enters the circuit; all ~122 IxVM
assert sites are labeled.
- Unreachable `_ => assert_eq!(0, 1)` match arms removed: an unmatched
runtime value already aborts (the kernel's reject), so the extra arm only
widened the circuit.
- address equality is a full 32-byte content comparison (never a pointer-
value compare, which is unsound for distinct-pointer/same-data), packed
as five base-256 field limbs (7+7+7+7+4) behind a byte-0 prefilter and
short-circuiting on the first mismatched group.
Crate reorganization
- PrimAddrs / reserved_marker_name moved verbatim from
crates/kernel/src/primitive.rs to crates/common/src/prim_addrs.rs (kernel
re-exports), so witness builders reach the primitive address table
without an ix-kernel -> ixon -> ix-kernel cycle.
- witness_scope reuses Env::bfs_closure -- the env's own dependency-closure
pass, which already follows refs, Prj->block, and the derived
Muts->member/ctor projection edges -- so the wrappers the kernel's get_ci
synthesizes ship as forward edges of a single O(closure) walk, with no
reverse scan of the whole env. Primitive-address seeding is unioned on
top: primitives are fabricated during reduction and need not appear in a
target's ref closure, so a closure-only scope aborts with `invalid IO
key`.
Tests
- New prim-addrs parity suite (Lean<->Aiur primitive address table).
- Codegen<->bytecode parity gate restored, green (198/198).
- FFT-cost pins re-pinned to current behavior; arena negative-fixture
corpus (77 bad_* constants) all reject.
…onstants
closureFromBase now walks the derived Muts->member/ctor projection
wrapper addresses as forward edges (projWrapperAddr: blake3 over the
serialized proj-only Constant, compile's construction), so closureFrom
and buildShardCheckEnvWitness drop their O(env) reverse scans that
parsed every env constant per witness build. Primitive addresses are
seeded bytes-only after the walk instead of being pushed onto the
worklist, so their transitive closures no longer inflate the byte scope
(a delta-unfold into an unshipped primitive body aborts with invalid IO
key rather than proving anything wrong).
bench-typecheck's constants metric now reads check_const's unique query
count (memoized per (ci, addr) = exactly the constants typechecked)
instead of closureFrom.size, which counted the shipped byte scope.
check_const is resolved up front and both lookups fail with IO.userError
instead of panicking.
Experiment: lake exe bench-typecheck --ixe seed.ixe --consts
Nat.add_comm --execute-only (seed.ixe = ix compile Tests.lean --consts
Nat.add_comm) reports constants=42, matching check_const's height in
lake exe ix check Nat.add_comm; fft-cost unchanged at 42747927.
The claim walk decided which of a constant's `refs` to recurse into by
SKIPPING every index the constant's Exprs used as a blob payload
(`Str(i)` / `Nat(i)`). That rule is unsound: the Exprs are authored by
the prover, and `refs` is one index space shared by `Ref`, `Prj`, `Str`
and `Nat`, so marking an index suppresses the walk of that index used as
a constant everywhere else in the same constant.
The marking need not even be live. `blob_idxs_of` traversed `sharing` in
full, but conversion reaches a `sharing` entry only through an
`Expr.Share` node, and nothing validates the array. A dead `Nat(i)`
entry is therefore free: it is never converted, never typechecked, and
needs no blob on channel 4. It just hides `refs[i]` from the checker.
`k_infer` then reads that dependency's DECLARED type without ever
checking it, so the kernel accepts a constant whose transitive closure
is ill-typed -- a wrong answer, not an abort. Both claim variants were
affected; for CheckEnv the skip happens before `env_walk` is entered, so
the strict owned-membership assert never fires either.
Replace the family with `const_idxs_*`: collect the indices `Ref(i, ..)`
and `Prj(i, ..)` use, and walk `refs[i]` iff `i` is among them. `Rec`
resolves through `recur_addrs` and a projection's block comes from the
ConstantInfo payload, both walked separately, so the positive set is
complete. Same single traversal as before, so no added cost. The failure
direction is now safe: over-approximating walks an address that faults
an unseeded key and aborts, which only costs a dishonest prover.
Adversarial Ixon test infrastructure
- New `Tests/Ix/IxVM/Exploits.lean`, wired into the ixvm suite. The
kernel-arena corpus attacks the layer above this one: its `bad_*`
fixtures are raw `Lean.ConstantInfo` declarations that skip the
elaborator, but they are still compiled by `rsCompileEnvFFI`, so the
Ixon they produce is always well-formed. This suite authors
`Ixon.Constant` values directly, which is what a prover can do.
- Each `ExploitCase` records the false statement acceptance would buy,
so a failure reads as a security statement. Rejection reasons are
printed: a fixture that aborts before reaching the mechanism under
attack is indistinguishable from a kernel that correctly refuses it,
and the first run of this fixture passed for exactly that wrong reason
(`invalid IO key` from an unseeded channel-3 hint, now seeded by
`storeAt`).
- The exploit ships with two controls that differ from it minimally: one
checking the smuggled dependency alone, one checking the same target
without the dead `sharing` entry. Both reject via the intended
in-kernel assert, so the exploit's acceptance was attributable to the
`sharing` entry alone.
- Two further cases probe what the fix does NOT cover. An index used as
both a `Nat` literal and a `Ref` — reachable because blob and constant
addresses share one un-domain-separated blake3 space and
`load_verified_blob` parses nothing — is now walked and rejected,
though by channel fault rather than type error, which is the safe
direction. A conditional Check claim naming its own target as its own
assumption IS accepted, and is pinned that way: `some root` means
"conditional on every leaf being well-typed", so the claim asserts
`A -> A` and is vacuous rather than false. The obligation sits with
whoever discharges assumptions, which must reject such cycles; sharded
CheckEnv is not exposed, since `shardClaimDigest` recomputes the claim
from (env, blocks).
Primitive closures are walked again
- The previous commit seeded primitive addresses as bytes only, without
walking their closures. That is too aggressive: def-eq delta-unfolds
primitive BODIES, and a body's refs are reachable from nowhere else in
the scope, so the `strAppend*` arena fixtures aborted with `invalid IO
key`. Both seeders now pass the primitives as walk ROOTS
(`closureFromRoots` in Lean, `bfs_closure` over target+primitives in
Rust); the Rust side had the same gap latently, since only the Lean
path had arena coverage.
- The removal of the O(env) reverse wrapper scan, which was that
commit's actual win, is unaffected.
Tests
- `lake test -- --ignored ixvm` green. FFT pins re-pinned across the 66
kernel-check entries and the shard pipeline; costs move in both
directions (e.g. `Eq.rec` 1_249_383 -> 1_249_094, `Vector.append`
1_994_697_378 -> 2_010_055_088), the increases coming from the
restored primitive closures.
- `ix codegen` regenerated: the Aiur kernel changed, so the committed
Rust kernel would otherwise be stale and the codegen/bytecode parity
gate would compare against it.
`Expr.Str` conversion wrapped any hash-valid blob in `KLiteral.Str`, and
`k_infer_lit` types a `KLiteral.Str` as `String` without inspecting the
bytes. Nothing on the ordinary checking path ever decodes a literal, so a
prover-authored definition carrying malformed bytes typechecked as
`x : String` even though no Lean `String` corresponds to those bytes. The
reference kernel rejects this at ingress via `String::from_utf8`
(crates/kernel/src/ingress.rs).
The UTF-8 validation added earlier lives in the string DECODER, which is
reached only during string reductions, so it did not cover this path.
`utf8_decode_one` already rejects out-of-range continuation bytes, stray
continuations used as leaders, and overlong forms. Walk the whole blob
with it at conversion (`utf8_validate`), which is the same rejection the
reference kernel performs.
Tests
- `invalid-utf8-accepted-as-string` in the adversarial Ixon suite: a
definition declaring `String` over the blob `[0xFF]`, which no
well-formed UTF-8 can contain. Accepted before this change, now
rejected.
- Paired with `control-valid-utf8-string-literal`, the same shape over
`[0x68, 0x69]`, which must still be accepted — so the verdict is
attributable to the bytes rather than to the fixture's shape. The
fixtures build on a base env supplying the real `String` constant, so
the declared type is the genuine one.
- Six FFT pins move, all string-touching, by between +17 and +66_530
(e.g. `IxVMPrim.str_size_lit` 567_999_042 -> 568_000_142,
`Vector.append` 2_010_055_088 -> 2_010_088_729, +0.0017%). Constants
with no string literals are unchanged, `Nat.add_comm` included.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`add_entries_parallel` returned early after a constant hit, so an address
present in both `env.consts` and `env.blobs` was seeded on channel 2 but
never on channel 4. The Lean seeder walks `consts` and `blobs` in
separate passes and seeds both, so the two builders disagreed, and the
Rust-native check/prove paths aborted with `invalid IO key` on such an
environment.
The two tables are not disjoint. Constant and blob addresses share a hash
codomain, and it is the Expr context — `Ref`/`Prj` versus `Str`/`Nat` —
that says which table a reference means, so the same bytes can legitimately
be read both ways. The kernel faults the two channels separately, so the
witness has to seed them separately. Reaching an address through both
kinds of use became expressible when the claim walk moved to selecting
indices by positive constant-use.
Corrects the module header and the `ChannelEntries` doc, which both
asserted the disjointness this relies on not having.
Tests
- `dual_use_address_is_seeded_on_both_channels`: an address stored as a
constant and as a blob must appear under both `(ch 2, key)` and
`(ch 4, key)` in the built IOBuffer. Fails on the previous code with
the ch-4 entry missing.
- `lake test -- --ignored ixvm` green; clippy `--all-targets` and fmt
clean.
The kernel walks a constant's `refs` by positive constant-use — index `i`
is followed only when some Expr names it via `Ref(i, ..)` or `Prj(i, ..)`
— but both host claim builders still treated every `refs` entry as a walk
edge. The host and the kernel therefore disagreed about the edge
relation.
A `refs` entry that is a real constant but that no Expr uses as one is
never walked in-circuit, so admitting it host-side puts a phantom member
in the thin frontier: a cross-shard assumption the kernel never makes,
and a spurious dependency edge that can close a cycle once assumptions
are discharged. `walk_check_set` likewise stopped computing the set the
verifier actually checks.
Mirror `const_idxs_of` on both hosts and emit an edge only for
positively-used indices. `Env::bfs_closure` is deliberately left alone:
it computes the witness BYTE scope, where over-approximating is correct
(shipping unused bytes is free, missing bytes aborts).
Honest constants are unaffected, since a compiler only puts an address in
`refs` when an Expr references it — the shard FFT pin is unchanged. The
divergence is reachable only through prover-authored `refs`.
Tests
- `thin_frontier_excludes_unused_ref_entries`: `refs` carries two
constants but only index 0 is named by a `Ref`; the frontier must
contain the used one and not the dead one. Verified red against the
previous edge relation ("unused refs entry became a phantom frontier
member").
- `const_with_refs` now names each of its `refs` entries via a `Ref`
node. The old helper left them unreferenced, which under the walk rule
would have produced no edges at all and quietly hollowed out the
existing frontier tests.
- `lake test -- --ignored ixvm` green; clippy `--all-targets` and fmt
clean.
`idx_to_u64` packed a synthesized member/constructor index into the
projection wrapper's `u64` field by populating only the low byte behind a
`u8_range_check`, so a mutual block with more than 256 members — or an
inductive with more than 256 constructors, which is the likelier case —
was rejected outright with `value 256 out of u8 range`.
Field-to-bytes is not free: the bytes have to be supplied as advice and
then pinned. Take them from `unconstrained_g_to_bytes` and discharge the
three obligations its contract names:
- range: four `u8_range_check` calls cover the eight bytes, the byte chip
taking a pair per lookup row;
- recomposition and canonicality: `flatten_u64(bytes) == idx`, since
`flatten_u64` asserts the top byte is zero and sums the low seven.
Canonicality is what makes the decomposition unique, and it is enforced
by RESTRICTING THE DOMAIN rather than by comparing against `p`. Goldilocks
is 2^64 - 2^32 + 1, so an arbitrary eight-byte string is not injective
into the field: any value and that value plus `p` recompose to the same
element, e.g. `[00]*8` and `[01,00,00,00,FF,FF,FF,FF]` both give 0.
Forcing the top byte to zero bounds the value by 2^56 - 1 < `p`, so the
sum cannot wrap and exactly one string encodes each index. That
uniqueness is the soundness property: the projection address is blake3
over these serialized bytes, so a prover free to choose a second string
for one index would obtain a second address for a member, or collide two
members onto one address. The pre-existing `u8_from_field_unsafe`
version did collide members 0 and 256, which the range check had already
fixed by rejecting instead.
Both halves are load-bearing. `flatten_u64` does NOT range-check its
input — `[U8; 8]` is nominal typing and `u8_from_field_unsafe` mints a
`U8` holding any field element — so without the range checks the sum is
satisfiable many ways (`b1 = k`, `b0 = idx - 256k`). Its doc comment now
records that its injectivity is conditional on the caller, and names this
call site.
Indices at or beyond 2^56 still fail the assert rather than truncating.
That is a liveness bound only, 2^48 times the previous cap and far past
any real block or constructor count; buying the remaining `[2^56, p)`
would cost a multi-byte comparison against `p` for no practical gain.
Tests
- `muts-block-past-single-byte-index`: a 257-member block of well-typed
definitions with a projection wrapper for each, which now checks.
Rejected before this change with `value 256 out of u8 range`.
- `control-small-muts-block`: the same construction at two members,
already accepted — so the big-block result is attributable to the index
width alone and not to a malformed fixture.
- `storeAt` also registers a reducibility hint for `DPrj` wrappers, which
`get_ci_dprj` faults under the wrapper's own address; without it these
fixtures aborted with `invalid IO key` before reaching the index logic.
- All 66 kernel-check FFT pins and the shard pin move by +17 to +485
(`Nat.add_comm` 42_967_279 -> 42_967_313, +0.00008%), the cost of the
hint plus four range checks per synthesized index.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`get_ci` handed every standalone `Defn` a one-element `recur_addrs` whose
only entry was its own address. `Rec` converts to a plain `Const`, and
`k_infer` discharges a `Const` against its DECLARED type without checking
it, so a definition could prove its own type by citing itself:
theorem bad : False := bad
typechecked. The ref-walk cannot catch it — `const_idxs_expr` does not
follow `Rec`, and the referent is the constant already under check. Lean
forbids exactly this by checking a definition against an environment that
does not yet contain it, admitting self-reference only for `unsafe` and
`partial`.
Restrict the self slot to `unsafe`/`partial`. Those are where
self-reference is both legitimate and contained:
- legitimate, because the compiler emits a singleton (or fully
collapsed) mutual block as a standalone `Defn` whose body still says
`Rec(0)` -- see the note at Ix/CompileM.lean:2000. Those blocks are
exactly Lean's `unsafe`/`partial` recursive definitions; a `safe`
recursive definition compiles its recursion into `.rec` applications.
Gating on `unsafe` alone breaks `Lean.extractMainModule._unsafe_rec`.
- contained, because safe code may not reference either kind.
That containment was itself missing for `partial`: `is_unsafe_ci`
classified it as safe, so `partial def bad : False := bad` plus a safe
theorem referencing `bad` would have walked straight around the first
half of this fix. Count `Partial` alongside `Unsafe`, matching the
reference's "safe definition references partial definition" rejection
(crates/kernel/src/check.rs:813-820). The predicate serves double duty --
the caller's own leniency and the referent's barrier -- and `Partial`
belongs on the non-safe side of both.
Tests
- `self-referential-theorem-proves-false`: `theorem bad : False := bad`
built directly in Ixon over the real `False` constant, submitted
through the production `verify_claim` entrypoint. Accepted before this
change -- a proof of False -- and rejected after.
- No safe constant in the corpus references a partial definition, and
the `_unsafe_rec` witnesses still check, so neither half of the fix
costs coverage.
- All 66 kernel-check FFT pins and the shard pin move (`Nat.add_comm`
42_967_313 -> 42_965_666); the self slot is no longer built for safe
definitions, so most costs drop slightly.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
The nested branch of the strict-positivity check dropped the parameter
arguments (`list_drop(args, n_ctor_params)`) and descended into the
nested inductive's constructors without substituting them. A nested
constructor's fields mention that inductive's parameters as bound
variables, so the block-occurrence test asked whether the block occurs in
`BVar 0` — it never does, and every parameter position passed vacuously.
The occurrence being hunted lives in the ARGUMENT substituted for the
parameter, which was exactly what got discarded.
So for `Inner α | mk : (α → False) → Inner α`, itself strictly positive,
`Host | mk : Inner Host → Host` was accepted: descending into `Inner.mk`
sees the field `α → False`, which mentions nothing tracked. With `α :=
Host` it reads `(Host → False) → …`, a negative occurrence, and Lean
rejects it. Accepting it admits `False`.
Mirror `check_nested_ctor_fields` (crates/kernel/src/inductive.rs): strip
`n_params` foralls, then simultaneously substitute the parameter
arguments at depth 0. After stripping, `BVar 0` is the innermost (last)
parameter, so the argument list is reversed — `expr_inst_many` maps
`substs[i]` to `BVar(depth + i)`, the same convention as `simul_subst`.
The parameters are substituted away rather than bound, so the field walk
now starts with an empty binder context instead of the peeled types.
Tests
- `nested-inductive-negative-occurrence`: the `Inner`/`Host` pair above,
built directly in Ixon as two single-inductive `Prop` blocks with their
projection wrappers. Accepted before this change, and now rejected by
the intended assert ("strict positivity: block occurs left of an
arrow").
- `control-nested-positive-inductive`: the same construction with
`Inner α | mk : α → Inner α`, which is strictly positive and must keep
checking. The two fixtures differ only in the inner constructor's
field, so the rejection is attributable to the negative occurrence
rather than to nesting.
- 63 FFT pins and the shard pin move, concentrated in inductive-heavy
constants (`IxVMInd.Tree` 767_167 -> 796_717, `IxVMInd.DepthM`
1_312_461 -> 1_347_394) — the nested check now does real work instead
of returning vacuously.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`KLimbs` is canonical in two independent ways — no trailing zero limbs,
and every limb byte in [0, 256) — and `klimbs_eq` is a raw limb compare
that establishes neither. Since it backs `Nat.beq`, `Nat.decEq` and
`literal_eq`, any producer emitting a value-correct but
representation-wrong `KLimbs` makes those answer `false` where Lean
answers `true`. That is an accepted false equation, not an abort. Two
producers did.
`String.utf8ByteSize` built the length as one unchecked byte
(`[u8_from_field_unsafe(n), 0, ...]`) with no normalization. The empty
string therefore reduced to the denormalized zero `[[0;8]]` instead of
the canonical empty limb list, and a string of 256 bytes or more to an
out-of-range digit — `u8_from_field_unsafe` is an erased coercion, so
nothing truncated or rejected. Build it with `klimbs_from_g`, whose byte
split is range-checked and pinned by a recomposition assert, then
normalize.
`klimbs_div_mod` pinned the quotient's VALUE (`q*b + r == a`, `r < b`)
but not its digits. The comment claimed `u8_mul` inside `klimbs_mul`
range-checked them, but `u64_mul` was rewritten to raw field products
plus `#split_carry`, whose u8 checks constrain the split outputs rather
than the input digits — and it re-canonicalizes while multiplying, so a
digit-wrong `q` still produces a canonical `q*b` and satisfies the
equality. Range-check the hint's limbs explicitly, quotient and
remainder both, via a new `klimbs_range_check`.
Tests
- `utf8-size-beq-false-accepted`: `Nat.beq (String.utf8ByteSize "") 0 =
false` proved by `Eq.refl`, over the real `Eq`/`Bool`/`Nat.beq`/
`String.utf8ByteSize` constants. The kernel had the polarity exactly
backwards — this FALSE equation was accepted while the true one was
rejected.
- `control-utf8-size-beq-true`: the same declaration ending in `true`,
which is what Lean computes and must be accepted. The pair differs only
in that operand, so each verdict is attributable to the reduction.
Both now land the right way round.
- The two fixtures share one blob for `""` and `0` — the same empty byte
string read as a string literal and as a nat literal, which is the
Expr-context disambiguation working as intended.
- 10 FFT pins and the shard pin move, confined to the div/mod/gcd/pow/
shift constants (the hint range-checks) and the string-size ones
(`IxVMPrim.str_size_lit` 567_992_937 -> 567_993_099).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT WAS DEMONSTRATED for this one. Unlike the other soundness
fixes in this series it ships without a red-then-green fixture; what
follows is why the rule is wrong as written, and what remains unproven.
`try_eta_struct` accepted `t ≡ S.mk p… a…` after checking only that the
right side is a fully-applied constructor of a struct-like inductive. It
never checked that the two sides have def-eq TYPES. All three references
do, citing lean4lean's `tryEtaStructCore`
(crates/kernel/src/def_eq.rs, Ix/Tc/DefEq.lean).
Without the guard the rule degenerates: for ANY `t` of ANY type,
`compare_struct_fields` builds precisely `Proj(S, i, t)` for each field,
so when the right side's arguments are those same projections the
pointer fast path matches them and the pair is accepted. Those
projections are synthesized inside def-eq and never inferred, so
`k_infer_proj`'s head-address gate — which would reject `Proj S i t` for
`t : Nat` — never sees them.
What is unproven is reachability. At the `k_check(e, expected)` boundary
both sides are types in the same context, and `de_args` compares type
arguments before value arguments, so I could not construct a term that
reaches this rule with genuinely mismatched types. The change is
therefore defense-in-depth against an unjustified rule, not a repair of a
demonstrated break.
The guard runs after the struct-like shape test, so it costs two
`k_infer_only` calls and one `k_is_def_eq` only on pairs that were about
to be accepted. `try_proof_irrel`, earlier in the same fallback chain,
already infers the left side unconditionally, so the added abort exposure
is small; the suite confirms no constant stops checking.
Tests
- No new fixture; the existing 14 adversarial cases and the arena corpus
are unchanged.
- 45 FFT pins and the shard pin move, the largest cost of this series
(`Nat.add_comm` 42_965_759 -> 42_966_970, +0.003%;
`IxVMPrim.str_size_lit` +26_687).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
A `Tag0` large-form header carries a 7-bit `small` field, so a payload
width up to 128 was expressible and unguarded (`Tag2`: 5 bits, up to 32).
A u64 payload is at most 8 bytes, and the three readers disagreed on the
surplus:
- Rust `u64_get_trimmed_le` rejects `len > 8` -- already correct, and
untouched here.
- The in-circuit `get_u64_le` consumed the surplus and DISCARDED it,
keeping only the first eight bytes.
- The Lean host folded it back in: `UInt64.shiftLeft` is taken mod 64,
so byte 8 ORs into bits 0-7, with no length guard.
Same bytes, same content address, three readings. An `Expr.Ref` whose
index is written overlong resolves as `refs[0]` in-circuit and `refs[42]`
on the Lean host, with both consuming the same span so nothing desyncs
afterwards. That also means the trailing-bytes assert does not imply
canonicity: up to 120 arbitrary bytes per `Tag0` site were consumed
inside the structure and dropped.
Bring the other two up to Rust: assert the width is under 9 in both
in-circuit readers, and give `getU64TrimmedLE` the guard it was missing.
No honest serializer is affected -- `putTag0` always emits the minimal
width -- so this narrows the accepted language to the format.
Tests
- `overlong-tag0-accepted`: an axiom `_ : Prop` whose `lvls` Tag0 is
rewritten to the nine-byte form. Accepted before this change, now
rejected by the intended assert.
- `control-canonical-tag0-axiom`: the same axiom with canonical bytes,
which must stay accepted. The pair differs only in that encoding.
- Both need bytes no `Ixon.ser` will emit, so the suite gained
`storeRawAt`, which stores a hand-authored byte string at its own
content address. Every previous fixture was built from a `Constant`
VALUE and was therefore canonical by construction, and structurally
could not reach this class of defect.
- All 66 kernel-check FFT pins and the shard pin move (`Nat.add_comm`
42_966_970 -> 42_980_767, +0.03%); every constant parses Tag0s.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
Four test-only fixes. No kernel code changes, so no FFT pins move.
prim-addrs asserted the literal `true`. It looked up each hardcoded
kernel address in the canonical table and, on a hit, recorded a pass —
but the table is keyed BY ADDRESS, so a permuted table still hits. Had
`nat_add_addr` held `Nat.mul`'s bytes it would have printed
"nat_add_addr = Nat.mul" and passed green, which is precisely the defect
the file's own docstring says it exists to catch: the kernel applying one
primitive's native semantics under another's name. Now the address must
resolve to the constant the function's NAME claims, comparing normalized
keys (`nat_add_addr` <-> `Nat.add`, and the final-component spelling for
`reduce_bool_addr` <-> `Lean.reduceBool`), with dispatch-site suffixes
(`_dec`/`_io`/`_iota`) and `_type`/`_ty` stripped, plus a short exception
list for the `Quot` constructors. Verified by temporarily swapping
`nat_add_addr` to `Nat.mul`'s bytes: red, then reverted. All 76 literals
pass, and the table is confirmed correct today.
The arena negative corpus counted ANY abort as a correct rejection while
its docstring claimed rejections come from an in-kernel `assert_eq!`. An
`invalid IO key` — the witness never supplied the bytes, so execution
died before reaching the logic under test — scored as a pass. That is how
a fixture rots into a green test that checks nothing, and it is the same
failure mode that made two exploit fixtures in this series pass for the
wrong reason. Now `invalid IO key` disqualifies. Unmatched-match aborts
still count: this kernel deliberately relies on an unmatched runtime
value aborting instead of spending width on `_ => assert_eq!(0, 1)`
arms, so those are genuine verdicts. No fixture currently relies on the
disqualified path.
`shardCheckEnvCase` returned `none` for two unrelated reasons and the
runner reported both as "SKIP (target absent)", a passing test. The
second — target present but the partition selected zero owned constants —
means the fixture stopped exercising the Rust witness builder and the
thin-frontier claim convention, and would have left the only test of that
path permanently green. It now throws; `none` keeps its one honest
meaning.
A `--ignored` filter naming no runner ran nothing and returned 0, so
`lake test -- --ignored ixvm2` reported success having executed nothing.
It is now an error. Exit 0 should mean tests ran and passed.
`ixVM`, the pruned production toplevel, carried `verify_const`,
`verify_check` and `verify_check_env` alongside `verify_claim`. Those are
DEBUG entrypoints that trust their dependencies instead of walking them,
so a proof made with one attests far less than a claim does — and
carrying them meant such a proof was verifiable under the SAME verifying
key. The only thing standing between it and acceptance was `ix verify`
pinning the funcIdx to `verify_claim`: a policy, enforced in one place,
with no test asserting it, and no policy at all in the recursive
verifier, which binds the claim bytes but applies no funcIdx check.
Prune to `verify_claim` alone. The guarantee is now structural: under the
production VK a debug-entrypoint proof cannot exist. No CLI path used
them, so nothing in `ix check`/`prove`/`verify` changes.
Callers that genuinely want a subject-only check build `ixVMFull` and are
thereby using a different VK, which is the honest signal — the arena
corpus, and `bench-typecheck`/`bench-recursion-debug` under
`--skip-deps`. That last one also means a `--skip-deps` benchmark is now
visibly measuring a different key than a claim run rather than silently
reporting `verify_const` rows in the claim schema.
`bench-typecheck` also reported success on failure, twice over. A proof
that failed to verify printed to stderr but left `failed` clear, so the
row was emitted as `"status": "ok"` with prove-time and proof-size
present and the run exited 0 — the only signal an absent `verify-time`
key, invisible to anything keying on status. And a constant whose execute
threw was dropped from the results entirely without setting `failed`,
so `execed.any (·.failed)` stayed clear; the `check_const`-missing throw
added earlier landed exactly there. Both now mark the row failed.
Tests
- `lake test -- --ignored ixvm` green with NO pin movement: the pruned
entrypoints were never executed, so removing them changes no cost.
Generated kernel drops from 733 to 731 functions.
- Arena repointed to `ixVMFull`, where `verify_const` still lives.
- `bench-typecheck --execute-only` smoke unchanged (`Nat.add_comm`,
constants=42).
`compare_rules` peeled the binders off the STORED rule and handed their
DOMAINS to `k_is_def_eq` as the local context for the comparison against
the canonical reconstruction. Those domains are prover-authored: nothing
checks them against the reconstruction, and reduction never consults them
(`try_iota` beta-reduces straight through). So the prover chose what the
two sides INFER to.
Retyping every binder to a Prop makes both sides infer to that Prop, so
`try_proof_irrel` fires and accepts ANY body. A recursor whose
computation rules are swapped between constructors was therefore
accepted, and iota then reduces each constructor to the other's branch.
Compare under an empty context instead, discarding the peeled domains. An
honest rule matches the reconstruction structurally, and the structural
path never consults the context — it compares `BVar i` against `BVar i`
directly. A comparison that instead needs to INFER a bound variable's
type now runs out of context and aborts, which is the safe direction.
Every recursor in the corpus still checks, including the mutual, nested
and aux-generated ones.
The reference avoids this differently, comparing whole Lam-wrapped terms
at empty context so `is_def_eq` descends binder-by-binder and compares
the domains themselves (crates/kernel/src/inductive.rs). Reconstructing
canonical domains here would mean re-deriving field domains from each
constructor's telescope and re-lifting their de Bruijn indices into the
rule's context; dropping the untrusted context achieves the same
guarantee without that machinery.
Tests
- `recursor-rule-rhs-doctored-binder-domains`: a real compiled recursor
with its first two computation rules swapped AND every peeled binder
domain replaced by a Prop. Accepted before this change; rejected now.
- `control-swapped-recursor-rules`: the identical swap with the
compiler's own binder domains, which was already rejected. The two
differ only in the domains, so the acceptance was attributable to them
alone.
- Both are built by rewriting a REAL block rather than hand-authoring a
recursor, since a hand-built one must survive `build_rec_type`'s def-eq
and constructor references inside a block are circular with the block's
own address. The fixture selects a recursor whose rules are field-free,
so no minor premise is applied and the retyping cannot abort as a
non-function before the comparison is reached — an earlier attempt did
exactly that and masked the acceptance.
- A guard case fails loudly if no such recursor is found; the first
version of this fixture silently corrupted nothing, and both its cases
"passed" meaninglessly.
- 57 FFT pins and the shard pin move (`Nat.add_comm` 42_980_767 ->
42_980_801).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT DEMONSTRATED. Defense-in-depth, like the struct-eta guard.
`compare_rules` now asserts three invariants that equivalents of already
exist in `check_rec_rules_walk` — which nothing calls. The live path had
none of them.
* `cidx == pos`. `find_rule` selects a rule at reduction time BY LABEL
while `compare_rules` pairs stored against canonical POSITIONALLY, so
permuted labels would validate one constructor's rule and then fire
it for another. It holds today only because ingress assigns `cidx`
positionally (`Convert.lean`) and the Ixon `RecursorRule` record
carries no cidx of its own — an accident of the pipeline rather than
a checked property, which is exactly what an assert is for.
* `cidx < num_ctors`, so a rule cannot name a constructor that does not
exist.
* `nf` equals the constructor's field count. `try_iota` splits the
constructor's arguments with `field_start = ctor_fields_len -
rfields`, so a wrong `nf` hands the minor premise the wrong slice; in
field arithmetic an over-large value wraps.
The reachability argument for the `nf` gap was "combined with the rule
RHS context hole"; that hole is fixed, and with the def-eq comparison
restored a mismatched peel is caught again. So this closes a route that
is currently blocked by another check rather than one that is open —
worth asserting locally rather than depending on a second mechanism.
Not fixed here: iota still selects rules by index rather than by
constructor IDENTITY, so a recursor applied to a foreign constructor
would reduce. That is not reachable through a claim —
`k_infer_app_spine_loop` runs `k_check` on every argument, so the
application fails inference before whnf sees it — and closing it properly
needs the parent inductive available at reduction time, which
`KConstantInfo.Rec` does not carry (its `Addr` is the recursor's own
block; the parent is derived at check time from the type). That is a KCI
layout change, left for its own pass.
Tests
- No new fixture; nothing demonstrates the hole.
- Every recursor in the corpus satisfies all three invariants, including
the mutual, nested and aux-generated ones, so they held by construction
and are now enforced.
- 57 FFT pins and the shard pin move (`Nat.add_comm` 42_980_801 ->
42_981_336) for the per-rule constructor lookup.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT DEMONSTRATED. Defense-in-depth.
`try_iota` selected a computation rule by constructor INDEX alone:
`collect_spine_of_ctor` returned the major's `cidx` and discarded which
inductive it came from, so `find_rule(rules, cidx)` fired whenever any
inductive's constructor happened to carry a matching index. On its face
`Bool.rec motive f t (Nat.succ n)` reduces to `t`. Lean's kernel selects
the rule by the constructor's NAME, which pins the inductive.
Not reachable through a claim: `k_infer_app_spine_loop` runs `k_check` on
every argument, so a recursor applied to a foreign constructor fails
inference long before whnf sees the redex — the ill-typed term cannot be
stated. The Rust reference selects by index too, consistent with that
being why it was tolerated. What the patch removes is the DEPENDENCE on
that: reduction no longer needs inference to have run first to be
correct.
`collect_spine_of_ctor` now also returns the constructor's parent, as the
address of its projection wrapper, and `try_iota` asserts it equals the
recursor's own parent. That parent is read off the recursor's type (the
major premise's head) with the existing `rec_to_parent_addr`, so no
KConstantInfo layout change was needed — its `Addr` field is the
recursor's own block, not the parent's. Aiur memoizes per argument tuple,
so the type walk is paid once per recursor rather than once per
reduction.
Tests
- No new fixture; nothing demonstrates the hole.
- Every recursor in the corpus still reduces, including the mutual,
nested and aux-generated ones, and the Nat/String literal paths that
synthesize a constructor application before iota (`nat_cases_big`,
`nat_pow_big`, `strOfListFold*` all unchanged in behaviour).
- 63 FFT pins and the shard pin move (`Nat.add_comm` 42_981_336 ->
42_986_025, +0.01%) for the parent comparison.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`shardsCover` gates every shard run: "exit 0 has to mean every env const
was checked by some shard". It skipped constants whose bytes fail to
parse (`LazyConstant.get?` discards the error and `.ixe` loading is
lazy, so such a constant sits in `consts` with its key present). A
skipped constant was assigned to no shard, counted as neither owned nor
unowned, and still included in the "covers all N consts" total — so the
gate reported success over constants it had not covered.
It would also enter a referring shard's frontier, since `thin_frontier`
admits an edge on key presence in `env.consts` without parsing, leaving
an assumption no shard discharges.
Count them separately and fail, rather than folding them into `unowned`:
"cannot be assigned to a shard" is a different problem from a partition
gap, and saying so points at the right cause.
`ownedConstsForBlocks` keeps its skip — it runs per shard and would
otherwise need the same error plumbing — but now documents that it is
safe only because this gate rejects such an env first.
Not changed: the composed verdict still has no assumption-cycle check,
and on analysis it does not need one. Assumptions are discharged per
CONSTANT along ref edges, which are acyclic by construction — a
constant's address is blake3 over bytes containing its refs' addresses,
so a reference cycle needs a hash cycle. A cycle in the shard-level graph
still stands for a well-founded per-constant chain, and
`verifyShardComposition` recomputes each digest via `shardClaimDigest`,
so a self-assuming shard claim cannot be substituted.
`compile_univ` memoized `Level -> Univ`, but a `Level::Param` compiles to
`Univ::Var(i)` where `i` is the parameter's POSITION in `univ_params` —
so the result depends on a context the key omitted. Unlike the expression
cache, which is cleared at every constant boundary, `univ_cache` lives
for a whole block, and `compile_definition` compiles each member under
its own `level_params`. `preseed_expr_tables` explicitly feeds
`(expr, univ_params)` pairs with differing param lists into the same
cache.
So two members of one block placing the same parameter NAME at different
positions would give the second member the first's index. Not an abort:
it compiles cleanly and registers a constant denoting a different
universe than the Lean source, under a correct-looking name.
The hazard is already recognized one level up — `collect_expr_tables`
keys `seen_exprs` by `(expr_hash, univ_params_key)` for exactly this
reason. Apply the same key here. The Lean mirror instead drops the cache
in `withUnivCtx`, matching how it already clears the expression cache per
constant.
Unreachable from ordinary Lean sources, since the elaborator gives every
member of a `mutual` block the same `levelParams` — but nothing checked
it, and under the prover threat model the environment is authored.
Also repairs `claim.rs`'s `defn_const`, whose Definition never referenced
its own `refs`. Since the claim walk admits an index only when some Expr
uses it as a constant, that fixture produced no walk edges and
`shard_check_env_claim_is_thin_frontier` has been failing since the walk
moved to positive constant-use. The identical flaw was fixed in
`shard_claim.rs`'s helper at the time and missed here, because only
`ixon` and `ixvm-codegen` were re-tested rather than the whole workspace.
Tests
- `cargo test --release` across all crates: 1195 passed, 0 failed.
- `lake test -- --ignored ixvm` green with NO pin movement and no address
changes, confirming no block in the corpus has members that order a
shared parameter name differently. The fix removes the dependence on
that remaining true.
- clippy `--all-targets` and fmt clean.
@johnchandlerburnham
johnchandlerburnham merged commit 80bb41f into mainAug 3, 2026
10 checks passed
@johnchandlerburnham
johnchandlerburnham deleted the ap/ixvm4 branch August 3, 2026 05:37
samuelburnham added a commit that referenced this pull request Aug 4, 2026
…acking
Port of sb/measured-ingress's planning layer onto the addr-first kernel
(#529), reworked for its lazy fault-in semantics.
Profiling (.ixprof v3): the recording kernel's try_get_const consults are
drained per constant into a touch graph — the measured set a lazy checker
faults in — persisted as a presence-flagged CSR section alongside a new
block-level reference graph and a subst_unique counter (splitmix64-keyed
dedup of substitution work items: the post-memoization volume a memoizing
executor pays). Both graph sections are optional, so older readers'
formats stay decodable by version gate alone.
Planner: partition_for_aiur_ram bin-packs to a host-RAM cap with
faulted-set byte accounting — under #529 the witness always ships the
full closure, but only faulted bytes cost ingress trace rows, so a
shard's byte feature is owned ∪ touched(owned) (full reference closure
as the over-approximating fallback when no touch graph was recorded).
The stub/promotion machinery of the original branch is dropped entirely:
lazy fault-in computes at run time exactly the set the stubs predicted,
with no replay divergence and no repair ladder, and the manifest format
stays byte-identical with main (owned-only).
Cost models: nlogn(x) = x·log2(x+2) feature form matching Aiur's
width·height·log2(height) FFT structure; prove/execute time and RAM
coefficients carried over from the pre-#529 aiur bench-suite fits.
They are placeholders pending a refit against this kernel — the circuit
shrank (deleted positional machinery), the byte feature narrowed to the
faulted set, and the recording counters moved under #442 — with
provenance noted at the definitions. The costs sidecar
(<out>.costs.csv) persists per-shard features incl. subst_unique and
nat_arith so the refit has its corpus.
CLI: ix shard --backend aiur --max-ram G; ix profile --backend/--top
(per-metric block leaderboards with display names) and ix profile sweep
(env-wide closure cost sweep: per-root feature CSV + feasibility,
min-root-per-hot-block, and diversity reports).
samuelburnham added a commit that referenced this pull request Aug 8, 2026
…acking
Port of sb/measured-ingress's planning layer onto the addr-first kernel
(#529), reworked for its lazy fault-in semantics.
Profiling (.ixprof v3): the recording kernel's try_get_const consults are
drained per constant into a touch graph — the measured set a lazy checker
faults in — persisted as a presence-flagged CSR section alongside a new
block-level reference graph and a subst_unique counter (splitmix64-keyed
dedup of substitution work items: the post-memoization volume a memoizing
executor pays). Both graph sections are optional, so older readers'
formats stay decodable by version gate alone.
Planner: partition_for_aiur_ram bin-packs to a host-RAM cap with
faulted-set byte accounting — under #529 the witness always ships the
full closure, but only faulted bytes cost ingress trace rows, so a
shard's byte feature is owned ∪ touched(owned) (full reference closure
as the over-approximating fallback when no touch graph was recorded).
The stub/promotion machinery of the original branch is dropped entirely:
lazy fault-in computes at run time exactly the set the stubs predicted,
with no replay divergence and no repair ladder, and the manifest format
stays byte-identical with main (owned-only).
Cost models: nlogn(x) = x·log2(x+2) feature form matching Aiur's
width·height·log2(height) FFT structure; prove/execute time and RAM
coefficients carried over from the pre-#529 aiur bench-suite fits.
They are placeholders pending a refit against this kernel — the circuit
shrank (deleted positional machinery), the byte feature narrowed to the
faulted set, and the recording counters moved under #442 — with
provenance noted at the definitions. The costs sidecar
(<out>.costs.csv) persists per-shard features incl. subst_unique and
nat_arith so the refit has its corpus.
CLI: ix shard --backend aiur --max-ram G; ix profile --backend/--top
(per-metric block leaderboards with display names) and ix profile sweep
(env-wide closure cost sweep: per-root feature CSV + feasibility,
min-root-per-hot-block, and diversity reports).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@arthurpaulino@johnchandlerburnham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

IxVM: cheaper shard checking via an addr-first kernel, plus soundness hardening - #529

Merged
johnchandlerburnham merged 19 commits into
mainfrom
ap/ixvm4
Aug 3, 2026
Merged

IxVM: cheaper shard checking via an addr-first kernel, plus soundness hardening#529
johnchandlerburnham merged 19 commits into
mainfrom
ap/ixvm4

Conversation

@arthurpaulino

@arthurpaulinoarthurpaulino commented Jul 31, 2026

Copy link
Copy Markdown
Member

Motivation

Make sharded environment checking cheap. A large environment is partitioned across shards, and each shard proves its own constants well-typed against a thin-frontier assumption cutoff, so every constant is checked in exactly one shard and cross-shard assumption trees stay minimal.

Delivering that required migrating the IxVM Aiur kernel from a positional constant table to a content-addressed, lazily-faulted design — and then auditing the result, which turned up five soundness holes with working exploits, including three separate routes to proving False.

Sharded environment checking

A shard proves CheckEnv{root, assumptions}: root is a merkle root over the shard's owned constants, assumptions a merkle root over the thin frontier — the direct out-of-owned walk edges. The kernel's env_walk starts from the owned set, checks each reachable constant, and stops at a frontier member (assumed well-typed by its owning shard: not re-checked, not recursed into).

  • Sub-frontier constants are never re-checked here; they are covered transitively by the frontier members' own shard claims. This is the cost win over checking each constant's full closure: no cross-shard duplication, and the assumption tree is one layer deep instead of the whole closure.
  • Strict membership: a walk-reachable constant in neither owned nor the frontier is a bad partition and hard-errors, so a shard cannot silently drop coverage of its own closure.
  • Byte scope ≠ check scope: delta unfolding reads through the frontier, so a witness ships the full dependency closure of owned even though only the strict-membership set is checked. The witness IOBuffer is built in Rust (aiur_ixvm_witness) rather than Lean, removing the per-byte boxing that dominated shard wall time.

Addr-first kernel

  • KExprNode.Const / Proj key on a 32-byte content Addr, not a numeric position; KConstantInfoInduct/Ctor/Rec carry a (block_addr, ind_idx) parent pair instead of indices. The positional top: List<KCI> table threaded through the kernel is gone.
  • get_ci(addr) is the sole resolver, Aiur-memoized per addr, faulting constant bytes lazily from an IOBuffer. load_verified_constant re-derives blake3 over the bytes and asserts key == addr; the io_get_info idx/len are unconstrained witness and are never branched on.
  • IOBuffer channels: 0 claim, 1 assumption tree, 2 constant bytes, 3 Defn reducibility hint (advisory), 4 blob bytes. The kernel derives blob-vs-constant from Expr context, so no wire discriminator ships. The two sets are not disjoint — one address can be read both ways — so each channel is seeded independently.
  • Kernel/Inductive.lean and Kernel/Primitive.lean deleted (~6.6k lines); inductive validation absorbed into Kernel/Check.lean, primitives split into Kernel/Klimbs.lean (bignum gadgets) and Kernel/NatPrim.lean (Nat/Str addr dispatch). New Kernel/InferOnly.lean breaks the DefEq↔Infer cycle. Ingress.lean collapsed 1700→298 lines.

Recursor validation

  • build_rec_type reconstructs a recursor's canonical type and asserts it def-eq to the declared type, on both the kind-aware and address-only lookup paths. Without it a recursor could declare ... -> False while its rules still reconstructed correctly.
  • spec_params are lowered from the major-premise frame into the recursor-param frame at extraction: the root cause of 17 nested/aux recursor reconstruction failures, and why the canonical check accepted a superset.
  • k_flag and parent-inductive-shape checks hoisted out of the attacker-selectable is_aux gate; the canonical rules comparison, which formerly never ran, now runs for every recursor.

Pre-existing criticals

IDFix
P1blob-vs-constant derived from Expr context instead of a prover-answered io_get_info probe, retiring the ch-2 len=0 sentinel; a prover can no longer mark a walk root or owned leaf "skip me"
P2Nat.div n 0 pins the quotient to zero (was prover-chosen)
P3klimbs_sub/le corrected on operands with trailing zero limbs; div/mod hints normalized before use
P4is_large_eliminator field-index arithmetic parenthesized (Aiur - is right-associative)
P5Reveal recursor rules assert the claimed ruleIdx equals its position

Robustness

  • L5: UTF-8 string-literal decode validates continuation-byte ranges and per-length overlong minimums, so distinct byte sequences (e.g. overlong NUL C0 80 vs 00) no longer decode to the same codepoint and def-eq as equal.
  • L6: run_claim dispatches on the full tag4 size (was the low byte only), matching Rust Claim::get.
  • L8: the K-recursor synth gate checks the major's inferred inductive against the recursor's own, not against itself, so I.rec no longer strips a major of a different inductive.

Aiur assert_eq! messages and circuit cleanup

  • assert_eq!(a, b, "why") gains an optional diagnostic label, threaded Source.Term → Typed → Simple → Concrete → Bytecode.Op → Rust Op::AssertEq(_, _, Option<String>) → ExecError::AssertEqMismatch. The message is diagnostic-only and never enters the circuit; all ~122 IxVM assert sites are labeled.
  • Unreachable _ => assert_eq!(0, 1) match arms removed: an unmatched runtime value already aborts (the kernel's reject), so the extra arm only widened the circuit.
  • Address equality is a full 32-byte content comparison — never a pointer-value compare, which is unsound for distinct-pointer/same-data — packed as five base-256 field limbs behind a byte-0 prefilter.

Witness construction

witness_scope reuses Env::bfs_closure, which already follows refs, Prj → block, and the derived Muts → member/ctor projection edges, so the wrappers get_ci synthesizes ship as forward edges of a single O(closure) walk with no reverse scan of the env. Primitive addresses are seeded as additional walk roots, because def-eq delta-unfolds primitive bodies whose refs are reachable from nowhere else. The Lean seeder computes the same scope by the same rule, so the two cannot drift.

bench-typecheck's constants metric now reads check_const's unique query count — exactly the constants typechecked — instead of the shipped byte scope, which is a superset and inflated throughput.

Soundness audit

Eight parallel read-only audits covered the pipeline end-to-end (compile, Ixon serde, ingress/convert, type-theory core, declaration checking, claim/witness layer, primitives, entry points), under the zkDSL threat model: a prover authors the data and pays for its own trace, so aborts and rejections are not threats and only a wrong answer counts. Thirty findings; every claim relayed here was re-verified against the code before acting on it, and several agent claims were corrected or withdrawn in the process.

Fixed, with a working exploit

Each of these shipped with a fixture that was red before the fix and green after, paired with a minimal-pair control so the verdict is attributable to one variable.

HoleWhat it bought an attacker
Self-referential definitionstheorem bad : False := bad typechecked. A standalone Defn was handed a recur slot naming itself, Rec erases to an ordinary Const, and k_infer reads a Const's declared type without checking it — so the body discharged its own type by citing itself. Now restricted to unsafe/partial, which is where Lean permits self-reference and where safe code cannot reach
Nested-inductive positivityThe nested branch dropped the parameter arguments and descended into the nested inductive's constructors without substituting them, so the occurrence being hunted was never present. Host | mk : Inner Host → Host was accepted for a negative Inner. Admits False
Recursor rules compared under the prover's contextcompare_rules handed the stored rule's own binder domains to k_is_def_eq as the comparison context. Retyping them to a Prop makes both sides infer to that Prop, proof irrelevance fires, and a recursor with its computation rules swapped between constructors was accepted
Non-canonical computed Nat literalsString.utf8ByteSize built a KLimbs as one unchecked byte, and klimbs_div_mod pinned the quotient's value but not its digits. Since klimbs_eq compares limbs rather than values, the kernel had the polarity backwards: Nat.beq (String.utf8ByteSize "") 0 = false was accepted while the true equation was rejected
Tag0/Tag2 overlong payloadsThree readers, three answers on the same bytes: the circuit discarded bytes past the eighth, the Lean host folded them back in (its shift is mod 64), Rust rejected. One address denoted different constants to different readers

Fixed as defense-in-depth

No exploit demonstrated for these; each commit says so in its first line. They remove a dependence on some other check happening to hold.

  • Struct-eta type equality.try_eta_struct omitted the infer(t) ≟ infer(s) side condition all three references carry, degenerating to t ≡ S.mk … (Proj S i t) … for any t of any type.
  • Recursor rule labels.cidx selects a rule at reduction time but was validated only positionally; nf decides how iota splits the constructor's arguments and was never checked against the constructor. Equivalents existed only in dead code.
  • Iota constructor identity. Rules were selected by index alone, so any inductive's constructor with a matching index fired one. Unreachable through a claim — inference k_checks every argument first — but reduction should not depend on that.
  • Universe cache context.compile_univ memoized Level → Univ while a Level::Param compiles to a position in univ_params, and the cache outlives a constant. Now keyed by (level, univ_params_key), matching what collect_expr_tables already does one level up.

Entry points and coverage

  • verify_const / verify_check / verify_check_env are DEBUG entrypoints that trust their dependencies, and they shipped in the production verifying key alongside verify_claim — leaving ix verify's funcIdx check as the only barrier, untested, with no equivalent in the recursive verifier. The production toplevel is now pruned to verify_claim alone, so such a proof cannot exist under that VK. Zero FFT cost.
  • bench-typecheck reported "status": "ok" and exit 0 when verification failed, and dropped thrown constants without marking them failed.
  • shardsCover skipped constants whose bytes do not parse — assigning them to no shard, counting them as neither owned nor unowned, and still including them in "covers all N consts". Now a hard failure.

Test guards that could not fail

Four of these were found by auditing the tests themselves, and they matter because they were the safety net for everything above.

  • prim-addrs asserted the literal true. Keyed by address, so a permuted primitive table passed — precisely the defect its docstring claimed to catch. Now the address must resolve to the constant the function's name claims; verified by temporarily swapping nat_add_addr to Nat.mul's bytes and watching it go red.
  • The arena negative corpus counted any abort as a correct rejection, so a fixture aborting on invalid IO key — never reaching the code under test — scored as a pass. That is the same failure mode that made two exploit fixtures in this series pass for the wrong reason before it was caught.
  • The shard-pipeline SKIP was a passing test conflating "target absent" with "partition selected zero constants".
  • A --ignored filter naming no runner executed nothing and returned 0, so a typo'd suite name reported success having run no tests.

Adversarial Ixon test suite

New Tests/Ix/IxVM/Exploits.lean, 19 cases, wired into the ixvm suite. The kernel-arena corpus attacks the layer above: its bad_* fixtures are raw Lean.ConstantInfo declarations that skip the elaborator, but they are still compiled by rsCompileEnvFFI, so the Ixon they produce is always well-formed. This suite authors Ixon.Constant values directly, and storeRawAt goes one level lower still, storing hand-authored byte strings — needed because every value-level fixture is canonical by construction and so cannot express a malformed encoding.

Each case records the false statement acceptance would buy, so a failure reads as a security statement. Rejection reasons are printed, which earned its keep repeatedly: fixtures that "passed" via invalid IO key without reaching the checker were caught this way, as was a fixture that silently corrupted nothing because non-mutual recursors are standalone constants rather than Muts members.

Testing

  • lake test -- --ignored ixvm green.
  • cargo test --release across all crates: 1195 passed, 0 failed. This also repaired shard_check_env_claim_is_thin_frontier, red since the claim walk moved to positive constant-use because its fixture's Exprs never referenced their own refs.
  • New prim-addrs parity suite; codegen ↔ bytecode parity gate green; arena negative corpus (77 bad_*) all reject, now only via genuine in-kernel refusals.
  • FFT-cost pins re-pinned per change. Cumulative effect is small (Nat.add_comm 42_747_927 → 42_986_025, +0.6% over the whole series); the largest single contributors are the nested-positivity substitution and the struct-eta guard.
  • ix codegen regenerated whenever the Aiur kernel changed, so the committed Rust kernel never lags the source the parity gate compares against.

Known limitations

Recorded rather than silently dropped.

  • Call-site surgery is not covered by the value pin. The compiler rewrites calls to aux-generated constants — permuting arguments, dropping some, replacing the head and its universe list — and the plans driving it live only in ConstantMeta, which prune_to_closure_anon excludes. So a verified claim says the term at address X is well-typed, not that term is the Lean declaration you wrote; the equivalence rests on the rewrite being faithful, checked today only by decompile round-trip tests. Closing it is a claim-format decision, not a patch.
  • Assumption cycles are not rejected, and a conditional Check claim naming its own target is accepted. That is the specified semantics — some root means "conditional on every leaf", so the claim asserts A → A, vacuous rather than false — and the obligation sits with whoever discharges assumptions. Sharded CheckEnv is not exposed, since shardClaimDigest recomputes the claim from (env, blocks). Pinned as ACCEPT so the boundary cannot move silently.
  • Alpha-collapse comparators ignore safety, so two members agreeing on kind/lvls/typ/value and disagreeing on safety collapse. Deliberately not fixed: a prover authors the environment directly and never goes through the compiler, so hardening it stops no attacker, while adding safety to the ordering would move every address and refusing the collapse would put a hard error in the compiler on the strength of an unverified assumption.
  • klimbs_le / klimbs_eq are in the logical trusted base, not just the performance path: Nat.decLe mints Decidable.isTrue with a proof term whose correctness is asserted rather than checked, so one wrong verdict yields False.
  • check_inductive_shape_ctors omits validate_expr_well_scoped / k_ensure_sort / assert_safety that check_const's Ctor arm performs; only an entirely unreferenced constructor escapes those.

@arthurpaulino
arthurpaulinoforce-pushed the ap/ixvm4 branch 4 times, most recently from 06e0489 to 559ced9CompareAugust 1, 2026 00:53
… hardening
Make sharded environment checking cheap. A large environment is partitioned
across shards, and each shard proves its own constants well-typed against a
thin-frontier assumption cutoff, so every constant is checked in exactly one
shard and cross-shard assumption trees stay minimal. Delivering that required
migrating the IxVM Aiur kernel from a positional constant table to a
content-addressed, lazily-faulted design -- and closing the soundness
regressions that migration introduced along with several pre-existing holes.
Sharded environment checking
- A shard proves `CheckEnv{root, assumptions}`: root is a merkle over the
shard's owned constants, assumptions a merkle over the THIN FRONTIER --
the direct out-of-owned walk edges. The kernel's env_walk starts from the
owned set, checks each reachable constant, and stops at a frontier member
(assumed well-typed by its owning shard, not re-checked, not recursed).
- Sub-frontier constants are never re-checked here; they are covered
transitively by the frontier members' own shard claims. This is the cost
win over checking each constant's full closure: no cross-shard duplication,
and the assumption tree is one layer deep instead of the whole closure.
- Strict membership: a walk-reachable constant in neither owned nor the
frontier is a bad partition and hard-errors, so a shard cannot silently
drop coverage of its own closure.
- Byte scope != check scope: delta unfolding reads through the frontier, so
a witness ships the full walk closure of owned even though only the
strict-membership set is checked. The witness IOBuffer is built in Rust
(aiur_ixvm_witness) rather than Lean, removing the per-byte boxing that
dominated shard wall time.
Addr-first kernel
- KExprNode.Const/Proj key on a 32-byte content Addr, not a numeric
position; KConstantInfo Induct/Ctor/Rec carry a (block_addr, ind_idx)
parent pair instead of indices. The positional `top: List<KCI>` table
threaded through the kernel is gone.
- get_ci(addr) is the sole resolver, Aiur-memoized per addr, faulting
constant bytes lazily from an IOBuffer. load_verified_constant re-derives
blake3 over the bytes and asserts key == addr; the io_get_info idx/len
are unconstrained witness and are never branched on.
- IOBuffer channels: 0 claim, 1 assumption tree, 2 constant bytes, 3 Defn
reducibility hint (advisory), 4 blob bytes. An address is on ch2 iff a
constant and ch4 iff a blob; the kernel derives which from Expr context
(Str/Nat -> blob, Ref/Prj -> constant), so no wire discriminator ships.
- Kernel/Inductive.lean and Kernel/Primitive.lean deleted (~6.6k lines);
inductive validation absorbed into Kernel/Check.lean, primitives split
into Kernel/Klimbs.lean (bignum gadgets) and Kernel/NatPrim.lean (Nat/Str
addr dispatch). New Kernel/InferOnly.lean breaks the DefEq<->Infer cycle.
Ingress.lean collapsed 1700->298 lines. V1/V2 naming removed throughout.
Recursor validation (R1, R2, R3)
- build_rec_type reconstructs a recursor's canonical type and asserts it
def-eq to the declared type, on both the kind-aware and address-only
lookup paths. Without it a recursor could declare `... -> False` while
its rules still reconstructed correctly.
- spec_params are lowered from the major-premise frame into the
recursor-param frame at extraction: the root cause of 17 nested/aux
recursor reconstruction failures, and why the canonical check accepted a
superset.
- k_flag and parent-inductive-shape checks hoisted out of the
attacker-selectable `is_aux` gate; the canonical rules comparison, which
formerly never ran, now runs for every recursor.
Pre-existing criticals (P1-P5)
- P1: blob-vs-constant is derived from Expr context instead of a prover-
answered io_get_info probe, retiring the ch-2 len=0 sentinel; a prover
can no longer mark a walk root or owned leaf "skip me".
- P2: Nat.div n 0 pins the quotient to zero (was prover-chosen).
- P3: klimbs_sub/le corrected on operands with trailing zero limbs; div/mod
hints normalized before use.
- P4: is_large_eliminator field-index arithmetic parenthesized (Aiur `-` is
right-associative).
- P5: Reveal recursor rules assert the claimed ruleIdx equals its position.
Robustness (L5, L6, L8)
- L5: UTF-8 string-literal decode validates continuation-byte ranges and
per-length overlong minimums, so distinct byte sequences (e.g. overlong
NUL C0 80 vs 00) no longer decode to the same codepoint and def-eq as
equal. Roughly zero added FFT cost.
- L6: run_claim dispatches on the full tag4 size (was the low byte only),
matching Rust Claim::get.
- L8: the K-recursor synth gate checks the major's inferred inductive
against the recursor's own (via se_parent_addr), not against itself, so
I.rec no longer strips a major of a different inductive.
Aiur assert_eq! messages + circuit cleanup
- assert_eq!(a, b, "why") gains an optional diagnostic label, threaded
Source.Term -> Typed -> Simple -> Concrete -> Bytecode.Op -> Rust
Op::AssertEq(_, _, Option<String>) -> ExecError::AssertEqMismatch. The
message is diagnostic-only and never enters the circuit; all ~122 IxVM
assert sites are labeled.
- Unreachable `_ => assert_eq!(0, 1)` match arms removed: an unmatched
runtime value already aborts (the kernel's reject), so the extra arm only
widened the circuit.
- address equality is a full 32-byte content comparison (never a pointer-
value compare, which is unsound for distinct-pointer/same-data), packed
as five base-256 field limbs (7+7+7+7+4) behind a byte-0 prefilter and
short-circuiting on the first mismatched group.
Crate reorganization
- PrimAddrs / reserved_marker_name moved verbatim from
crates/kernel/src/primitive.rs to crates/common/src/prim_addrs.rs (kernel
re-exports), so witness builders reach the primitive address table
without an ix-kernel -> ixon -> ix-kernel cycle.
- witness_scope reuses Env::bfs_closure -- the env's own dependency-closure
pass, which already follows refs, Prj->block, and the derived
Muts->member/ctor projection edges -- so the wrappers the kernel's get_ci
synthesizes ship as forward edges of a single O(closure) walk, with no
reverse scan of the whole env. Primitive-address seeding is unioned on
top: primitives are fabricated during reduction and need not appear in a
target's ref closure, so a closure-only scope aborts with `invalid IO
key`.
Tests
- New prim-addrs parity suite (Lean<->Aiur primitive address table).
- Codegen<->bytecode parity gate restored, green (198/198).
- FFT-cost pins re-pinned to current behavior; arena negative-fixture
corpus (77 bad_* constants) all reject.
…onstants
closureFromBase now walks the derived Muts->member/ctor projection
wrapper addresses as forward edges (projWrapperAddr: blake3 over the
serialized proj-only Constant, compile's construction), so closureFrom
and buildShardCheckEnvWitness drop their O(env) reverse scans that
parsed every env constant per witness build. Primitive addresses are
seeded bytes-only after the walk instead of being pushed onto the
worklist, so their transitive closures no longer inflate the byte scope
(a delta-unfold into an unshipped primitive body aborts with invalid IO
key rather than proving anything wrong).
bench-typecheck's constants metric now reads check_const's unique query
count (memoized per (ci, addr) = exactly the constants typechecked)
instead of closureFrom.size, which counted the shipped byte scope.
check_const is resolved up front and both lookups fail with IO.userError
instead of panicking.
Experiment: lake exe bench-typecheck --ixe seed.ixe --consts
Nat.add_comm --execute-only (seed.ixe = ix compile Tests.lean --consts
Nat.add_comm) reports constants=42, matching check_const's height in
lake exe ix check Nat.add_comm; fft-cost unchanged at 42747927.
The claim walk decided which of a constant's `refs` to recurse into by
SKIPPING every index the constant's Exprs used as a blob payload
(`Str(i)` / `Nat(i)`). That rule is unsound: the Exprs are authored by
the prover, and `refs` is one index space shared by `Ref`, `Prj`, `Str`
and `Nat`, so marking an index suppresses the walk of that index used as
a constant everywhere else in the same constant.
The marking need not even be live. `blob_idxs_of` traversed `sharing` in
full, but conversion reaches a `sharing` entry only through an
`Expr.Share` node, and nothing validates the array. A dead `Nat(i)`
entry is therefore free: it is never converted, never typechecked, and
needs no blob on channel 4. It just hides `refs[i]` from the checker.
`k_infer` then reads that dependency's DECLARED type without ever
checking it, so the kernel accepts a constant whose transitive closure
is ill-typed -- a wrong answer, not an abort. Both claim variants were
affected; for CheckEnv the skip happens before `env_walk` is entered, so
the strict owned-membership assert never fires either.
Replace the family with `const_idxs_*`: collect the indices `Ref(i, ..)`
and `Prj(i, ..)` use, and walk `refs[i]` iff `i` is among them. `Rec`
resolves through `recur_addrs` and a projection's block comes from the
ConstantInfo payload, both walked separately, so the positive set is
complete. Same single traversal as before, so no added cost. The failure
direction is now safe: over-approximating walks an address that faults
an unseeded key and aborts, which only costs a dishonest prover.
Adversarial Ixon test infrastructure
- New `Tests/Ix/IxVM/Exploits.lean`, wired into the ixvm suite. The
kernel-arena corpus attacks the layer above this one: its `bad_*`
fixtures are raw `Lean.ConstantInfo` declarations that skip the
elaborator, but they are still compiled by `rsCompileEnvFFI`, so the
Ixon they produce is always well-formed. This suite authors
`Ixon.Constant` values directly, which is what a prover can do.
- Each `ExploitCase` records the false statement acceptance would buy,
so a failure reads as a security statement. Rejection reasons are
printed: a fixture that aborts before reaching the mechanism under
attack is indistinguishable from a kernel that correctly refuses it,
and the first run of this fixture passed for exactly that wrong reason
(`invalid IO key` from an unseeded channel-3 hint, now seeded by
`storeAt`).
- The exploit ships with two controls that differ from it minimally: one
checking the smuggled dependency alone, one checking the same target
without the dead `sharing` entry. Both reject via the intended
in-kernel assert, so the exploit's acceptance was attributable to the
`sharing` entry alone.
- Two further cases probe what the fix does NOT cover. An index used as
both a `Nat` literal and a `Ref` — reachable because blob and constant
addresses share one un-domain-separated blake3 space and
`load_verified_blob` parses nothing — is now walked and rejected,
though by channel fault rather than type error, which is the safe
direction. A conditional Check claim naming its own target as its own
assumption IS accepted, and is pinned that way: `some root` means
"conditional on every leaf being well-typed", so the claim asserts
`A -> A` and is vacuous rather than false. The obligation sits with
whoever discharges assumptions, which must reject such cycles; sharded
CheckEnv is not exposed, since `shardClaimDigest` recomputes the claim
from (env, blocks).
Primitive closures are walked again
- The previous commit seeded primitive addresses as bytes only, without
walking their closures. That is too aggressive: def-eq delta-unfolds
primitive BODIES, and a body's refs are reachable from nowhere else in
the scope, so the `strAppend*` arena fixtures aborted with `invalid IO
key`. Both seeders now pass the primitives as walk ROOTS
(`closureFromRoots` in Lean, `bfs_closure` over target+primitives in
Rust); the Rust side had the same gap latently, since only the Lean
path had arena coverage.
- The removal of the O(env) reverse wrapper scan, which was that
commit's actual win, is unaffected.
Tests
- `lake test -- --ignored ixvm` green. FFT pins re-pinned across the 66
kernel-check entries and the shard pipeline; costs move in both
directions (e.g. `Eq.rec` 1_249_383 -> 1_249_094, `Vector.append`
1_994_697_378 -> 2_010_055_088), the increases coming from the
restored primitive closures.
- `ix codegen` regenerated: the Aiur kernel changed, so the committed
Rust kernel would otherwise be stale and the codegen/bytecode parity
gate would compare against it.
`Expr.Str` conversion wrapped any hash-valid blob in `KLiteral.Str`, and
`k_infer_lit` types a `KLiteral.Str` as `String` without inspecting the
bytes. Nothing on the ordinary checking path ever decodes a literal, so a
prover-authored definition carrying malformed bytes typechecked as
`x : String` even though no Lean `String` corresponds to those bytes. The
reference kernel rejects this at ingress via `String::from_utf8`
(crates/kernel/src/ingress.rs).
The UTF-8 validation added earlier lives in the string DECODER, which is
reached only during string reductions, so it did not cover this path.
`utf8_decode_one` already rejects out-of-range continuation bytes, stray
continuations used as leaders, and overlong forms. Walk the whole blob
with it at conversion (`utf8_validate`), which is the same rejection the
reference kernel performs.
Tests
- `invalid-utf8-accepted-as-string` in the adversarial Ixon suite: a
definition declaring `String` over the blob `[0xFF]`, which no
well-formed UTF-8 can contain. Accepted before this change, now
rejected.
- Paired with `control-valid-utf8-string-literal`, the same shape over
`[0x68, 0x69]`, which must still be accepted — so the verdict is
attributable to the bytes rather than to the fixture's shape. The
fixtures build on a base env supplying the real `String` constant, so
the declared type is the genuine one.
- Six FFT pins move, all string-touching, by between +17 and +66_530
(e.g. `IxVMPrim.str_size_lit` 567_999_042 -> 568_000_142,
`Vector.append` 2_010_055_088 -> 2_010_088_729, +0.0017%). Constants
with no string literals are unchanged, `Nat.add_comm` included.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`add_entries_parallel` returned early after a constant hit, so an address
present in both `env.consts` and `env.blobs` was seeded on channel 2 but
never on channel 4. The Lean seeder walks `consts` and `blobs` in
separate passes and seeds both, so the two builders disagreed, and the
Rust-native check/prove paths aborted with `invalid IO key` on such an
environment.
The two tables are not disjoint. Constant and blob addresses share a hash
codomain, and it is the Expr context — `Ref`/`Prj` versus `Str`/`Nat` —
that says which table a reference means, so the same bytes can legitimately
be read both ways. The kernel faults the two channels separately, so the
witness has to seed them separately. Reaching an address through both
kinds of use became expressible when the claim walk moved to selecting
indices by positive constant-use.
Corrects the module header and the `ChannelEntries` doc, which both
asserted the disjointness this relies on not having.
Tests
- `dual_use_address_is_seeded_on_both_channels`: an address stored as a
constant and as a blob must appear under both `(ch 2, key)` and
`(ch 4, key)` in the built IOBuffer. Fails on the previous code with
the ch-4 entry missing.
- `lake test -- --ignored ixvm` green; clippy `--all-targets` and fmt
clean.
The kernel walks a constant's `refs` by positive constant-use — index `i`
is followed only when some Expr names it via `Ref(i, ..)` or `Prj(i, ..)`
— but both host claim builders still treated every `refs` entry as a walk
edge. The host and the kernel therefore disagreed about the edge
relation.
A `refs` entry that is a real constant but that no Expr uses as one is
never walked in-circuit, so admitting it host-side puts a phantom member
in the thin frontier: a cross-shard assumption the kernel never makes,
and a spurious dependency edge that can close a cycle once assumptions
are discharged. `walk_check_set` likewise stopped computing the set the
verifier actually checks.
Mirror `const_idxs_of` on both hosts and emit an edge only for
positively-used indices. `Env::bfs_closure` is deliberately left alone:
it computes the witness BYTE scope, where over-approximating is correct
(shipping unused bytes is free, missing bytes aborts).
Honest constants are unaffected, since a compiler only puts an address in
`refs` when an Expr references it — the shard FFT pin is unchanged. The
divergence is reachable only through prover-authored `refs`.
Tests
- `thin_frontier_excludes_unused_ref_entries`: `refs` carries two
constants but only index 0 is named by a `Ref`; the frontier must
contain the used one and not the dead one. Verified red against the
previous edge relation ("unused refs entry became a phantom frontier
member").
- `const_with_refs` now names each of its `refs` entries via a `Ref`
node. The old helper left them unreferenced, which under the walk rule
would have produced no edges at all and quietly hollowed out the
existing frontier tests.
- `lake test -- --ignored ixvm` green; clippy `--all-targets` and fmt
clean.
`idx_to_u64` packed a synthesized member/constructor index into the
projection wrapper's `u64` field by populating only the low byte behind a
`u8_range_check`, so a mutual block with more than 256 members — or an
inductive with more than 256 constructors, which is the likelier case —
was rejected outright with `value 256 out of u8 range`.
Field-to-bytes is not free: the bytes have to be supplied as advice and
then pinned. Take them from `unconstrained_g_to_bytes` and discharge the
three obligations its contract names:
- range: four `u8_range_check` calls cover the eight bytes, the byte chip
taking a pair per lookup row;
- recomposition and canonicality: `flatten_u64(bytes) == idx`, since
`flatten_u64` asserts the top byte is zero and sums the low seven.
Canonicality is what makes the decomposition unique, and it is enforced
by RESTRICTING THE DOMAIN rather than by comparing against `p`. Goldilocks
is 2^64 - 2^32 + 1, so an arbitrary eight-byte string is not injective
into the field: any value and that value plus `p` recompose to the same
element, e.g. `[00]*8` and `[01,00,00,00,FF,FF,FF,FF]` both give 0.
Forcing the top byte to zero bounds the value by 2^56 - 1 < `p`, so the
sum cannot wrap and exactly one string encodes each index. That
uniqueness is the soundness property: the projection address is blake3
over these serialized bytes, so a prover free to choose a second string
for one index would obtain a second address for a member, or collide two
members onto one address. The pre-existing `u8_from_field_unsafe`
version did collide members 0 and 256, which the range check had already
fixed by rejecting instead.
Both halves are load-bearing. `flatten_u64` does NOT range-check its
input — `[U8; 8]` is nominal typing and `u8_from_field_unsafe` mints a
`U8` holding any field element — so without the range checks the sum is
satisfiable many ways (`b1 = k`, `b0 = idx - 256k`). Its doc comment now
records that its injectivity is conditional on the caller, and names this
call site.
Indices at or beyond 2^56 still fail the assert rather than truncating.
That is a liveness bound only, 2^48 times the previous cap and far past
any real block or constructor count; buying the remaining `[2^56, p)`
would cost a multi-byte comparison against `p` for no practical gain.
Tests
- `muts-block-past-single-byte-index`: a 257-member block of well-typed
definitions with a projection wrapper for each, which now checks.
Rejected before this change with `value 256 out of u8 range`.
- `control-small-muts-block`: the same construction at two members,
already accepted — so the big-block result is attributable to the index
width alone and not to a malformed fixture.
- `storeAt` also registers a reducibility hint for `DPrj` wrappers, which
`get_ci_dprj` faults under the wrapper's own address; without it these
fixtures aborted with `invalid IO key` before reaching the index logic.
- All 66 kernel-check FFT pins and the shard pin move by +17 to +485
(`Nat.add_comm` 42_967_279 -> 42_967_313, +0.00008%), the cost of the
hint plus four range checks per synthesized index.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`get_ci` handed every standalone `Defn` a one-element `recur_addrs` whose
only entry was its own address. `Rec` converts to a plain `Const`, and
`k_infer` discharges a `Const` against its DECLARED type without checking
it, so a definition could prove its own type by citing itself:
theorem bad : False := bad
typechecked. The ref-walk cannot catch it — `const_idxs_expr` does not
follow `Rec`, and the referent is the constant already under check. Lean
forbids exactly this by checking a definition against an environment that
does not yet contain it, admitting self-reference only for `unsafe` and
`partial`.
Restrict the self slot to `unsafe`/`partial`. Those are where
self-reference is both legitimate and contained:
- legitimate, because the compiler emits a singleton (or fully
collapsed) mutual block as a standalone `Defn` whose body still says
`Rec(0)` -- see the note at Ix/CompileM.lean:2000. Those blocks are
exactly Lean's `unsafe`/`partial` recursive definitions; a `safe`
recursive definition compiles its recursion into `.rec` applications.
Gating on `unsafe` alone breaks `Lean.extractMainModule._unsafe_rec`.
- contained, because safe code may not reference either kind.
That containment was itself missing for `partial`: `is_unsafe_ci`
classified it as safe, so `partial def bad : False := bad` plus a safe
theorem referencing `bad` would have walked straight around the first
half of this fix. Count `Partial` alongside `Unsafe`, matching the
reference's "safe definition references partial definition" rejection
(crates/kernel/src/check.rs:813-820). The predicate serves double duty --
the caller's own leniency and the referent's barrier -- and `Partial`
belongs on the non-safe side of both.
Tests
- `self-referential-theorem-proves-false`: `theorem bad : False := bad`
built directly in Ixon over the real `False` constant, submitted
through the production `verify_claim` entrypoint. Accepted before this
change -- a proof of False -- and rejected after.
- No safe constant in the corpus references a partial definition, and
the `_unsafe_rec` witnesses still check, so neither half of the fix
costs coverage.
- All 66 kernel-check FFT pins and the shard pin move (`Nat.add_comm`
42_967_313 -> 42_965_666); the self slot is no longer built for safe
definitions, so most costs drop slightly.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
The nested branch of the strict-positivity check dropped the parameter
arguments (`list_drop(args, n_ctor_params)`) and descended into the
nested inductive's constructors without substituting them. A nested
constructor's fields mention that inductive's parameters as bound
variables, so the block-occurrence test asked whether the block occurs in
`BVar 0` — it never does, and every parameter position passed vacuously.
The occurrence being hunted lives in the ARGUMENT substituted for the
parameter, which was exactly what got discarded.
So for `Inner α | mk : (α → False) → Inner α`, itself strictly positive,
`Host | mk : Inner Host → Host` was accepted: descending into `Inner.mk`
sees the field `α → False`, which mentions nothing tracked. With `α :=
Host` it reads `(Host → False) → …`, a negative occurrence, and Lean
rejects it. Accepting it admits `False`.
Mirror `check_nested_ctor_fields` (crates/kernel/src/inductive.rs): strip
`n_params` foralls, then simultaneously substitute the parameter
arguments at depth 0. After stripping, `BVar 0` is the innermost (last)
parameter, so the argument list is reversed — `expr_inst_many` maps
`substs[i]` to `BVar(depth + i)`, the same convention as `simul_subst`.
The parameters are substituted away rather than bound, so the field walk
now starts with an empty binder context instead of the peeled types.
Tests
- `nested-inductive-negative-occurrence`: the `Inner`/`Host` pair above,
built directly in Ixon as two single-inductive `Prop` blocks with their
projection wrappers. Accepted before this change, and now rejected by
the intended assert ("strict positivity: block occurs left of an
arrow").
- `control-nested-positive-inductive`: the same construction with
`Inner α | mk : α → Inner α`, which is strictly positive and must keep
checking. The two fixtures differ only in the inner constructor's
field, so the rejection is attributable to the negative occurrence
rather than to nesting.
- 63 FFT pins and the shard pin move, concentrated in inductive-heavy
constants (`IxVMInd.Tree` 767_167 -> 796_717, `IxVMInd.DepthM`
1_312_461 -> 1_347_394) — the nested check now does real work instead
of returning vacuously.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`KLimbs` is canonical in two independent ways — no trailing zero limbs,
and every limb byte in [0, 256) — and `klimbs_eq` is a raw limb compare
that establishes neither. Since it backs `Nat.beq`, `Nat.decEq` and
`literal_eq`, any producer emitting a value-correct but
representation-wrong `KLimbs` makes those answer `false` where Lean
answers `true`. That is an accepted false equation, not an abort. Two
producers did.
`String.utf8ByteSize` built the length as one unchecked byte
(`[u8_from_field_unsafe(n), 0, ...]`) with no normalization. The empty
string therefore reduced to the denormalized zero `[[0;8]]` instead of
the canonical empty limb list, and a string of 256 bytes or more to an
out-of-range digit — `u8_from_field_unsafe` is an erased coercion, so
nothing truncated or rejected. Build it with `klimbs_from_g`, whose byte
split is range-checked and pinned by a recomposition assert, then
normalize.
`klimbs_div_mod` pinned the quotient's VALUE (`q*b + r == a`, `r < b`)
but not its digits. The comment claimed `u8_mul` inside `klimbs_mul`
range-checked them, but `u64_mul` was rewritten to raw field products
plus `#split_carry`, whose u8 checks constrain the split outputs rather
than the input digits — and it re-canonicalizes while multiplying, so a
digit-wrong `q` still produces a canonical `q*b` and satisfies the
equality. Range-check the hint's limbs explicitly, quotient and
remainder both, via a new `klimbs_range_check`.
Tests
- `utf8-size-beq-false-accepted`: `Nat.beq (String.utf8ByteSize "") 0 =
false` proved by `Eq.refl`, over the real `Eq`/`Bool`/`Nat.beq`/
`String.utf8ByteSize` constants. The kernel had the polarity exactly
backwards — this FALSE equation was accepted while the true one was
rejected.
- `control-utf8-size-beq-true`: the same declaration ending in `true`,
which is what Lean computes and must be accepted. The pair differs only
in that operand, so each verdict is attributable to the reduction.
Both now land the right way round.
- The two fixtures share one blob for `""` and `0` — the same empty byte
string read as a string literal and as a nat literal, which is the
Expr-context disambiguation working as intended.
- 10 FFT pins and the shard pin move, confined to the div/mod/gcd/pow/
shift constants (the hint range-checks) and the string-size ones
(`IxVMPrim.str_size_lit` 567_992_937 -> 567_993_099).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT WAS DEMONSTRATED for this one. Unlike the other soundness
fixes in this series it ships without a red-then-green fixture; what
follows is why the rule is wrong as written, and what remains unproven.
`try_eta_struct` accepted `t ≡ S.mk p… a…` after checking only that the
right side is a fully-applied constructor of a struct-like inductive. It
never checked that the two sides have def-eq TYPES. All three references
do, citing lean4lean's `tryEtaStructCore`
(crates/kernel/src/def_eq.rs, Ix/Tc/DefEq.lean).
Without the guard the rule degenerates: for ANY `t` of ANY type,
`compare_struct_fields` builds precisely `Proj(S, i, t)` for each field,
so when the right side's arguments are those same projections the
pointer fast path matches them and the pair is accepted. Those
projections are synthesized inside def-eq and never inferred, so
`k_infer_proj`'s head-address gate — which would reject `Proj S i t` for
`t : Nat` — never sees them.
What is unproven is reachability. At the `k_check(e, expected)` boundary
both sides are types in the same context, and `de_args` compares type
arguments before value arguments, so I could not construct a term that
reaches this rule with genuinely mismatched types. The change is
therefore defense-in-depth against an unjustified rule, not a repair of a
demonstrated break.
The guard runs after the struct-like shape test, so it costs two
`k_infer_only` calls and one `k_is_def_eq` only on pairs that were about
to be accepted. `try_proof_irrel`, earlier in the same fallback chain,
already infers the left side unconditionally, so the added abort exposure
is small; the suite confirms no constant stops checking.
Tests
- No new fixture; the existing 14 adversarial cases and the arena corpus
are unchanged.
- 45 FFT pins and the shard pin move, the largest cost of this series
(`Nat.add_comm` 42_965_759 -> 42_966_970, +0.003%;
`IxVMPrim.str_size_lit` +26_687).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
A `Tag0` large-form header carries a 7-bit `small` field, so a payload
width up to 128 was expressible and unguarded (`Tag2`: 5 bits, up to 32).
A u64 payload is at most 8 bytes, and the three readers disagreed on the
surplus:
- Rust `u64_get_trimmed_le` rejects `len > 8` -- already correct, and
untouched here.
- The in-circuit `get_u64_le` consumed the surplus and DISCARDED it,
keeping only the first eight bytes.
- The Lean host folded it back in: `UInt64.shiftLeft` is taken mod 64,
so byte 8 ORs into bits 0-7, with no length guard.
Same bytes, same content address, three readings. An `Expr.Ref` whose
index is written overlong resolves as `refs[0]` in-circuit and `refs[42]`
on the Lean host, with both consuming the same span so nothing desyncs
afterwards. That also means the trailing-bytes assert does not imply
canonicity: up to 120 arbitrary bytes per `Tag0` site were consumed
inside the structure and dropped.
Bring the other two up to Rust: assert the width is under 9 in both
in-circuit readers, and give `getU64TrimmedLE` the guard it was missing.
No honest serializer is affected -- `putTag0` always emits the minimal
width -- so this narrows the accepted language to the format.
Tests
- `overlong-tag0-accepted`: an axiom `_ : Prop` whose `lvls` Tag0 is
rewritten to the nine-byte form. Accepted before this change, now
rejected by the intended assert.
- `control-canonical-tag0-axiom`: the same axiom with canonical bytes,
which must stay accepted. The pair differs only in that encoding.
- Both need bytes no `Ixon.ser` will emit, so the suite gained
`storeRawAt`, which stores a hand-authored byte string at its own
content address. Every previous fixture was built from a `Constant`
VALUE and was therefore canonical by construction, and structurally
could not reach this class of defect.
- All 66 kernel-check FFT pins and the shard pin move (`Nat.add_comm`
42_966_970 -> 42_980_767, +0.03%); every constant parses Tag0s.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
Four test-only fixes. No kernel code changes, so no FFT pins move.
prim-addrs asserted the literal `true`. It looked up each hardcoded
kernel address in the canonical table and, on a hit, recorded a pass —
but the table is keyed BY ADDRESS, so a permuted table still hits. Had
`nat_add_addr` held `Nat.mul`'s bytes it would have printed
"nat_add_addr = Nat.mul" and passed green, which is precisely the defect
the file's own docstring says it exists to catch: the kernel applying one
primitive's native semantics under another's name. Now the address must
resolve to the constant the function's NAME claims, comparing normalized
keys (`nat_add_addr` <-> `Nat.add`, and the final-component spelling for
`reduce_bool_addr` <-> `Lean.reduceBool`), with dispatch-site suffixes
(`_dec`/`_io`/`_iota`) and `_type`/`_ty` stripped, plus a short exception
list for the `Quot` constructors. Verified by temporarily swapping
`nat_add_addr` to `Nat.mul`'s bytes: red, then reverted. All 76 literals
pass, and the table is confirmed correct today.
The arena negative corpus counted ANY abort as a correct rejection while
its docstring claimed rejections come from an in-kernel `assert_eq!`. An
`invalid IO key` — the witness never supplied the bytes, so execution
died before reaching the logic under test — scored as a pass. That is how
a fixture rots into a green test that checks nothing, and it is the same
failure mode that made two exploit fixtures in this series pass for the
wrong reason. Now `invalid IO key` disqualifies. Unmatched-match aborts
still count: this kernel deliberately relies on an unmatched runtime
value aborting instead of spending width on `_ => assert_eq!(0, 1)`
arms, so those are genuine verdicts. No fixture currently relies on the
disqualified path.
`shardCheckEnvCase` returned `none` for two unrelated reasons and the
runner reported both as "SKIP (target absent)", a passing test. The
second — target present but the partition selected zero owned constants —
means the fixture stopped exercising the Rust witness builder and the
thin-frontier claim convention, and would have left the only test of that
path permanently green. It now throws; `none` keeps its one honest
meaning.
A `--ignored` filter naming no runner ran nothing and returned 0, so
`lake test -- --ignored ixvm2` reported success having executed nothing.
It is now an error. Exit 0 should mean tests ran and passed.
`ixVM`, the pruned production toplevel, carried `verify_const`,
`verify_check` and `verify_check_env` alongside `verify_claim`. Those are
DEBUG entrypoints that trust their dependencies instead of walking them,
so a proof made with one attests far less than a claim does — and
carrying them meant such a proof was verifiable under the SAME verifying
key. The only thing standing between it and acceptance was `ix verify`
pinning the funcIdx to `verify_claim`: a policy, enforced in one place,
with no test asserting it, and no policy at all in the recursive
verifier, which binds the claim bytes but applies no funcIdx check.
Prune to `verify_claim` alone. The guarantee is now structural: under the
production VK a debug-entrypoint proof cannot exist. No CLI path used
them, so nothing in `ix check`/`prove`/`verify` changes.
Callers that genuinely want a subject-only check build `ixVMFull` and are
thereby using a different VK, which is the honest signal — the arena
corpus, and `bench-typecheck`/`bench-recursion-debug` under
`--skip-deps`. That last one also means a `--skip-deps` benchmark is now
visibly measuring a different key than a claim run rather than silently
reporting `verify_const` rows in the claim schema.
`bench-typecheck` also reported success on failure, twice over. A proof
that failed to verify printed to stderr but left `failed` clear, so the
row was emitted as `"status": "ok"` with prove-time and proof-size
present and the run exited 0 — the only signal an absent `verify-time`
key, invisible to anything keying on status. And a constant whose execute
threw was dropped from the results entirely without setting `failed`,
so `execed.any (·.failed)` stayed clear; the `check_const`-missing throw
added earlier landed exactly there. Both now mark the row failed.
Tests
- `lake test -- --ignored ixvm` green with NO pin movement: the pruned
entrypoints were never executed, so removing them changes no cost.
Generated kernel drops from 733 to 731 functions.
- Arena repointed to `ixVMFull`, where `verify_const` still lives.
- `bench-typecheck --execute-only` smoke unchanged (`Nat.add_comm`,
constants=42).
`compare_rules` peeled the binders off the STORED rule and handed their
DOMAINS to `k_is_def_eq` as the local context for the comparison against
the canonical reconstruction. Those domains are prover-authored: nothing
checks them against the reconstruction, and reduction never consults them
(`try_iota` beta-reduces straight through). So the prover chose what the
two sides INFER to.
Retyping every binder to a Prop makes both sides infer to that Prop, so
`try_proof_irrel` fires and accepts ANY body. A recursor whose
computation rules are swapped between constructors was therefore
accepted, and iota then reduces each constructor to the other's branch.
Compare under an empty context instead, discarding the peeled domains. An
honest rule matches the reconstruction structurally, and the structural
path never consults the context — it compares `BVar i` against `BVar i`
directly. A comparison that instead needs to INFER a bound variable's
type now runs out of context and aborts, which is the safe direction.
Every recursor in the corpus still checks, including the mutual, nested
and aux-generated ones.
The reference avoids this differently, comparing whole Lam-wrapped terms
at empty context so `is_def_eq` descends binder-by-binder and compares
the domains themselves (crates/kernel/src/inductive.rs). Reconstructing
canonical domains here would mean re-deriving field domains from each
constructor's telescope and re-lifting their de Bruijn indices into the
rule's context; dropping the untrusted context achieves the same
guarantee without that machinery.
Tests
- `recursor-rule-rhs-doctored-binder-domains`: a real compiled recursor
with its first two computation rules swapped AND every peeled binder
domain replaced by a Prop. Accepted before this change; rejected now.
- `control-swapped-recursor-rules`: the identical swap with the
compiler's own binder domains, which was already rejected. The two
differ only in the domains, so the acceptance was attributable to them
alone.
- Both are built by rewriting a REAL block rather than hand-authoring a
recursor, since a hand-built one must survive `build_rec_type`'s def-eq
and constructor references inside a block are circular with the block's
own address. The fixture selects a recursor whose rules are field-free,
so no minor premise is applied and the retyping cannot abort as a
non-function before the comparison is reached — an earlier attempt did
exactly that and masked the acceptance.
- A guard case fails loudly if no such recursor is found; the first
version of this fixture silently corrupted nothing, and both its cases
"passed" meaninglessly.
- 57 FFT pins and the shard pin move (`Nat.add_comm` 42_980_767 ->
42_980_801).
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT DEMONSTRATED. Defense-in-depth, like the struct-eta guard.
`compare_rules` now asserts three invariants that equivalents of already
exist in `check_rec_rules_walk` — which nothing calls. The live path had
none of them.
* `cidx == pos`. `find_rule` selects a rule at reduction time BY LABEL
while `compare_rules` pairs stored against canonical POSITIONALLY, so
permuted labels would validate one constructor's rule and then fire
it for another. It holds today only because ingress assigns `cidx`
positionally (`Convert.lean`) and the Ixon `RecursorRule` record
carries no cidx of its own — an accident of the pipeline rather than
a checked property, which is exactly what an assert is for.
* `cidx < num_ctors`, so a rule cannot name a constructor that does not
exist.
* `nf` equals the constructor's field count. `try_iota` splits the
constructor's arguments with `field_start = ctor_fields_len -
rfields`, so a wrong `nf` hands the minor premise the wrong slice; in
field arithmetic an over-large value wraps.
The reachability argument for the `nf` gap was "combined with the rule
RHS context hole"; that hole is fixed, and with the def-eq comparison
restored a mismatched peel is caught again. So this closes a route that
is currently blocked by another check rather than one that is open —
worth asserting locally rather than depending on a second mechanism.
Not fixed here: iota still selects rules by index rather than by
constructor IDENTITY, so a recursor applied to a foreign constructor
would reduce. That is not reachable through a claim —
`k_infer_app_spine_loop` runs `k_check` on every argument, so the
application fails inference before whnf sees it — and closing it properly
needs the parent inductive available at reduction time, which
`KConstantInfo.Rec` does not carry (its `Addr` is the recursor's own
block; the parent is derived at check time from the type). That is a KCI
layout change, left for its own pass.
Tests
- No new fixture; nothing demonstrates the hole.
- Every recursor in the corpus satisfies all three invariants, including
the mutual, nested and aux-generated ones, so they held by construction
and are now enforced.
- 57 FFT pins and the shard pin move (`Nat.add_comm` 42_980_801 ->
42_981_336) for the per-rule constructor lookup.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
NO EXPLOIT DEMONSTRATED. Defense-in-depth.
`try_iota` selected a computation rule by constructor INDEX alone:
`collect_spine_of_ctor` returned the major's `cidx` and discarded which
inductive it came from, so `find_rule(rules, cidx)` fired whenever any
inductive's constructor happened to carry a matching index. On its face
`Bool.rec motive f t (Nat.succ n)` reduces to `t`. Lean's kernel selects
the rule by the constructor's NAME, which pins the inductive.
Not reachable through a claim: `k_infer_app_spine_loop` runs `k_check` on
every argument, so a recursor applied to a foreign constructor fails
inference long before whnf sees the redex — the ill-typed term cannot be
stated. The Rust reference selects by index too, consistent with that
being why it was tolerated. What the patch removes is the DEPENDENCE on
that: reduction no longer needs inference to have run first to be
correct.
`collect_spine_of_ctor` now also returns the constructor's parent, as the
address of its projection wrapper, and `try_iota` asserts it equals the
recursor's own parent. That parent is read off the recursor's type (the
major premise's head) with the existing `rec_to_parent_addr`, so no
KConstantInfo layout change was needed — its `Addr` field is the
recursor's own block, not the parent's. Aiur memoizes per argument tuple,
so the type walk is paid once per recursor rather than once per
reduction.
Tests
- No new fixture; nothing demonstrates the hole.
- Every recursor in the corpus still reduces, including the mutual,
nested and aux-generated ones, and the Nat/String literal paths that
synthesize a constructor application before iota (`nat_cases_big`,
`nat_pow_big`, `strOfListFold*` all unchanged in behaviour).
- 63 FFT pins and the shard pin move (`Nat.add_comm` 42_981_336 ->
42_986_025, +0.01%) for the parent comparison.
- `ix codegen` regenerated; `lake test -- --ignored ixvm` green.
`shardsCover` gates every shard run: "exit 0 has to mean every env const
was checked by some shard". It skipped constants whose bytes fail to
parse (`LazyConstant.get?` discards the error and `.ixe` loading is
lazy, so such a constant sits in `consts` with its key present). A
skipped constant was assigned to no shard, counted as neither owned nor
unowned, and still included in the "covers all N consts" total — so the
gate reported success over constants it had not covered.
It would also enter a referring shard's frontier, since `thin_frontier`
admits an edge on key presence in `env.consts` without parsing, leaving
an assumption no shard discharges.
Count them separately and fail, rather than folding them into `unowned`:
"cannot be assigned to a shard" is a different problem from a partition
gap, and saying so points at the right cause.
`ownedConstsForBlocks` keeps its skip — it runs per shard and would
otherwise need the same error plumbing — but now documents that it is
safe only because this gate rejects such an env first.
Not changed: the composed verdict still has no assumption-cycle check,
and on analysis it does not need one. Assumptions are discharged per
CONSTANT along ref edges, which are acyclic by construction — a
constant's address is blake3 over bytes containing its refs' addresses,
so a reference cycle needs a hash cycle. A cycle in the shard-level graph
still stands for a well-founded per-constant chain, and
`verifyShardComposition` recomputes each digest via `shardClaimDigest`,
so a self-assuming shard claim cannot be substituted.
`compile_univ` memoized `Level -> Univ`, but a `Level::Param` compiles to
`Univ::Var(i)` where `i` is the parameter's POSITION in `univ_params` —
so the result depends on a context the key omitted. Unlike the expression
cache, which is cleared at every constant boundary, `univ_cache` lives
for a whole block, and `compile_definition` compiles each member under
its own `level_params`. `preseed_expr_tables` explicitly feeds
`(expr, univ_params)` pairs with differing param lists into the same
cache.
So two members of one block placing the same parameter NAME at different
positions would give the second member the first's index. Not an abort:
it compiles cleanly and registers a constant denoting a different
universe than the Lean source, under a correct-looking name.
The hazard is already recognized one level up — `collect_expr_tables`
keys `seen_exprs` by `(expr_hash, univ_params_key)` for exactly this
reason. Apply the same key here. The Lean mirror instead drops the cache
in `withUnivCtx`, matching how it already clears the expression cache per
constant.
Unreachable from ordinary Lean sources, since the elaborator gives every
member of a `mutual` block the same `levelParams` — but nothing checked
it, and under the prover threat model the environment is authored.
Also repairs `claim.rs`'s `defn_const`, whose Definition never referenced
its own `refs`. Since the claim walk admits an index only when some Expr
uses it as a constant, that fixture produced no walk edges and
`shard_check_env_claim_is_thin_frontier` has been failing since the walk
moved to positive constant-use. The identical flaw was fixed in
`shard_claim.rs`'s helper at the time and missed here, because only
`ixon` and `ixvm-codegen` were re-tested rather than the whole workspace.
Tests
- `cargo test --release` across all crates: 1195 passed, 0 failed.
- `lake test -- --ignored ixvm` green with NO pin movement and no address
changes, confirming no block in the corpus has members that order a
shared parameter name differently. The fix removes the dependence on
that remaining true.
- clippy `--all-targets` and fmt clean.
@johnchandlerburnham
johnchandlerburnham merged commit 80bb41f into mainAug 3, 2026
10 checks passed
@johnchandlerburnham
johnchandlerburnham deleted the ap/ixvm4 branch August 3, 2026 05:37
samuelburnham added a commit that referenced this pull request Aug 4, 2026
…acking
Port of sb/measured-ingress's planning layer onto the addr-first kernel
(#529), reworked for its lazy fault-in semantics.
Profiling (.ixprof v3): the recording kernel's try_get_const consults are
drained per constant into a touch graph — the measured set a lazy checker
faults in — persisted as a presence-flagged CSR section alongside a new
block-level reference graph and a subst_unique counter (splitmix64-keyed
dedup of substitution work items: the post-memoization volume a memoizing
executor pays). Both graph sections are optional, so older readers'
formats stay decodable by version gate alone.
Planner: partition_for_aiur_ram bin-packs to a host-RAM cap with
faulted-set byte accounting — under #529 the witness always ships the
full closure, but only faulted bytes cost ingress trace rows, so a
shard's byte feature is owned ∪ touched(owned) (full reference closure
as the over-approximating fallback when no touch graph was recorded).
The stub/promotion machinery of the original branch is dropped entirely:
lazy fault-in computes at run time exactly the set the stubs predicted,
with no replay divergence and no repair ladder, and the manifest format
stays byte-identical with main (owned-only).
Cost models: nlogn(x) = x·log2(x+2) feature form matching Aiur's
width·height·log2(height) FFT structure; prove/execute time and RAM
coefficients carried over from the pre-#529 aiur bench-suite fits.
They are placeholders pending a refit against this kernel — the circuit
shrank (deleted positional machinery), the byte feature narrowed to the
faulted set, and the recording counters moved under #442 — with
provenance noted at the definitions. The costs sidecar
(<out>.costs.csv) persists per-shard features incl. subst_unique and
nat_arith so the refit has its corpus.
CLI: ix shard --backend aiur --max-ram G; ix profile --backend/--top
(per-metric block leaderboards with display names) and ix profile sweep
(env-wide closure cost sweep: per-root feature CSV + feasibility,
min-root-per-hot-block, and diversity reports).
samuelburnham added a commit that referenced this pull request Aug 8, 2026
…acking
Port of sb/measured-ingress's planning layer onto the addr-first kernel
(#529), reworked for its lazy fault-in semantics.
Profiling (.ixprof v3): the recording kernel's try_get_const consults are
drained per constant into a touch graph — the measured set a lazy checker
faults in — persisted as a presence-flagged CSR section alongside a new
block-level reference graph and a subst_unique counter (splitmix64-keyed
dedup of substitution work items: the post-memoization volume a memoizing
executor pays). Both graph sections are optional, so older readers'
formats stay decodable by version gate alone.
Planner: partition_for_aiur_ram bin-packs to a host-RAM cap with
faulted-set byte accounting — under #529 the witness always ships the
full closure, but only faulted bytes cost ingress trace rows, so a
shard's byte feature is owned ∪ touched(owned) (full reference closure
as the over-approximating fallback when no touch graph was recorded).
The stub/promotion machinery of the original branch is dropped entirely:
lazy fault-in computes at run time exactly the set the stubs predicted,
with no replay divergence and no repair ladder, and the manifest format
stays byte-identical with main (owned-only).
Cost models: nlogn(x) = x·log2(x+2) feature form matching Aiur's
width·height·log2(height) FFT structure; prove/execute time and RAM
coefficients carried over from the pre-#529 aiur bench-suite fits.
They are placeholders pending a refit against this kernel — the circuit
shrank (deleted positional machinery), the byte feature narrowed to the
faulted set, and the recording counters moved under #442 — with
provenance noted at the definitions. The costs sidecar
(<out>.costs.csv) persists per-shard features incl. subst_unique and
nat_arith so the refit has its corpus.
CLI: ix shard --backend aiur --max-ram G; ix profile --backend/--top
(per-metric block leaderboards with display names) and ix profile sweep
(env-wide closure cost sweep: per-root feature CSV + feasibility,
min-root-per-hot-block, and diversity reports).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@arthurpaulino@johnchandlerburnham