IxVM Aiur kernel: drive mathlib shard 0 to green - #485

Merged
arthurpaulino merged 8 commits into
mainfrom
ap/probe-mathlib
Jul 17, 2026
Merged

IxVM Aiur kernel: drive mathlib shard 0 to green#485
arthurpaulino merged 8 commits into
mainfrom
ap/probe-mathlib

Conversation

@arthurpaulino

@arthurpaulinoarthurpaulino commented Jul 11, 2026

Copy link
Copy Markdown
Member

Summary

ix check --ixe mathlib.ixe --ixes mathlib.ixes --shard 0 died with an
opaque assert_eq mismatch: 0 != 1. Driving it to green surfaced four
independent correctness divergences between the Aiur kernel and its Rust
mirror, one shard-scale performance pathology, and some missing debugging
infrastructure. All are fixed here: shard 0 (9,357 owned constants of the
647,127-const mathlib env) typechecks end-to-end through the native
codegen kernel, lake test -- --ignored ixvm is green (599 assertions),
and the FFT pins land within ±1.1% of main (median ratio 0.997) — the
kernel surgery is cost-neutral.

Kernel fixes

  1. Canonical block sort compares constructors (CanonicalCheck.lean).
    The Indc comparator stopped at the inductive's type, so same-shape
    mutual inductives (Mathlib.Tactic.Ring's ExBase/ExProd/ExSum:
    identical flags, arities and types, differing only in ctor types)
    collapsed into one alpha class and the block was rejected. Now mirrors
    compare_kindc's ctors tail, with a real KMutCtx (same-class members
    share one index; ctors mapped at i + cidx), address-based external-ref
    comparison, (fields, rhs)-only rec-rule comparison, and a provably
    sufficient refinement fuel bound.

  2. whnf-aware index walks (Inductive.lean). get_result_sort_level
    and the motive/rec-type index-domain collectors peeled literal Foralls
    only; index binders can hide under definitional wrappers —
    inductive εClosure (S : Set σ) : Set σ stores a type ending in
    Set σ that only whnf exposes as σ → Prop. whnf before every peel,
    mirroring the Rust walks.

  3. Aux synthesis parity + stored-aux validation removal
    (Inductive.lean). Synthesized aux types never instantiated the
    occurrence's universe args (Type u vs the stored monomorphized
    Type 0) and peeled binders structurally; fixed per
    canonical_aux_order's construction. The stored-aux validation pass
    (validate_block_auxes / is_aux_inductive) is removed outright:
    stored Muts blocks only ever contain the source originals (Rust seeds
    every stored member is_aux: false; auxes are transient detection
    results), and the classification heuristic misclassified originals in
    mixed blocks (Lean.Compiler.LCNF's Alt/FunDecl/Cases/Code),
    rejecting legitimate envs.

  4. Canonical aux order (Inductive.lean). Stored recursors bake the
    compiler's canonical aux order into their motive/minor layout, while
    build_flat_block discovers auxes in traversal order. The aux suffix
    is now re-sorted by partition refinement over synthetic aux views,
    with sentinel positions (|top| + ordinal) standing in for Rust's
    synthetic addresses and stable tie handling — an unstable sort
    reverses content-identical auxes whose spec params differ only in
    phantom parameters (IxVMInd.DedupM) and breaks rec-type matching.

Performance

  • Block-membership table. derive_block_member_idxs rescanned all of
    top per block with per-block memo keys — |blocks| × |top| query-map
    entries ate 99% of shard-scale CPU. One shared memoized walk now builds
    an rbtree from addr_key(block_addr) to address_eq-confirmed buckets;
    each query is an O(log N) lookup.

Tooling

  • ix name-of: new subcommand resolving a content address back to
    its Lean names against a .ixe env — potentially many, since
    structurally equivalent constants collapse to the same address (every
    matching entry in the env's named table is printed; the one-slot
    addrToName index would drop aliases). Unnamed Muts blocks fall back
    to scanning for projections into the block — turns an anonymized
    failing address into an ix check <name> fast repro. ix addr-of
    (name → address) is unchanged.
  • dbg! prints in the codegen kernel: emitDebug was a silent stub;
    it now emits the same println! the bytecode interpreter produces, so
    probe builds run at native speed. Production kernels contain no dbg!,
    so generated output is unchanged there.
  • lake exe check shim dropped: it had drifted from ix check
    (bool interp flag vs mode string); one entrypoint, no drift.

Validation

  • Full mathlib shard-0 native run: green, 9,357/9,357 owned consts.
  • Bytecode repros green for every constant that failed along the way:
    Mathlib.Tactic.Ring.ExSum, εNFA.εClosure.step,
    Lean.Compiler.LCNF.Cases, plus Lean.Syntax.rec /
    IxVMInd.DedupM.rec / IxVMInd.DepthM.rec for the aux machinery.
  • lake test -- --ignored ixvm: 599 assertions, 0 failures.
  • verify_bytes_against was stubbed during the debugging campaign and is
    restored; final pins include blake3 verification and sit within ±1.1%
    of main.

Known follow-ups

  • Shards 1–63 have not been run (pure machine time; ~300 GB peak RSS per
    shard, so sequential on a single box).

@arthurpaulino
arthurpaulinoforce-pushed the ap/probe-mathlib branch 3 times, most recently from 309e1c4 to de33f23CompareJuly 11, 2026 11:28
@arthurpaulino

Copy link
Copy Markdown
MemberAuthor

!benchmark

@argument-ci-bot

Copy link
Copy Markdown
Contributor

!benchmark — main vs de33f23

backends: aiur=prove · envs: InitStd · set: primary · shard: 0

aiur · InitStd · prove — main from: bencher @ 16455ba + base run (1 new)

constantprove-time (main)prove-time (PR)Δ%throughput (main)throughput (PR)Δ%peak-ram (main)peak-ram (PR)Δ%execute-time (main)execute-time (PR)Δ%verify-time (main)verify-time (PR)Δ%proof-size (main)proof-size (PR)Δ%fft-cost (main)fft-cost (PR)Δ%
ByteArray.utf8DecodeChar?_utf8EncodeChar_append54.933 s56.778 s+3.4% ⚠️52.61050.900-3.3% ⚠️100.89 GiB100.61 GiB-0.3%11.817 s12.508 s+5.8% (1.06× slower) ⚠️208.4 ms219.8 ms+5.5% (1.05× slower) ⚠️33.27 MiB34.43 MiB+3.5% ⚠️42.43B42.29B-0.3%
Array.extract_append53.157 s55.123 s+3.7% ⚠️32.09030.950-3.6% ⚠️92.86 GiB92.66 GiB-0.2%12.727 s13.021 s+2.3%210.3 ms214.2 ms+1.9%33.27 MiB34.43 MiB+3.5% ⚠️40.03B39.96B-0.2%
Char.ofOrdinal_le_of_le43.097 s43.778 s+1.6%66.41065.370-1.6%76.04 GiB75.49 GiB-0.7%8.947 s9.087 s+1.6%209.1 ms240.6 ms+15.1% (1.15× slower) ⚠️33.27 MiB34.43 MiB+3.5% ⚠️32.73B32.54B-0.6%
Vector.extract_append._proof_231.148 s32.048 s+2.9%46.26044.960-2.8%53.50 GiB53.32 GiB-0.3%6.820 s7.167 s+5.1% (1.05× slower) ⚠️215.8 ms238.4 ms+10.5% (1.10× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️23.34B23.29B-0.2%
_private.Init.Data.Range.Polymorphic.SInt.0.Int64.instRxcHasSize_eq26.216 s26.605 s+1.5%75.26074.160-1.5%48.88 GiB48.34 GiB-1.1%4.401 s4.443 s+0.9%205.7 ms222.9 ms+8.4% (1.08× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️17.43B17.35B-0.5%
String.split24.576 s25.436 s+3.5% ⚠️79.30076.620-3.4% ⚠️47.09 GiB46.75 GiB-0.7%4.030 s4.182 s+3.8% ⚠️205.0 ms219.6 ms+7.1% (1.07× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️15.99B15.89B-0.6%
List.mergeSort16.628 s17.267 s+3.8% ⚠️96.10092.540-3.7% ⚠️29.84 GiB29.71 GiB-0.4%2.782 s2.867 s+3.1% ⚠️205.2 ms215.4 ms+5.0% ⚠️33.19 MiB34.35 MiB+3.5% ⚠️11.63B11.58B-0.4%
Vector.append5.218 s5.439 s+4.2% ⚠️108.480104.060-4.1% ⚠️8.72 GiB8.90 GiB+2.0%529.2 ms541.8 ms+2.4%203.3 ms233.7 ms+15.0% (1.15× slower) ⚠️32.96 MiB34.12 MiB+3.5% ⚠️2.60B2.58B-0.5%
Nat.gcd_comm4.158 s4.382 s+5.4% (1.05× slower) ⚠️99.80094.700-5.1% (1.05× slower) ⚠️7.09 GiB7.01 GiB-1.0%349.0 ms366.4 ms+5.0% ⚠️218.0 ms215.4 ms-1.2%32.96 MiB34.12 MiB+3.5% ⚠️1.75B1.75B-0.2%
String.append2.973 s3.063 s+3.0% ⚠️118.750115.250-2.9%4.42 GiB4.57 GiB+3.5% ⚠️186.0 ms189.3 ms+1.8%201.0 ms223.1 ms+11.0% (1.11× slower) ⚠️32.89 MiB34.05 MiB+3.5% ⚠️967.65M963.43M-0.4%
Int.gcd2.350 s2.469 s+5.0% (1.05× slower) ⚠️97.43092.760-4.8% (1.05× slower) ⚠️3.25 GiB3.27 GiB+0.6%121.2 ms126.6 ms+4.5% ⚠️203.0 ms206.7 ms+1.8%32.89 MiB34.05 MiB+3.5% ⚠️605.84M604.12M-0.3%
Nat.sub_le_of_le_add2.347 s2.287 s-2.6%80.94083.090+2.7%3.06 GiB3.06 GiB-0.0%95.2 ms103.9 ms+9.2% (1.09× slower) ⚠️204.7 ms218.8 ms+6.9% (1.07× slower) ⚠️32.89 MiB34.05 MiB+3.5% ⚠️510.60M509.28M-0.3%
Nat.add_comm1.156 s1.258 s+8.8% (1.09× slower) ⚠️44.10040.530-8.1% (1.09× slower) ⚠️1.43 GiB1.46 GiB+2.3%15.5 ms16.0 ms+3.0% ⚠️194.5 ms208.9 ms+7.4% (1.07× slower) ⚠️32.76 MiB33.91 MiB+3.5% ⚠️53.90M53.75M-0.3%
Std.Tactic.BVDecide.BVExpr.bitblast.goCache_Inv_of_Inv._mutualOOMOOMn/aOOMOOMn/aOOMOOMn/a1m 21.4s1m 21.4s+0.1%OOMOOMn/aOOMOOMn/a261.03B260.46B-0.2%

14 constants · 13 regressed · 0 improved (|Δ| > 3.0% on any metric).

Workflow logs

@arthurpaulino
arthurpaulinoforce-pushed the ap/probe-mathlib branch 2 times, most recently from 6051448 to 6c5daa4CompareJuly 14, 2026 00:13
`ix name-of --ixe env.ixe <64-hex-addr>` resolves a content address
back to its Lean names. Potentially MANY names: structurally
equivalent constants collapse to the same content address, so every
entry in the env's `named` table pointing at the address is printed,
one per line (the `addrToName` reverse index keeps only one name per
address and would silently drop the aliases). Addresses of unnamed
Muts blocks fall back to scanning for projection constants into the
block and printing their names. Turns an anonymized failing address
surfaced by the IxVM kernel into an `ix check <name>` fast repro.
Lives in its own subcommand instead of overloading `ix addr-of`
(name -> address), which stays as-is.
…x walks
Two mathlib-blocking divergences from the Rust kernel, found via shard 0
of the mathlib env:
1. Canonical block sort (CanonicalCheck.lean): the Indc comparator never
descended into constructors, so same-shape mutual inductives (e.g.
Mathlib.Tactic.Ring's ExBase/ExProd/ExSum: identical flags, arities
and types, differing only in ctor types) collapsed into one alpha
class and the block was rejected (assert 0 != 1 in
validate_block_canonical). Mirror compare_kindc's ctors tail
(canonical_check.rs:299-338) via compare_kctor_idxs_ctx /
compare_kctor_pair_ctx, resolving ctor positions through `top`.
Further parity fixes in the same pass:
* ctx is now a real KMutCtx mirror (from_id_classes): (position,
class idx) pairs where same-class members share ONE index (weak-
Equal, no position-derived tiebreaks) and each Indc member's ctors
map at i + cidx.
* External const refs compare by 32-byte address (addr_cmp,
lexicographic like Rust's Address Ord) instead of by ingress
position, matching the order the compile-side sort produced.
* Rec-rule comparison drops the ingress-artifact global ctor idx —
(fields, rhs) only, mirroring canonical_check.rs:280-289.
* Refinement fuel is 1 + |members| (provably reaches fixpoint)
instead of a fixed 32.
* Ctx-less compare_kexpr Const arm compares levels before ref
(field-order parity; equality-only callers unaffected).
2. Inductive index walks (Inductive.lean): get_result_sort_level and
the motive/rec-type index-dom collectors peeled literal Foralls only.
Index binders can hide under definitional wrappers — Mathlib's
`inductive εClosure (S : Set σ) : Set σ` stores a type ending in
`Set σ` that only whnf exposes as `σ → Prop` — so εNFA.εClosure died
with `no match case for value 3` (KExprNode.App). whnf before every
peel, mirroring inductive.rs get_result_sort_level (line 2101+) and
build_motive_type_flat (2521-2531).
Repros (both now pass):
ix check --ixe mathlib.ixe Mathlib.Tactic.Ring.ExSum --interp bytecode
ix check --ixe mathlib.ixe εNFA.εClosure.step --interp bytecode
`lake test -- --ignored ixvm` green (599 assertions). Codegen
regenerated; FFT pins re-bumped — pin values reflect this branch's
stubbed verify_bytes_against and need one more bump when the stub is
reverted.
Known remaining issue: shard-scale runs crawl inside
collect_block_members (per-block O(|top|) rescans whose memo keys make
the query map blow up); fix planned separately.
emitDebug was a silent stub, so dbg! probes only printed under
--interp bytecode — kernel debugging was pinned to the slowest engine.
Emit the same println! the bytecode interpreter's Op::Debug arm
produces (label + comma-separated values), with the label escaped for
a Rust format-string literal. Production kernels contain no dbg!, so
generated output is unchanged there; probe builds now print at native
speed.
…red-aux validation
Three fixes surfaced by driving mathlib shard 0 to green:
1. Perf: derive_block_member_idxs rescanned the whole `top` list per
block, and each scan step's memo key included the block addr —
|blocks| x |top| query-map entries dominated shard-scale runs (99%
of CPU in QueryMap ops). One memoized walk now builds an rbtree from
addr_key(block_addr) to buckets of (full addr, ascending member
positions); every query is an O(log N) lookup + address_eq-confirmed
bucket walk (4-byte key collisions cost a short walk, never a wrong
member list).
2. Aux synthesis (Lean.Compiler.LCNF.Cases): synth_aux_ind_ty /
synth_aux_ctor_ty never instantiated the occurrence's universe args,
so synthesized types kept `Type u` (Param 0) where the block stores
monomorphized `Type 0`; binder peels also assumed literal Foralls.
Mirror canonical_aux_order's construction (inductive.rs:1169-1234):
instantiate occurrence_us, whnf before each peel. addrs is threaded
through the build_flat_block chain for the whnf calls.
3. Stored-aux validation removed: validate_block_auxes assumed Muts
blocks store synthesized aux inductives and classified members with
is_aux_inductive ("no own nested occurrence but some member has
one"). Stored blocks only ever contain the source originals — the
Rust kernel seeds every stored member is_aux:false (rs:537) and
flags auxes only on transient detection (rs:755) — so the heuristic
misclassified originals in mixed blocks (LCNF's Alt/FunDecl/Code,
where only Cases carries the nested occurrence) and asserted on a
legitimate block. Smuggled extra members remain rejected by the
recursor-vs-canonical-type equality over the detected flat block.
Validation: full shard-0 native run green (9357/9357 owned consts);
`lake test -- --ignored ixvm` green (599 assertions; codegen
regenerated, FFT pins re-bumped — still relative to this branch's
stubbed verify_bytes_against).
Known residual divergence (not yet observed failing): Rust reorders
the detected aux portion of a flat block via canonical_aux_order
before recursor checks; Aiur keeps discovery order.
The thin Cli wrapper around Ix.Cli.CheckCmd.runCheckCmd had drifted
from the real command (its `interp` flag was still a bare bool while
`ix check --interp` takes a mode string), and `lake exe ix check`
rebuilds just as incrementally for day-to-day kernel iteration. One
entrypoint, no drift.
Stored recursors bake the COMPILER's canonical aux order into their
motive/minor layout, but build_flat_block discovers auxes in queue
(traversal) order — position-by-position recursor matching only worked
when the two orders happened to coincide. Mirror inductive.rs
canonical_aux_order (rs:1058+, applied at rs:2293-2321): after the
queue pass, re-sort the aux suffix by partition refinement over
synthetic aux views.
Aiur-isms vs the Rust original:
* Synthetic addresses are replaced by SENTINEL positions
`|top| + ordinal`: each aux's view (ext type/ctors with occurrence
universe args instantiated, spec_params substituted, block params
wrapped) rewrites nested aux occurrences to sentinel Consts, fixed
per ordinal so views synthesize once; each refinement round's ctx
maps sentinels to their current class, so same-class refs compare
weak-Equal — the same mechanism the canonical block sort uses for
block-local refs.
* Ties keep DISCOVERY order (stable insert). Rust's sort_by_compare is
a stable merge sort; a first cut with an unstable insertion sort
REVERSED tied pairs, swapping content-identical auxes whose spec
params differ only in phantom parameters (IxVMInd.DedupM's
Bar2⟨·,Nat⟩ / Bar2⟨·,Bool⟩) and breaking their rec-type match.
* The reorder is unconditional: every env Aiur checks comes through
the Ix compile pipeline (RecursorAuxOrder::Canonical); the Lean-
source order case Rust skips (rs:2293) cannot reach this kernel.
Validation: `lake test -- --ignored ixvm` green (599 assertions; FFT
pins re-bumped, still relative to this branch's stubbed
verify_bytes_against); bytecode repros green for the multi-aux blocks
Lean.Compiler.LCNF.Cases, Lean.Syntax.rec, IxVMInd.DedupM.rec,
IxVMInd.DepthM.rec; full mathlib shard-0 native run green
(9357/9357 owned consts). Codegen regenerated.
Side finding for the compile side: copying the DedupM/Bar2 fixtures
into the CLI env under a different namespace (IxDbgFixtures) makes
compile_env fail that block with "compute_aux_perm: no canonical match
for in-SCC source aux #1" while the identical structure compiles fine
as IxVMInd.* in the test env — compile-side aux matching looks
name-order sensitive. Repro: re-add the four fixtures from
Tests/Ix/IxVM.lean:93-108 to any ix-CLI-visible module under a fresh
namespace and run `ix check <ns>.DedupM.rec`.
Reverts the probe-branch stub (b7a0336) that disabled blake3
verification of constant/blob/claim bytes during the mathlib shard-0
debugging campaign. Codegen regenerated; FFT pins re-bumped with
hashing back in the circuit.
The pins land within ±1.1% of main (median ratio 0.997): the kernel
work on this branch — canonical-sort comparator parity, whnf-aware
index walks, the block-members table, aux-synthesis parity and
canonical aux order — is cost-neutral. Nested-aux-heavy targets
(DedupM, DepthM, AuxDedup*, Lean.Syntax.rec) got 0.5-1.1% cheaper;
small stdlib targets (HEq, Nat, Eq.rec) pay 0.4-0.7% for the richer
comparator and whnf walks.
`lake test -- --ignored ixvm` green (599 assertions).
@arthurpaulino
arthurpaulino enabled auto-merge (squash) July 17, 2026 17:58
@arthurpaulino
arthurpaulino merged commit 4e554d2 into mainJul 17, 2026
8 of 9 checks passed
@arthurpaulino
arthurpaulino deleted the ap/probe-mathlib branch July 17, 2026 17:58
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@arthurpaulino@johnchandlerburnham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

IxVM Aiur kernel: drive mathlib shard 0 to green - #485

Merged
arthurpaulino merged 8 commits into
mainfrom
ap/probe-mathlib
Jul 17, 2026
Merged

IxVM Aiur kernel: drive mathlib shard 0 to green#485
arthurpaulino merged 8 commits into
mainfrom
ap/probe-mathlib

Conversation

@arthurpaulino

@arthurpaulinoarthurpaulino commented Jul 11, 2026

Copy link
Copy Markdown
Member

Summary

ix check --ixe mathlib.ixe --ixes mathlib.ixes --shard 0 died with an
opaque assert_eq mismatch: 0 != 1. Driving it to green surfaced four
independent correctness divergences between the Aiur kernel and its Rust
mirror, one shard-scale performance pathology, and some missing debugging
infrastructure. All are fixed here: shard 0 (9,357 owned constants of the
647,127-const mathlib env) typechecks end-to-end through the native
codegen kernel, lake test -- --ignored ixvm is green (599 assertions),
and the FFT pins land within ±1.1% of main (median ratio 0.997) — the
kernel surgery is cost-neutral.

Kernel fixes

  1. Canonical block sort compares constructors (CanonicalCheck.lean).
    The Indc comparator stopped at the inductive's type, so same-shape
    mutual inductives (Mathlib.Tactic.Ring's ExBase/ExProd/ExSum:
    identical flags, arities and types, differing only in ctor types)
    collapsed into one alpha class and the block was rejected. Now mirrors
    compare_kindc's ctors tail, with a real KMutCtx (same-class members
    share one index; ctors mapped at i + cidx), address-based external-ref
    comparison, (fields, rhs)-only rec-rule comparison, and a provably
    sufficient refinement fuel bound.

  2. whnf-aware index walks (Inductive.lean). get_result_sort_level
    and the motive/rec-type index-domain collectors peeled literal Foralls
    only; index binders can hide under definitional wrappers —
    inductive εClosure (S : Set σ) : Set σ stores a type ending in
    Set σ that only whnf exposes as σ → Prop. whnf before every peel,
    mirroring the Rust walks.

  3. Aux synthesis parity + stored-aux validation removal
    (Inductive.lean). Synthesized aux types never instantiated the
    occurrence's universe args (Type u vs the stored monomorphized
    Type 0) and peeled binders structurally; fixed per
    canonical_aux_order's construction. The stored-aux validation pass
    (validate_block_auxes / is_aux_inductive) is removed outright:
    stored Muts blocks only ever contain the source originals (Rust seeds
    every stored member is_aux: false; auxes are transient detection
    results), and the classification heuristic misclassified originals in
    mixed blocks (Lean.Compiler.LCNF's Alt/FunDecl/Cases/Code),
    rejecting legitimate envs.

  4. Canonical aux order (Inductive.lean). Stored recursors bake the
    compiler's canonical aux order into their motive/minor layout, while
    build_flat_block discovers auxes in traversal order. The aux suffix
    is now re-sorted by partition refinement over synthetic aux views,
    with sentinel positions (|top| + ordinal) standing in for Rust's
    synthetic addresses and stable tie handling — an unstable sort
    reverses content-identical auxes whose spec params differ only in
    phantom parameters (IxVMInd.DedupM) and breaks rec-type matching.

Performance

  • Block-membership table. derive_block_member_idxs rescanned all of
    top per block with per-block memo keys — |blocks| × |top| query-map
    entries ate 99% of shard-scale CPU. One shared memoized walk now builds
    an rbtree from addr_key(block_addr) to address_eq-confirmed buckets;
    each query is an O(log N) lookup.

Tooling

  • ix name-of: new subcommand resolving a content address back to
    its Lean names against a .ixe env — potentially many, since
    structurally equivalent constants collapse to the same address (every
    matching entry in the env's named table is printed; the one-slot
    addrToName index would drop aliases). Unnamed Muts blocks fall back
    to scanning for projections into the block — turns an anonymized
    failing address into an ix check <name> fast repro. ix addr-of
    (name → address) is unchanged.
  • dbg! prints in the codegen kernel: emitDebug was a silent stub;
    it now emits the same println! the bytecode interpreter produces, so
    probe builds run at native speed. Production kernels contain no dbg!,
    so generated output is unchanged there.
  • lake exe check shim dropped: it had drifted from ix check
    (bool interp flag vs mode string); one entrypoint, no drift.

Validation

  • Full mathlib shard-0 native run: green, 9,357/9,357 owned consts.
  • Bytecode repros green for every constant that failed along the way:
    Mathlib.Tactic.Ring.ExSum, εNFA.εClosure.step,
    Lean.Compiler.LCNF.Cases, plus Lean.Syntax.rec /
    IxVMInd.DedupM.rec / IxVMInd.DepthM.rec for the aux machinery.
  • lake test -- --ignored ixvm: 599 assertions, 0 failures.
  • verify_bytes_against was stubbed during the debugging campaign and is
    restored; final pins include blake3 verification and sit within ±1.1%
    of main.

Known follow-ups

  • Shards 1–63 have not been run (pure machine time; ~300 GB peak RSS per
    shard, so sequential on a single box).

@arthurpaulino
arthurpaulinoforce-pushed the ap/probe-mathlib branch 3 times, most recently from 309e1c4 to de33f23CompareJuly 11, 2026 11:28
@arthurpaulino

Copy link
Copy Markdown
MemberAuthor

!benchmark

@argument-ci-bot

Copy link
Copy Markdown
Contributor

!benchmark — main vs de33f23

backends: aiur=prove · envs: InitStd · set: primary · shard: 0

aiur · InitStd · prove — main from: bencher @ 16455ba + base run (1 new)

constantprove-time (main)prove-time (PR)Δ%throughput (main)throughput (PR)Δ%peak-ram (main)peak-ram (PR)Δ%execute-time (main)execute-time (PR)Δ%verify-time (main)verify-time (PR)Δ%proof-size (main)proof-size (PR)Δ%fft-cost (main)fft-cost (PR)Δ%
ByteArray.utf8DecodeChar?_utf8EncodeChar_append54.933 s56.778 s+3.4% ⚠️52.61050.900-3.3% ⚠️100.89 GiB100.61 GiB-0.3%11.817 s12.508 s+5.8% (1.06× slower) ⚠️208.4 ms219.8 ms+5.5% (1.05× slower) ⚠️33.27 MiB34.43 MiB+3.5% ⚠️42.43B42.29B-0.3%
Array.extract_append53.157 s55.123 s+3.7% ⚠️32.09030.950-3.6% ⚠️92.86 GiB92.66 GiB-0.2%12.727 s13.021 s+2.3%210.3 ms214.2 ms+1.9%33.27 MiB34.43 MiB+3.5% ⚠️40.03B39.96B-0.2%
Char.ofOrdinal_le_of_le43.097 s43.778 s+1.6%66.41065.370-1.6%76.04 GiB75.49 GiB-0.7%8.947 s9.087 s+1.6%209.1 ms240.6 ms+15.1% (1.15× slower) ⚠️33.27 MiB34.43 MiB+3.5% ⚠️32.73B32.54B-0.6%
Vector.extract_append._proof_231.148 s32.048 s+2.9%46.26044.960-2.8%53.50 GiB53.32 GiB-0.3%6.820 s7.167 s+5.1% (1.05× slower) ⚠️215.8 ms238.4 ms+10.5% (1.10× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️23.34B23.29B-0.2%
_private.Init.Data.Range.Polymorphic.SInt.0.Int64.instRxcHasSize_eq26.216 s26.605 s+1.5%75.26074.160-1.5%48.88 GiB48.34 GiB-1.1%4.401 s4.443 s+0.9%205.7 ms222.9 ms+8.4% (1.08× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️17.43B17.35B-0.5%
String.split24.576 s25.436 s+3.5% ⚠️79.30076.620-3.4% ⚠️47.09 GiB46.75 GiB-0.7%4.030 s4.182 s+3.8% ⚠️205.0 ms219.6 ms+7.1% (1.07× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️15.99B15.89B-0.6%
List.mergeSort16.628 s17.267 s+3.8% ⚠️96.10092.540-3.7% ⚠️29.84 GiB29.71 GiB-0.4%2.782 s2.867 s+3.1% ⚠️205.2 ms215.4 ms+5.0% ⚠️33.19 MiB34.35 MiB+3.5% ⚠️11.63B11.58B-0.4%
Vector.append5.218 s5.439 s+4.2% ⚠️108.480104.060-4.1% ⚠️8.72 GiB8.90 GiB+2.0%529.2 ms541.8 ms+2.4%203.3 ms233.7 ms+15.0% (1.15× slower) ⚠️32.96 MiB34.12 MiB+3.5% ⚠️2.60B2.58B-0.5%
Nat.gcd_comm4.158 s4.382 s+5.4% (1.05× slower) ⚠️99.80094.700-5.1% (1.05× slower) ⚠️7.09 GiB7.01 GiB-1.0%349.0 ms366.4 ms+5.0% ⚠️218.0 ms215.4 ms-1.2%32.96 MiB34.12 MiB+3.5% ⚠️1.75B1.75B-0.2%
String.append2.973 s3.063 s+3.0% ⚠️118.750115.250-2.9%4.42 GiB4.57 GiB+3.5% ⚠️186.0 ms189.3 ms+1.8%201.0 ms223.1 ms+11.0% (1.11× slower) ⚠️32.89 MiB34.05 MiB+3.5% ⚠️967.65M963.43M-0.4%
Int.gcd2.350 s2.469 s+5.0% (1.05× slower) ⚠️97.43092.760-4.8% (1.05× slower) ⚠️3.25 GiB3.27 GiB+0.6%121.2 ms126.6 ms+4.5% ⚠️203.0 ms206.7 ms+1.8%32.89 MiB34.05 MiB+3.5% ⚠️605.84M604.12M-0.3%
Nat.sub_le_of_le_add2.347 s2.287 s-2.6%80.94083.090+2.7%3.06 GiB3.06 GiB-0.0%95.2 ms103.9 ms+9.2% (1.09× slower) ⚠️204.7 ms218.8 ms+6.9% (1.07× slower) ⚠️32.89 MiB34.05 MiB+3.5% ⚠️510.60M509.28M-0.3%
Nat.add_comm1.156 s1.258 s+8.8% (1.09× slower) ⚠️44.10040.530-8.1% (1.09× slower) ⚠️1.43 GiB1.46 GiB+2.3%15.5 ms16.0 ms+3.0% ⚠️194.5 ms208.9 ms+7.4% (1.07× slower) ⚠️32.76 MiB33.91 MiB+3.5% ⚠️53.90M53.75M-0.3%
Std.Tactic.BVDecide.BVExpr.bitblast.goCache_Inv_of_Inv._mutualOOMOOMn/aOOMOOMn/aOOMOOMn/a1m 21.4s1m 21.4s+0.1%OOMOOMn/aOOMOOMn/a261.03B260.46B-0.2%

14 constants · 13 regressed · 0 improved (|Δ| > 3.0% on any metric).

Workflow logs

@arthurpaulino
arthurpaulinoforce-pushed the ap/probe-mathlib branch 2 times, most recently from 6051448 to 6c5daa4CompareJuly 14, 2026 00:13
`ix name-of --ixe env.ixe <64-hex-addr>` resolves a content address
back to its Lean names. Potentially MANY names: structurally
equivalent constants collapse to the same content address, so every
entry in the env's `named` table pointing at the address is printed,
one per line (the `addrToName` reverse index keeps only one name per
address and would silently drop the aliases). Addresses of unnamed
Muts blocks fall back to scanning for projection constants into the
block and printing their names. Turns an anonymized failing address
surfaced by the IxVM kernel into an `ix check <name>` fast repro.
Lives in its own subcommand instead of overloading `ix addr-of`
(name -> address), which stays as-is.
…x walks
Two mathlib-blocking divergences from the Rust kernel, found via shard 0
of the mathlib env:
1. Canonical block sort (CanonicalCheck.lean): the Indc comparator never
descended into constructors, so same-shape mutual inductives (e.g.
Mathlib.Tactic.Ring's ExBase/ExProd/ExSum: identical flags, arities
and types, differing only in ctor types) collapsed into one alpha
class and the block was rejected (assert 0 != 1 in
validate_block_canonical). Mirror compare_kindc's ctors tail
(canonical_check.rs:299-338) via compare_kctor_idxs_ctx /
compare_kctor_pair_ctx, resolving ctor positions through `top`.
Further parity fixes in the same pass:
* ctx is now a real KMutCtx mirror (from_id_classes): (position,
class idx) pairs where same-class members share ONE index (weak-
Equal, no position-derived tiebreaks) and each Indc member's ctors
map at i + cidx.
* External const refs compare by 32-byte address (addr_cmp,
lexicographic like Rust's Address Ord) instead of by ingress
position, matching the order the compile-side sort produced.
* Rec-rule comparison drops the ingress-artifact global ctor idx —
(fields, rhs) only, mirroring canonical_check.rs:280-289.
* Refinement fuel is 1 + |members| (provably reaches fixpoint)
instead of a fixed 32.
* Ctx-less compare_kexpr Const arm compares levels before ref
(field-order parity; equality-only callers unaffected).
2. Inductive index walks (Inductive.lean): get_result_sort_level and
the motive/rec-type index-dom collectors peeled literal Foralls only.
Index binders can hide under definitional wrappers — Mathlib's
`inductive εClosure (S : Set σ) : Set σ` stores a type ending in
`Set σ` that only whnf exposes as `σ → Prop` — so εNFA.εClosure died
with `no match case for value 3` (KExprNode.App). whnf before every
peel, mirroring inductive.rs get_result_sort_level (line 2101+) and
build_motive_type_flat (2521-2531).
Repros (both now pass):
ix check --ixe mathlib.ixe Mathlib.Tactic.Ring.ExSum --interp bytecode
ix check --ixe mathlib.ixe εNFA.εClosure.step --interp bytecode
`lake test -- --ignored ixvm` green (599 assertions). Codegen
regenerated; FFT pins re-bumped — pin values reflect this branch's
stubbed verify_bytes_against and need one more bump when the stub is
reverted.
Known remaining issue: shard-scale runs crawl inside
collect_block_members (per-block O(|top|) rescans whose memo keys make
the query map blow up); fix planned separately.
emitDebug was a silent stub, so dbg! probes only printed under
--interp bytecode — kernel debugging was pinned to the slowest engine.
Emit the same println! the bytecode interpreter's Op::Debug arm
produces (label + comma-separated values), with the label escaped for
a Rust format-string literal. Production kernels contain no dbg!, so
generated output is unchanged there; probe builds now print at native
speed.
…red-aux validation
Three fixes surfaced by driving mathlib shard 0 to green:
1. Perf: derive_block_member_idxs rescanned the whole `top` list per
block, and each scan step's memo key included the block addr —
|blocks| x |top| query-map entries dominated shard-scale runs (99%
of CPU in QueryMap ops). One memoized walk now builds an rbtree from
addr_key(block_addr) to buckets of (full addr, ascending member
positions); every query is an O(log N) lookup + address_eq-confirmed
bucket walk (4-byte key collisions cost a short walk, never a wrong
member list).
2. Aux synthesis (Lean.Compiler.LCNF.Cases): synth_aux_ind_ty /
synth_aux_ctor_ty never instantiated the occurrence's universe args,
so synthesized types kept `Type u` (Param 0) where the block stores
monomorphized `Type 0`; binder peels also assumed literal Foralls.
Mirror canonical_aux_order's construction (inductive.rs:1169-1234):
instantiate occurrence_us, whnf before each peel. addrs is threaded
through the build_flat_block chain for the whnf calls.
3. Stored-aux validation removed: validate_block_auxes assumed Muts
blocks store synthesized aux inductives and classified members with
is_aux_inductive ("no own nested occurrence but some member has
one"). Stored blocks only ever contain the source originals — the
Rust kernel seeds every stored member is_aux:false (rs:537) and
flags auxes only on transient detection (rs:755) — so the heuristic
misclassified originals in mixed blocks (LCNF's Alt/FunDecl/Code,
where only Cases carries the nested occurrence) and asserted on a
legitimate block. Smuggled extra members remain rejected by the
recursor-vs-canonical-type equality over the detected flat block.
Validation: full shard-0 native run green (9357/9357 owned consts);
`lake test -- --ignored ixvm` green (599 assertions; codegen
regenerated, FFT pins re-bumped — still relative to this branch's
stubbed verify_bytes_against).
Known residual divergence (not yet observed failing): Rust reorders
the detected aux portion of a flat block via canonical_aux_order
before recursor checks; Aiur keeps discovery order.
The thin Cli wrapper around Ix.Cli.CheckCmd.runCheckCmd had drifted
from the real command (its `interp` flag was still a bare bool while
`ix check --interp` takes a mode string), and `lake exe ix check`
rebuilds just as incrementally for day-to-day kernel iteration. One
entrypoint, no drift.
Stored recursors bake the COMPILER's canonical aux order into their
motive/minor layout, but build_flat_block discovers auxes in queue
(traversal) order — position-by-position recursor matching only worked
when the two orders happened to coincide. Mirror inductive.rs
canonical_aux_order (rs:1058+, applied at rs:2293-2321): after the
queue pass, re-sort the aux suffix by partition refinement over
synthetic aux views.
Aiur-isms vs the Rust original:
* Synthetic addresses are replaced by SENTINEL positions
`|top| + ordinal`: each aux's view (ext type/ctors with occurrence
universe args instantiated, spec_params substituted, block params
wrapped) rewrites nested aux occurrences to sentinel Consts, fixed
per ordinal so views synthesize once; each refinement round's ctx
maps sentinels to their current class, so same-class refs compare
weak-Equal — the same mechanism the canonical block sort uses for
block-local refs.
* Ties keep DISCOVERY order (stable insert). Rust's sort_by_compare is
a stable merge sort; a first cut with an unstable insertion sort
REVERSED tied pairs, swapping content-identical auxes whose spec
params differ only in phantom parameters (IxVMInd.DedupM's
Bar2⟨·,Nat⟩ / Bar2⟨·,Bool⟩) and breaking their rec-type match.
* The reorder is unconditional: every env Aiur checks comes through
the Ix compile pipeline (RecursorAuxOrder::Canonical); the Lean-
source order case Rust skips (rs:2293) cannot reach this kernel.
Validation: `lake test -- --ignored ixvm` green (599 assertions; FFT
pins re-bumped, still relative to this branch's stubbed
verify_bytes_against); bytecode repros green for the multi-aux blocks
Lean.Compiler.LCNF.Cases, Lean.Syntax.rec, IxVMInd.DedupM.rec,
IxVMInd.DepthM.rec; full mathlib shard-0 native run green
(9357/9357 owned consts). Codegen regenerated.
Side finding for the compile side: copying the DedupM/Bar2 fixtures
into the CLI env under a different namespace (IxDbgFixtures) makes
compile_env fail that block with "compute_aux_perm: no canonical match
for in-SCC source aux #1" while the identical structure compiles fine
as IxVMInd.* in the test env — compile-side aux matching looks
name-order sensitive. Repro: re-add the four fixtures from
Tests/Ix/IxVM.lean:93-108 to any ix-CLI-visible module under a fresh
namespace and run `ix check <ns>.DedupM.rec`.
Reverts the probe-branch stub (b7a0336) that disabled blake3
verification of constant/blob/claim bytes during the mathlib shard-0
debugging campaign. Codegen regenerated; FFT pins re-bumped with
hashing back in the circuit.
The pins land within ±1.1% of main (median ratio 0.997): the kernel
work on this branch — canonical-sort comparator parity, whnf-aware
index walks, the block-members table, aux-synthesis parity and
canonical aux order — is cost-neutral. Nested-aux-heavy targets
(DedupM, DepthM, AuxDedup*, Lean.Syntax.rec) got 0.5-1.1% cheaper;
small stdlib targets (HEq, Nat, Eq.rec) pay 0.4-0.7% for the richer
comparator and whnf walks.
`lake test -- --ignored ixvm` green (599 assertions).
@arthurpaulino
arthurpaulino enabled auto-merge (squash) July 17, 2026 17:58
@arthurpaulino
arthurpaulino merged commit 4e554d2 into mainJul 17, 2026
8 of 9 checks passed
@arthurpaulino
arthurpaulino deleted the ap/probe-mathlib branch July 17, 2026 17:58
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@arthurpaulino@johnchandlerburnham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

IxVM Aiur kernel: drive mathlib shard 0 to green - #485

Merged
arthurpaulino merged 8 commits into
mainfrom
ap/probe-mathlib
Jul 17, 2026
Merged

IxVM Aiur kernel: drive mathlib shard 0 to green#485
arthurpaulino merged 8 commits into
mainfrom
ap/probe-mathlib

Conversation

@arthurpaulino

@arthurpaulinoarthurpaulino commented Jul 11, 2026

Copy link
Copy Markdown
Member

Summary

ix check --ixe mathlib.ixe --ixes mathlib.ixes --shard 0 died with an
opaque assert_eq mismatch: 0 != 1. Driving it to green surfaced four
independent correctness divergences between the Aiur kernel and its Rust
mirror, one shard-scale performance pathology, and some missing debugging
infrastructure. All are fixed here: shard 0 (9,357 owned constants of the
647,127-const mathlib env) typechecks end-to-end through the native
codegen kernel, lake test -- --ignored ixvm is green (599 assertions),
and the FFT pins land within ±1.1% of main (median ratio 0.997) — the
kernel surgery is cost-neutral.

Kernel fixes

  1. Canonical block sort compares constructors (CanonicalCheck.lean).
    The Indc comparator stopped at the inductive's type, so same-shape
    mutual inductives (Mathlib.Tactic.Ring's ExBase/ExProd/ExSum:
    identical flags, arities and types, differing only in ctor types)
    collapsed into one alpha class and the block was rejected. Now mirrors
    compare_kindc's ctors tail, with a real KMutCtx (same-class members
    share one index; ctors mapped at i + cidx), address-based external-ref
    comparison, (fields, rhs)-only rec-rule comparison, and a provably
    sufficient refinement fuel bound.

  2. whnf-aware index walks (Inductive.lean). get_result_sort_level
    and the motive/rec-type index-domain collectors peeled literal Foralls
    only; index binders can hide under definitional wrappers —
    inductive εClosure (S : Set σ) : Set σ stores a type ending in
    Set σ that only whnf exposes as σ → Prop. whnf before every peel,
    mirroring the Rust walks.

  3. Aux synthesis parity + stored-aux validation removal
    (Inductive.lean). Synthesized aux types never instantiated the
    occurrence's universe args (Type u vs the stored monomorphized
    Type 0) and peeled binders structurally; fixed per
    canonical_aux_order's construction. The stored-aux validation pass
    (validate_block_auxes / is_aux_inductive) is removed outright:
    stored Muts blocks only ever contain the source originals (Rust seeds
    every stored member is_aux: false; auxes are transient detection
    results), and the classification heuristic misclassified originals in
    mixed blocks (Lean.Compiler.LCNF's Alt/FunDecl/Cases/Code),
    rejecting legitimate envs.

  4. Canonical aux order (Inductive.lean). Stored recursors bake the
    compiler's canonical aux order into their motive/minor layout, while
    build_flat_block discovers auxes in traversal order. The aux suffix
    is now re-sorted by partition refinement over synthetic aux views,
    with sentinel positions (|top| + ordinal) standing in for Rust's
    synthetic addresses and stable tie handling — an unstable sort
    reverses content-identical auxes whose spec params differ only in
    phantom parameters (IxVMInd.DedupM) and breaks rec-type matching.

Performance

  • Block-membership table. derive_block_member_idxs rescanned all of
    top per block with per-block memo keys — |blocks| × |top| query-map
    entries ate 99% of shard-scale CPU. One shared memoized walk now builds
    an rbtree from addr_key(block_addr) to address_eq-confirmed buckets;
    each query is an O(log N) lookup.

Tooling

  • ix name-of: new subcommand resolving a content address back to
    its Lean names against a .ixe env — potentially many, since
    structurally equivalent constants collapse to the same address (every
    matching entry in the env's named table is printed; the one-slot
    addrToName index would drop aliases). Unnamed Muts blocks fall back
    to scanning for projections into the block — turns an anonymized
    failing address into an ix check <name> fast repro. ix addr-of
    (name → address) is unchanged.
  • dbg! prints in the codegen kernel: emitDebug was a silent stub;
    it now emits the same println! the bytecode interpreter produces, so
    probe builds run at native speed. Production kernels contain no dbg!,
    so generated output is unchanged there.
  • lake exe check shim dropped: it had drifted from ix check
    (bool interp flag vs mode string); one entrypoint, no drift.

Validation

  • Full mathlib shard-0 native run: green, 9,357/9,357 owned consts.
  • Bytecode repros green for every constant that failed along the way:
    Mathlib.Tactic.Ring.ExSum, εNFA.εClosure.step,
    Lean.Compiler.LCNF.Cases, plus Lean.Syntax.rec /
    IxVMInd.DedupM.rec / IxVMInd.DepthM.rec for the aux machinery.
  • lake test -- --ignored ixvm: 599 assertions, 0 failures.
  • verify_bytes_against was stubbed during the debugging campaign and is
    restored; final pins include blake3 verification and sit within ±1.1%
    of main.

Known follow-ups

  • Shards 1–63 have not been run (pure machine time; ~300 GB peak RSS per
    shard, so sequential on a single box).

@arthurpaulino
arthurpaulinoforce-pushed the ap/probe-mathlib branch 3 times, most recently from 309e1c4 to de33f23CompareJuly 11, 2026 11:28
@arthurpaulino

Copy link
Copy Markdown
MemberAuthor

!benchmark

@argument-ci-bot

Copy link
Copy Markdown
Contributor

!benchmark — main vs de33f23

backends: aiur=prove · envs: InitStd · set: primary · shard: 0

aiur · InitStd · prove — main from: bencher @ 16455ba + base run (1 new)

constantprove-time (main)prove-time (PR)Δ%throughput (main)throughput (PR)Δ%peak-ram (main)peak-ram (PR)Δ%execute-time (main)execute-time (PR)Δ%verify-time (main)verify-time (PR)Δ%proof-size (main)proof-size (PR)Δ%fft-cost (main)fft-cost (PR)Δ%
ByteArray.utf8DecodeChar?_utf8EncodeChar_append54.933 s56.778 s+3.4% ⚠️52.61050.900-3.3% ⚠️100.89 GiB100.61 GiB-0.3%11.817 s12.508 s+5.8% (1.06× slower) ⚠️208.4 ms219.8 ms+5.5% (1.05× slower) ⚠️33.27 MiB34.43 MiB+3.5% ⚠️42.43B42.29B-0.3%
Array.extract_append53.157 s55.123 s+3.7% ⚠️32.09030.950-3.6% ⚠️92.86 GiB92.66 GiB-0.2%12.727 s13.021 s+2.3%210.3 ms214.2 ms+1.9%33.27 MiB34.43 MiB+3.5% ⚠️40.03B39.96B-0.2%
Char.ofOrdinal_le_of_le43.097 s43.778 s+1.6%66.41065.370-1.6%76.04 GiB75.49 GiB-0.7%8.947 s9.087 s+1.6%209.1 ms240.6 ms+15.1% (1.15× slower) ⚠️33.27 MiB34.43 MiB+3.5% ⚠️32.73B32.54B-0.6%
Vector.extract_append._proof_231.148 s32.048 s+2.9%46.26044.960-2.8%53.50 GiB53.32 GiB-0.3%6.820 s7.167 s+5.1% (1.05× slower) ⚠️215.8 ms238.4 ms+10.5% (1.10× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️23.34B23.29B-0.2%
_private.Init.Data.Range.Polymorphic.SInt.0.Int64.instRxcHasSize_eq26.216 s26.605 s+1.5%75.26074.160-1.5%48.88 GiB48.34 GiB-1.1%4.401 s4.443 s+0.9%205.7 ms222.9 ms+8.4% (1.08× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️17.43B17.35B-0.5%
String.split24.576 s25.436 s+3.5% ⚠️79.30076.620-3.4% ⚠️47.09 GiB46.75 GiB-0.7%4.030 s4.182 s+3.8% ⚠️205.0 ms219.6 ms+7.1% (1.07× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️15.99B15.89B-0.6%
List.mergeSort16.628 s17.267 s+3.8% ⚠️96.10092.540-3.7% ⚠️29.84 GiB29.71 GiB-0.4%2.782 s2.867 s+3.1% ⚠️205.2 ms215.4 ms+5.0% ⚠️33.19 MiB34.35 MiB+3.5% ⚠️11.63B11.58B-0.4%
Vector.append5.218 s5.439 s+4.2% ⚠️108.480104.060-4.1% ⚠️8.72 GiB8.90 GiB+2.0%529.2 ms541.8 ms+2.4%203.3 ms233.7 ms+15.0% (1.15× slower) ⚠️32.96 MiB34.12 MiB+3.5% ⚠️2.60B2.58B-0.5%
Nat.gcd_comm4.158 s4.382 s+5.4% (1.05× slower) ⚠️99.80094.700-5.1% (1.05× slower) ⚠️7.09 GiB7.01 GiB-1.0%349.0 ms366.4 ms+5.0% ⚠️218.0 ms215.4 ms-1.2%32.96 MiB34.12 MiB+3.5% ⚠️1.75B1.75B-0.2%
String.append2.973 s3.063 s+3.0% ⚠️118.750115.250-2.9%4.42 GiB4.57 GiB+3.5% ⚠️186.0 ms189.3 ms+1.8%201.0 ms223.1 ms+11.0% (1.11× slower) ⚠️32.89 MiB34.05 MiB+3.5% ⚠️967.65M963.43M-0.4%
Int.gcd2.350 s2.469 s+5.0% (1.05× slower) ⚠️97.43092.760-4.8% (1.05× slower) ⚠️3.25 GiB3.27 GiB+0.6%121.2 ms126.6 ms+4.5% ⚠️203.0 ms206.7 ms+1.8%32.89 MiB34.05 MiB+3.5% ⚠️605.84M604.12M-0.3%
Nat.sub_le_of_le_add2.347 s2.287 s-2.6%80.94083.090+2.7%3.06 GiB3.06 GiB-0.0%95.2 ms103.9 ms+9.2% (1.09× slower) ⚠️204.7 ms218.8 ms+6.9% (1.07× slower) ⚠️32.89 MiB34.05 MiB+3.5% ⚠️510.60M509.28M-0.3%
Nat.add_comm1.156 s1.258 s+8.8% (1.09× slower) ⚠️44.10040.530-8.1% (1.09× slower) ⚠️1.43 GiB1.46 GiB+2.3%15.5 ms16.0 ms+3.0% ⚠️194.5 ms208.9 ms+7.4% (1.07× slower) ⚠️32.76 MiB33.91 MiB+3.5% ⚠️53.90M53.75M-0.3%
Std.Tactic.BVDecide.BVExpr.bitblast.goCache_Inv_of_Inv._mutualOOMOOMn/aOOMOOMn/aOOMOOMn/a1m 21.4s1m 21.4s+0.1%OOMOOMn/aOOMOOMn/a261.03B260.46B-0.2%

14 constants · 13 regressed · 0 improved (|Δ| > 3.0% on any metric).

Workflow logs

@arthurpaulino
arthurpaulinoforce-pushed the ap/probe-mathlib branch 2 times, most recently from 6051448 to 6c5daa4CompareJuly 14, 2026 00:13
`ix name-of --ixe env.ixe <64-hex-addr>` resolves a content address
back to its Lean names. Potentially MANY names: structurally
equivalent constants collapse to the same content address, so every
entry in the env's `named` table pointing at the address is printed,
one per line (the `addrToName` reverse index keeps only one name per
address and would silently drop the aliases). Addresses of unnamed
Muts blocks fall back to scanning for projection constants into the
block and printing their names. Turns an anonymized failing address
surfaced by the IxVM kernel into an `ix check <name>` fast repro.
Lives in its own subcommand instead of overloading `ix addr-of`
(name -> address), which stays as-is.
…x walks
Two mathlib-blocking divergences from the Rust kernel, found via shard 0
of the mathlib env:
1. Canonical block sort (CanonicalCheck.lean): the Indc comparator never
descended into constructors, so same-shape mutual inductives (e.g.
Mathlib.Tactic.Ring's ExBase/ExProd/ExSum: identical flags, arities
and types, differing only in ctor types) collapsed into one alpha
class and the block was rejected (assert 0 != 1 in
validate_block_canonical). Mirror compare_kindc's ctors tail
(canonical_check.rs:299-338) via compare_kctor_idxs_ctx /
compare_kctor_pair_ctx, resolving ctor positions through `top`.
Further parity fixes in the same pass:
* ctx is now a real KMutCtx mirror (from_id_classes): (position,
class idx) pairs where same-class members share ONE index (weak-
Equal, no position-derived tiebreaks) and each Indc member's ctors
map at i + cidx.
* External const refs compare by 32-byte address (addr_cmp,
lexicographic like Rust's Address Ord) instead of by ingress
position, matching the order the compile-side sort produced.
* Rec-rule comparison drops the ingress-artifact global ctor idx —
(fields, rhs) only, mirroring canonical_check.rs:280-289.
* Refinement fuel is 1 + |members| (provably reaches fixpoint)
instead of a fixed 32.
* Ctx-less compare_kexpr Const arm compares levels before ref
(field-order parity; equality-only callers unaffected).
2. Inductive index walks (Inductive.lean): get_result_sort_level and
the motive/rec-type index-dom collectors peeled literal Foralls only.
Index binders can hide under definitional wrappers — Mathlib's
`inductive εClosure (S : Set σ) : Set σ` stores a type ending in
`Set σ` that only whnf exposes as `σ → Prop` — so εNFA.εClosure died
with `no match case for value 3` (KExprNode.App). whnf before every
peel, mirroring inductive.rs get_result_sort_level (line 2101+) and
build_motive_type_flat (2521-2531).
Repros (both now pass):
ix check --ixe mathlib.ixe Mathlib.Tactic.Ring.ExSum --interp bytecode
ix check --ixe mathlib.ixe εNFA.εClosure.step --interp bytecode
`lake test -- --ignored ixvm` green (599 assertions). Codegen
regenerated; FFT pins re-bumped — pin values reflect this branch's
stubbed verify_bytes_against and need one more bump when the stub is
reverted.
Known remaining issue: shard-scale runs crawl inside
collect_block_members (per-block O(|top|) rescans whose memo keys make
the query map blow up); fix planned separately.
emitDebug was a silent stub, so dbg! probes only printed under
--interp bytecode — kernel debugging was pinned to the slowest engine.
Emit the same println! the bytecode interpreter's Op::Debug arm
produces (label + comma-separated values), with the label escaped for
a Rust format-string literal. Production kernels contain no dbg!, so
generated output is unchanged there; probe builds now print at native
speed.
…red-aux validation
Three fixes surfaced by driving mathlib shard 0 to green:
1. Perf: derive_block_member_idxs rescanned the whole `top` list per
block, and each scan step's memo key included the block addr —
|blocks| x |top| query-map entries dominated shard-scale runs (99%
of CPU in QueryMap ops). One memoized walk now builds an rbtree from
addr_key(block_addr) to buckets of (full addr, ascending member
positions); every query is an O(log N) lookup + address_eq-confirmed
bucket walk (4-byte key collisions cost a short walk, never a wrong
member list).
2. Aux synthesis (Lean.Compiler.LCNF.Cases): synth_aux_ind_ty /
synth_aux_ctor_ty never instantiated the occurrence's universe args,
so synthesized types kept `Type u` (Param 0) where the block stores
monomorphized `Type 0`; binder peels also assumed literal Foralls.
Mirror canonical_aux_order's construction (inductive.rs:1169-1234):
instantiate occurrence_us, whnf before each peel. addrs is threaded
through the build_flat_block chain for the whnf calls.
3. Stored-aux validation removed: validate_block_auxes assumed Muts
blocks store synthesized aux inductives and classified members with
is_aux_inductive ("no own nested occurrence but some member has
one"). Stored blocks only ever contain the source originals — the
Rust kernel seeds every stored member is_aux:false (rs:537) and
flags auxes only on transient detection (rs:755) — so the heuristic
misclassified originals in mixed blocks (LCNF's Alt/FunDecl/Code,
where only Cases carries the nested occurrence) and asserted on a
legitimate block. Smuggled extra members remain rejected by the
recursor-vs-canonical-type equality over the detected flat block.
Validation: full shard-0 native run green (9357/9357 owned consts);
`lake test -- --ignored ixvm` green (599 assertions; codegen
regenerated, FFT pins re-bumped — still relative to this branch's
stubbed verify_bytes_against).
Known residual divergence (not yet observed failing): Rust reorders
the detected aux portion of a flat block via canonical_aux_order
before recursor checks; Aiur keeps discovery order.
The thin Cli wrapper around Ix.Cli.CheckCmd.runCheckCmd had drifted
from the real command (its `interp` flag was still a bare bool while
`ix check --interp` takes a mode string), and `lake exe ix check`
rebuilds just as incrementally for day-to-day kernel iteration. One
entrypoint, no drift.
Stored recursors bake the COMPILER's canonical aux order into their
motive/minor layout, but build_flat_block discovers auxes in queue
(traversal) order — position-by-position recursor matching only worked
when the two orders happened to coincide. Mirror inductive.rs
canonical_aux_order (rs:1058+, applied at rs:2293-2321): after the
queue pass, re-sort the aux suffix by partition refinement over
synthetic aux views.
Aiur-isms vs the Rust original:
* Synthetic addresses are replaced by SENTINEL positions
`|top| + ordinal`: each aux's view (ext type/ctors with occurrence
universe args instantiated, spec_params substituted, block params
wrapped) rewrites nested aux occurrences to sentinel Consts, fixed
per ordinal so views synthesize once; each refinement round's ctx
maps sentinels to their current class, so same-class refs compare
weak-Equal — the same mechanism the canonical block sort uses for
block-local refs.
* Ties keep DISCOVERY order (stable insert). Rust's sort_by_compare is
a stable merge sort; a first cut with an unstable insertion sort
REVERSED tied pairs, swapping content-identical auxes whose spec
params differ only in phantom parameters (IxVMInd.DedupM's
Bar2⟨·,Nat⟩ / Bar2⟨·,Bool⟩) and breaking their rec-type match.
* The reorder is unconditional: every env Aiur checks comes through
the Ix compile pipeline (RecursorAuxOrder::Canonical); the Lean-
source order case Rust skips (rs:2293) cannot reach this kernel.
Validation: `lake test -- --ignored ixvm` green (599 assertions; FFT
pins re-bumped, still relative to this branch's stubbed
verify_bytes_against); bytecode repros green for the multi-aux blocks
Lean.Compiler.LCNF.Cases, Lean.Syntax.rec, IxVMInd.DedupM.rec,
IxVMInd.DepthM.rec; full mathlib shard-0 native run green
(9357/9357 owned consts). Codegen regenerated.
Side finding for the compile side: copying the DedupM/Bar2 fixtures
into the CLI env under a different namespace (IxDbgFixtures) makes
compile_env fail that block with "compute_aux_perm: no canonical match
for in-SCC source aux #1" while the identical structure compiles fine
as IxVMInd.* in the test env — compile-side aux matching looks
name-order sensitive. Repro: re-add the four fixtures from
Tests/Ix/IxVM.lean:93-108 to any ix-CLI-visible module under a fresh
namespace and run `ix check <ns>.DedupM.rec`.
Reverts the probe-branch stub (b7a0336) that disabled blake3
verification of constant/blob/claim bytes during the mathlib shard-0
debugging campaign. Codegen regenerated; FFT pins re-bumped with
hashing back in the circuit.
The pins land within ±1.1% of main (median ratio 0.997): the kernel
work on this branch — canonical-sort comparator parity, whnf-aware
index walks, the block-members table, aux-synthesis parity and
canonical aux order — is cost-neutral. Nested-aux-heavy targets
(DedupM, DepthM, AuxDedup*, Lean.Syntax.rec) got 0.5-1.1% cheaper;
small stdlib targets (HEq, Nat, Eq.rec) pay 0.4-0.7% for the richer
comparator and whnf walks.
`lake test -- --ignored ixvm` green (599 assertions).
@arthurpaulino
arthurpaulino enabled auto-merge (squash) July 17, 2026 17:58
@arthurpaulino
arthurpaulino merged commit 4e554d2 into mainJul 17, 2026
8 of 9 checks passed
@arthurpaulino
arthurpaulino deleted the ap/probe-mathlib branch July 17, 2026 17:58
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@arthurpaulino@johnchandlerburnham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

IxVM Aiur kernel: drive mathlib shard 0 to green - #485

Merged
arthurpaulino merged 8 commits into
mainfrom
ap/probe-mathlib
Jul 17, 2026
Merged

IxVM Aiur kernel: drive mathlib shard 0 to green#485
arthurpaulino merged 8 commits into
mainfrom
ap/probe-mathlib

Conversation

@arthurpaulino

@arthurpaulinoarthurpaulino commented Jul 11, 2026

Copy link
Copy Markdown
Member

Summary

ix check --ixe mathlib.ixe --ixes mathlib.ixes --shard 0 died with an
opaque assert_eq mismatch: 0 != 1. Driving it to green surfaced four
independent correctness divergences between the Aiur kernel and its Rust
mirror, one shard-scale performance pathology, and some missing debugging
infrastructure. All are fixed here: shard 0 (9,357 owned constants of the
647,127-const mathlib env) typechecks end-to-end through the native
codegen kernel, lake test -- --ignored ixvm is green (599 assertions),
and the FFT pins land within ±1.1% of main (median ratio 0.997) — the
kernel surgery is cost-neutral.

Kernel fixes

  1. Canonical block sort compares constructors (CanonicalCheck.lean).
    The Indc comparator stopped at the inductive's type, so same-shape
    mutual inductives (Mathlib.Tactic.Ring's ExBase/ExProd/ExSum:
    identical flags, arities and types, differing only in ctor types)
    collapsed into one alpha class and the block was rejected. Now mirrors
    compare_kindc's ctors tail, with a real KMutCtx (same-class members
    share one index; ctors mapped at i + cidx), address-based external-ref
    comparison, (fields, rhs)-only rec-rule comparison, and a provably
    sufficient refinement fuel bound.

  2. whnf-aware index walks (Inductive.lean). get_result_sort_level
    and the motive/rec-type index-domain collectors peeled literal Foralls
    only; index binders can hide under definitional wrappers —
    inductive εClosure (S : Set σ) : Set σ stores a type ending in
    Set σ that only whnf exposes as σ → Prop. whnf before every peel,
    mirroring the Rust walks.

  3. Aux synthesis parity + stored-aux validation removal
    (Inductive.lean). Synthesized aux types never instantiated the
    occurrence's universe args (Type u vs the stored monomorphized
    Type 0) and peeled binders structurally; fixed per
    canonical_aux_order's construction. The stored-aux validation pass
    (validate_block_auxes / is_aux_inductive) is removed outright:
    stored Muts blocks only ever contain the source originals (Rust seeds
    every stored member is_aux: false; auxes are transient detection
    results), and the classification heuristic misclassified originals in
    mixed blocks (Lean.Compiler.LCNF's Alt/FunDecl/Cases/Code),
    rejecting legitimate envs.

  4. Canonical aux order (Inductive.lean). Stored recursors bake the
    compiler's canonical aux order into their motive/minor layout, while
    build_flat_block discovers auxes in traversal order. The aux suffix
    is now re-sorted by partition refinement over synthetic aux views,
    with sentinel positions (|top| + ordinal) standing in for Rust's
    synthetic addresses and stable tie handling — an unstable sort
    reverses content-identical auxes whose spec params differ only in
    phantom parameters (IxVMInd.DedupM) and breaks rec-type matching.

Performance

  • Block-membership table. derive_block_member_idxs rescanned all of
    top per block with per-block memo keys — |blocks| × |top| query-map
    entries ate 99% of shard-scale CPU. One shared memoized walk now builds
    an rbtree from addr_key(block_addr) to address_eq-confirmed buckets;
    each query is an O(log N) lookup.

Tooling

  • ix name-of: new subcommand resolving a content address back to
    its Lean names against a .ixe env — potentially many, since
    structurally equivalent constants collapse to the same address (every
    matching entry in the env's named table is printed; the one-slot
    addrToName index would drop aliases). Unnamed Muts blocks fall back
    to scanning for projections into the block — turns an anonymized
    failing address into an ix check <name> fast repro. ix addr-of
    (name → address) is unchanged.
  • dbg! prints in the codegen kernel: emitDebug was a silent stub;
    it now emits the same println! the bytecode interpreter produces, so
    probe builds run at native speed. Production kernels contain no dbg!,
    so generated output is unchanged there.
  • lake exe check shim dropped: it had drifted from ix check
    (bool interp flag vs mode string); one entrypoint, no drift.

Validation

  • Full mathlib shard-0 native run: green, 9,357/9,357 owned consts.
  • Bytecode repros green for every constant that failed along the way:
    Mathlib.Tactic.Ring.ExSum, εNFA.εClosure.step,
    Lean.Compiler.LCNF.Cases, plus Lean.Syntax.rec /
    IxVMInd.DedupM.rec / IxVMInd.DepthM.rec for the aux machinery.
  • lake test -- --ignored ixvm: 599 assertions, 0 failures.
  • verify_bytes_against was stubbed during the debugging campaign and is
    restored; final pins include blake3 verification and sit within ±1.1%
    of main.

Known follow-ups

  • Shards 1–63 have not been run (pure machine time; ~300 GB peak RSS per
    shard, so sequential on a single box).

@arthurpaulino
arthurpaulinoforce-pushed the ap/probe-mathlib branch 3 times, most recently from 309e1c4 to de33f23CompareJuly 11, 2026 11:28
@arthurpaulino

Copy link
Copy Markdown
MemberAuthor

!benchmark

@argument-ci-bot

Copy link
Copy Markdown
Contributor

!benchmark — main vs de33f23

backends: aiur=prove · envs: InitStd · set: primary · shard: 0

aiur · InitStd · prove — main from: bencher @ 16455ba + base run (1 new)

constantprove-time (main)prove-time (PR)Δ%throughput (main)throughput (PR)Δ%peak-ram (main)peak-ram (PR)Δ%execute-time (main)execute-time (PR)Δ%verify-time (main)verify-time (PR)Δ%proof-size (main)proof-size (PR)Δ%fft-cost (main)fft-cost (PR)Δ%
ByteArray.utf8DecodeChar?_utf8EncodeChar_append54.933 s56.778 s+3.4% ⚠️52.61050.900-3.3% ⚠️100.89 GiB100.61 GiB-0.3%11.817 s12.508 s+5.8% (1.06× slower) ⚠️208.4 ms219.8 ms+5.5% (1.05× slower) ⚠️33.27 MiB34.43 MiB+3.5% ⚠️42.43B42.29B-0.3%
Array.extract_append53.157 s55.123 s+3.7% ⚠️32.09030.950-3.6% ⚠️92.86 GiB92.66 GiB-0.2%12.727 s13.021 s+2.3%210.3 ms214.2 ms+1.9%33.27 MiB34.43 MiB+3.5% ⚠️40.03B39.96B-0.2%
Char.ofOrdinal_le_of_le43.097 s43.778 s+1.6%66.41065.370-1.6%76.04 GiB75.49 GiB-0.7%8.947 s9.087 s+1.6%209.1 ms240.6 ms+15.1% (1.15× slower) ⚠️33.27 MiB34.43 MiB+3.5% ⚠️32.73B32.54B-0.6%
Vector.extract_append._proof_231.148 s32.048 s+2.9%46.26044.960-2.8%53.50 GiB53.32 GiB-0.3%6.820 s7.167 s+5.1% (1.05× slower) ⚠️215.8 ms238.4 ms+10.5% (1.10× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️23.34B23.29B-0.2%
_private.Init.Data.Range.Polymorphic.SInt.0.Int64.instRxcHasSize_eq26.216 s26.605 s+1.5%75.26074.160-1.5%48.88 GiB48.34 GiB-1.1%4.401 s4.443 s+0.9%205.7 ms222.9 ms+8.4% (1.08× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️17.43B17.35B-0.5%
String.split24.576 s25.436 s+3.5% ⚠️79.30076.620-3.4% ⚠️47.09 GiB46.75 GiB-0.7%4.030 s4.182 s+3.8% ⚠️205.0 ms219.6 ms+7.1% (1.07× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️15.99B15.89B-0.6%
List.mergeSort16.628 s17.267 s+3.8% ⚠️96.10092.540-3.7% ⚠️29.84 GiB29.71 GiB-0.4%2.782 s2.867 s+3.1% ⚠️205.2 ms215.4 ms+5.0% ⚠️33.19 MiB34.35 MiB+3.5% ⚠️11.63B11.58B-0.4%
Vector.append5.218 s5.439 s+4.2% ⚠️108.480104.060-4.1% ⚠️8.72 GiB8.90 GiB+2.0%529.2 ms541.8 ms+2.4%203.3 ms233.7 ms+15.0% (1.15× slower) ⚠️32.96 MiB34.12 MiB+3.5% ⚠️2.60B2.58B-0.5%
Nat.gcd_comm4.158 s4.382 s+5.4% (1.05× slower) ⚠️99.80094.700-5.1% (1.05× slower) ⚠️7.09 GiB7.01 GiB-1.0%349.0 ms366.4 ms+5.0% ⚠️218.0 ms215.4 ms-1.2%32.96 MiB34.12 MiB+3.5% ⚠️1.75B1.75B-0.2%
String.append2.973 s3.063 s+3.0% ⚠️118.750115.250-2.9%4.42 GiB4.57 GiB+3.5% ⚠️186.0 ms189.3 ms+1.8%201.0 ms223.1 ms+11.0% (1.11× slower) ⚠️32.89 MiB34.05 MiB+3.5% ⚠️967.65M963.43M-0.4%
Int.gcd2.350 s2.469 s+5.0% (1.05× slower) ⚠️97.43092.760-4.8% (1.05× slower) ⚠️3.25 GiB3.27 GiB+0.6%121.2 ms126.6 ms+4.5% ⚠️203.0 ms206.7 ms+1.8%32.89 MiB34.05 MiB+3.5% ⚠️605.84M604.12M-0.3%
Nat.sub_le_of_le_add2.347 s2.287 s-2.6%80.94083.090+2.7%3.06 GiB3.06 GiB-0.0%95.2 ms103.9 ms+9.2% (1.09× slower) ⚠️204.7 ms218.8 ms+6.9% (1.07× slower) ⚠️32.89 MiB34.05 MiB+3.5% ⚠️510.60M509.28M-0.3%
Nat.add_comm1.156 s1.258 s+8.8% (1.09× slower) ⚠️44.10040.530-8.1% (1.09× slower) ⚠️1.43 GiB1.46 GiB+2.3%15.5 ms16.0 ms+3.0% ⚠️194.5 ms208.9 ms+7.4% (1.07× slower) ⚠️32.76 MiB33.91 MiB+3.5% ⚠️53.90M53.75M-0.3%
Std.Tactic.BVDecide.BVExpr.bitblast.goCache_Inv_of_Inv._mutualOOMOOMn/aOOMOOMn/aOOMOOMn/a1m 21.4s1m 21.4s+0.1%OOMOOMn/aOOMOOMn/a261.03B260.46B-0.2%

14 constants · 13 regressed · 0 improved (|Δ| > 3.0% on any metric).

Workflow logs

@arthurpaulino
arthurpaulinoforce-pushed the ap/probe-mathlib branch 2 times, most recently from 6051448 to 6c5daa4CompareJuly 14, 2026 00:13
`ix name-of --ixe env.ixe <64-hex-addr>` resolves a content address
back to its Lean names. Potentially MANY names: structurally
equivalent constants collapse to the same content address, so every
entry in the env's `named` table pointing at the address is printed,
one per line (the `addrToName` reverse index keeps only one name per
address and would silently drop the aliases). Addresses of unnamed
Muts blocks fall back to scanning for projection constants into the
block and printing their names. Turns an anonymized failing address
surfaced by the IxVM kernel into an `ix check <name>` fast repro.
Lives in its own subcommand instead of overloading `ix addr-of`
(name -> address), which stays as-is.
…x walks
Two mathlib-blocking divergences from the Rust kernel, found via shard 0
of the mathlib env:
1. Canonical block sort (CanonicalCheck.lean): the Indc comparator never
descended into constructors, so same-shape mutual inductives (e.g.
Mathlib.Tactic.Ring's ExBase/ExProd/ExSum: identical flags, arities
and types, differing only in ctor types) collapsed into one alpha
class and the block was rejected (assert 0 != 1 in
validate_block_canonical). Mirror compare_kindc's ctors tail
(canonical_check.rs:299-338) via compare_kctor_idxs_ctx /
compare_kctor_pair_ctx, resolving ctor positions through `top`.
Further parity fixes in the same pass:
* ctx is now a real KMutCtx mirror (from_id_classes): (position,
class idx) pairs where same-class members share ONE index (weak-
Equal, no position-derived tiebreaks) and each Indc member's ctors
map at i + cidx.
* External const refs compare by 32-byte address (addr_cmp,
lexicographic like Rust's Address Ord) instead of by ingress
position, matching the order the compile-side sort produced.
* Rec-rule comparison drops the ingress-artifact global ctor idx —
(fields, rhs) only, mirroring canonical_check.rs:280-289.
* Refinement fuel is 1 + |members| (provably reaches fixpoint)
instead of a fixed 32.
* Ctx-less compare_kexpr Const arm compares levels before ref
(field-order parity; equality-only callers unaffected).
2. Inductive index walks (Inductive.lean): get_result_sort_level and
the motive/rec-type index-dom collectors peeled literal Foralls only.
Index binders can hide under definitional wrappers — Mathlib's
`inductive εClosure (S : Set σ) : Set σ` stores a type ending in
`Set σ` that only whnf exposes as `σ → Prop` — so εNFA.εClosure died
with `no match case for value 3` (KExprNode.App). whnf before every
peel, mirroring inductive.rs get_result_sort_level (line 2101+) and
build_motive_type_flat (2521-2531).
Repros (both now pass):
ix check --ixe mathlib.ixe Mathlib.Tactic.Ring.ExSum --interp bytecode
ix check --ixe mathlib.ixe εNFA.εClosure.step --interp bytecode
`lake test -- --ignored ixvm` green (599 assertions). Codegen
regenerated; FFT pins re-bumped — pin values reflect this branch's
stubbed verify_bytes_against and need one more bump when the stub is
reverted.
Known remaining issue: shard-scale runs crawl inside
collect_block_members (per-block O(|top|) rescans whose memo keys make
the query map blow up); fix planned separately.
emitDebug was a silent stub, so dbg! probes only printed under
--interp bytecode — kernel debugging was pinned to the slowest engine.
Emit the same println! the bytecode interpreter's Op::Debug arm
produces (label + comma-separated values), with the label escaped for
a Rust format-string literal. Production kernels contain no dbg!, so
generated output is unchanged there; probe builds now print at native
speed.
…red-aux validation
Three fixes surfaced by driving mathlib shard 0 to green:
1. Perf: derive_block_member_idxs rescanned the whole `top` list per
block, and each scan step's memo key included the block addr —
|blocks| x |top| query-map entries dominated shard-scale runs (99%
of CPU in QueryMap ops). One memoized walk now builds an rbtree from
addr_key(block_addr) to buckets of (full addr, ascending member
positions); every query is an O(log N) lookup + address_eq-confirmed
bucket walk (4-byte key collisions cost a short walk, never a wrong
member list).
2. Aux synthesis (Lean.Compiler.LCNF.Cases): synth_aux_ind_ty /
synth_aux_ctor_ty never instantiated the occurrence's universe args,
so synthesized types kept `Type u` (Param 0) where the block stores
monomorphized `Type 0`; binder peels also assumed literal Foralls.
Mirror canonical_aux_order's construction (inductive.rs:1169-1234):
instantiate occurrence_us, whnf before each peel. addrs is threaded
through the build_flat_block chain for the whnf calls.
3. Stored-aux validation removed: validate_block_auxes assumed Muts
blocks store synthesized aux inductives and classified members with
is_aux_inductive ("no own nested occurrence but some member has
one"). Stored blocks only ever contain the source originals — the
Rust kernel seeds every stored member is_aux:false (rs:537) and
flags auxes only on transient detection (rs:755) — so the heuristic
misclassified originals in mixed blocks (LCNF's Alt/FunDecl/Code,
where only Cases carries the nested occurrence) and asserted on a
legitimate block. Smuggled extra members remain rejected by the
recursor-vs-canonical-type equality over the detected flat block.
Validation: full shard-0 native run green (9357/9357 owned consts);
`lake test -- --ignored ixvm` green (599 assertions; codegen
regenerated, FFT pins re-bumped — still relative to this branch's
stubbed verify_bytes_against).
Known residual divergence (not yet observed failing): Rust reorders
the detected aux portion of a flat block via canonical_aux_order
before recursor checks; Aiur keeps discovery order.
The thin Cli wrapper around Ix.Cli.CheckCmd.runCheckCmd had drifted
from the real command (its `interp` flag was still a bare bool while
`ix check --interp` takes a mode string), and `lake exe ix check`
rebuilds just as incrementally for day-to-day kernel iteration. One
entrypoint, no drift.
Stored recursors bake the COMPILER's canonical aux order into their
motive/minor layout, but build_flat_block discovers auxes in queue
(traversal) order — position-by-position recursor matching only worked
when the two orders happened to coincide. Mirror inductive.rs
canonical_aux_order (rs:1058+, applied at rs:2293-2321): after the
queue pass, re-sort the aux suffix by partition refinement over
synthetic aux views.
Aiur-isms vs the Rust original:
* Synthetic addresses are replaced by SENTINEL positions
`|top| + ordinal`: each aux's view (ext type/ctors with occurrence
universe args instantiated, spec_params substituted, block params
wrapped) rewrites nested aux occurrences to sentinel Consts, fixed
per ordinal so views synthesize once; each refinement round's ctx
maps sentinels to their current class, so same-class refs compare
weak-Equal — the same mechanism the canonical block sort uses for
block-local refs.
* Ties keep DISCOVERY order (stable insert). Rust's sort_by_compare is
a stable merge sort; a first cut with an unstable insertion sort
REVERSED tied pairs, swapping content-identical auxes whose spec
params differ only in phantom parameters (IxVMInd.DedupM's
Bar2⟨·,Nat⟩ / Bar2⟨·,Bool⟩) and breaking their rec-type match.
* The reorder is unconditional: every env Aiur checks comes through
the Ix compile pipeline (RecursorAuxOrder::Canonical); the Lean-
source order case Rust skips (rs:2293) cannot reach this kernel.
Validation: `lake test -- --ignored ixvm` green (599 assertions; FFT
pins re-bumped, still relative to this branch's stubbed
verify_bytes_against); bytecode repros green for the multi-aux blocks
Lean.Compiler.LCNF.Cases, Lean.Syntax.rec, IxVMInd.DedupM.rec,
IxVMInd.DepthM.rec; full mathlib shard-0 native run green
(9357/9357 owned consts). Codegen regenerated.
Side finding for the compile side: copying the DedupM/Bar2 fixtures
into the CLI env under a different namespace (IxDbgFixtures) makes
compile_env fail that block with "compute_aux_perm: no canonical match
for in-SCC source aux #1" while the identical structure compiles fine
as IxVMInd.* in the test env — compile-side aux matching looks
name-order sensitive. Repro: re-add the four fixtures from
Tests/Ix/IxVM.lean:93-108 to any ix-CLI-visible module under a fresh
namespace and run `ix check <ns>.DedupM.rec`.
Reverts the probe-branch stub (b7a0336) that disabled blake3
verification of constant/blob/claim bytes during the mathlib shard-0
debugging campaign. Codegen regenerated; FFT pins re-bumped with
hashing back in the circuit.
The pins land within ±1.1% of main (median ratio 0.997): the kernel
work on this branch — canonical-sort comparator parity, whnf-aware
index walks, the block-members table, aux-synthesis parity and
canonical aux order — is cost-neutral. Nested-aux-heavy targets
(DedupM, DepthM, AuxDedup*, Lean.Syntax.rec) got 0.5-1.1% cheaper;
small stdlib targets (HEq, Nat, Eq.rec) pay 0.4-0.7% for the richer
comparator and whnf walks.
`lake test -- --ignored ixvm` green (599 assertions).
@arthurpaulino
arthurpaulino enabled auto-merge (squash) July 17, 2026 17:58
@arthurpaulino
arthurpaulino merged commit 4e554d2 into mainJul 17, 2026
8 of 9 checks passed
@arthurpaulino
arthurpaulino deleted the ap/probe-mathlib branch July 17, 2026 17:58
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@arthurpaulino@johnchandlerburnham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

IxVM Aiur kernel: drive mathlib shard 0 to green - #485

Merged
arthurpaulino merged 8 commits into
mainfrom
ap/probe-mathlib
Jul 17, 2026
Merged

IxVM Aiur kernel: drive mathlib shard 0 to green#485
arthurpaulino merged 8 commits into
mainfrom
ap/probe-mathlib

Conversation

@arthurpaulino

@arthurpaulinoarthurpaulino commented Jul 11, 2026

Copy link
Copy Markdown
Member

Summary

ix check --ixe mathlib.ixe --ixes mathlib.ixes --shard 0 died with an
opaque assert_eq mismatch: 0 != 1. Driving it to green surfaced four
independent correctness divergences between the Aiur kernel and its Rust
mirror, one shard-scale performance pathology, and some missing debugging
infrastructure. All are fixed here: shard 0 (9,357 owned constants of the
647,127-const mathlib env) typechecks end-to-end through the native
codegen kernel, lake test -- --ignored ixvm is green (599 assertions),
and the FFT pins land within ±1.1% of main (median ratio 0.997) — the
kernel surgery is cost-neutral.

Kernel fixes

  1. Canonical block sort compares constructors (CanonicalCheck.lean).
    The Indc comparator stopped at the inductive's type, so same-shape
    mutual inductives (Mathlib.Tactic.Ring's ExBase/ExProd/ExSum:
    identical flags, arities and types, differing only in ctor types)
    collapsed into one alpha class and the block was rejected. Now mirrors
    compare_kindc's ctors tail, with a real KMutCtx (same-class members
    share one index; ctors mapped at i + cidx), address-based external-ref
    comparison, (fields, rhs)-only rec-rule comparison, and a provably
    sufficient refinement fuel bound.

  2. whnf-aware index walks (Inductive.lean). get_result_sort_level
    and the motive/rec-type index-domain collectors peeled literal Foralls
    only; index binders can hide under definitional wrappers —
    inductive εClosure (S : Set σ) : Set σ stores a type ending in
    Set σ that only whnf exposes as σ → Prop. whnf before every peel,
    mirroring the Rust walks.

  3. Aux synthesis parity + stored-aux validation removal
    (Inductive.lean). Synthesized aux types never instantiated the
    occurrence's universe args (Type u vs the stored monomorphized
    Type 0) and peeled binders structurally; fixed per
    canonical_aux_order's construction. The stored-aux validation pass
    (validate_block_auxes / is_aux_inductive) is removed outright:
    stored Muts blocks only ever contain the source originals (Rust seeds
    every stored member is_aux: false; auxes are transient detection
    results), and the classification heuristic misclassified originals in
    mixed blocks (Lean.Compiler.LCNF's Alt/FunDecl/Cases/Code),
    rejecting legitimate envs.

  4. Canonical aux order (Inductive.lean). Stored recursors bake the
    compiler's canonical aux order into their motive/minor layout, while
    build_flat_block discovers auxes in traversal order. The aux suffix
    is now re-sorted by partition refinement over synthetic aux views,
    with sentinel positions (|top| + ordinal) standing in for Rust's
    synthetic addresses and stable tie handling — an unstable sort
    reverses content-identical auxes whose spec params differ only in
    phantom parameters (IxVMInd.DedupM) and breaks rec-type matching.

Performance

  • Block-membership table. derive_block_member_idxs rescanned all of
    top per block with per-block memo keys — |blocks| × |top| query-map
    entries ate 99% of shard-scale CPU. One shared memoized walk now builds
    an rbtree from addr_key(block_addr) to address_eq-confirmed buckets;
    each query is an O(log N) lookup.

Tooling

  • ix name-of: new subcommand resolving a content address back to
    its Lean names against a .ixe env — potentially many, since
    structurally equivalent constants collapse to the same address (every
    matching entry in the env's named table is printed; the one-slot
    addrToName index would drop aliases). Unnamed Muts blocks fall back
    to scanning for projections into the block — turns an anonymized
    failing address into an ix check <name> fast repro. ix addr-of
    (name → address) is unchanged.
  • dbg! prints in the codegen kernel: emitDebug was a silent stub;
    it now emits the same println! the bytecode interpreter produces, so
    probe builds run at native speed. Production kernels contain no dbg!,
    so generated output is unchanged there.
  • lake exe check shim dropped: it had drifted from ix check
    (bool interp flag vs mode string); one entrypoint, no drift.

Validation

  • Full mathlib shard-0 native run: green, 9,357/9,357 owned consts.
  • Bytecode repros green for every constant that failed along the way:
    Mathlib.Tactic.Ring.ExSum, εNFA.εClosure.step,
    Lean.Compiler.LCNF.Cases, plus Lean.Syntax.rec /
    IxVMInd.DedupM.rec / IxVMInd.DepthM.rec for the aux machinery.
  • lake test -- --ignored ixvm: 599 assertions, 0 failures.
  • verify_bytes_against was stubbed during the debugging campaign and is
    restored; final pins include blake3 verification and sit within ±1.1%
    of main.

Known follow-ups

  • Shards 1–63 have not been run (pure machine time; ~300 GB peak RSS per
    shard, so sequential on a single box).

@arthurpaulino
arthurpaulinoforce-pushed the ap/probe-mathlib branch 3 times, most recently from 309e1c4 to de33f23CompareJuly 11, 2026 11:28
@arthurpaulino

Copy link
Copy Markdown
MemberAuthor

!benchmark

@argument-ci-bot

Copy link
Copy Markdown
Contributor

!benchmark — main vs de33f23

backends: aiur=prove · envs: InitStd · set: primary · shard: 0

aiur · InitStd · prove — main from: bencher @ 16455ba + base run (1 new)

constantprove-time (main)prove-time (PR)Δ%throughput (main)throughput (PR)Δ%peak-ram (main)peak-ram (PR)Δ%execute-time (main)execute-time (PR)Δ%verify-time (main)verify-time (PR)Δ%proof-size (main)proof-size (PR)Δ%fft-cost (main)fft-cost (PR)Δ%
ByteArray.utf8DecodeChar?_utf8EncodeChar_append54.933 s56.778 s+3.4% ⚠️52.61050.900-3.3% ⚠️100.89 GiB100.61 GiB-0.3%11.817 s12.508 s+5.8% (1.06× slower) ⚠️208.4 ms219.8 ms+5.5% (1.05× slower) ⚠️33.27 MiB34.43 MiB+3.5% ⚠️42.43B42.29B-0.3%
Array.extract_append53.157 s55.123 s+3.7% ⚠️32.09030.950-3.6% ⚠️92.86 GiB92.66 GiB-0.2%12.727 s13.021 s+2.3%210.3 ms214.2 ms+1.9%33.27 MiB34.43 MiB+3.5% ⚠️40.03B39.96B-0.2%
Char.ofOrdinal_le_of_le43.097 s43.778 s+1.6%66.41065.370-1.6%76.04 GiB75.49 GiB-0.7%8.947 s9.087 s+1.6%209.1 ms240.6 ms+15.1% (1.15× slower) ⚠️33.27 MiB34.43 MiB+3.5% ⚠️32.73B32.54B-0.6%
Vector.extract_append._proof_231.148 s32.048 s+2.9%46.26044.960-2.8%53.50 GiB53.32 GiB-0.3%6.820 s7.167 s+5.1% (1.05× slower) ⚠️215.8 ms238.4 ms+10.5% (1.10× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️23.34B23.29B-0.2%
_private.Init.Data.Range.Polymorphic.SInt.0.Int64.instRxcHasSize_eq26.216 s26.605 s+1.5%75.26074.160-1.5%48.88 GiB48.34 GiB-1.1%4.401 s4.443 s+0.9%205.7 ms222.9 ms+8.4% (1.08× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️17.43B17.35B-0.5%
String.split24.576 s25.436 s+3.5% ⚠️79.30076.620-3.4% ⚠️47.09 GiB46.75 GiB-0.7%4.030 s4.182 s+3.8% ⚠️205.0 ms219.6 ms+7.1% (1.07× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️15.99B15.89B-0.6%
List.mergeSort16.628 s17.267 s+3.8% ⚠️96.10092.540-3.7% ⚠️29.84 GiB29.71 GiB-0.4%2.782 s2.867 s+3.1% ⚠️205.2 ms215.4 ms+5.0% ⚠️33.19 MiB34.35 MiB+3.5% ⚠️11.63B11.58B-0.4%
Vector.append5.218 s5.439 s+4.2% ⚠️108.480104.060-4.1% ⚠️8.72 GiB8.90 GiB+2.0%529.2 ms541.8 ms+2.4%203.3 ms233.7 ms+15.0% (1.15× slower) ⚠️32.96 MiB34.12 MiB+3.5% ⚠️2.60B2.58B-0.5%
Nat.gcd_comm4.158 s4.382 s+5.4% (1.05× slower) ⚠️99.80094.700-5.1% (1.05× slower) ⚠️7.09 GiB7.01 GiB-1.0%349.0 ms366.4 ms+5.0% ⚠️218.0 ms215.4 ms-1.2%32.96 MiB34.12 MiB+3.5% ⚠️1.75B1.75B-0.2%
String.append2.973 s3.063 s+3.0% ⚠️118.750115.250-2.9%4.42 GiB4.57 GiB+3.5% ⚠️186.0 ms189.3 ms+1.8%201.0 ms223.1 ms+11.0% (1.11× slower) ⚠️32.89 MiB34.05 MiB+3.5% ⚠️967.65M963.43M-0.4%
Int.gcd2.350 s2.469 s+5.0% (1.05× slower) ⚠️97.43092.760-4.8% (1.05× slower) ⚠️3.25 GiB3.27 GiB+0.6%121.2 ms126.6 ms+4.5% ⚠️203.0 ms206.7 ms+1.8%32.89 MiB34.05 MiB+3.5% ⚠️605.84M604.12M-0.3%
Nat.sub_le_of_le_add2.347 s2.287 s-2.6%80.94083.090+2.7%3.06 GiB3.06 GiB-0.0%95.2 ms103.9 ms+9.2% (1.09× slower) ⚠️204.7 ms218.8 ms+6.9% (1.07× slower) ⚠️32.89 MiB34.05 MiB+3.5% ⚠️510.60M509.28M-0.3%
Nat.add_comm1.156 s1.258 s+8.8% (1.09× slower) ⚠️44.10040.530-8.1% (1.09× slower) ⚠️1.43 GiB1.46 GiB+2.3%15.5 ms16.0 ms+3.0% ⚠️194.5 ms208.9 ms+7.4% (1.07× slower) ⚠️32.76 MiB33.91 MiB+3.5% ⚠️53.90M53.75M-0.3%
Std.Tactic.BVDecide.BVExpr.bitblast.goCache_Inv_of_Inv._mutualOOMOOMn/aOOMOOMn/aOOMOOMn/a1m 21.4s1m 21.4s+0.1%OOMOOMn/aOOMOOMn/a261.03B260.46B-0.2%

14 constants · 13 regressed · 0 improved (|Δ| > 3.0% on any metric).

Workflow logs

@arthurpaulino
arthurpaulinoforce-pushed the ap/probe-mathlib branch 2 times, most recently from 6051448 to 6c5daa4CompareJuly 14, 2026 00:13
`ix name-of --ixe env.ixe <64-hex-addr>` resolves a content address
back to its Lean names. Potentially MANY names: structurally
equivalent constants collapse to the same content address, so every
entry in the env's `named` table pointing at the address is printed,
one per line (the `addrToName` reverse index keeps only one name per
address and would silently drop the aliases). Addresses of unnamed
Muts blocks fall back to scanning for projection constants into the
block and printing their names. Turns an anonymized failing address
surfaced by the IxVM kernel into an `ix check <name>` fast repro.
Lives in its own subcommand instead of overloading `ix addr-of`
(name -> address), which stays as-is.
…x walks
Two mathlib-blocking divergences from the Rust kernel, found via shard 0
of the mathlib env:
1. Canonical block sort (CanonicalCheck.lean): the Indc comparator never
descended into constructors, so same-shape mutual inductives (e.g.
Mathlib.Tactic.Ring's ExBase/ExProd/ExSum: identical flags, arities
and types, differing only in ctor types) collapsed into one alpha
class and the block was rejected (assert 0 != 1 in
validate_block_canonical). Mirror compare_kindc's ctors tail
(canonical_check.rs:299-338) via compare_kctor_idxs_ctx /
compare_kctor_pair_ctx, resolving ctor positions through `top`.
Further parity fixes in the same pass:
* ctx is now a real KMutCtx mirror (from_id_classes): (position,
class idx) pairs where same-class members share ONE index (weak-
Equal, no position-derived tiebreaks) and each Indc member's ctors
map at i + cidx.
* External const refs compare by 32-byte address (addr_cmp,
lexicographic like Rust's Address Ord) instead of by ingress
position, matching the order the compile-side sort produced.
* Rec-rule comparison drops the ingress-artifact global ctor idx —
(fields, rhs) only, mirroring canonical_check.rs:280-289.
* Refinement fuel is 1 + |members| (provably reaches fixpoint)
instead of a fixed 32.
* Ctx-less compare_kexpr Const arm compares levels before ref
(field-order parity; equality-only callers unaffected).
2. Inductive index walks (Inductive.lean): get_result_sort_level and
the motive/rec-type index-dom collectors peeled literal Foralls only.
Index binders can hide under definitional wrappers — Mathlib's
`inductive εClosure (S : Set σ) : Set σ` stores a type ending in
`Set σ` that only whnf exposes as `σ → Prop` — so εNFA.εClosure died
with `no match case for value 3` (KExprNode.App). whnf before every
peel, mirroring inductive.rs get_result_sort_level (line 2101+) and
build_motive_type_flat (2521-2531).
Repros (both now pass):
ix check --ixe mathlib.ixe Mathlib.Tactic.Ring.ExSum --interp bytecode
ix check --ixe mathlib.ixe εNFA.εClosure.step --interp bytecode
`lake test -- --ignored ixvm` green (599 assertions). Codegen
regenerated; FFT pins re-bumped — pin values reflect this branch's
stubbed verify_bytes_against and need one more bump when the stub is
reverted.
Known remaining issue: shard-scale runs crawl inside
collect_block_members (per-block O(|top|) rescans whose memo keys make
the query map blow up); fix planned separately.
emitDebug was a silent stub, so dbg! probes only printed under
--interp bytecode — kernel debugging was pinned to the slowest engine.
Emit the same println! the bytecode interpreter's Op::Debug arm
produces (label + comma-separated values), with the label escaped for
a Rust format-string literal. Production kernels contain no dbg!, so
generated output is unchanged there; probe builds now print at native
speed.
…red-aux validation
Three fixes surfaced by driving mathlib shard 0 to green:
1. Perf: derive_block_member_idxs rescanned the whole `top` list per
block, and each scan step's memo key included the block addr —
|blocks| x |top| query-map entries dominated shard-scale runs (99%
of CPU in QueryMap ops). One memoized walk now builds an rbtree from
addr_key(block_addr) to buckets of (full addr, ascending member
positions); every query is an O(log N) lookup + address_eq-confirmed
bucket walk (4-byte key collisions cost a short walk, never a wrong
member list).
2. Aux synthesis (Lean.Compiler.LCNF.Cases): synth_aux_ind_ty /
synth_aux_ctor_ty never instantiated the occurrence's universe args,
so synthesized types kept `Type u` (Param 0) where the block stores
monomorphized `Type 0`; binder peels also assumed literal Foralls.
Mirror canonical_aux_order's construction (inductive.rs:1169-1234):
instantiate occurrence_us, whnf before each peel. addrs is threaded
through the build_flat_block chain for the whnf calls.
3. Stored-aux validation removed: validate_block_auxes assumed Muts
blocks store synthesized aux inductives and classified members with
is_aux_inductive ("no own nested occurrence but some member has
one"). Stored blocks only ever contain the source originals — the
Rust kernel seeds every stored member is_aux:false (rs:537) and
flags auxes only on transient detection (rs:755) — so the heuristic
misclassified originals in mixed blocks (LCNF's Alt/FunDecl/Code,
where only Cases carries the nested occurrence) and asserted on a
legitimate block. Smuggled extra members remain rejected by the
recursor-vs-canonical-type equality over the detected flat block.
Validation: full shard-0 native run green (9357/9357 owned consts);
`lake test -- --ignored ixvm` green (599 assertions; codegen
regenerated, FFT pins re-bumped — still relative to this branch's
stubbed verify_bytes_against).
Known residual divergence (not yet observed failing): Rust reorders
the detected aux portion of a flat block via canonical_aux_order
before recursor checks; Aiur keeps discovery order.
The thin Cli wrapper around Ix.Cli.CheckCmd.runCheckCmd had drifted
from the real command (its `interp` flag was still a bare bool while
`ix check --interp` takes a mode string), and `lake exe ix check`
rebuilds just as incrementally for day-to-day kernel iteration. One
entrypoint, no drift.
Stored recursors bake the COMPILER's canonical aux order into their
motive/minor layout, but build_flat_block discovers auxes in queue
(traversal) order — position-by-position recursor matching only worked
when the two orders happened to coincide. Mirror inductive.rs
canonical_aux_order (rs:1058+, applied at rs:2293-2321): after the
queue pass, re-sort the aux suffix by partition refinement over
synthetic aux views.
Aiur-isms vs the Rust original:
* Synthetic addresses are replaced by SENTINEL positions
`|top| + ordinal`: each aux's view (ext type/ctors with occurrence
universe args instantiated, spec_params substituted, block params
wrapped) rewrites nested aux occurrences to sentinel Consts, fixed
per ordinal so views synthesize once; each refinement round's ctx
maps sentinels to their current class, so same-class refs compare
weak-Equal — the same mechanism the canonical block sort uses for
block-local refs.
* Ties keep DISCOVERY order (stable insert). Rust's sort_by_compare is
a stable merge sort; a first cut with an unstable insertion sort
REVERSED tied pairs, swapping content-identical auxes whose spec
params differ only in phantom parameters (IxVMInd.DedupM's
Bar2⟨·,Nat⟩ / Bar2⟨·,Bool⟩) and breaking their rec-type match.
* The reorder is unconditional: every env Aiur checks comes through
the Ix compile pipeline (RecursorAuxOrder::Canonical); the Lean-
source order case Rust skips (rs:2293) cannot reach this kernel.
Validation: `lake test -- --ignored ixvm` green (599 assertions; FFT
pins re-bumped, still relative to this branch's stubbed
verify_bytes_against); bytecode repros green for the multi-aux blocks
Lean.Compiler.LCNF.Cases, Lean.Syntax.rec, IxVMInd.DedupM.rec,
IxVMInd.DepthM.rec; full mathlib shard-0 native run green
(9357/9357 owned consts). Codegen regenerated.
Side finding for the compile side: copying the DedupM/Bar2 fixtures
into the CLI env under a different namespace (IxDbgFixtures) makes
compile_env fail that block with "compute_aux_perm: no canonical match
for in-SCC source aux #1" while the identical structure compiles fine
as IxVMInd.* in the test env — compile-side aux matching looks
name-order sensitive. Repro: re-add the four fixtures from
Tests/Ix/IxVM.lean:93-108 to any ix-CLI-visible module under a fresh
namespace and run `ix check <ns>.DedupM.rec`.
Reverts the probe-branch stub (b7a0336) that disabled blake3
verification of constant/blob/claim bytes during the mathlib shard-0
debugging campaign. Codegen regenerated; FFT pins re-bumped with
hashing back in the circuit.
The pins land within ±1.1% of main (median ratio 0.997): the kernel
work on this branch — canonical-sort comparator parity, whnf-aware
index walks, the block-members table, aux-synthesis parity and
canonical aux order — is cost-neutral. Nested-aux-heavy targets
(DedupM, DepthM, AuxDedup*, Lean.Syntax.rec) got 0.5-1.1% cheaper;
small stdlib targets (HEq, Nat, Eq.rec) pay 0.4-0.7% for the richer
comparator and whnf walks.
`lake test -- --ignored ixvm` green (599 assertions).
@arthurpaulino
arthurpaulino enabled auto-merge (squash) July 17, 2026 17:58
@arthurpaulino
arthurpaulino merged commit 4e554d2 into mainJul 17, 2026
8 of 9 checks passed
@arthurpaulino
arthurpaulino deleted the ap/probe-mathlib branch July 17, 2026 17:58
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@arthurpaulino@johnchandlerburnham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

IxVM Aiur kernel: drive mathlib shard 0 to green - #485

Merged
arthurpaulino merged 8 commits into
mainfrom
ap/probe-mathlib
Jul 17, 2026
Merged

IxVM Aiur kernel: drive mathlib shard 0 to green#485
arthurpaulino merged 8 commits into
mainfrom
ap/probe-mathlib

Conversation

@arthurpaulino

@arthurpaulinoarthurpaulino commented Jul 11, 2026

Copy link
Copy Markdown
Member

Summary

ix check --ixe mathlib.ixe --ixes mathlib.ixes --shard 0 died with an
opaque assert_eq mismatch: 0 != 1. Driving it to green surfaced four
independent correctness divergences between the Aiur kernel and its Rust
mirror, one shard-scale performance pathology, and some missing debugging
infrastructure. All are fixed here: shard 0 (9,357 owned constants of the
647,127-const mathlib env) typechecks end-to-end through the native
codegen kernel, lake test -- --ignored ixvm is green (599 assertions),
and the FFT pins land within ±1.1% of main (median ratio 0.997) — the
kernel surgery is cost-neutral.

Kernel fixes

  1. Canonical block sort compares constructors (CanonicalCheck.lean).
    The Indc comparator stopped at the inductive's type, so same-shape
    mutual inductives (Mathlib.Tactic.Ring's ExBase/ExProd/ExSum:
    identical flags, arities and types, differing only in ctor types)
    collapsed into one alpha class and the block was rejected. Now mirrors
    compare_kindc's ctors tail, with a real KMutCtx (same-class members
    share one index; ctors mapped at i + cidx), address-based external-ref
    comparison, (fields, rhs)-only rec-rule comparison, and a provably
    sufficient refinement fuel bound.

  2. whnf-aware index walks (Inductive.lean). get_result_sort_level
    and the motive/rec-type index-domain collectors peeled literal Foralls
    only; index binders can hide under definitional wrappers —
    inductive εClosure (S : Set σ) : Set σ stores a type ending in
    Set σ that only whnf exposes as σ → Prop. whnf before every peel,
    mirroring the Rust walks.

  3. Aux synthesis parity + stored-aux validation removal
    (Inductive.lean). Synthesized aux types never instantiated the
    occurrence's universe args (Type u vs the stored monomorphized
    Type 0) and peeled binders structurally; fixed per
    canonical_aux_order's construction. The stored-aux validation pass
    (validate_block_auxes / is_aux_inductive) is removed outright:
    stored Muts blocks only ever contain the source originals (Rust seeds
    every stored member is_aux: false; auxes are transient detection
    results), and the classification heuristic misclassified originals in
    mixed blocks (Lean.Compiler.LCNF's Alt/FunDecl/Cases/Code),
    rejecting legitimate envs.

  4. Canonical aux order (Inductive.lean). Stored recursors bake the
    compiler's canonical aux order into their motive/minor layout, while
    build_flat_block discovers auxes in traversal order. The aux suffix
    is now re-sorted by partition refinement over synthetic aux views,
    with sentinel positions (|top| + ordinal) standing in for Rust's
    synthetic addresses and stable tie handling — an unstable sort
    reverses content-identical auxes whose spec params differ only in
    phantom parameters (IxVMInd.DedupM) and breaks rec-type matching.

Performance

  • Block-membership table. derive_block_member_idxs rescanned all of
    top per block with per-block memo keys — |blocks| × |top| query-map
    entries ate 99% of shard-scale CPU. One shared memoized walk now builds
    an rbtree from addr_key(block_addr) to address_eq-confirmed buckets;
    each query is an O(log N) lookup.

Tooling

  • ix name-of: new subcommand resolving a content address back to
    its Lean names against a .ixe env — potentially many, since
    structurally equivalent constants collapse to the same address (every
    matching entry in the env's named table is printed; the one-slot
    addrToName index would drop aliases). Unnamed Muts blocks fall back
    to scanning for projections into the block — turns an anonymized
    failing address into an ix check <name> fast repro. ix addr-of
    (name → address) is unchanged.
  • dbg! prints in the codegen kernel: emitDebug was a silent stub;
    it now emits the same println! the bytecode interpreter produces, so
    probe builds run at native speed. Production kernels contain no dbg!,
    so generated output is unchanged there.
  • lake exe check shim dropped: it had drifted from ix check
    (bool interp flag vs mode string); one entrypoint, no drift.

Validation

  • Full mathlib shard-0 native run: green, 9,357/9,357 owned consts.
  • Bytecode repros green for every constant that failed along the way:
    Mathlib.Tactic.Ring.ExSum, εNFA.εClosure.step,
    Lean.Compiler.LCNF.Cases, plus Lean.Syntax.rec /
    IxVMInd.DedupM.rec / IxVMInd.DepthM.rec for the aux machinery.
  • lake test -- --ignored ixvm: 599 assertions, 0 failures.
  • verify_bytes_against was stubbed during the debugging campaign and is
    restored; final pins include blake3 verification and sit within ±1.1%
    of main.

Known follow-ups

  • Shards 1–63 have not been run (pure machine time; ~300 GB peak RSS per
    shard, so sequential on a single box).

@arthurpaulino
arthurpaulinoforce-pushed the ap/probe-mathlib branch 3 times, most recently from 309e1c4 to de33f23CompareJuly 11, 2026 11:28
@arthurpaulino

Copy link
Copy Markdown
MemberAuthor

!benchmark

@argument-ci-bot

Copy link
Copy Markdown
Contributor

!benchmark — main vs de33f23

backends: aiur=prove · envs: InitStd · set: primary · shard: 0

aiur · InitStd · prove — main from: bencher @ 16455ba + base run (1 new)

constantprove-time (main)prove-time (PR)Δ%throughput (main)throughput (PR)Δ%peak-ram (main)peak-ram (PR)Δ%execute-time (main)execute-time (PR)Δ%verify-time (main)verify-time (PR)Δ%proof-size (main)proof-size (PR)Δ%fft-cost (main)fft-cost (PR)Δ%
ByteArray.utf8DecodeChar?_utf8EncodeChar_append54.933 s56.778 s+3.4% ⚠️52.61050.900-3.3% ⚠️100.89 GiB100.61 GiB-0.3%11.817 s12.508 s+5.8% (1.06× slower) ⚠️208.4 ms219.8 ms+5.5% (1.05× slower) ⚠️33.27 MiB34.43 MiB+3.5% ⚠️42.43B42.29B-0.3%
Array.extract_append53.157 s55.123 s+3.7% ⚠️32.09030.950-3.6% ⚠️92.86 GiB92.66 GiB-0.2%12.727 s13.021 s+2.3%210.3 ms214.2 ms+1.9%33.27 MiB34.43 MiB+3.5% ⚠️40.03B39.96B-0.2%
Char.ofOrdinal_le_of_le43.097 s43.778 s+1.6%66.41065.370-1.6%76.04 GiB75.49 GiB-0.7%8.947 s9.087 s+1.6%209.1 ms240.6 ms+15.1% (1.15× slower) ⚠️33.27 MiB34.43 MiB+3.5% ⚠️32.73B32.54B-0.6%
Vector.extract_append._proof_231.148 s32.048 s+2.9%46.26044.960-2.8%53.50 GiB53.32 GiB-0.3%6.820 s7.167 s+5.1% (1.05× slower) ⚠️215.8 ms238.4 ms+10.5% (1.10× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️23.34B23.29B-0.2%
_private.Init.Data.Range.Polymorphic.SInt.0.Int64.instRxcHasSize_eq26.216 s26.605 s+1.5%75.26074.160-1.5%48.88 GiB48.34 GiB-1.1%4.401 s4.443 s+0.9%205.7 ms222.9 ms+8.4% (1.08× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️17.43B17.35B-0.5%
String.split24.576 s25.436 s+3.5% ⚠️79.30076.620-3.4% ⚠️47.09 GiB46.75 GiB-0.7%4.030 s4.182 s+3.8% ⚠️205.0 ms219.6 ms+7.1% (1.07× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️15.99B15.89B-0.6%
List.mergeSort16.628 s17.267 s+3.8% ⚠️96.10092.540-3.7% ⚠️29.84 GiB29.71 GiB-0.4%2.782 s2.867 s+3.1% ⚠️205.2 ms215.4 ms+5.0% ⚠️33.19 MiB34.35 MiB+3.5% ⚠️11.63B11.58B-0.4%
Vector.append5.218 s5.439 s+4.2% ⚠️108.480104.060-4.1% ⚠️8.72 GiB8.90 GiB+2.0%529.2 ms541.8 ms+2.4%203.3 ms233.7 ms+15.0% (1.15× slower) ⚠️32.96 MiB34.12 MiB+3.5% ⚠️2.60B2.58B-0.5%
Nat.gcd_comm4.158 s4.382 s+5.4% (1.05× slower) ⚠️99.80094.700-5.1% (1.05× slower) ⚠️7.09 GiB7.01 GiB-1.0%349.0 ms366.4 ms+5.0% ⚠️218.0 ms215.4 ms-1.2%32.96 MiB34.12 MiB+3.5% ⚠️1.75B1.75B-0.2%
String.append2.973 s3.063 s+3.0% ⚠️118.750115.250-2.9%4.42 GiB4.57 GiB+3.5% ⚠️186.0 ms189.3 ms+1.8%201.0 ms223.1 ms+11.0% (1.11× slower) ⚠️32.89 MiB34.05 MiB+3.5% ⚠️967.65M963.43M-0.4%
Int.gcd2.350 s2.469 s+5.0% (1.05× slower) ⚠️97.43092.760-4.8% (1.05× slower) ⚠️3.25 GiB3.27 GiB+0.6%121.2 ms126.6 ms+4.5% ⚠️203.0 ms206.7 ms+1.8%32.89 MiB34.05 MiB+3.5% ⚠️605.84M604.12M-0.3%
Nat.sub_le_of_le_add2.347 s2.287 s-2.6%80.94083.090+2.7%3.06 GiB3.06 GiB-0.0%95.2 ms103.9 ms+9.2% (1.09× slower) ⚠️204.7 ms218.8 ms+6.9% (1.07× slower) ⚠️32.89 MiB34.05 MiB+3.5% ⚠️510.60M509.28M-0.3%
Nat.add_comm1.156 s1.258 s+8.8% (1.09× slower) ⚠️44.10040.530-8.1% (1.09× slower) ⚠️1.43 GiB1.46 GiB+2.3%15.5 ms16.0 ms+3.0% ⚠️194.5 ms208.9 ms+7.4% (1.07× slower) ⚠️32.76 MiB33.91 MiB+3.5% ⚠️53.90M53.75M-0.3%
Std.Tactic.BVDecide.BVExpr.bitblast.goCache_Inv_of_Inv._mutualOOMOOMn/aOOMOOMn/aOOMOOMn/a1m 21.4s1m 21.4s+0.1%OOMOOMn/aOOMOOMn/a261.03B260.46B-0.2%

14 constants · 13 regressed · 0 improved (|Δ| > 3.0% on any metric).

Workflow logs

@arthurpaulino
arthurpaulinoforce-pushed the ap/probe-mathlib branch 2 times, most recently from 6051448 to 6c5daa4CompareJuly 14, 2026 00:13
`ix name-of --ixe env.ixe <64-hex-addr>` resolves a content address
back to its Lean names. Potentially MANY names: structurally
equivalent constants collapse to the same content address, so every
entry in the env's `named` table pointing at the address is printed,
one per line (the `addrToName` reverse index keeps only one name per
address and would silently drop the aliases). Addresses of unnamed
Muts blocks fall back to scanning for projection constants into the
block and printing their names. Turns an anonymized failing address
surfaced by the IxVM kernel into an `ix check <name>` fast repro.
Lives in its own subcommand instead of overloading `ix addr-of`
(name -> address), which stays as-is.
…x walks
Two mathlib-blocking divergences from the Rust kernel, found via shard 0
of the mathlib env:
1. Canonical block sort (CanonicalCheck.lean): the Indc comparator never
descended into constructors, so same-shape mutual inductives (e.g.
Mathlib.Tactic.Ring's ExBase/ExProd/ExSum: identical flags, arities
and types, differing only in ctor types) collapsed into one alpha
class and the block was rejected (assert 0 != 1 in
validate_block_canonical). Mirror compare_kindc's ctors tail
(canonical_check.rs:299-338) via compare_kctor_idxs_ctx /
compare_kctor_pair_ctx, resolving ctor positions through `top`.
Further parity fixes in the same pass:
* ctx is now a real KMutCtx mirror (from_id_classes): (position,
class idx) pairs where same-class members share ONE index (weak-
Equal, no position-derived tiebreaks) and each Indc member's ctors
map at i + cidx.
* External const refs compare by 32-byte address (addr_cmp,
lexicographic like Rust's Address Ord) instead of by ingress
position, matching the order the compile-side sort produced.
* Rec-rule comparison drops the ingress-artifact global ctor idx —
(fields, rhs) only, mirroring canonical_check.rs:280-289.
* Refinement fuel is 1 + |members| (provably reaches fixpoint)
instead of a fixed 32.
* Ctx-less compare_kexpr Const arm compares levels before ref
(field-order parity; equality-only callers unaffected).
2. Inductive index walks (Inductive.lean): get_result_sort_level and
the motive/rec-type index-dom collectors peeled literal Foralls only.
Index binders can hide under definitional wrappers — Mathlib's
`inductive εClosure (S : Set σ) : Set σ` stores a type ending in
`Set σ` that only whnf exposes as `σ → Prop` — so εNFA.εClosure died
with `no match case for value 3` (KExprNode.App). whnf before every
peel, mirroring inductive.rs get_result_sort_level (line 2101+) and
build_motive_type_flat (2521-2531).
Repros (both now pass):
ix check --ixe mathlib.ixe Mathlib.Tactic.Ring.ExSum --interp bytecode
ix check --ixe mathlib.ixe εNFA.εClosure.step --interp bytecode
`lake test -- --ignored ixvm` green (599 assertions). Codegen
regenerated; FFT pins re-bumped — pin values reflect this branch's
stubbed verify_bytes_against and need one more bump when the stub is
reverted.
Known remaining issue: shard-scale runs crawl inside
collect_block_members (per-block O(|top|) rescans whose memo keys make
the query map blow up); fix planned separately.
emitDebug was a silent stub, so dbg! probes only printed under
--interp bytecode — kernel debugging was pinned to the slowest engine.
Emit the same println! the bytecode interpreter's Op::Debug arm
produces (label + comma-separated values), with the label escaped for
a Rust format-string literal. Production kernels contain no dbg!, so
generated output is unchanged there; probe builds now print at native
speed.
…red-aux validation
Three fixes surfaced by driving mathlib shard 0 to green:
1. Perf: derive_block_member_idxs rescanned the whole `top` list per
block, and each scan step's memo key included the block addr —
|blocks| x |top| query-map entries dominated shard-scale runs (99%
of CPU in QueryMap ops). One memoized walk now builds an rbtree from
addr_key(block_addr) to buckets of (full addr, ascending member
positions); every query is an O(log N) lookup + address_eq-confirmed
bucket walk (4-byte key collisions cost a short walk, never a wrong
member list).
2. Aux synthesis (Lean.Compiler.LCNF.Cases): synth_aux_ind_ty /
synth_aux_ctor_ty never instantiated the occurrence's universe args,
so synthesized types kept `Type u` (Param 0) where the block stores
monomorphized `Type 0`; binder peels also assumed literal Foralls.
Mirror canonical_aux_order's construction (inductive.rs:1169-1234):
instantiate occurrence_us, whnf before each peel. addrs is threaded
through the build_flat_block chain for the whnf calls.
3. Stored-aux validation removed: validate_block_auxes assumed Muts
blocks store synthesized aux inductives and classified members with
is_aux_inductive ("no own nested occurrence but some member has
one"). Stored blocks only ever contain the source originals — the
Rust kernel seeds every stored member is_aux:false (rs:537) and
flags auxes only on transient detection (rs:755) — so the heuristic
misclassified originals in mixed blocks (LCNF's Alt/FunDecl/Code,
where only Cases carries the nested occurrence) and asserted on a
legitimate block. Smuggled extra members remain rejected by the
recursor-vs-canonical-type equality over the detected flat block.
Validation: full shard-0 native run green (9357/9357 owned consts);
`lake test -- --ignored ixvm` green (599 assertions; codegen
regenerated, FFT pins re-bumped — still relative to this branch's
stubbed verify_bytes_against).
Known residual divergence (not yet observed failing): Rust reorders
the detected aux portion of a flat block via canonical_aux_order
before recursor checks; Aiur keeps discovery order.
The thin Cli wrapper around Ix.Cli.CheckCmd.runCheckCmd had drifted
from the real command (its `interp` flag was still a bare bool while
`ix check --interp` takes a mode string), and `lake exe ix check`
rebuilds just as incrementally for day-to-day kernel iteration. One
entrypoint, no drift.
Stored recursors bake the COMPILER's canonical aux order into their
motive/minor layout, but build_flat_block discovers auxes in queue
(traversal) order — position-by-position recursor matching only worked
when the two orders happened to coincide. Mirror inductive.rs
canonical_aux_order (rs:1058+, applied at rs:2293-2321): after the
queue pass, re-sort the aux suffix by partition refinement over
synthetic aux views.
Aiur-isms vs the Rust original:
* Synthetic addresses are replaced by SENTINEL positions
`|top| + ordinal`: each aux's view (ext type/ctors with occurrence
universe args instantiated, spec_params substituted, block params
wrapped) rewrites nested aux occurrences to sentinel Consts, fixed
per ordinal so views synthesize once; each refinement round's ctx
maps sentinels to their current class, so same-class refs compare
weak-Equal — the same mechanism the canonical block sort uses for
block-local refs.
* Ties keep DISCOVERY order (stable insert). Rust's sort_by_compare is
a stable merge sort; a first cut with an unstable insertion sort
REVERSED tied pairs, swapping content-identical auxes whose spec
params differ only in phantom parameters (IxVMInd.DedupM's
Bar2⟨·,Nat⟩ / Bar2⟨·,Bool⟩) and breaking their rec-type match.
* The reorder is unconditional: every env Aiur checks comes through
the Ix compile pipeline (RecursorAuxOrder::Canonical); the Lean-
source order case Rust skips (rs:2293) cannot reach this kernel.
Validation: `lake test -- --ignored ixvm` green (599 assertions; FFT
pins re-bumped, still relative to this branch's stubbed
verify_bytes_against); bytecode repros green for the multi-aux blocks
Lean.Compiler.LCNF.Cases, Lean.Syntax.rec, IxVMInd.DedupM.rec,
IxVMInd.DepthM.rec; full mathlib shard-0 native run green
(9357/9357 owned consts). Codegen regenerated.
Side finding for the compile side: copying the DedupM/Bar2 fixtures
into the CLI env under a different namespace (IxDbgFixtures) makes
compile_env fail that block with "compute_aux_perm: no canonical match
for in-SCC source aux #1" while the identical structure compiles fine
as IxVMInd.* in the test env — compile-side aux matching looks
name-order sensitive. Repro: re-add the four fixtures from
Tests/Ix/IxVM.lean:93-108 to any ix-CLI-visible module under a fresh
namespace and run `ix check <ns>.DedupM.rec`.
Reverts the probe-branch stub (b7a0336) that disabled blake3
verification of constant/blob/claim bytes during the mathlib shard-0
debugging campaign. Codegen regenerated; FFT pins re-bumped with
hashing back in the circuit.
The pins land within ±1.1% of main (median ratio 0.997): the kernel
work on this branch — canonical-sort comparator parity, whnf-aware
index walks, the block-members table, aux-synthesis parity and
canonical aux order — is cost-neutral. Nested-aux-heavy targets
(DedupM, DepthM, AuxDedup*, Lean.Syntax.rec) got 0.5-1.1% cheaper;
small stdlib targets (HEq, Nat, Eq.rec) pay 0.4-0.7% for the richer
comparator and whnf walks.
`lake test -- --ignored ixvm` green (599 assertions).
@arthurpaulino
arthurpaulino enabled auto-merge (squash) July 17, 2026 17:58
@arthurpaulino
arthurpaulino merged commit 4e554d2 into mainJul 17, 2026
8 of 9 checks passed
@arthurpaulino
arthurpaulino deleted the ap/probe-mathlib branch July 17, 2026 17:58
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@arthurpaulino@johnchandlerburnham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

IxVM Aiur kernel: drive mathlib shard 0 to green - #485

Merged
arthurpaulino merged 8 commits into
mainfrom
ap/probe-mathlib
Jul 17, 2026
Merged

IxVM Aiur kernel: drive mathlib shard 0 to green#485
arthurpaulino merged 8 commits into
mainfrom
ap/probe-mathlib

Conversation

@arthurpaulino

@arthurpaulinoarthurpaulino commented Jul 11, 2026

Copy link
Copy Markdown
Member

Summary

ix check --ixe mathlib.ixe --ixes mathlib.ixes --shard 0 died with an
opaque assert_eq mismatch: 0 != 1. Driving it to green surfaced four
independent correctness divergences between the Aiur kernel and its Rust
mirror, one shard-scale performance pathology, and some missing debugging
infrastructure. All are fixed here: shard 0 (9,357 owned constants of the
647,127-const mathlib env) typechecks end-to-end through the native
codegen kernel, lake test -- --ignored ixvm is green (599 assertions),
and the FFT pins land within ±1.1% of main (median ratio 0.997) — the
kernel surgery is cost-neutral.

Kernel fixes

  1. Canonical block sort compares constructors (CanonicalCheck.lean).
    The Indc comparator stopped at the inductive's type, so same-shape
    mutual inductives (Mathlib.Tactic.Ring's ExBase/ExProd/ExSum:
    identical flags, arities and types, differing only in ctor types)
    collapsed into one alpha class and the block was rejected. Now mirrors
    compare_kindc's ctors tail, with a real KMutCtx (same-class members
    share one index; ctors mapped at i + cidx), address-based external-ref
    comparison, (fields, rhs)-only rec-rule comparison, and a provably
    sufficient refinement fuel bound.

  2. whnf-aware index walks (Inductive.lean). get_result_sort_level
    and the motive/rec-type index-domain collectors peeled literal Foralls
    only; index binders can hide under definitional wrappers —
    inductive εClosure (S : Set σ) : Set σ stores a type ending in
    Set σ that only whnf exposes as σ → Prop. whnf before every peel,
    mirroring the Rust walks.

  3. Aux synthesis parity + stored-aux validation removal
    (Inductive.lean). Synthesized aux types never instantiated the
    occurrence's universe args (Type u vs the stored monomorphized
    Type 0) and peeled binders structurally; fixed per
    canonical_aux_order's construction. The stored-aux validation pass
    (validate_block_auxes / is_aux_inductive) is removed outright:
    stored Muts blocks only ever contain the source originals (Rust seeds
    every stored member is_aux: false; auxes are transient detection
    results), and the classification heuristic misclassified originals in
    mixed blocks (Lean.Compiler.LCNF's Alt/FunDecl/Cases/Code),
    rejecting legitimate envs.

  4. Canonical aux order (Inductive.lean). Stored recursors bake the
    compiler's canonical aux order into their motive/minor layout, while
    build_flat_block discovers auxes in traversal order. The aux suffix
    is now re-sorted by partition refinement over synthetic aux views,
    with sentinel positions (|top| + ordinal) standing in for Rust's
    synthetic addresses and stable tie handling — an unstable sort
    reverses content-identical auxes whose spec params differ only in
    phantom parameters (IxVMInd.DedupM) and breaks rec-type matching.

Performance

  • Block-membership table. derive_block_member_idxs rescanned all of
    top per block with per-block memo keys — |blocks| × |top| query-map
    entries ate 99% of shard-scale CPU. One shared memoized walk now builds
    an rbtree from addr_key(block_addr) to address_eq-confirmed buckets;
    each query is an O(log N) lookup.

Tooling

  • ix name-of: new subcommand resolving a content address back to
    its Lean names against a .ixe env — potentially many, since
    structurally equivalent constants collapse to the same address (every
    matching entry in the env's named table is printed; the one-slot
    addrToName index would drop aliases). Unnamed Muts blocks fall back
    to scanning for projections into the block — turns an anonymized
    failing address into an ix check <name> fast repro. ix addr-of
    (name → address) is unchanged.
  • dbg! prints in the codegen kernel: emitDebug was a silent stub;
    it now emits the same println! the bytecode interpreter produces, so
    probe builds run at native speed. Production kernels contain no dbg!,
    so generated output is unchanged there.
  • lake exe check shim dropped: it had drifted from ix check
    (bool interp flag vs mode string); one entrypoint, no drift.

Validation

  • Full mathlib shard-0 native run: green, 9,357/9,357 owned consts.
  • Bytecode repros green for every constant that failed along the way:
    Mathlib.Tactic.Ring.ExSum, εNFA.εClosure.step,
    Lean.Compiler.LCNF.Cases, plus Lean.Syntax.rec /
    IxVMInd.DedupM.rec / IxVMInd.DepthM.rec for the aux machinery.
  • lake test -- --ignored ixvm: 599 assertions, 0 failures.
  • verify_bytes_against was stubbed during the debugging campaign and is
    restored; final pins include blake3 verification and sit within ±1.1%
    of main.

Known follow-ups

  • Shards 1–63 have not been run (pure machine time; ~300 GB peak RSS per
    shard, so sequential on a single box).

@arthurpaulino
arthurpaulinoforce-pushed the ap/probe-mathlib branch 3 times, most recently from 309e1c4 to de33f23CompareJuly 11, 2026 11:28
@arthurpaulino

Copy link
Copy Markdown
MemberAuthor

!benchmark

@argument-ci-bot

Copy link
Copy Markdown
Contributor

!benchmark — main vs de33f23

backends: aiur=prove · envs: InitStd · set: primary · shard: 0

aiur · InitStd · prove — main from: bencher @ 16455ba + base run (1 new)

constantprove-time (main)prove-time (PR)Δ%throughput (main)throughput (PR)Δ%peak-ram (main)peak-ram (PR)Δ%execute-time (main)execute-time (PR)Δ%verify-time (main)verify-time (PR)Δ%proof-size (main)proof-size (PR)Δ%fft-cost (main)fft-cost (PR)Δ%
ByteArray.utf8DecodeChar?_utf8EncodeChar_append54.933 s56.778 s+3.4% ⚠️52.61050.900-3.3% ⚠️100.89 GiB100.61 GiB-0.3%11.817 s12.508 s+5.8% (1.06× slower) ⚠️208.4 ms219.8 ms+5.5% (1.05× slower) ⚠️33.27 MiB34.43 MiB+3.5% ⚠️42.43B42.29B-0.3%
Array.extract_append53.157 s55.123 s+3.7% ⚠️32.09030.950-3.6% ⚠️92.86 GiB92.66 GiB-0.2%12.727 s13.021 s+2.3%210.3 ms214.2 ms+1.9%33.27 MiB34.43 MiB+3.5% ⚠️40.03B39.96B-0.2%
Char.ofOrdinal_le_of_le43.097 s43.778 s+1.6%66.41065.370-1.6%76.04 GiB75.49 GiB-0.7%8.947 s9.087 s+1.6%209.1 ms240.6 ms+15.1% (1.15× slower) ⚠️33.27 MiB34.43 MiB+3.5% ⚠️32.73B32.54B-0.6%
Vector.extract_append._proof_231.148 s32.048 s+2.9%46.26044.960-2.8%53.50 GiB53.32 GiB-0.3%6.820 s7.167 s+5.1% (1.05× slower) ⚠️215.8 ms238.4 ms+10.5% (1.10× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️23.34B23.29B-0.2%
_private.Init.Data.Range.Polymorphic.SInt.0.Int64.instRxcHasSize_eq26.216 s26.605 s+1.5%75.26074.160-1.5%48.88 GiB48.34 GiB-1.1%4.401 s4.443 s+0.9%205.7 ms222.9 ms+8.4% (1.08× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️17.43B17.35B-0.5%
String.split24.576 s25.436 s+3.5% ⚠️79.30076.620-3.4% ⚠️47.09 GiB46.75 GiB-0.7%4.030 s4.182 s+3.8% ⚠️205.0 ms219.6 ms+7.1% (1.07× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️15.99B15.89B-0.6%
List.mergeSort16.628 s17.267 s+3.8% ⚠️96.10092.540-3.7% ⚠️29.84 GiB29.71 GiB-0.4%2.782 s2.867 s+3.1% ⚠️205.2 ms215.4 ms+5.0% ⚠️33.19 MiB34.35 MiB+3.5% ⚠️11.63B11.58B-0.4%
Vector.append5.218 s5.439 s+4.2% ⚠️108.480104.060-4.1% ⚠️8.72 GiB8.90 GiB+2.0%529.2 ms541.8 ms+2.4%203.3 ms233.7 ms+15.0% (1.15× slower) ⚠️32.96 MiB34.12 MiB+3.5% ⚠️2.60B2.58B-0.5%
Nat.gcd_comm4.158 s4.382 s+5.4% (1.05× slower) ⚠️99.80094.700-5.1% (1.05× slower) ⚠️7.09 GiB7.01 GiB-1.0%349.0 ms366.4 ms+5.0% ⚠️218.0 ms215.4 ms-1.2%32.96 MiB34.12 MiB+3.5% ⚠️1.75B1.75B-0.2%
String.append2.973 s3.063 s+3.0% ⚠️118.750115.250-2.9%4.42 GiB4.57 GiB+3.5% ⚠️186.0 ms189.3 ms+1.8%201.0 ms223.1 ms+11.0% (1.11× slower) ⚠️32.89 MiB34.05 MiB+3.5% ⚠️967.65M963.43M-0.4%
Int.gcd2.350 s2.469 s+5.0% (1.05× slower) ⚠️97.43092.760-4.8% (1.05× slower) ⚠️3.25 GiB3.27 GiB+0.6%121.2 ms126.6 ms+4.5% ⚠️203.0 ms206.7 ms+1.8%32.89 MiB34.05 MiB+3.5% ⚠️605.84M604.12M-0.3%
Nat.sub_le_of_le_add2.347 s2.287 s-2.6%80.94083.090+2.7%3.06 GiB3.06 GiB-0.0%95.2 ms103.9 ms+9.2% (1.09× slower) ⚠️204.7 ms218.8 ms+6.9% (1.07× slower) ⚠️32.89 MiB34.05 MiB+3.5% ⚠️510.60M509.28M-0.3%
Nat.add_comm1.156 s1.258 s+8.8% (1.09× slower) ⚠️44.10040.530-8.1% (1.09× slower) ⚠️1.43 GiB1.46 GiB+2.3%15.5 ms16.0 ms+3.0% ⚠️194.5 ms208.9 ms+7.4% (1.07× slower) ⚠️32.76 MiB33.91 MiB+3.5% ⚠️53.90M53.75M-0.3%
Std.Tactic.BVDecide.BVExpr.bitblast.goCache_Inv_of_Inv._mutualOOMOOMn/aOOMOOMn/aOOMOOMn/a1m 21.4s1m 21.4s+0.1%OOMOOMn/aOOMOOMn/a261.03B260.46B-0.2%

14 constants · 13 regressed · 0 improved (|Δ| > 3.0% on any metric).

Workflow logs

@arthurpaulino
arthurpaulinoforce-pushed the ap/probe-mathlib branch 2 times, most recently from 6051448 to 6c5daa4CompareJuly 14, 2026 00:13
`ix name-of --ixe env.ixe <64-hex-addr>` resolves a content address
back to its Lean names. Potentially MANY names: structurally
equivalent constants collapse to the same content address, so every
entry in the env's `named` table pointing at the address is printed,
one per line (the `addrToName` reverse index keeps only one name per
address and would silently drop the aliases). Addresses of unnamed
Muts blocks fall back to scanning for projection constants into the
block and printing their names. Turns an anonymized failing address
surfaced by the IxVM kernel into an `ix check <name>` fast repro.
Lives in its own subcommand instead of overloading `ix addr-of`
(name -> address), which stays as-is.
…x walks
Two mathlib-blocking divergences from the Rust kernel, found via shard 0
of the mathlib env:
1. Canonical block sort (CanonicalCheck.lean): the Indc comparator never
descended into constructors, so same-shape mutual inductives (e.g.
Mathlib.Tactic.Ring's ExBase/ExProd/ExSum: identical flags, arities
and types, differing only in ctor types) collapsed into one alpha
class and the block was rejected (assert 0 != 1 in
validate_block_canonical). Mirror compare_kindc's ctors tail
(canonical_check.rs:299-338) via compare_kctor_idxs_ctx /
compare_kctor_pair_ctx, resolving ctor positions through `top`.
Further parity fixes in the same pass:
* ctx is now a real KMutCtx mirror (from_id_classes): (position,
class idx) pairs where same-class members share ONE index (weak-
Equal, no position-derived tiebreaks) and each Indc member's ctors
map at i + cidx.
* External const refs compare by 32-byte address (addr_cmp,
lexicographic like Rust's Address Ord) instead of by ingress
position, matching the order the compile-side sort produced.
* Rec-rule comparison drops the ingress-artifact global ctor idx —
(fields, rhs) only, mirroring canonical_check.rs:280-289.
* Refinement fuel is 1 + |members| (provably reaches fixpoint)
instead of a fixed 32.
* Ctx-less compare_kexpr Const arm compares levels before ref
(field-order parity; equality-only callers unaffected).
2. Inductive index walks (Inductive.lean): get_result_sort_level and
the motive/rec-type index-dom collectors peeled literal Foralls only.
Index binders can hide under definitional wrappers — Mathlib's
`inductive εClosure (S : Set σ) : Set σ` stores a type ending in
`Set σ` that only whnf exposes as `σ → Prop` — so εNFA.εClosure died
with `no match case for value 3` (KExprNode.App). whnf before every
peel, mirroring inductive.rs get_result_sort_level (line 2101+) and
build_motive_type_flat (2521-2531).
Repros (both now pass):
ix check --ixe mathlib.ixe Mathlib.Tactic.Ring.ExSum --interp bytecode
ix check --ixe mathlib.ixe εNFA.εClosure.step --interp bytecode
`lake test -- --ignored ixvm` green (599 assertions). Codegen
regenerated; FFT pins re-bumped — pin values reflect this branch's
stubbed verify_bytes_against and need one more bump when the stub is
reverted.
Known remaining issue: shard-scale runs crawl inside
collect_block_members (per-block O(|top|) rescans whose memo keys make
the query map blow up); fix planned separately.
emitDebug was a silent stub, so dbg! probes only printed under
--interp bytecode — kernel debugging was pinned to the slowest engine.
Emit the same println! the bytecode interpreter's Op::Debug arm
produces (label + comma-separated values), with the label escaped for
a Rust format-string literal. Production kernels contain no dbg!, so
generated output is unchanged there; probe builds now print at native
speed.
…red-aux validation
Three fixes surfaced by driving mathlib shard 0 to green:
1. Perf: derive_block_member_idxs rescanned the whole `top` list per
block, and each scan step's memo key included the block addr —
|blocks| x |top| query-map entries dominated shard-scale runs (99%
of CPU in QueryMap ops). One memoized walk now builds an rbtree from
addr_key(block_addr) to buckets of (full addr, ascending member
positions); every query is an O(log N) lookup + address_eq-confirmed
bucket walk (4-byte key collisions cost a short walk, never a wrong
member list).
2. Aux synthesis (Lean.Compiler.LCNF.Cases): synth_aux_ind_ty /
synth_aux_ctor_ty never instantiated the occurrence's universe args,
so synthesized types kept `Type u` (Param 0) where the block stores
monomorphized `Type 0`; binder peels also assumed literal Foralls.
Mirror canonical_aux_order's construction (inductive.rs:1169-1234):
instantiate occurrence_us, whnf before each peel. addrs is threaded
through the build_flat_block chain for the whnf calls.
3. Stored-aux validation removed: validate_block_auxes assumed Muts
blocks store synthesized aux inductives and classified members with
is_aux_inductive ("no own nested occurrence but some member has
one"). Stored blocks only ever contain the source originals — the
Rust kernel seeds every stored member is_aux:false (rs:537) and
flags auxes only on transient detection (rs:755) — so the heuristic
misclassified originals in mixed blocks (LCNF's Alt/FunDecl/Code,
where only Cases carries the nested occurrence) and asserted on a
legitimate block. Smuggled extra members remain rejected by the
recursor-vs-canonical-type equality over the detected flat block.
Validation: full shard-0 native run green (9357/9357 owned consts);
`lake test -- --ignored ixvm` green (599 assertions; codegen
regenerated, FFT pins re-bumped — still relative to this branch's
stubbed verify_bytes_against).
Known residual divergence (not yet observed failing): Rust reorders
the detected aux portion of a flat block via canonical_aux_order
before recursor checks; Aiur keeps discovery order.
The thin Cli wrapper around Ix.Cli.CheckCmd.runCheckCmd had drifted
from the real command (its `interp` flag was still a bare bool while
`ix check --interp` takes a mode string), and `lake exe ix check`
rebuilds just as incrementally for day-to-day kernel iteration. One
entrypoint, no drift.
Stored recursors bake the COMPILER's canonical aux order into their
motive/minor layout, but build_flat_block discovers auxes in queue
(traversal) order — position-by-position recursor matching only worked
when the two orders happened to coincide. Mirror inductive.rs
canonical_aux_order (rs:1058+, applied at rs:2293-2321): after the
queue pass, re-sort the aux suffix by partition refinement over
synthetic aux views.
Aiur-isms vs the Rust original:
* Synthetic addresses are replaced by SENTINEL positions
`|top| + ordinal`: each aux's view (ext type/ctors with occurrence
universe args instantiated, spec_params substituted, block params
wrapped) rewrites nested aux occurrences to sentinel Consts, fixed
per ordinal so views synthesize once; each refinement round's ctx
maps sentinels to their current class, so same-class refs compare
weak-Equal — the same mechanism the canonical block sort uses for
block-local refs.
* Ties keep DISCOVERY order (stable insert). Rust's sort_by_compare is
a stable merge sort; a first cut with an unstable insertion sort
REVERSED tied pairs, swapping content-identical auxes whose spec
params differ only in phantom parameters (IxVMInd.DedupM's
Bar2⟨·,Nat⟩ / Bar2⟨·,Bool⟩) and breaking their rec-type match.
* The reorder is unconditional: every env Aiur checks comes through
the Ix compile pipeline (RecursorAuxOrder::Canonical); the Lean-
source order case Rust skips (rs:2293) cannot reach this kernel.
Validation: `lake test -- --ignored ixvm` green (599 assertions; FFT
pins re-bumped, still relative to this branch's stubbed
verify_bytes_against); bytecode repros green for the multi-aux blocks
Lean.Compiler.LCNF.Cases, Lean.Syntax.rec, IxVMInd.DedupM.rec,
IxVMInd.DepthM.rec; full mathlib shard-0 native run green
(9357/9357 owned consts). Codegen regenerated.
Side finding for the compile side: copying the DedupM/Bar2 fixtures
into the CLI env under a different namespace (IxDbgFixtures) makes
compile_env fail that block with "compute_aux_perm: no canonical match
for in-SCC source aux #1" while the identical structure compiles fine
as IxVMInd.* in the test env — compile-side aux matching looks
name-order sensitive. Repro: re-add the four fixtures from
Tests/Ix/IxVM.lean:93-108 to any ix-CLI-visible module under a fresh
namespace and run `ix check <ns>.DedupM.rec`.
Reverts the probe-branch stub (b7a0336) that disabled blake3
verification of constant/blob/claim bytes during the mathlib shard-0
debugging campaign. Codegen regenerated; FFT pins re-bumped with
hashing back in the circuit.
The pins land within ±1.1% of main (median ratio 0.997): the kernel
work on this branch — canonical-sort comparator parity, whnf-aware
index walks, the block-members table, aux-synthesis parity and
canonical aux order — is cost-neutral. Nested-aux-heavy targets
(DedupM, DepthM, AuxDedup*, Lean.Syntax.rec) got 0.5-1.1% cheaper;
small stdlib targets (HEq, Nat, Eq.rec) pay 0.4-0.7% for the richer
comparator and whnf walks.
`lake test -- --ignored ixvm` green (599 assertions).
@arthurpaulino
arthurpaulino enabled auto-merge (squash) July 17, 2026 17:58
@arthurpaulino
arthurpaulino merged commit 4e554d2 into mainJul 17, 2026
8 of 9 checks passed
@arthurpaulino
arthurpaulino deleted the ap/probe-mathlib branch July 17, 2026 17:58
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@arthurpaulino@johnchandlerburnham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

IxVM Aiur kernel: drive mathlib shard 0 to green - #485

Merged
arthurpaulino merged 8 commits into
mainfrom
ap/probe-mathlib
Jul 17, 2026
Merged

IxVM Aiur kernel: drive mathlib shard 0 to green#485
arthurpaulino merged 8 commits into
mainfrom
ap/probe-mathlib

Conversation

@arthurpaulino

@arthurpaulinoarthurpaulino commented Jul 11, 2026

Copy link
Copy Markdown
Member

Summary

ix check --ixe mathlib.ixe --ixes mathlib.ixes --shard 0 died with an
opaque assert_eq mismatch: 0 != 1. Driving it to green surfaced four
independent correctness divergences between the Aiur kernel and its Rust
mirror, one shard-scale performance pathology, and some missing debugging
infrastructure. All are fixed here: shard 0 (9,357 owned constants of the
647,127-const mathlib env) typechecks end-to-end through the native
codegen kernel, lake test -- --ignored ixvm is green (599 assertions),
and the FFT pins land within ±1.1% of main (median ratio 0.997) — the
kernel surgery is cost-neutral.

Kernel fixes

  1. Canonical block sort compares constructors (CanonicalCheck.lean).
    The Indc comparator stopped at the inductive's type, so same-shape
    mutual inductives (Mathlib.Tactic.Ring's ExBase/ExProd/ExSum:
    identical flags, arities and types, differing only in ctor types)
    collapsed into one alpha class and the block was rejected. Now mirrors
    compare_kindc's ctors tail, with a real KMutCtx (same-class members
    share one index; ctors mapped at i + cidx), address-based external-ref
    comparison, (fields, rhs)-only rec-rule comparison, and a provably
    sufficient refinement fuel bound.

  2. whnf-aware index walks (Inductive.lean). get_result_sort_level
    and the motive/rec-type index-domain collectors peeled literal Foralls
    only; index binders can hide under definitional wrappers —
    inductive εClosure (S : Set σ) : Set σ stores a type ending in
    Set σ that only whnf exposes as σ → Prop. whnf before every peel,
    mirroring the Rust walks.

  3. Aux synthesis parity + stored-aux validation removal
    (Inductive.lean). Synthesized aux types never instantiated the
    occurrence's universe args (Type u vs the stored monomorphized
    Type 0) and peeled binders structurally; fixed per
    canonical_aux_order's construction. The stored-aux validation pass
    (validate_block_auxes / is_aux_inductive) is removed outright:
    stored Muts blocks only ever contain the source originals (Rust seeds
    every stored member is_aux: false; auxes are transient detection
    results), and the classification heuristic misclassified originals in
    mixed blocks (Lean.Compiler.LCNF's Alt/FunDecl/Cases/Code),
    rejecting legitimate envs.

  4. Canonical aux order (Inductive.lean). Stored recursors bake the
    compiler's canonical aux order into their motive/minor layout, while
    build_flat_block discovers auxes in traversal order. The aux suffix
    is now re-sorted by partition refinement over synthetic aux views,
    with sentinel positions (|top| + ordinal) standing in for Rust's
    synthetic addresses and stable tie handling — an unstable sort
    reverses content-identical auxes whose spec params differ only in
    phantom parameters (IxVMInd.DedupM) and breaks rec-type matching.

Performance

  • Block-membership table. derive_block_member_idxs rescanned all of
    top per block with per-block memo keys — |blocks| × |top| query-map
    entries ate 99% of shard-scale CPU. One shared memoized walk now builds
    an rbtree from addr_key(block_addr) to address_eq-confirmed buckets;
    each query is an O(log N) lookup.

Tooling

  • ix name-of: new subcommand resolving a content address back to
    its Lean names against a .ixe env — potentially many, since
    structurally equivalent constants collapse to the same address (every
    matching entry in the env's named table is printed; the one-slot
    addrToName index would drop aliases). Unnamed Muts blocks fall back
    to scanning for projections into the block — turns an anonymized
    failing address into an ix check <name> fast repro. ix addr-of
    (name → address) is unchanged.
  • dbg! prints in the codegen kernel: emitDebug was a silent stub;
    it now emits the same println! the bytecode interpreter produces, so
    probe builds run at native speed. Production kernels contain no dbg!,
    so generated output is unchanged there.
  • lake exe check shim dropped: it had drifted from ix check
    (bool interp flag vs mode string); one entrypoint, no drift.

Validation

  • Full mathlib shard-0 native run: green, 9,357/9,357 owned consts.
  • Bytecode repros green for every constant that failed along the way:
    Mathlib.Tactic.Ring.ExSum, εNFA.εClosure.step,
    Lean.Compiler.LCNF.Cases, plus Lean.Syntax.rec /
    IxVMInd.DedupM.rec / IxVMInd.DepthM.rec for the aux machinery.
  • lake test -- --ignored ixvm: 599 assertions, 0 failures.
  • verify_bytes_against was stubbed during the debugging campaign and is
    restored; final pins include blake3 verification and sit within ±1.1%
    of main.

Known follow-ups

  • Shards 1–63 have not been run (pure machine time; ~300 GB peak RSS per
    shard, so sequential on a single box).

@arthurpaulino
arthurpaulinoforce-pushed the ap/probe-mathlib branch 3 times, most recently from 309e1c4 to de33f23CompareJuly 11, 2026 11:28
@arthurpaulino

Copy link
Copy Markdown
MemberAuthor

!benchmark

@argument-ci-bot

Copy link
Copy Markdown
Contributor

!benchmark — main vs de33f23

backends: aiur=prove · envs: InitStd · set: primary · shard: 0

aiur · InitStd · prove — main from: bencher @ 16455ba + base run (1 new)

constantprove-time (main)prove-time (PR)Δ%throughput (main)throughput (PR)Δ%peak-ram (main)peak-ram (PR)Δ%execute-time (main)execute-time (PR)Δ%verify-time (main)verify-time (PR)Δ%proof-size (main)proof-size (PR)Δ%fft-cost (main)fft-cost (PR)Δ%
ByteArray.utf8DecodeChar?_utf8EncodeChar_append54.933 s56.778 s+3.4% ⚠️52.61050.900-3.3% ⚠️100.89 GiB100.61 GiB-0.3%11.817 s12.508 s+5.8% (1.06× slower) ⚠️208.4 ms219.8 ms+5.5% (1.05× slower) ⚠️33.27 MiB34.43 MiB+3.5% ⚠️42.43B42.29B-0.3%
Array.extract_append53.157 s55.123 s+3.7% ⚠️32.09030.950-3.6% ⚠️92.86 GiB92.66 GiB-0.2%12.727 s13.021 s+2.3%210.3 ms214.2 ms+1.9%33.27 MiB34.43 MiB+3.5% ⚠️40.03B39.96B-0.2%
Char.ofOrdinal_le_of_le43.097 s43.778 s+1.6%66.41065.370-1.6%76.04 GiB75.49 GiB-0.7%8.947 s9.087 s+1.6%209.1 ms240.6 ms+15.1% (1.15× slower) ⚠️33.27 MiB34.43 MiB+3.5% ⚠️32.73B32.54B-0.6%
Vector.extract_append._proof_231.148 s32.048 s+2.9%46.26044.960-2.8%53.50 GiB53.32 GiB-0.3%6.820 s7.167 s+5.1% (1.05× slower) ⚠️215.8 ms238.4 ms+10.5% (1.10× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️23.34B23.29B-0.2%
_private.Init.Data.Range.Polymorphic.SInt.0.Int64.instRxcHasSize_eq26.216 s26.605 s+1.5%75.26074.160-1.5%48.88 GiB48.34 GiB-1.1%4.401 s4.443 s+0.9%205.7 ms222.9 ms+8.4% (1.08× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️17.43B17.35B-0.5%
String.split24.576 s25.436 s+3.5% ⚠️79.30076.620-3.4% ⚠️47.09 GiB46.75 GiB-0.7%4.030 s4.182 s+3.8% ⚠️205.0 ms219.6 ms+7.1% (1.07× slower) ⚠️33.19 MiB34.35 MiB+3.5% ⚠️15.99B15.89B-0.6%
List.mergeSort16.628 s17.267 s+3.8% ⚠️96.10092.540-3.7% ⚠️29.84 GiB29.71 GiB-0.4%2.782 s2.867 s+3.1% ⚠️205.2 ms215.4 ms+5.0% ⚠️33.19 MiB34.35 MiB+3.5% ⚠️11.63B11.58B-0.4%
Vector.append5.218 s5.439 s+4.2% ⚠️108.480104.060-4.1% ⚠️8.72 GiB8.90 GiB+2.0%529.2 ms541.8 ms+2.4%203.3 ms233.7 ms+15.0% (1.15× slower) ⚠️32.96 MiB34.12 MiB+3.5% ⚠️2.60B2.58B-0.5%
Nat.gcd_comm4.158 s4.382 s+5.4% (1.05× slower) ⚠️99.80094.700-5.1% (1.05× slower) ⚠️7.09 GiB7.01 GiB-1.0%349.0 ms366.4 ms+5.0% ⚠️218.0 ms215.4 ms-1.2%32.96 MiB34.12 MiB+3.5% ⚠️1.75B1.75B-0.2%
String.append2.973 s3.063 s+3.0% ⚠️118.750115.250-2.9%4.42 GiB4.57 GiB+3.5% ⚠️186.0 ms189.3 ms+1.8%201.0 ms223.1 ms+11.0% (1.11× slower) ⚠️32.89 MiB34.05 MiB+3.5% ⚠️967.65M963.43M-0.4%
Int.gcd2.350 s2.469 s+5.0% (1.05× slower) ⚠️97.43092.760-4.8% (1.05× slower) ⚠️3.25 GiB3.27 GiB+0.6%121.2 ms126.6 ms+4.5% ⚠️203.0 ms206.7 ms+1.8%32.89 MiB34.05 MiB+3.5% ⚠️605.84M604.12M-0.3%
Nat.sub_le_of_le_add2.347 s2.287 s-2.6%80.94083.090+2.7%3.06 GiB3.06 GiB-0.0%95.2 ms103.9 ms+9.2% (1.09× slower) ⚠️204.7 ms218.8 ms+6.9% (1.07× slower) ⚠️32.89 MiB34.05 MiB+3.5% ⚠️510.60M509.28M-0.3%
Nat.add_comm1.156 s1.258 s+8.8% (1.09× slower) ⚠️44.10040.530-8.1% (1.09× slower) ⚠️1.43 GiB1.46 GiB+2.3%15.5 ms16.0 ms+3.0% ⚠️194.5 ms208.9 ms+7.4% (1.07× slower) ⚠️32.76 MiB33.91 MiB+3.5% ⚠️53.90M53.75M-0.3%
Std.Tactic.BVDecide.BVExpr.bitblast.goCache_Inv_of_Inv._mutualOOMOOMn/aOOMOOMn/aOOMOOMn/a1m 21.4s1m 21.4s+0.1%OOMOOMn/aOOMOOMn/a261.03B260.46B-0.2%

14 constants · 13 regressed · 0 improved (|Δ| > 3.0% on any metric).

Workflow logs

@arthurpaulino
arthurpaulinoforce-pushed the ap/probe-mathlib branch 2 times, most recently from 6051448 to 6c5daa4CompareJuly 14, 2026 00:13
`ix name-of --ixe env.ixe <64-hex-addr>` resolves a content address
back to its Lean names. Potentially MANY names: structurally
equivalent constants collapse to the same content address, so every
entry in the env's `named` table pointing at the address is printed,
one per line (the `addrToName` reverse index keeps only one name per
address and would silently drop the aliases). Addresses of unnamed
Muts blocks fall back to scanning for projection constants into the
block and printing their names. Turns an anonymized failing address
surfaced by the IxVM kernel into an `ix check <name>` fast repro.
Lives in its own subcommand instead of overloading `ix addr-of`
(name -> address), which stays as-is.
…x walks
Two mathlib-blocking divergences from the Rust kernel, found via shard 0
of the mathlib env:
1. Canonical block sort (CanonicalCheck.lean): the Indc comparator never
descended into constructors, so same-shape mutual inductives (e.g.
Mathlib.Tactic.Ring's ExBase/ExProd/ExSum: identical flags, arities
and types, differing only in ctor types) collapsed into one alpha
class and the block was rejected (assert 0 != 1 in
validate_block_canonical). Mirror compare_kindc's ctors tail
(canonical_check.rs:299-338) via compare_kctor_idxs_ctx /
compare_kctor_pair_ctx, resolving ctor positions through `top`.
Further parity fixes in the same pass:
* ctx is now a real KMutCtx mirror (from_id_classes): (position,
class idx) pairs where same-class members share ONE index (weak-
Equal, no position-derived tiebreaks) and each Indc member's ctors
map at i + cidx.
* External const refs compare by 32-byte address (addr_cmp,
lexicographic like Rust's Address Ord) instead of by ingress
position, matching the order the compile-side sort produced.
* Rec-rule comparison drops the ingress-artifact global ctor idx —
(fields, rhs) only, mirroring canonical_check.rs:280-289.
* Refinement fuel is 1 + |members| (provably reaches fixpoint)
instead of a fixed 32.
* Ctx-less compare_kexpr Const arm compares levels before ref
(field-order parity; equality-only callers unaffected).
2. Inductive index walks (Inductive.lean): get_result_sort_level and
the motive/rec-type index-dom collectors peeled literal Foralls only.
Index binders can hide under definitional wrappers — Mathlib's
`inductive εClosure (S : Set σ) : Set σ` stores a type ending in
`Set σ` that only whnf exposes as `σ → Prop` — so εNFA.εClosure died
with `no match case for value 3` (KExprNode.App). whnf before every
peel, mirroring inductive.rs get_result_sort_level (line 2101+) and
build_motive_type_flat (2521-2531).
Repros (both now pass):
ix check --ixe mathlib.ixe Mathlib.Tactic.Ring.ExSum --interp bytecode
ix check --ixe mathlib.ixe εNFA.εClosure.step --interp bytecode
`lake test -- --ignored ixvm` green (599 assertions). Codegen
regenerated; FFT pins re-bumped — pin values reflect this branch's
stubbed verify_bytes_against and need one more bump when the stub is
reverted.
Known remaining issue: shard-scale runs crawl inside
collect_block_members (per-block O(|top|) rescans whose memo keys make
the query map blow up); fix planned separately.
emitDebug was a silent stub, so dbg! probes only printed under
--interp bytecode — kernel debugging was pinned to the slowest engine.
Emit the same println! the bytecode interpreter's Op::Debug arm
produces (label + comma-separated values), with the label escaped for
a Rust format-string literal. Production kernels contain no dbg!, so
generated output is unchanged there; probe builds now print at native
speed.
…red-aux validation
Three fixes surfaced by driving mathlib shard 0 to green:
1. Perf: derive_block_member_idxs rescanned the whole `top` list per
block, and each scan step's memo key included the block addr —
|blocks| x |top| query-map entries dominated shard-scale runs (99%
of CPU in QueryMap ops). One memoized walk now builds an rbtree from
addr_key(block_addr) to buckets of (full addr, ascending member
positions); every query is an O(log N) lookup + address_eq-confirmed
bucket walk (4-byte key collisions cost a short walk, never a wrong
member list).
2. Aux synthesis (Lean.Compiler.LCNF.Cases): synth_aux_ind_ty /
synth_aux_ctor_ty never instantiated the occurrence's universe args,
so synthesized types kept `Type u` (Param 0) where the block stores
monomorphized `Type 0`; binder peels also assumed literal Foralls.
Mirror canonical_aux_order's construction (inductive.rs:1169-1234):
instantiate occurrence_us, whnf before each peel. addrs is threaded
through the build_flat_block chain for the whnf calls.
3. Stored-aux validation removed: validate_block_auxes assumed Muts
blocks store synthesized aux inductives and classified members with
is_aux_inductive ("no own nested occurrence but some member has
one"). Stored blocks only ever contain the source originals — the
Rust kernel seeds every stored member is_aux:false (rs:537) and
flags auxes only on transient detection (rs:755) — so the heuristic
misclassified originals in mixed blocks (LCNF's Alt/FunDecl/Code,
where only Cases carries the nested occurrence) and asserted on a
legitimate block. Smuggled extra members remain rejected by the
recursor-vs-canonical-type equality over the detected flat block.
Validation: full shard-0 native run green (9357/9357 owned consts);
`lake test -- --ignored ixvm` green (599 assertions; codegen
regenerated, FFT pins re-bumped — still relative to this branch's
stubbed verify_bytes_against).
Known residual divergence (not yet observed failing): Rust reorders
the detected aux portion of a flat block via canonical_aux_order
before recursor checks; Aiur keeps discovery order.
The thin Cli wrapper around Ix.Cli.CheckCmd.runCheckCmd had drifted
from the real command (its `interp` flag was still a bare bool while
`ix check --interp` takes a mode string), and `lake exe ix check`
rebuilds just as incrementally for day-to-day kernel iteration. One
entrypoint, no drift.
Stored recursors bake the COMPILER's canonical aux order into their
motive/minor layout, but build_flat_block discovers auxes in queue
(traversal) order — position-by-position recursor matching only worked
when the two orders happened to coincide. Mirror inductive.rs
canonical_aux_order (rs:1058+, applied at rs:2293-2321): after the
queue pass, re-sort the aux suffix by partition refinement over
synthetic aux views.
Aiur-isms vs the Rust original:
* Synthetic addresses are replaced by SENTINEL positions
`|top| + ordinal`: each aux's view (ext type/ctors with occurrence
universe args instantiated, spec_params substituted, block params
wrapped) rewrites nested aux occurrences to sentinel Consts, fixed
per ordinal so views synthesize once; each refinement round's ctx
maps sentinels to their current class, so same-class refs compare
weak-Equal — the same mechanism the canonical block sort uses for
block-local refs.
* Ties keep DISCOVERY order (stable insert). Rust's sort_by_compare is
a stable merge sort; a first cut with an unstable insertion sort
REVERSED tied pairs, swapping content-identical auxes whose spec
params differ only in phantom parameters (IxVMInd.DedupM's
Bar2⟨·,Nat⟩ / Bar2⟨·,Bool⟩) and breaking their rec-type match.
* The reorder is unconditional: every env Aiur checks comes through
the Ix compile pipeline (RecursorAuxOrder::Canonical); the Lean-
source order case Rust skips (rs:2293) cannot reach this kernel.
Validation: `lake test -- --ignored ixvm` green (599 assertions; FFT
pins re-bumped, still relative to this branch's stubbed
verify_bytes_against); bytecode repros green for the multi-aux blocks
Lean.Compiler.LCNF.Cases, Lean.Syntax.rec, IxVMInd.DedupM.rec,
IxVMInd.DepthM.rec; full mathlib shard-0 native run green
(9357/9357 owned consts). Codegen regenerated.
Side finding for the compile side: copying the DedupM/Bar2 fixtures
into the CLI env under a different namespace (IxDbgFixtures) makes
compile_env fail that block with "compute_aux_perm: no canonical match
for in-SCC source aux #1" while the identical structure compiles fine
as IxVMInd.* in the test env — compile-side aux matching looks
name-order sensitive. Repro: re-add the four fixtures from
Tests/Ix/IxVM.lean:93-108 to any ix-CLI-visible module under a fresh
namespace and run `ix check <ns>.DedupM.rec`.
Reverts the probe-branch stub (b7a0336) that disabled blake3
verification of constant/blob/claim bytes during the mathlib shard-0
debugging campaign. Codegen regenerated; FFT pins re-bumped with
hashing back in the circuit.
The pins land within ±1.1% of main (median ratio 0.997): the kernel
work on this branch — canonical-sort comparator parity, whnf-aware
index walks, the block-members table, aux-synthesis parity and
canonical aux order — is cost-neutral. Nested-aux-heavy targets
(DedupM, DepthM, AuxDedup*, Lean.Syntax.rec) got 0.5-1.1% cheaper;
small stdlib targets (HEq, Nat, Eq.rec) pay 0.4-0.7% for the richer
comparator and whnf walks.
`lake test -- --ignored ixvm` green (599 assertions).
@arthurpaulino
arthurpaulino enabled auto-merge (squash) July 17, 2026 17:58
@arthurpaulino
arthurpaulino merged commit 4e554d2 into mainJul 17, 2026
8 of 9 checks passed
@arthurpaulino
arthurpaulino deleted the ap/probe-mathlib branch July 17, 2026 17:58
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@arthurpaulino@johnchandlerburnham