vindex 0.7.0 — export: the GGUF compiler target - #363
Merged
Merged
Conversation
First step toward a GGUF target. This commit does one thing: express a
VINDEX3 NVFP4 pack in GGML's block layout without changing what it
denotes.
**Both formats are called NVFP4 and they do not agree.**
VINDEX3 three contiguous regions over the whole matrix
[ E2M1 codes ][ E4M3 group scales ][ f32 tensor scale ]
one scale per 16 elements, adjacent-pair nibbles
GGML interleaved 64-element blocks, 36 bytes each
[ 4 x UE4M3 ][ 32 x E2M1 ] no per-tensor level
Three transformations, none of them arithmetic on values: nibbles move
from adjacent pairs to planar halves, E4M3 loses a sign bit it never
used, and four 16-element groups become one 64-element block.
**The tensor scale is carried out, not folded in.** GGML has no
per-tensor level, so the f32 leaves as a sibling `.scale` tensor — the
arrangement llama.cpp's own converter uses. Folding it into the UE4M3
bytes would have been a re-quantization: four exponent bits and three
mantissa bits cannot generally hold the product, and that the two
levels do not collapse is the entire reason the two-level recipe
exists. Getting this wrong would have silently shipped a different
representation than the one that was measured.
So: **export is ABI lowering, not quantization.** No floating-point
weight passes through `repack_nvfp4`. The codes that leave are the
codes that arrived. The representation that measured KL 0.02411 across
1,740 positions is the representation that reaches the GGUF, because
nothing here is free to choose another one.
Three kinds of test, because two of them are not enough:
- **Golden bytes.** All 36 pinned, over a fixture whose every nibble
value appears and whose four group scales differ — a nibble swap or a
mis-grouping cannot survive by symmetry. One scale has its sign bit
set so the strip is exercised rather than assumed.
- **Semantic equivalence.** Decode the VINDEX pack through its own
reference decoder, decode the GGML block by hand from its layout, and
require agreement element by element. Byte pinning catches a changed
arrangement; only this catches an arrangement that is
self-consistently wrong.
- **Refusals.** K of 16, 32 and 48 refuse and name the constraint; 64
and 128 pass. No padding — padding would invent weights the source
does not have.
`TYPE_NVFP4 = 40` joins the type table so `tensor_data_size` stops
answering "unsupported type id" for it.
Building a GGUF target asked the artifact a question it could not answer: how long a context does this model run to? Every other field llama.cpp's qwen35 lowering needs was already there — head counts, score scale, output-gate placement, norm epsilon and kind, m_rope theta and sections, GDN key/value geometry, conv kernel, state dtype, vocab, the per-layer operator. Context extent was not. **The encoder already knew it mattered.** `max_position_embeddings` is in EXECUTION_SEMANTIC_KEYS — classified as changing what a forward pass computes — and `inventory::components` reads it. It was judged, read, held, and then dropped at the graph boundary. A judged fact with nowhere to land is indistinguishable from an unread one, which is the failure this instrument exists to catch, found in the instrument. So the fact now has a home: `ExecutionSurface.context_length`. **On the component, not on `attention`.** Context extent is a property of the execution programme, not of softmax. Qwen3.8 makes the point itself — forty-eight of its layers never attend, and hanging the field off the attention surface would put it out of reach for exactly the architectures that most need it. A Mamba or KDA stack has an extent without attending at all. **Additive within GRAPH_SCHEMA 6, not a bump.** It is new information, not a reinterpretation of existing bytes: no v6 field changes meaning and no v6 reader must read an existing graph differently. Nothing in the format uses `deny_unknown_fields`, and the field is `skip_serializing_if = "Option::is_none"`, so a graph written before it existed is byte-identical to one written after where it is absent. Both directions are tested. **`model_max_length` is not the authority.** The tokenizer config carries a serving bound that usually agrees, and the disagreement is planted deliberately in a test: a lowering that reached for the tokenizer's copy would pass on every model where the two match and be wrong on the one where they do not. The execution semantic wins, because it is the one that changes the forward pass. An undeclared extent records `None`. Absence is a fact about the source, and choosing a number here would be the substitution the surface exists to refuse.
`GgufWriter::to_bytes` builds the whole file in a `Vec<u8>`. The module called that a future optimisation; at Qwen3.8's size it is not an optimisation, it is whether the export can happen at all. `write_streaming` is a sibling rather than a replacement. GGUF puts every tensor's offset in a table ahead of any payload, so the sizes must be known in advance either way — which means streaming needs only a descriptor without the bytes, and a callback that supplies them as the file is written. The header, metadata and tensor table are kilobytes and still built in memory; only the data section streams. Memory is constant in the model. **The test is byte identity, not validity.** Producing "a valid GGUF" would let the two writers drift into two formats, one of which nobody exercises. The fixture uses deliberately unaligned payload lengths — 37, 64 and 5 bytes — so the inter-tensor padding is compared rather than coincidentally zero on both sides. An emitter that writes the wrong number of bytes is refused by name and by both numbers. That is not defensive tidiness: the offsets are already committed to the table, so a short write leaves every later tensor pointing into the middle of its neighbour, and a reader would follow those offsets and decode plausible nonsense rather than fail. The existing buffered API and its tests are untouched.
Records what building the GGUF target taught, as a spec section rather than as tribal knowledge from one afternoon. A SystemGraph is complete when an independent backend can be lowered from it without returning to source-family configuration. That is a sharper test than checking the graph against a list of expected fields, and it caught a defect months of inference use had not: context length was judged execution-semantic, read into the inventory, and then dropped at the graph boundary because ExecutionSurface had no slot for it. The reason it survived is the point. A family-specific loader never noticed, because it can quietly know the answer — a Qwen loader reaches for a Qwen config. An independent target cannot. So each lowering target doubles as a conformance instrument, and any fact it cannot get from the graph alone is either a lowering bug or a graph that has not finished being honest. With the corollary for target authors: semantic concepts in, target vocabulary out. The moment a lowering needs to know it is looking at Qwen in order to find a fact, that fact is missing from the graph.
Exportability turns out to be two questions that fail for different
reasons, and conflating them would produce the wrong diagnosis:
semantic completeness does the graph carry the facts?
representation compat does the target's layout survive
contact with how these weights are
actually quantised?
The second is the one nothing internal ever had to ask, and it nearly
went unnoticed. qwen35 wants V heads tiled rather than grouped, which
means permuting the INPUT axis of `out_proj` — and the input axis is
exactly the axis NVFP4 groups run along. Permute it carelessly and an
element lands in a new group while its E4M3 scale belongs to the old
one. Nothing errors. The weights stay finite and plausible, decode to
noise, and the fidelity number measured before the export describes a
representation the file does not contain.
**The general invariant, of which Qwen's arithmetic is one instance:**
every permutation boundary must lie on a quantisation-group boundary
For this reorder the permuted axis reshapes to
`[k_heads, v_per_k, head_dim]` and the swap moves the outer two, leaving
`head_dim` contiguous — so it reduces to `head_dim % group == 0`.
Qwen3.8 passes at 128 % 16, eight whole groups per head. A model at 120
would not, and is refused by name rather than shipped.
Stating the general form in the module rather than the Qwen special case
is deliberate: the next architecture will want a stranger permutation,
and "128 happens to divide" is not a rule anyone can apply to it.
The report is a value rather than a series of early returns, so `export`
can print the whole assessment before writing a byte — and so a refusal
can show its working:
tensor 0.linear_attn.out_proj.weight
operation qwen35 V-head reorder
axis columns
NVFP4 group 16
head dimension 120
reason column permutation would split NVFP4 groups
Refusals distinguish where the defect lives. A missing semantic is the
artifact not finished being honest; an incompatible geometry is the
source and target genuinely disagreeing. Both are tested for saying so.
Run against the hero container today, the preflight passes every
geometry constraint and refuses on one missing semantic:
`execution.context_length` — because that container was encoded before
the previous commit gave the fact a home. The remedy is re-encoding,
which is the correct answer and not a flag.
Your closing point on the target contract: the qwen35 lowering does not only rename tensors. It materialises `-exp(log decay)` into `ssm_a` and folds a norm offset into the trunk norms — arithmetic on weights, both of which are legitimate only because the graph says which operand is a log decay and what offset each norm carries. Without requiring those, the two transforms become "`-exp` because the tensor is called `A_log`" and "`+ 1.0` because qwen35" — source-family assumptions smuggled back in one commit after they were removed. So `TransformFacts` sits beside the metadata requirements, and a missing `log decay` role refuses while naming what the target would have done with it. **The norm case turned out better than a requirement.** llama.cpp makes an exception: the trunk norms get `+1` and the GDN's internal gated norm does not. In this graph that is not an exception at all — `norm.pre` and `norm.final_norm` declare `weight_offset: 1.0`, and the linear-attention surface declares no offset for its gated norm. The distinction falls out of the artifact rather than being written down a second time on the target side. `weight_offset` is `f32` rather than `Option<f32>`, so absence is not representable and a presence check would have been theatre. It is recorded as a value instead, with the point stated where it matters: the lowering folds whichever number is there, and never a literal 1.0 justified by the family name.
Planning sits between preflight and emission, and each layer answers a
different question:
preflight should this target exist for this artifact?
planner is this particular lowering semantically legal?
writer can these bytes be emitted?
**Layout and value transforms are separate types, deliberately.** The
V-head permutation and the NVFP4 ABI repack move bytes; `-exp(log
decay)` and folding a norm offset change numbers. Collapsing both under
"lossless lowering" is the refactor that would later let someone put
`ApplyWeightOffset` on a measured projection slab without noticing, so
`LoweredTensorPlan::new` refuses it:
NVFP4 + ValueTransform -> illegal plan
and says what the damage would be — decode, compute, re-quantise, and a
new representation wearing the measured one's name. That is the
planner's own defence, and it holds even where preflight was skipped.
**The name table is pinned because three entries fail silently rather
than loudly:**
attn_q.weight the FUSED Q + output gate, double width.
Splitting the gate makes a file llama.cpp loads
and then misreads.
ssm_a no `.weight` suffix.
ssm_dt.bias a bias, not a weight.
None of the three would error. They would fail to bind, or bind the
wrong thing, and no shape check would notice. A separate
full-attention gate role deliberately maps to nothing, because it does
not exist as its own tensor.
The gated norm's exemption from the offset is tested as a fact rather
than an exception: the linear-attention surface declares no offset, so
the planner receives an empty transform list. llama.cpp writes the
exception down; here the graph makes it unnecessary.
An unknown role has no target rather than a guessed one.
Twenty-two keys, and the rule the module exists to enforce: **no literal
unless it is a target constant.** `general.architecture = "qwen35"` is a
fact about llama.cpp. `qwen35.ssm.state_size = 128` is a fact about
Qwen3.8, and writing it here rather than deriving it would put model
knowledge on the target side — the leak the independent-backend test
exists to catch.
Every entry carries `derived_from`, so a literal creeping in is visible
as `"target constant"` on something that is not one. A test asserts the
constant set is exactly the four `general.*` keys.
block_count <- component.num_layers
embedding_length <- component.hidden_size
feed_forward_length <- ffn.intermediate_size
attention.head_count <- attention.num_q_heads
key/value_length <- attention.head_dim
rms_epsilon <- norm.pre.eps
rope.freq_base <- layer position.theta
rope.dimension_count <- head_dim x position.rotary_fraction
ssm.inner_size <- value_heads x value_head_dim
ssm.state_size <- linear_attention.key_head_dim
ssm.time_step_rank <- linear_attention.value_heads
ssm.group_count <- linear_attention.key_heads
Two conventions genuinely belong to the target and are stated as such.
The graph declares three MRoPE sections and llama.cpp's array has four
slots, so the trailing zero is the target's spelling rather than a
fourth section the model has. And `full_attention_interval` is derived
from the declared per-layer operators, not taken from the converter's
default of 4 — Qwen3.8 happens to be 4, and a model that is not would
otherwise export a layer programme it does not run. An irregular
cadence refuses, as does a regular one that stops before the end of the
stack, because such an interval describes a prefix and lies about the
tail.
One namespace trap is pinned by its own test: `general.file_type = 39`
is `LLAMA_FTYPE_MOSTLY_NVFP4`, `GGML_TYPE_NVFP4` is 40. Adjacent
numbers, unrelated enumerations, and swapping them yields a file that
loads and misreads.
The Qwen-shaped surface moves to a shared `tests_support` module. Its
first version omitted `attention` and `ffn` and the preflight refused
it — correctly, since a qwen35 hybrid attends every fourth layer — so
it is kept complete: a failing test should be telling you about the
code, not about the fixture.
The fixtures got the planner to coherent. The real container found what
they could not see: **160 of 848 decoder tensors had no target.**
input_layernorm.weight x64 -> blk.N.attn_norm.weight
post_attention_layernorm.weight x64 -> blk.N.post_attention_norm.weight
self_attn.q_norm.weight x16 -> blk.N.attn_q_norm.weight
self_attn.k_norm.weight x16 -> blk.N.attn_k_norm.weight
The name table was built from mixer-operand roles, so the layer norms
fell outside it entirely. None would have errored — llama.cpp would find
no norm where it expects one, which is the "loads and produces plausible
output" class again.
**The denominator is 851, not 848.** The decoder stack is one object of
four: embedding, final norm and an untied output head are separate, and
`output.weight` is the dangerous omission because llama.cpp's qwen35
loader treats it as optional and ties the embedding when absent. That
runs. It is not the model that was exported. So it is a required target
derived from `head_reuses_embedding = false`, and the walk refuses
without it regardless of what the runtime is willing to tolerate.
Vision's 333 tensors are excluded by name and reason rather than by
silence, so "we planned all 848" cannot quietly become "we planned the
model".
Three coverage invariants, failing by category: every physical source
tensor consumed exactly once or excluded with a reason, every required
target produced exactly once, no two plans claiming one name.
**Physical, not semantic** — a full-attention `q_proj` carries two roles
in one tensor, and counting roles would see two things to emit. NVFP4
runs the other way and adds sibling `.scale` tensors, so source and
target counts differ in both directions and neither is wrong.
Two tests are named for the mistake rather than the assertion:
post_attention_norm_is_a_trunk_norm_not_an_attention_layer_only_norm
untied_output_head_is_required_even_when_the_target_runtime_can_fallback
The first pins x64 rather than x16, because the tensor's name does not
describe its applicability and the obvious "simplification" is wrong.
Roles arrive as input. The walk never derives one from a tensor name;
that is the operation plan's job, and re-deriving here would reintroduce
the mistake the film spends a chapter explaining.
851/851 accounted, 0 unplanned, 0 duplicates, 0 missing.
Walking the represented hero found the third thing fixtures could not
see. `1696` sources for an `848`-tensor object, and 848 duplicate-target
errors:
DuplicateTarget blk.0.ffn_down.weight
<- 0.mlp.down_proj.weight
<- 0.mlp.down_proj.weight
The walk was right to complain and wrong to have been asked. `represent`
is archival by design: the compiled pack lands BESIDE the canonical
bytes, so a represented container holds `target.decoder_stack@BF16` and
`@NVFP4` at once. That is the point — it is what makes representation
first-class, and what lets `diff` compare the two. But it means
**"present in the index" is not "selected for execution"**, and the walk
had no selection step at all. It took the catalogue for the programme.
Selecting is the precision programme's job, so `inventory_from_container`
now takes it as an answer rather than making it:
object -> available representations -> programme selects exactly one
-> inventory reads that one -> planner lowers it
Three guardrails, because a selector that answers badly must not be able
to quietly shrink the model: declining to choose refuses rather than
omitting the object, naming a representation the object does not have
refuses, and exactly one must be selected per included object. Declining
is not the same as having nothing to choose from, and the difference
between them is 848 tensors.
The counts are now diagnostic rather than mysterious:
canonical represented
primary-text sources 851 851
NVFP4 scale siblings 0 848
GGUF target tensors 851 1,699
errors 0 0
1,699 is 851 selected physical tensors plus 848 target-generated scale
siblings — a number with an explanation.
The canary's selector ("prefer NVFP4, else canonical") is scaffolding,
labelled as such, standing in for the precision map exactly as
`role_for()` stands in for the operation plan's role assignment. The
authoritative canary is this same 851/851 ledger with both callbacks
gone.
… each other
Coverage proves every tensor has a target. This proves the target is the
right shape — from two derivations that are independent by construction:
planned physical source tensor + its layout transforms
expected graph-derived target metadata + the target ABI
Both modules can be individually correct and still disagree about the
model. Nothing else catches that: a `q_proj` at ordinary width is a
perfectly normal tensor, passes coverage, maps to a unique name, and is
wrong.
**Geometry is representation-independent**, so the two selections of one
model can be compared directly. `TargetGeometry` carries a name and dims
and nothing about encoding, which gives the sharper invariant:
> Representation choice may change target encoding and auxiliary
> tensors; it must not change the model's semantic geometry.
Against the real hero, both selections agree:
canonical represented
semantic targets 851 851
scale siblings 0 848
semantic-shape digest 6abf7cac566d1ef3 6abf7cac566d1ef3
`.scale` siblings stay out of the digest. They are target-ABI
auxiliaries, not model tensors, and including them would make the
invariant untestable by construction.
**Value transforms cannot move a dimension, structurally.** Target shape
is folded from the source through the LAYOUT transforms only, so
`MaterializeLogDecay` is not in the loop and could not affect geometry
even by mistake. The type system keeps it out rather than a comment
asking nicely.
`SqueezeSingletonAxis` now refuses in `LoweredTensorPlan::new` rather
than at write time. By the time the writer sees a plan it should be
unable to discover a semantic problem — only an I/O one.
Three near-misses pinned, each showing both derivations because "shape
mismatch" sends the reader to the wrong module:
head_dim disagreement planned [6144,5120] vs expected [12288,5120]
non-singleton squeeze [10240,2,4] — never collapse real channels
unfused query width "a plausible tensor and the wrong one"
The fused-Q rule is pinned semantically as 2 x q_heads x head_dim rather
than at Qwen's particular 12288, so another model's geometry still
holds.
One bookkeeping note: the canary printed a "shape digest" for one commit
while passing empty dims. It was renamed to "target-name digest" until
the shapes were actually plumbed through, then earned the better name.
4,661 tests pass.
…container
The geometry gate proved planned == planned: both hero selections
derive their shapes the same way, so matching digests proved
representation-independence and nothing more. `reconcile()` compared
two vectors a unit test typed. Nothing on the container path compared
the planner against the metadata's model — the loophole named in the
previous commit, still open.
This joins them. `ModelGeometry` reads the facts the metadata table
reads — hidden size, head counts, head dims, DeltaNet head geometry,
vocabulary — off the execution surface, never off a tensor.
`qwen35_expected_shape` turns a role into the shape the target ABI
demands from those facts alone. The walk calls `check_target` on every
plan as it is made, so the comparison now runs where the loophole
lived:
canonical represented
plans 851 851
reconciled with graph 851 851
disagreements 0 0
semantic-shape digest bc0009582ccabee1 bc0009582ccabee1
The digest moved from 6abf7cac566d1ef3, and should have: the conv
tensor's singleton axis is now actually squeezed in the plan
(`qwen35_layout` attaches it by role, beside the name table), so the
planned geometry changed on both selections identically. The
expectation for `ssm_conv1d` is the post-squeeze `[channels, kernel]`
llama.cpp binds — a plan that never squeezed disagrees here, which is
the point.
**The fused-Q factor is a graph fact, not a Qwen assumption.** The
surface declares `attention.output_gate.source =
fused_query_projection`, and that is what doubles the expected query
rows. A model without the gate expects an ordinary-width Q and is not
refused for having one. The semantic refusal still runs first on the
walk, so an ordinary-width Q on a gated model says why it is dangerous
rather than printing a bare mismatch.
Two tables must cover the same roles. A role with a target name and no
expected shape reaches the file named and unchecked, so
`NoExpectation` refuses it and a test pins the 23 roles across both
tables.
Pinned on the walk rather than in isolation: an ordinary-width `q_proj`
at layer 3 leaves coverage at 851/851 and geometry at 850, with
`UnfusedQueryWidth`; a `k_proj` at head_dim 128 shows `[512, 5120]`
against `[1024, 5120]` and names `kv_heads x head_dim`; a `[10240, 2,
4]` conv is refused by the constructor before any writer sees it. The
encoded-container test now derives `ModelGeometry` from the container's
own graph — the production path — and every roled tensor reconciles
against facts the encoder wrote.
The walk fixture's shapes were `[5120, 5120]` throughout. They are now
the hero's physical shapes, typed from its segment headers; deriving
them from the model facts would have made the comparison check nothing.
`Ledger::ready` now also requires `geometry_reconciled == accounted`.
13,807 tests pass across the workspace.
The walk read `entry.encoding` — the object's catalogue entry — and stamped every member tensor with it. An NVFP4 pack quantises the 2-D projections and leaves norms, the convolution and the 1-D parameters at source precision, per tensor, in the segment header. The object-level reading therefore promised 848 scale siblings on the represented hero where the pack actually holds 496. Nothing downstream had failed yet because nothing downstream consumes the representation until emission — which is exactly when a `.scale` sibling for a BF16 layer norm would have been written, and an NVFP4 repack attempted on bfloat16 bytes. The walk now reads each tensor's own dtype and refuses a spelling it does not understand, rather than defaulting. Pinned through the real pipeline: the fixture container is compiled to NVFP4 by `compile_representation`, and every walked source's representation must match its segment header's dtype — with the guards that the pack quantised something, that NVFP4 appears only on rank-2 tensors, and that the @NVFP4 object genuinely holds mixed dtypes, since a uniform object would make the test vacuous. Both hero selections still walk 851/851 with geometry reconciled and the same semantic-shape digest; the represented selection's ledger now says 496 scale siblings, which is what its segments say.
The plan types could express every transform since 1df50ea; nothing attached them. The walk stitched in the conv squeeze by hand and left layout and value empty — plans that were geometrically proven and semantically incomplete. `qwen35_transforms` is now the single table from role to programme, and the walk attaches whatever it says. Two families, both forced by llama.cpp's qwen35 graph: **The V-head permutation touches every tensor indexed by value head.** llama.cpp broadcasts K-head state across value heads tiled (`ggml_repeat`); the checkpoint stores V heads grouped by K head. The reorder therefore reaches eight surfaces per recurrent layer: the fused QKV's V rows (offset past 2 x 16 x 128 = 4096 Q/K rows), the gate's rows, the decay and write-strength projections' per-head rows, the 1-D log decay and timestep bias elements, the convolution's V channels after its squeeze, and the output projection's INPUT columns — by whole heads, which for NVFP4 means eight intact 16-element groups per head, the invariant the preflight established before any of this was expressible. A tensor left in grouped order binds cleanly and mixes heads; no shape check can see it, which is why the table is pinned role by role with the hero's numbers. When v_per_k == 1 the reorder is the identity and the programme says nothing. Heads that do not group refuse (`VHeadsDoNotGroup`) rather than half-permuting. **Value arithmetic folds declared facts only.** -exp on the log decay; each norm family's declared weight offset. Three offsets from three authorities — norm.pre, norm.final_norm, attention.qk_norm_weight_offset — and the gated norm's absence from the table is the graph speaking: its surface declares no offset, so none is folded. llama.cpp's converter writes that same distinction as a name-based exception; the table never needs to. A declared zero offset attaches nothing, because the operation is the identity and carrying it would change bytes for no semantic reason. Target encoding follows: a quantised source keeps its encoding — that is the point of the export — and an unquantised source stays BF16 only as a plain 2-D projection; norms, 1-D parameters, the convolution and anything value-transformed are stored F32, llama.cpp's convention and the exact representation of the f32 arithmetic's result. The hero canary now tallies the programme on both selections: layout-transformed 384 = 8 surfaces x 48 recurrent layers value-transformed 209 = 128 trunk + 32 q/k + 1 final + 48 A_log identical across representations, semantic digest unchanged — the reorders move bytes, not geometry. 3,756 tests pass in larql-vindex.
Code generation. The emitter receives already-resolved metadata and
LoweredTensorPlans and does what they say — no semantic lookup, no
representation choice, no family fallback, no role inference, no
target-name construction, no shape correction. It does not contain a
single role name. If it ever needs to know something a plan does not
carry, that is a missing plan field, never a lookup below the boundary.
Executing a plan means three things:
layout permute bytes as instructed — one head permutation
behind every V transform (new[a·K+b] = old[b·r+a]),
applied to row blocks or per-row column blocks
value f32 arithmetic on an unquantised lattice, stored F32
encoding BF16 passes through (streamed, unbuffered), F32
re-encodes, NVFP4 repacks losslessly with the tensor
scale carried out as the sibling the plan named
On the quantised path codes are moved, never decoded: scale rows move
with their code rows, whole 16-element groups move with their heads,
and a plan whose group accounting disagrees with the tensor refuses.
A lowering nobody defined refuses by name rather than being invented
at the bottom of the pipeline.
`verify_emitted` is the other half of the contract: parse the finished
file back through the INDEPENDENT reader — the one written for foreign
GGUFs, sharing nothing with the writer's descriptors — and require the
metadata to be exactly what was resolved and the tensor inventory to
be exactly what was planned, names, dims and types, nothing extra,
nothing missing, every NVFP4 tensor still holding its sibling. Every
defect is collected and named, not just the first.
The tests earn the claims the hard way:
- Both permutations are compared against an independently written
reshape/transpose reference, on marker lattices where any moved or
unmoved byte is visible, offsets included.
- The NVFP4 reorder is proven as *the same permutation the floats
undergo*: decode the source pack through its reference decoder,
permute the floats; decode the emitted GGML blocks through a decoder
written from the spec in the test; require exact equality. A code
moved without its scale cannot pass this, and no byte comparison
needs to notice.
- Value arithmetic is pinned bit-exact, and the emitted sibling is
read back and compared against the pack's own tensor scale.
- The round trip hands the verifier wrong expectations — a foreign
shape, a dropped plan, an altered metadata value, a missing required
tensor — and requires each named.
- A short write is refused at the byte it happens, because the offsets
are already committed.
4,677 tests pass across larql-vindex and larql-models.
Same rule as the metadata table: no literal unless it is a target
constant. The tokens, merges, token types and special ids come from
the capability snapshot the container carries; the two constants are
llama.cpp's vocabulary-model name (`gpt2`) and pre-tokenizer id
(`qwen2`), both facts about llama.cpp.
Two decisions stated rather than buried:
- **The token table pads to the model's vocabulary.** The embedding
carries vocab_size rows (a graph fact, 248,320 on the hero); the
tokenizer defines 248,077 ids. llama.cpp sizes the model from the
token list, so the gap becomes explicit `[PAD{id}]` entries marked
UNUSED — its own converter's spelling. An id the tokenizer defines
BEYOND the model's vocabulary refuses: that is a tokenizer for a
different model.
- **Special ids resolve through the files, in order** —
tokenizer_config.json's named tokens, else generation_config.json's
ids — and a container with neither refuses. An unterminated chat
model is not a guessable convention.
Types follow the files too: vocabulary tokens NORMAL, special added
tokens CONTROL, non-special added USER_DEFINED, padding UNUSED. Both
merge spellings (verbatim string, two-element pair) serialise to the
"left right" form llama.cpp reads. The chat template rides along from
chat_template.jinja when the container carries one.
… the capability snapshot Two related repairs, both found on the road to emission. **The hero graphs predate `context_length`.** ce69301 gave the fact a home on the execution surface; the containers were encoded before it, so the metadata gate refuses them — correctly. The fix is not to edit an artifact by hand: `graph_refresh` re-derives the graph from the same checkpoint with the same builder and prints it beside its validation counts, so the diff can be inspected before anything is copied. On the hero the diff was exactly one additive field per container: `context_length: 262144`, read from the config the graph was always built from. The canonical container took the rebuilt graph verbatim; the represented one took a surgical merge of the derived fact, because its stored graph additionally records the compiled NVFP4 representation, which a rebuild from the checkpoint cannot know. **`represent` kept tokenizer.json and dropped the rest.** A compiled container that keeps only the tokenizer can tokenise and not chat — no eos, no template — which reads as a broken model rather than a missing file. The aux list now carries the whole capability snapshot (tokenizer_config, special_tokens_map, generation_config, chat_template), matching what encode snapshots from the checkpoint. The hero's represented container received the same files from its checkpoint by hand, verified against the snapshot encode carries. Both hero containers now walk ready with context_length recorded.
…sees it
18.80 GB, emitted in 24.3s, and the independent reader agrees with the
plan exactly:
walk 851 plans, 496 scale siblings, geometry 851/851
vocab 248,077 tokens + 243 pad, 247,587 merges
emitted 851 tensors + 496 scale siblings, 32 metadata keys
VERIFIED 1,347 tensors (496 NVFP4, 496 siblings), 32 keys
The tensor count was deliberately not predicted: the plan produced it,
the file was parsed back, and 1,347 is now the observed figure this
message pins. Everything in the file traces up: names, dims and types
to the plans; metadata values to the graph-derived table and the
container's tokenizer snapshot; every NVFP4 tensor to its sibling.
What it took to get an honest file, each refusing loudly on the way:
- **Plan sources are object-qualified** (`object/name`): three
model-scope surfaces all name their tensor `weight`, and a payload
source keyed on bare names would have handed the embedding to the
output head — same bytes count, silently transposed model.
- **`general.file_type` derives from the selection.** It is display
metadata, but a BF16 export stamped MOSTLY_NVFP4 would still be the
file lying about itself.
- **A truncated segment refuses at inventory time.** The first
emission attempt died 500 tensors in with a "0-byte payload": the
represented container's NVFP4 segment held 7.66 GB of a declared
13.70 — an interrupted represent run, discovered only when read.
`inventory_from_container` now compares each segment's tensor-table
extent against its file and names the truncation before any plan
exists. The complete uniform-programme container took its place
(byte-exact 13,702,559,904, same encoder, same source); the
truncated one is on the repair list.
- The uniform container received the same graph refresh
(context_length 262144, derived) and capability snapshot as its
sibling.
`hero_emit` is the assembled pipeline in one example: inventory →
walk → graph-derived metadata (MRoPE theta/sections/fraction and the
attention cadence read from the per-layer policies) → tokenizer table
→ emit → verify. The one judgment it holds is the representation
selection, spelled once and passed to both the walk and the payload
source.
Next: llama.cpp.
…ntation, two runtimes
Generation proving "Paris" is the opening shot, not the acceptance
condition. This is the acceptance condition, and it is deliberately
narrow: exact token ids, full logits, predeclared tolerances.
Two gates. TOKENIZATION first: the GGUF's own tokenizer (via libllama,
add_special=false) and the container's tokenizer.json produce identical
ids for the fixed prompt — 12 ids, byte-equal — so the numerical
comparison owes nothing to chat templates or tokenizer drift. Those ids
are then frozen and fed to both runtimes:
same stored NVFP4 pack
↓ frozen ids
├── larql vindex3 exec --backend production-nvfp4
│ --representation-source stored (0 tensors
│ runtime-compiled — served from stored bytes)
└── llama_logits: a ~100-line libllama program, ids in,
raw [positions, vocab] f32 out — written because the
CLI surfaces only presentation, and tying the gate to
presentation behaviour would measure the wrong thing
EXECUTION, per position and overall, first recorded run
(canary-2026-08-31.json, gguf sha256 9494a78a…):
positions compared 12
top-1 agreement 100%
top-5 overlap 100% at every position
cosine mean 0.999956 min 0.999863
KL(A||B) mean 0.00014 max 0.00028 nats
max |dlogit| 0.2118 RMS 0.0296
Bit-identical logits are deliberately NOT the bar — different kernels
and accumulation orders move low bits while both implementations are
correct. The declared bar: top-1 everywhere, top-5 >= 80%, KL <= 0.05,
cosine >= 0.995. The observed KL sits two orders of magnitude inside
it, uniformly across positions — no systematic divergence.
The exported GGUF is semantically lowered to llama.cpp's ABI: on a
fixed token sequence, llama.cpp and VINDEX3 produce matching token
decisions and closely agreeing logit distributions from the same
selected representation.
Same model. Same representation. Two independent runtimes.
The pipeline that produced the verified hero file, assembled behind one
boundary (`gguf::export::export_qwen35`) and exposed as the CLI verb
the film types:
vindex export $QWEN_NVFP4 qwen3.8-27b-vindex-nvfp4.gguf
Two authorities moved to where they always belonged:
- **Roles come from the operation plan.** The walk's docs said from the
start that roles arrive as input and are never inferred from tensor
names; until now the only caller matched name fragments. The verb
reads `plan_component_ops` — the same closure-gated plan `describe
layer.N.mixer` renders — and maps each operand reference to its
target role. A fused attention gate is accepted only when it IS the
query tensor; a standalone gate, a third trunk norm, a routed FFN, a
KDA/MLA/Mamba2 layer each refuse by name, because qwen35 has no
binding for them.
- **Selection comes from the precision map.** The container's own
programme says which encoding executes; the exporter takes that
answer, falling back to canonical bytes when no programme is
recorded. No caller preference survives.
The regression that proves the move changed nothing: the re-exported
hero file is byte-identical to the first emission —
sha256 9494a78a209408b714f5e0f6b1a45d74b7dda0778fdf5fba7cede9a0da6e7c64
18.80 GB, 851 tensors + 496 siblings, verified, 28.0s
so the plan-derived role assignment agrees with the hand-written table
on all 851 tensors, and everything downstream is untouched.
Pinned on a container the test encodes itself: the fixture's roles
close, its walk passes coverage and geometry, and the export stops at
the honest gate — its graph declares plain RoPE where qwen35 needs
MRoPE, the refusal names both, and no file is left behind to mistake
for a finished one.
`hero_emit` is now a thin caller of the same function.
The verb the film types at §12:45. `vindex export` compiles the container's selected representation to a qwen35 GGUF — roles from the operation plan, selection from the precision map, metadata and tokenizer from the artifact, the file verified through the independent reader before the command returns. 0.7.0 rather than 0.6.2 because a new compiler target is a new capability, not a patch.
Named types for the transform programme, the role map and the walk's role-assignment callback; an allow with its reason on the eight-fact metadata builder; tidier emit tests; one needless mut in the streaming writer. The five findings in the Kimi Metal test files arrived with today's upstream merge and are main's existing state — not touched mid-release.
This was referenced Aug 31, 2026
twistios
pushed a commit
to twistios/larql
that referenced
this pull request
Sep 4, 2026
…bt, baseline the structurally unreachable The gate has been red on `main` since chrishayuk#363, which means it can no longer tell anyone about a real regression: the next genuine coverage drop is indistinguishable from the standing failure. Seven entries, added for TWO different reasons that must not be conflated. **DEBT — issue chrishayuk#367.** The five gguf/* files landed with ~2100 lines carrying four inline tests between them (emit/, preflight/ and walk/ have none): gguf/export.rs 50.8% 419 lines, 1 test gguf/emit/mod.rs 83.0% 517 lines, 0 gguf/vocab.rs 83.4% 400 lines, 3 gguf/walk/mod.rs 88.5% 434 lines, 0 gguf/preflight/mod.rs 88.8% 346 lines, 0 They are baselined AT TODAY'S LEVELS. That is a ratchet, not an exemption: the gate is green now and any drop below today still fails, so regression detection is restored while the debt stays visible and owned. Raise them as tests land; do not lower them. export.rs at 50.8% is the priority — half the export path is unexercised. **STRUCTURAL — do not chase with more tests.** represent/kda_candidate_real.rs (4.4%) is a #[cfg(test)] driver gated on a real 48B container the ubuntu job does not have, and kda_candidate.rs (75.2%) carries gpu-gated paths the non-gpu coverage run cannot reach. Same class as the integer.rs/stationary.rs/physical.rs entries the policy note already documents. Verified with CI's own script against a measured report: `Coverage policy passed: total 93.42% lines, 365 files checked, 304 files at 90.0% default, 61 debt baselines.`
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The new compiler target: `vindex export` compiles a container's selected representation to a qwen35 GGUF, verified through an independent reader before the command returns.
The chain this branch builds, each gate forced by a concrete failure before bytes were emitted:
Acceptance: llama.cpp loads and generates from the exported file, and on a frozen 12-token sequence its full logits agree with VINDEX3's own execution of the same stored NVFP4 pack — 12/12 top-1, 100% top-5 at every position, KL ≤ 0.00028 nats (tools/gguf-canary, recorded run committed).
13,842 workspace tests pass; clippy clean on the touched crates (the five findings in the Kimi Metal test files are main's existing state).
🤖 Generated with Claude Code
https://claude.ai/code/session_01WY5Bz971Hc9jucuvpK3Chx