Uh oh!
There was an error while loading. Please reload this page.
docs: design the micro-compiler architecture for the compilers - #158
Merged
Conversation
The OpenAPI compiler is one Go package whose ~20 files are a filing convention rather than a boundary, and one struct has accumulated 151 methods. The per-file split did not reduce that count; it grew. The individual functions are not the problem, since no body in the package exceeds 50 lines against a 70-line cap. The failure is type surface, which no rule in force measures. This records the architecture that replaces it: every lowering step is a pure function over an explicit position, returning its diagnostics rather than accumulating them, with interning as the single shared effect. Concerns become packages so the compiler enforces the boundary, since file boundaries demonstrably do not. It also records three findings that shape the work. The canonical naming grammar is written three times across the OpenAPI, GraphQL and Protobuf compilers, two copies identical and one behaviourally divergent, while the verifier that guards neutral naming checks only casing and is silent on segmentation. The architecture test's prefix matching lets one compiler import another undetected. And the general two-order diff the testing rules prescribe does not exist, which matters because moving interning is exactly what order dependence is sensitive to. The document is tracked under docs/ rather than left as a working note, because the previous design for this work was lost to an ignored path and had to be recovered from the history of a deleted file.
The first draft asserted a shape without specifying it. This settles the parts a reader would otherwise have to invent: what a micro-compiler is, what Ctx looks like, which packages exist and which way they import, and what each step has to prove before it lands. Testing carried the least weight and now carries the most, because most of this work is proving behaviour did not change rather than changing it. Two oracles the repository lacks are specified here. The general two-order diff exists only as three hand-written cases at the site where the pointer-collision bug was found, and the determinism check that looks like its general form recompiles the same bytes, which is a different property. Nothing at all asserts that two distinct source constructs cannot mint the same type ID, and moving lowering between packages is exactly when an ID derivation gets rewritten. Three claims from the first draft were wrong and are corrected. The annotation readers cannot move without the site type they read, which is declared alongside reference resolution. Promoting the naming grammar is not behaviour-neutral, since the three copies disagree and at most one survives unchanged. And constraints carries lowerer methods, so it joins a package during the conversion phase rather than founding one. Also records the blind spots that must not be mistaken for coverage: the harness returns before the invariant checks when a fixture trips an error diagnostic, the annotation matrix cannot address a carrier position, and the verifier checks that provenance is in range but never that a pointer is correct.
The design says what to build; this says what order it lands in and what each step waits on. The detail for each unit of work lives in its issue rather than being repeated here, and the blocking relationships are recorded as real issue dependencies rather than only as prose, so the next available piece of work can be queried instead of inferred. Three things gate everything else and are worth stating plainly. The architecture test cannot currently enforce compiler isolation, so the package boundaries this rests on are unenforceable until it can. The two-order diff and the type-ID collision check both have to exist and be proven against the current code before any lowering moves, because a guard written during a refactor and first observed passing has demonstrated nothing. The lint caps land last rather than first. They are calibrated against the finished shape, and enforcing them earlier would only encode the shape being replaced.
An audit of the filed work against the design found four issues already in the backlog that land inside this restructuring rather than beside it, and one piece of work with no issue at all. The provenance report is the one worth naming. It proposes a method on the lowerer, which the restructuring removes, so its direction no longer holds — but its problem gets sharper, because hand-built provenance is the single change here with nothing behind it: the verifier checks that a source index is in range and never that a pointer is right. The reference-resolution cleanup and the test-scaffolding report are likewise reframed rather than dropped, and the golden line-ending gap is noted because byte equality is what proves roughly twenty pull requests neutral. The missing work was documentation. Both the architecture document and CLAUDE.md carry a package-layout tree that this effort invalidates, and CLAUDE.md states the documents are the spec and are read first, so a diagram that no longer matches the tree misleads rather than merely ages. Also unblocks the naming-divergence defect, which had been made to wait on the promotion that would fix it. That contradicted the reason it was filed separately: it ships today and must not depend on this programme running to completion.
Provenance pointer correctness was the one property in the restructuring with nothing behind it. The verifier checks that a source index is in range and never that a pointer is right, so a signature change passing the wrong coordinate produced a node that was wrong but valid, and only a human reading a golden diff would notice. It does not have to be that way. Every named entity derives its ID from its source pointer, so an ID and its own provenance must agree, and that is exactly checkable. The type-ID oracle now carries both assertions: an ID minted from the wrong pointer disagrees with its provenance, and a grammar that collapses two pointers breaks injectivity. Neither implies the other and one traversal gets both. Where it lives is left open deliberately. The verifier imports only the IR while the grammar is headed for the compiler framework, so the harness is the fallback. The better answer is to decide whether the identifier shape belongs in the IR itself, which would enforce it for every compiler rather than only under test — that belongs with the promotion, not here. The golden line-ending gap is promoted from background cleanup to a prerequisite, blocking both entry points. Byte equality is the sole proof of neutrality across all fifty-nine snapshots, and a comparison that can fail environmentally teaches a reader to dismiss golden diffs at the one point where every diff has to be read as a finding.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The OpenAPI compiler is one Go package whose files are a filing convention rather than a boundary,
and a single struct has accumulated every lowering concern. The per-file split did not reduce that;
the method count grew. The individual functions are not the problem — no body in the package exceeds
50 lines against a 70-line cap. The failure is type surface, which no rule in force measures.
This adds two documents.
docs/micro-compiler-design.mdis the architecture: every lowering stepbecomes a function over an explicit position, returning its diagnostics instead of accumulating
them, with type interning as the single shared effect, and concerns become packages so the compiler
enforces the boundary.
docs/micro-compiler-plan.mdis the order the work lands in and what eachstep waits on; the detail for each unit lives in its issue rather than being duplicated, and the
blocking relationships are recorded as issue dependencies so the next available piece of work can be
queried rather than inferred.
Three findings surfaced while writing them, each verified against the tree rather than inferred:
compilers. Two copies are identical; the third treats
.as a word separator and the other two donot.
Naming.Canonicalis part of the IR contract, so the same field means different thingsdepending on which compiler produced it, and
irverifychecks only casing — never segmentation —so both spellings pass.
compilersas licence forcompilers/graphql, so one compiler may import another today with nothing to catch it.hand-written cases at the site where the pointer-collision bug was found, and the determinism check
that resembles its general form recompiles the same bytes, which is a different property.
Separately, nothing anywhere asserts that two distinct source constructs cannot mint the same type
ID.
Each is dispositioned rather than noted: the first two are prerequisites, and the naming divergence
is filed on its own since it is a live contract violation regardless of whether this work proceeds.
The documents are tracked under
docs/rather than kept as working notes because the previous designfor this area was written to an ignored path, could not be committed, and had to be recovered from
the history of a since-deleted file.
Test plan
Documentation only — no Go files are touched, so no code path changes and the gate is unaffected:
Every derivation command in the design document was executed against
a095636and its outputmatches the surrounding claim; the documents give the command rather than the number wherever a
count would otherwise rot. Every issue referenced in the plan was checked to resolve to the issue it
is described as.