Skip to content

feat(compiler): make outputs write-only with live input references - #104

Open
thiremani wants to merge 12 commits into
masterfrom
feat-write-only-outputs
Open

feat(compiler): make outputs write-only with live input references#104
thiremani wants to merge 12 commits into
masterfrom
feat-write-only-outputs

Conversation

@thiremani

@thiremani thiremani commented Sep 11, 2026

Copy link
Copy Markdown
Owner

Declared outputs are write-only inside their template. A caller can explicitly share an input with an output by reusing the same binding; later input reads observe earlier output writes in both ordinary and ranged calls. This replaces the seed-read analysis proposed in the closed #102 and fixes #103.

Behavior

For out, seen = Fold(current, item), statement order determines what seen receives:

Body order Plain call from 10 with item 5 Ranged call from 10 with items 1:3
seen = current, then out = current + item 15 10 13 11
out = current + item, then seen = current 15 15 13 13

An unaliased input keeps its own value. Sharing stays inside the call's staged result slots: sibling right-hand sides still read the caller's pre-assignment values. Empty ranges and skipped writes preserve existing destinations; fresh destinations start at zero.

What master did. Ordinary calls returned 15 10 for both body orders, while ranged calls returned 13 13 for the output-first body and 15 15 for a one-element range 5:6: the same read observed different values depending on whether the argument was 5 or a range containing only 5. Master's documented rule was the pre-call snapshot for ordinary calls; the live behavior in ranged variants was an artifact of routing the input to the output's staged slot. This PR removes that inconsistency by making every call follow the live-reference rule; the earlier commits on this branch had resolved it the other way, toward the snapshot.

Contract. Inputs cannot be assigned through their input names. Writing a shared output changes what subsequent input reads observe. Every simultaneous assignment evaluates its reads before applying its writes, inside a body as at the call site, so out, before = current + item, current still gives before the prior value. Keeping an old value across a write is an explicit assignment that creates an independent value; any copy it costs sits at that assignment.

With two shared inputs, sequential a = y then b = x produces 2 2 from 1 2. Simultaneous a, b = y, x swaps them to 2 1.

Output-name reads remain errors, including conditions, arguments, prints, and formatting markers. Inputs are read-only bindings, so %n writes into an input or iterator parameter are now rejected; acc_fmt uses a writable local copy.

Implementation and migration

  • Remove per-iteration input snapshots. Whether an argument shares a binding with a destination is decided at compile time from the call's names and lowered as a private variant of the specialization (internal linkage) named with the mangling scheme's marker-plus-count suffixes, _aN_<slots> for the alias pattern and _oN_<types> for widened output storage, both demangled by Demangle: inside it a direct scalar input reads the output's current value and a compatible indirect input receives the staged output pointer. Nested calls and caller-driven ranges forward sharing by name through the same alias bindings, so there are no hidden arguments, no run-time selects, and no pointer comparisons.
  • Specialize binding arguments on settled storage types and revisit calls after storage widening. Private output-storage variants preserve sharing across compatible ownership or array-shape representations. This fixes the static-to-heap and nested refinement cases in Call arguments are specialized on the binding's flow type, not its merged slot type #103.
  • Preserve prior output writes in CFG liveness when an explicit compatible input can observe them. Local copies retain their own binding permissions.
  • Native ABI 2.1: exported prototypes carry no alias selectors: a direct-return function is (params..., seed) and an indirect-return function keeps its leading result carrier followed by params..., with no seed. Functions without a range parameter are unchanged from master. Range-bearing functions on master carried a hidden i32 alias selector per direct scalar parameter before the seed; those selectors are removed, so their prototype changes and the seed moves one position earlier. This is a documented break in the C ABI spec ("Changes in 2.1"); there are no shipped native callers. An intermediate revision of this PR had instead added selectors to every function and resolved aliasing at run time, which cost about 10% on fib_tail.
  • Native pointer sharing holds within the called body's own statements; a nested Pluto call inside it stages its outputs, so it does not extend the sharing to a native caller. Pluto callers get sharing across nested calls statically. Documented in the spec; a generic pointer entry that resolves unknown sharing at run time, beside the private variants, is recorded as outstanding in the effects plan.
  • Templates that previously read output names use locals or single expressions. The Rebuild rewrite in this PR changes [26] to [37] and [2] to [6]. Live sharing also changes mem_alias_refine from Sibling: hello! to Sibling: static!.
  • Preserve existing LLVM loop metadata when adding unroll hints (separate commit). It was required while the selector signatures changed the optimized Fib loop's shape; with the original prototypes restored it is a robustness fix and stays.

The memory model, README, ABI specifications, IR plan, and effects plan describe the final rule. The effects plan distinguishes the implemented %n input restriction from pending write-effect modeling.

Verification

  • Race-enabled lexer, parser, compiler, and root tests pass; go vet ./... passes.
  • Full suite: 76 passed, 0 failed; all 75 runtime leak checks report zero leaks.
  • Existing fixtures cover both Fold orders, ownership, nested sharing, conditional writes, array resets, and caller-side staging. The only additional behavior pair from review is sequential versus simultaneous swap.
  • IR tests cover the array-copy regressions, the alias variant reading its output's storage, an unaliased call using the public specialization, and a variant skipping an incompatible output; metadata tests cover preservation, explicit policies, and idempotence.
  • Benchmarks against master (bench repo fib, fib_tail, harmonic; median of 10 with 5 warm-ups, native CPU, output parity checked): fib 0.97x, fib_tail 1.01x, harmonic 1.01x. With the earlier run-time selectors fib_tail was 1.10x slower (stable across three re-runs); restoring the plain prototype through compile-time variants recovered it.

thiremani and others added 6 commits September 11, 2026 21:31
…-stable

Declared outputs can no longer be read inside their template: values,
conditions, call arguments, prints, and formatting markers (including
dynamic width and precision) are rejected with
`output "y" is read inside its function; outputs are write-only, use a
local`. A body transforms inputs into outputs and never observes an
output's value, so the incoming destination seed can never leak in; the
seed and destination-seeded staging slots stay as an unobservable
keep-old carrier and the ABI is unchanged. This supersedes the seed-read
effect analysis of the closed #102: the reproducer `y = x > 0 x` then
`y = y + 1` is now a template error.

Range-bearing variants snapshot every non-iterator input at the start of
each scalar iteration. A direct scalar aliased to an output reads the
carried output once; an indirect input the body reads after an output
write keeps a private copy for the iteration. Before, an aliased input
read the output's storage on every read, so `out = current + item` then
`seen = current` observed the new value: `value, before = Fold(value,
1:3)` printed `13 13` and now prints `13 11`, and a ranged heap-string
accumulator prints `abc ab` instead of `ac`. The promoted-alias path is
deleted; nothing reaches it once inputs are snapshotted.

Fixtures that read an output are rewritten with locals or single
expressions. `Rebuild` changes from reading its freshly written output
to reading the previous iteration's result (`[26]` -> `[37]`, `[2]` ->
`[6]`), and `shareStaticOutput(0)` no longer blanks its second output.
The flow-versus-slot call specialization gap found in review is #103.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
snapshotIterationInputs copied every indirect input read after an
output write, even one no output could share storage with, so
`count, value = Read(data, 0:100000)` with integer outputs copied and
freed the whole array on every iteration: 2.2s for the probe against
milliseconds on master. The caller aliases an input to an output only
when the two types lower identically (setCallArgAliasSelectors), so
mirror that check before allocating a snapshot. The probe is back under
the timer's resolution and its IR carries no arr_i64_copy; a heap-string
input read after a heap-string output write still copies per iteration.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The structural CFG recorded which inputs a template reads after its
first output write so lowering could skip the per-iteration copy for
inputs read only before it. Measurement shows ranged string
accumulation is quadratic with or without that copy, because today's
lowering allocates a fresh string every iteration; the amortized
in-place append is planned, not implemented. The fact therefore bought
about a 2x constant on an already-quadratic path (0.99s to 1.97s at
160000 appends) while making correctness depend on a read-ordering
analysis being complete.

Remove the fact and its plumbing from the CFG, CodeCompiler, and
FuncArgs. An indirect input that some output could back is now copied
at the start of every scalar iteration; direct scalars still re-read
the carried output once. Inputs no output can alias are still left in
place. Linear accumulation will come from consuming the input on its
last use when carried appends land.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Live input alias selectors change the optimized Fib tail loop so its latch
already carries llvm.loop.peeled.count. Preserve existing loop properties
when adding scalar unroll hints, while respecting explicit unroll policies.

Read metadata operands through a native C pointer buffer rather than relying
on the layout of llvm.Value. Existing metadata tests cover preservation,
policy handling, and idempotence.
Keep outputs write-only while allowing explicitly aliased inputs to observe
earlier output writes in ordinary and ranged calls. Remove iteration
snapshots, preserve nested reference identity, and specialize calls on
settled binding storage, including compatible wider output slots.

Reject formatting %n writes to input and iterator parameters; writable local
copies retain their own permissions. Document the live-reference rule and
cover both Fold orders and sequential versus simultaneous swaps without
adding duplicate test combinations.

BREAKING CHANGE: native ABI 2.1 adds an i32 alias selector for every direct
scalar parameter in ordinary as well as ranged variants. Native callers
must supply zero for inputs that do not alias an output.

Fixes #103.
@thiremani thiremani changed the title feat(compiler): make function outputs write-only and inputs iteration-stable feat(compiler): make outputs write-only with live input references Sep 12, 2026
thiremani and others added 6 commits September 12, 2026 18:32
Reads precede writes in every simultaneous assignment, inside a body as at
the call site, so a shared input still yields its prior value within the
statement that writes the output. Keeping an old value across a write is
an explicit assignment; that is where any copy is paid.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Whether a call's argument shares a binding with one of its destinations is
known from the call's names, so it no longer travels as a hidden i32 alias
selector on every direct scalar parameter. A call whose argument names its
own destination lowers to a private variant of the specialization,
`<mangled>$alias$<pattern>` with internal linkage, in which a direct scalar
input reads the output's current value and a compatible indirect input
receives the staged output pointer. Nested calls and caller-driven ranges
forward the sharing by name through the same alias bindings, replacing the
run-time selects and pointer comparisons.

The exported prototypes return to ABI 2.0, `(params..., seed)`, and the
plain variant's IR matches master's, which recovers the 10% loss the
selectors had caused on fib_tail: master-relative timings are now fib
0.97x, fib_tail 1.01x, harmonic 1.01x. Behavior is unchanged; every alias
fixture prints the same output.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The private alias and output-storage variants were named with `$`, which
LLVM accepts but which is not a C identifier character and did not follow
the mangling scheme. Name them with the scheme's lowercase-marker-plus-
count form, alongside `_fN`, `_tN`, and the reserved `_cN`: `_oN_<types>`
lists every output slot's storage type for a widened-storage variant and
`_aN_<slot>...` carries the per-parameter alias pattern, `_oN` before
`_aN` when both apply. MangleVariant builds the suffixes, Demangle parses
them back into OutputStorage and AliasPattern and renders them as
`-> (StrH, StrH)` and `[in1->out1]`, and the C ABI spec gains §5.2 plus
grammar rules for VariantSym. Emitted symbols change; behavior does not.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…onals

Two lowerings replaced the name a shared input was recorded under, so a
nested call inside them selected the unshared variant. Ranged staging
rebinds an aliased input to the staged output slot, which no longer
matched paramAlias.Base; `out, seen = Fold(current, (1:3) + 0)` in a
wrapper called with a shared input gave `13 11` instead of `13 13`, and
a second ranged call in the same body gave `20 16` instead of `20 20`.
Conditional lowering targets synthetic `$c_cond_<name>` destinations
that never equal the recorded output name; `item > 0 Fold(current, item)`
gave `15 10` instead of `15 15`.

A parameter may now carry several alias bases, and ranged staging
registers the staged slot as one. Conditional temps record the source
destination they commit into, and call-site aliasing resolves through
that map before comparing names. Fixtures cover both ranged shapes,
arrays, and the conditional call taken and skipped.

The C ABI spec is versioned to 2.1: master's range-bearing variants
carried hidden alias selectors before the seed, and removing them moves
the seed, so those prototypes change. Native pointer sharing is scoped
to the called body's own statements; a nested Pluto call stages its
outputs and does not extend it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Only direct-return functions end in a seed; indirect returns keep their
leading result carrier and have none. The effects plan and the ABI
optimization plan no longer claim the calling convention is unchanged:
range-bearing prototypes change in 2.1, and the generic pointer entry for
native callers with unknown sharing is recorded as outstanding. The alias
fixture gains the two-level array wrapper from review.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…tor wording

NestedArrayRange now stages the ranged call and makes its nested call
inside that loop, which is the shape that lost the alias before ceb8a89:
it prints `[10 1 2] [10 1]` on 8c862a2 and the expected `[10 1 2]
[10 1 2]` on the fix. The previous extra wrapper forwarded the alias
before staging and passed on both. The mismatched-sibling IR test now
shares the input with its second output so the variant has to skip the
incompatible first one, and the unrelated-array test asserts the public
specialization is called rather than a retired selector name. A stale
comment describing a run-time pointer select is corrected.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Call arguments are specialized on the binding's flow type, not its merged slot type

1 participant