The union's absent-column contract is undefined-or-throws, not null (#731)

[philcunliffe]

Corrects a documented invariant that was never true of the code, and pins the real contract with tests. No runtime behaviour changed.

What the docs claimed

LLP 0015#multi-partition-union and the comment at src/core/query/union-source.js:28 both said:

columns is always forwarded: projecting an absent column reads as null, never throws.

Neither half was ever true.

What actually happens, measured

Built a fixture of two real parquet partitions unioned by unionSources, with additive drift: partition A {id, score}, partition B {id, score, extra}. Instrumented the sub-scans to see the options each partition received and the row.columns it yielded.

The issue's headline claim is directionally right but wrong in one important detail: the projected key is not missing. It is present with the value undefined.JSON.stringify drops undefined-valued keys, which is how the reporter saw [{}, {extra:"x"}].

SELECT extra FROM t
rows=[{},{},{"extra":"x"}] ownKeys=[["extra"],["extra"],["extra"]]
hasOwnProperty('extra')=[true,true,true] ===undefined=[true,true,false] ===null=[false,false,false]
SELECT extra FROM t WHERE score > 1 -> identical
SELECT extra AS e / LIMIT 1 / subquery / ORDER BY id -> same, no throw
SELECT extra FROM t WHERE extra IS NOT NULL -> THROW ColumnNotFoundError (row 1)
SELECT extra FROM t WHERE extra IS NULL -> THROW
SELECT id FROM t WHERE extra = 'x' -> THROW
SELECT coalesce(extra,'none') AS e FROM t -> THROW
SELECT extra || 'y' AS e FROM t -> THROW
SELECT id, extra FROM t ORDER BY extra -> THROW
SELECT DISTINCT extra FROM t -> THROW
SELECT extra, count(*) FROM t GROUP BY extra -> THROW
SELECT max(extra) / count(extra) FROM t -> THROW
SELECT * FROM t -> OK, hasOwnProperty('extra')=[false,false,true] (key truly absent)
SELECT extra FROM t, single partition A -> THROW at plan time

The boundary is copy-versus-evaluate.executeProject takes a fast path for bare identifier projections: it copies row.resolved[name], which is undefined for a partition lacking the column, so nothing ever invokes the cell that would throw. Any other use goes through evaluateExpr, whose identifier lookup misses row.cells and throws at the first row from the lacking partition.

Mechanically: parquetDataSource builds each row's advertised columns from Object.keys(data[0]) (what hyparquet actually returned), and hyparquet returns {} for a file lacking a requested column, so the drifted partition's rows never carry an extra cell at all. The column is addressable in the first place only because union.columns advertises the superset; with no partition having it, planning fails.

The choice made

Option (a) from the issue: correct the doc and the comment to describe the real contract. Not option (b), changing the union layer to null-pad.

The reasoning: the code throws loudly rather than returning silently-wrong data, so current behaviour is defensible; correcting a doc that states a false invariant is strictly an improvement regardless of which option is ultimately chosen; and null-padding is a behaviour change a maintainer should make deliberately, not one slipped in under a doc-accuracy issue. If you would rather have (b), this PR does not foreclose it - it makes the current contract explicit, which is what a behaviour change would then have to supersede.

The LLP 0015 edit is an in-place factual repair rather than a superseding doc, since it corrects a statement that was never true of the code rather than changing anything the spec settled. Minimal and surgical; the section is not restructured.

Also fixed

The inline import('squirreling/src/types.js') type CLAUDE.md forbids. Note it is at line 366 on current master, not the 439 the issue cites - the file has shifted since filing. ScanColumnResults moved into the file's existing @import block.

Tests

Three added, two carrying @ref LLP 0015#multi-partition-union [tests], all over real parquet partitions:

• a projected column one partition lacks reads as undefined, never null (asserts own-key present, values, !== null, and the exact JSON.stringify output)
• evaluating such a column throws ColumnNotFoundError (five shapes: WHERE, WHERE-not-projected, coalesce, ORDER BY, aggregate)
• SELECT * keeps each partition's row shape, so a drifted key is absent rather than undefined

Suite 3978 pass / 0 fail / 1 pre-existing skip (+3 new); typecheck clean; llp-ref-hygiene 11/11, resolving both new anchors.

Fixes#731

[philcunliffe]

Review round 1 of 76cc970. Verdict: findings - 2 major, 3 minor. All five fixed and pushed as 763c931.

The reviewer re-derived the contract on its own fixtures rather than re-running the PR's, probed roughly 40 SQL shapes, and read executeProject/evaluateExpr/collect in the installed squirreling. Every observable claim held. The documented mechanism did not - and the precondition that mechanism hid is load-bearing.

1. major - the mechanism was backwards, and it hid a load-bearing precondition. FIXED

Both texts said a bare identifier projection "copies the value through and yields undefined". Nothing copies anything, and the projected cell does throw.

executeProject takes the copy path only if (sourceName in row.cells). For a partition lacking the column that is false, so control goes to the else branch, which installs a throwing evaluator thunk and writes noresolved[alias]. squirreling's own comment says why: "fall through to the evaluator so suffix-search and the proper ColumnNotFoundError apply instead of emitting an undefined cell."

projected AsyncRow for SELECT extra FROM t
resolvedHasKey=false cells.extra()= THROW ColumnNotFoundError (row 1)
resolvedHasKey=false cells.extra()= THROW ColumnNotFoundError (row 2)
resolvedHasKey=true cells.extra()= "x"

The undefined and the present own-key come entirely from collect(), whose all-rows-pre-materialized fast path builds item[col] = row.resolved[col] over row.columns, creating the own key from a missing one and never invoking the cell.

Two consequences the old text denied:

• The real boundary is consumer reads resolved versus invokes cells, not copy-versus-evaluate. A caller iterating AsyncRow.cells gets ColumnNotFoundError on a bare projection - the shape the doc promised never throws.
• collect()'s fast path requires every row to carry resolved. Built from partitions yielding hand-rolled AsyncRows without it - a legal AsyncDataSource, and this very test file already exercises legacy bare-iterable sources - SELECT extra FROM t, SELECT extra AS e FROM t and LIMIT 1all throw. The claim is true today only because every in-repo partition goes through squirreling's asyncRow. unionSources accepts arbitrary sources, so the next partition implementation that skips asyncRow silently converts "reads undefined" into "throws".

Fixed in both places, now naming the resolved-versus-cells seam and stating the asyncRow precondition explicitly. The fixer independently re-derived it before writing and agreed in full, adding one precision: the projected row does carry a resolved object; what is missing is the key.

2. major - the same false invariant survived on the flagship dataset. FIXED

ai-gateway/src/dataset.js:167 still read: "a row object that lacks the key simply reads as null, which is the correct value for 'this partition predates the column'."

withSchemaColumns wraps unionSources and its scan delegates straight through, so this is the identical row path. Measured: key-present/undefined, not null, and WHERE extra IS NULL throws.

After this PR the repo would have held two comments about the same code path flatly contradicting each other, with the surviving wrong one on ai_gateway_messages - the dataset a user chasing this symptom actually queries. Issue #731's option (a) said "correct the doc and comment"; this was the second, more consequential comment. Fixed, now also pointing at scanColumn as the layer that genuinely does normalize holes to null, which is the whole reason that forwarding exists.

3. minor - the plan-failure claim was false of the shipped path. FIXED

"When no partition has it, planning fails with the same error" is true of a bare unionSources and false of every production dataset, each of which is wrapped by a layer advertising the declared schema precisely so planning does not fail:

bare SELECT extra FROM t THROW ColumnNotFoundError
wrapped SELECT extra FROM t OK [{},{}] own=[true,true]

Fixed with the withSchemaColumns qualification.

4. minor - the LLP edit needed a provenance note. FIXED

LLP 0015 is Status: Active, and CLAUDE.md allows in-place edits for changes that do not change meaning; this reverses a stated invariant. The in-place repair is still right (the sentence described the code, wrongly - it was never a decision the spec settled), but as landed the record read as if the spec always said this. Fixed with a one-line corrected-by note.

5. minor - test helper duplication, and "Pinned by" was not honest. FIXED

asyncBufferFromBytes was a byte-identical copy of the one in parquet-source.test.js. test/helpers/ already exists, so both now import a shared parquet_source_fixture.js. And the LLP named "with or without an alias" and "a LIMIT" as part of the pinned contract while neither shape was in the test; both are now in the case table, alongside two new tests - one inspecting the unresolved cell and throwing thunk directly, one proving a partition without resolved makes all three bare shapes throw.

Also checked, clean

• Roughly 40 shapes re-derived independently. Non-throwing, all key-present/undefined/not-null: bare projection, alias, LIMIT 1, LIMIT 0, subquery, ORDER BY id, ORDER BY score DESC, WHERE score > 1, LIMIT 2 OFFSET 1, SELECT extra, extra, SELECT t.extra, SELECT a.extra FROM t a, SELECT id, extra, SELECT *, extra, SELECT count(*), UNION ALL.
• Shapes the PR did not list, all consistent with the rule - so the rule is complete even though the enumeration is not: CASE WHEN extra IS NULL, NOT extra, extra IN (...), cast(extra AS TEXT), HAVING max(extra) IS NULL, GROUP BY id, extra, DISTINCT id, extra, row_number() OVER (PARTITION BY extra ...), and a JOIN projecting a.extra. Every one throws. No shape contradicts the documented rule.
• SELECT * genuinely distinguishes absent from undefined (hasOwnProperty[false,false,true]), and that assertion fails under a null-padding implementation.
• Test discrimination is honest, not loose.assert.deepEqual here is node:assert/strict, so [undefined, undefined, 'x'] does not match [null, null, 'x']; combined with the explicit === null check, the exact JSON.stringify string, and the hasOwnProperty array, the tests fail against a null-padding union and against the old claims.
• Consumer behaviour for undefined versus null.format.js:238 maps both to '', so table and markdown render identically - no bug. json/jsonl, and therefore the MCP query_sql surface, do differ visibly: JSON.stringify drops the undefined key, producing ragged objects, which is exactly the reported symptom. The LLP text already calls that out. rowBytes, truncateRow and inferColumns all tolerate undefined. In-repo AsyncRow-level consumers are safe: visibility.js:187 prefers resolved and falls back only when the key is genuinely present, and iceberg/store.js:565 runs against a single source, never a union projection.
• The @import fix: zero import( occurrences remain in that file, and tsconfig.json includes test, so typecheck genuinely covers it.
• Conventions clean in all changed files; the LLP edit is minimal and the #multi-partition-union anchor resolves.
• Ran: npm test 3980 pass / 0 fail / 1 skip; typecheck clean; llp-ref-hygiene 11/11.

One nuance from the fix worth recording: typecheck initially failed on the new hand-rolled source because AsyncCell is () => Promise<SqlPrimitive>, not sync. Caught by npm run typecheck, which npm test alone does not cover.

The head has moved to 763c931, so the next tick reviews that head (round 2).

[philcunliffe]

Review round 2 of 763c931. Verdict: findings - 1 major, 3 minor. All fixed and pushed as a08857e. The fixer also corrected two claims in the review itself, with evidence; both corrections are recorded below because they changed what landed.

1. major - the resolveable gate is a missing third precondition. FIXED

The corrected text named two preconditions for the undefined read and stopped. There is a third, in the very function the text cites. executeProject computes up front:

constresolveable=plan.columns.every(col=>col.type==='star'||col.type==='derived'&&col.expr.type==='identifier')

and emits no resolved map at all when that fails. So one non-identifier output column - an expression, a function, even a literal - collapses collect()'s fast path for the whole result, and the drifted column's thunk is invoked and throws:

SELECT extra FROM t -> extra=UNDEF | UNDEF | "x"
SELECT id, extra FROM t -> fine (all identifiers)
SELECT *, extra FROM t -> fine (star + identifier)
SELECT extra, 1 AS lit FROM t -> THROW ColumnNotFoundError
SELECT extra, score * 2 AS d FROM t -> THROW
SELECT extra, coalesce(score,0) AS c -> THROW

Why it mattered: the text's dichotomy was "bare identifier projection reads undefined" versus "anything that evaluates the absent column throws", and everything it listed on the throwing side was an operation on the absent column. SELECT extra, 1 AS n evaluates nothing on extra and throws. A maintainer using the paragraph to judge whether a query is safe would get the wrong answer for a common shape. And the corrected text is more likely to be trusted than the old one precisely because it cites internals.

Fixed in both places with the three-condition formulation, plus tests for both throwing shapes and a SELECT score, extra FROM t case so the gate's boundary is pinned on both sides.

2. minor - "only while every partition yields rows built by asyncRow" was a false necessary condition. FIXED

asyncRow is not required. resolved is originated only by asyncRow and propagated by cachedDataSource and executeProject, but nothing checks provenance - collect() tests if (!rows[i].resolved) and nothing more. Verified: a hand-rolled source that never calls asyncRow but attaches its own resolved keeps the fast path. That is the difference between a rule a new partition implementation must satisfy and an implementation detail it happens to satisfy today. Reworded to name the real rule.

3. minor - the false invariant survived in three more places. FIXED

Round 1's major #2 was this exact thing one file over, and the sweep stopped at the instance it named:

• message_projector.js:25 - the comment a developer reads while adding a nullable column, i.e. the moment the claim is load-bearing.
• llp/0055:77-80 - the parenthetical "the same additive schema-drift rule withSchemaColumns already applies to row reads", which dataset.js:202@refs straight at, so a reader following the corrected comment's own reference landed on the uncorrected claim.
• test/core/ai-gateway-dataset.test.js:287 - assert.equal(seen[0].git_remote ?? null, null, 'absent column reads as null'), where ?? null launders undefined into null and the message asserts the untrue half.

All three fixed. For LLP 0055 the fixer took both offered options - dropped the parenthetical and appended a corrected-by note - reasoning that dropping alone would leave a reader arriving via the @ref unable to tell the doc was ever wrong, while a note alone leaves the false claim as the first thing they read. The note states explicitly that what the decision settled is unchanged.

4. minor - dataset.js overclaimed why scanColumn is forwarded. FIXED

"...which is the whole reason the scanColumn forwarding exists" contradicted the block comment 30 lines below in the same function. The forwarding exists for LLP 0055's streaming aggregates; the null normalization is a correctness duty inside it, not its motivation. Corrected.

Two corrections the fixer made to this review, both verified and both material

The flagship-wrapper example in finding 1 was wrong.SELECT git_remote, 1 AS n FROM ai_gateway_messages does not throw. createDataSource builds only from storage.dataSourceForTable, which is icebird, not parquetDataSource (used solely by the s3 plugin). Icebird answers a scan for a column it lacks with row.columns containing the name and a cell resolving to undefined - it never omits the cell, so there is no throwing thunk. Measured over two drifted cache partitions:

SELECT git_remote FROM t -> NULL | g (scanColumn path)
SELECT git_remote, 1 AS n FROM t -> NULL,n=1 | g,n=1
SELECT date, git_remote, 1 AS n FROM t -> git_remote=UNDEF | g
WHERE / ORDER BY / DISTINCT / coalesce / max / count on git_remote -> all OK

The resolveable gate still fires (hence the UNDEF), it just cannot produce a throw when the underlying cell exists.

Consequence: the PR's own dataset.js text was a fourth wrong mechanism. Its claim that "anything that evaluates it throws ColumnNotFoundError" is false for that stack. The fixer corrected it beyond the ask rather than leave it, and the comment now distinguishes the two backends explicitly: icebird-backed partitions resolve to undefined, and the throwing half of LLP 0015 belongs to parquet-backed partitions, which omit the cell entirely.

Finding 3's proposed message_projector.js wording repeated that same error ("reads as undefined or throws"). The fixer applied the finding's intent with verified wording instead - hole reads undefined on the row path, JSON.stringify drops it, only scanColumn normalizes to null, and SQL predicates treat it as NULL either way (verified: WHERE git_remote IS NULL returns the old row).

That is the right call. This is the third attempt at describing this mechanism, and writing prescribed text that had been measured false would have made it the fourth.

Verified from round 1

• Major 1 (mechanism backwards) - landed, and re-derived from the squirreling source rather than accepted. The else-branch installs the throwing thunk and writes no resolved entry; evaluateExpr throws after exact-match and suffix-search both miss; collect()'s fast path creates the own-key-with-undefined. Incomplete only in the way finding 1 describes.
• Major 2 (dataset.js) - landed; the scanColumn normalization claim is true (dataset.js:216-218 rewrites undefined holes to null). Its rationale clause was wrong (finding 4) and its throws clause was wrong (the correction above).
• Minor 3 ("planning fails" qualified) - landed correctly, both halves verified empirically.
• Minor 4 (provenance note) - landed as a blockquote note on an Active doc rather than a silent rewrite.
• Minor 5 (helper extraction) - a faithful merge. parquet-source.test.js keeps rowGroupSize: 2; the union test's implicit undefined hits hyparquet-writer's default parameter, so behaviour is byte-identical. The no-resolved test does discriminate: the negative control (identical hand-rolled source with resolved added, nothing else changed) returns undefined instead of throwing, so it measures resolved and not an artifact of the row shape.

Also checked, clean

• One non-resolved partition really does disable the fast path for the whole result - utils.js:62-67 breaks out of the allMaterialized loop on the first such row and takes the slow path for every row. The doc's "for the whole result" is exact.
• collect() is the consumer on the shipped path (query/verb.js, commands/local_only.js, commands/clients.js, query/overview.js).
• ~20 further SQL shapes beyond round 1's set all agree with the rule as stated, except the mixed-projection class in finding 1.
• One pre-existing observation, not a finding: withSchemaColumns.scan forwards where verbatim, so only the scanColumn path strips a predicate naming a declared-but-absent column. Predates this PR and the corrected text describes it correctly; flagged only so it is not mistaken for a regression later.
• Conventions clean across all seven changed files; npm test 3980 pass / 0 fail / 1 skip; typecheck clean; llp-ref-hygiene 11/11.

The head has moved to a08857e, so the round budget (2) is spent at an unreviewed head: the next tick triages rather than opening a round 3.

[philcunliffe]

Triage after the review budget (LLP 0017). Two rounds ran; both cleared. The triage re-derived the whole contract itself and found the icebird-facing text added in round 2's fix is false. This is a doc-accuracy PR, so a false documented claim is its entire defect surface.

The blocker

Three sentences are demonstrably false at this head:

• dataset.js:170-171 - "a projection of it yields undefined (key present, not null)"
• message_projector.js:28-31 - "the value a projection hands back is undefined ... which JSON.stringify drops instead of rendering as null. Only the scanColumnaggregate path normalizes the hole to null"
• llp/0015:102-104 - "unless a wrapper advertises the declared schema ... in which case the same undefined-or-throws contract applies"

Measured on a real icebird-backed ai_gateway_messages fixture:

SELECT git_remote FROM t -> [{"git_remote":null}, ...] null, rendered, NOT undefined
SELECT git_remote, 1 AS n FROM t -> {"git_remote":null,"n":1} no throw
SELECT id, git_remote FROM t -> undefined, key dropped (matches the doc)

The cause: squirreling routes any single-column scan through scanColumn when the source offers one - execute.js:284 gates only on plan.hints.columns?.length === 1, no aggregate required - and withSchemaColumns normalizes holes to null on exactly that path (dataset.js:222-224). So calling it "the scanColumnaggregate path" tells a maintainer plain projections are unaffected, when the most common projection of an absent column is precisely the one that contradicts the doc.

A maintainer acting on the LLP sentence would predict undefined-or-throws for the flagship dataset and observe null-or-null.

Why two rounds missed it

The PR's icebird tests pin only the raw scan() rows and the scanColumn chunks (ai-gateway-dataset.test.js:254-296, 298+), never a full executeSql + collect SELECT on the icebird fixture. SQL-surface tests exist only for the parquet union. The false sentences live precisely in the untested gap - and were written late, in response to a reviewer error, which is exactly the condition under which a plausible-but-wrong sentence gets in.

What is verified good

The parquet half is right, and every sentence of it survived independent re-derivation: the undefined read with alias, LIMIT, WHERE on a shared column and a bare sibling; the SELECT extra, 1 AS n sibling collapse; every evaluating shape throwing; SELECT * keeping per-row shape. That is the counterintuitive trap the old "reads as null, never throws" line would have set, and documenting it earns its keep.

The merge is clean: #724's narrowed projection is intact, a parquet partition asked for a column it lacks still emits its rows rather than throwing, #730's NULL-guard tests pass through the shared fixture refactor, and both comment-block edits to union-source.js coexist correctly. Suite 4000 tests, 0 fail.

The decision needed

How to dispose of the false icebird text before merge.

1. Fix in place. A fifth revision stating the measured icebird contract: single-column projection reads null via the single-column scanColumn fast path (which is not aggregate-only); multi-column bare projection reads undefined; nothing throws, because icebird supplies a resolving cell. Amend the LLP 0015 wrapper sentence to say the wrapper changes the contract to null-or-undefined-never-throws. Add executeSql + collect SQL-surface tests on the icebird fixture so the backend split is pinned where it was previously untested.
2. Descope. Revert the icebird-facing additions to neutral wording ("stays addressable; exact value depends on the read path, see LLP 0015") and ship only the parquet-union correction, which is verified true. Smaller and safer, but leaves the question that spawned round 2's dispute unanswered in the docs.

Neutral is not choosing between these unilaterally. This PR has now described the mechanism wrongly five times - the original claim, and four successive corrections, each of which passed a review round before the next one caught it. Whether the icebird half is worth a fifth attempt or should be descoped is a judgement about how much more effort this document deserves, and that is yours.

Option 3, shipping as-is, is rejected on neutral's own standard: for a documentation PR a false documented claim is a production defect, and this one sits on the dataset users actually query.

How to unstick

Reply with a comment on this PR (or push to the branch). Neutral monitors this thread and will re-engage with your guidance on its next tick.

[philcunliffe]

Decision on the triage question: option 2, descope.

Revert the icebird-facing additions to neutral wording ("stays addressable; exact value depends on the read path, see LLP 0015") and ship only the parquet-union correction, which is verified true. After five wrong descriptions of the icebird mechanism, this document has earned the smaller scope; the icebird contract can be documented later in its own change, with the executeSql + collect SQL-surface tests on the icebird fixture landing alongside it so the claim is pinned where it was previously untested.

[philcunliffe]

Re-engaging with your guidance - taking option 2, descope: revert the icebird-facing additions to neutral wording ("stays addressable; exact value depends on the read path, see LLP 0015") and ship only the verified parquet-union correction, leaving the icebird contract to a later change that lands its executeSql plus collect SQL-surface tests on the icebird fixture alongside the claim.

The stuck label is removed. Neutral re-runs the rung at the current head on its next tick and the worker gets this decision as its input.

[philcunliffe]

Applied your decision: option 2, descope. Pushed as cdbda18.

Only the icebird-facing text is gone. The parquet-union correction, which independent re-derivation confirmed true, ships unchanged.

Neutralized to "stays addressable; exact value depends on the read path"

• hypaware-core/plugins-workspace/ai-gateway/src/dataset.js:166-169 - the withSchemaColumns block comment. The nine lines asserting that the partitions are icebird-backed, that icebird resolves an absent column's cell to undefined with no resolved entry, that a projection yields undefined and not null, and that the throwing half of LLP 0015 belongs only to parquet-backed partitions are replaced with: "The scan itself is unchanged: a column an old partition physically lacks stays addressable, and the exact value a read of it yields depends on the read path (LLP 0015#multi-partition-union)."
• hypaware-core/plugins-workspace/ai-gateway/src/message_projector.js:25-28 - same treatment. The claims that a projection hands back undefined, that JSON.stringify drops it rather than rendering null, and that only the scanColumnaggregate path normalizes the hole (the sentence the triage measured false, since squirreling routes any single-column scan through scanColumn) are gone. What remains names withSchemaColumns as the reason the columns stay addressable, and defers the value to the read path.
• llp/0015-query-and-datasets.spec.md:102-105 - the bolded wrapper clause "in which case the same undefined-or-throws contract applies to a column no partition has" now reads that the wrapper "keeps such a column addressable; the exact value a read of it then yields depends on the read path and is not settled here."
• llp/0055-stream-aggregates-via-scancolumn.decision.md - reverted to its accepted text, byte-identical to master. The corrected-by note this PR added made the same icebird claim ("the row path ... surfaces the same physical hole as undefined, or throws"), and with the icebird contract deferred there is nothing left for this change to say about that decision. The bullet's original parenthetical is restored rather than silently dropped.
• test/core/ai-gateway-dataset.test.js:257,280-282,289-290,296-300 - the two tests that asserted on the removed wording. The hasOwnProperty(...) === false assertions ("absent column is a hole, not a null") are back to the tolerant ?? null form under a neutral message, with a comment saying the exact value depends on the read path instead of naming one; the scanColumn test's comment is back to its master text. No coverage was deleted: both tests still prove the declared columns are advertised and that neither the row scan nor the column stream throws over a pre-v7 partition.

Kept, untouched

The parquet-union correction in full: the LLP 0015 union paragraph and its corrected-by note, the union-source.js comment block, all five new/extended tests in test/core/union-source.test.js (undefined-never-null across the bare-projection shapes, the unresolved throwing cell, the no-resolved partition, the evaluating and non-identifier-sibling throws, SELECT * keeping per-row shape), the shared test/helpers/parquet_source_fixture.js, and the inline-import(...) type fix.

Deliberately deferred, per your note

Documenting the icebird contract at all, and the executeSql plus collect SQL-surface tests on the icebird fixture. Both belong to the later change so the claim lands pinned where it is currently untested. Nothing in this branch now describes the icebird mechanism, so that change starts from a clean slate rather than a sixth revision.

Checks

npm test: 3956 pass / 4 fail / 1 skip, identical failure set to the pre-change head in this environment (the two report-render files, pushed-down comparisons do not leak NULL rows, and the hyparquet-writer dictionary-encoding test - all environmental, missing marked and a parquet/hyparquet version skew). npm run typecheck: the same single pre-existing Cannot find module 'marked' error, no delta. llp-ref-hygiene passes in-suite; no @ref was added or removed, and the #multi-partition-union anchor still resolves.

[philcunliffe]

Triage of f073338 (LLP 0017/0026): ship. The maintainer's descope decision (option 2) was applied as cdbda18 and this triage re-verified it independently at the current head rather than taking the applied-report on trust.

Descope verified at head

• No PR-added icebird prose survives.grep -i icebird over the full head-versus-master diff returns nothing. dataset.js:166-169, message_projector.js:25-28, and llp/0015:102-105 all carry the neutral wording: the column stays addressable and the exact value depends on the read path, not settled here.
• llp/0055 is byte-identical to master, verified with a direct diff of the two blobs, not just the diffstat.
• The parquet-union correction is intact and whole: the LLP 0015 union paragraph with its corrected-by note, the union-source.js comment block (three-condition formulation including the resolveable gate), all five new/extended tests in test/core/union-source.test.js (undefined-never-null across six bare-projection shapes, the unresolved throwing cell, the no-resolved partition, evaluating and non-identifier-sibling throws, SELECT * row shape), the shared test/helpers/parquet_source_fixture.js, and the inline-import(...) type fix.
• The relaxed tests lost no coverage: ai-gateway-dataset.test.js:283-291 is back to the tolerant ?? null form with a comment deferring the exact value to the read path; it still proves the declared columns are advertised and that neither the row scan nor the column stream throws over a pre-v7 partition.
• CI is green at this head: test and typecheck on Node 22 and 24, plus duplicate-numbers, all pass.
• Head f073338 is cdbda18 plus a clean merge of origin/master (which brought the LLP 0223/0226 renumber that fixed CI); the merge introduced no changes to the PR's files.

Findings ledger

• Round 1 (5 findings) and round 2 (4 findings): all fixed in 763c931 / a08857e and cross-verified by the following round. Resolved.
• Triage blocker (false icebird text in three places): removed by the descope. Resolved, not deferred; nothing in this branch now describes the icebird mechanism.
• Round 2's pre-existing observation (withSchemaColumns.scan forwards where verbatim): predates the PR, correctly described by the surviving text, not a finding.

One deferred item, now tracked

The maintainer's decision defers documenting the icebird contract to its own later change, with executeSql plus collect SQL-surface tests on the icebird fixture landing alongside the claim. Nothing tracked that direction, and after five consecutive wrong descriptions of the mechanism it should not live only in a PR thread: #778 now records the deferral, the measured behaviour to re-verify, and the exact test gap (ai-gateway-dataset.test.js pins raw scan() and scanColumn only, no SQL-surface SELECT over the icebird fixture).

Non-blocking, so the PR ships. Issue #731's two items are both delivered (option (a) doc correction with pinning tests, and the inline-import fix) and it closes on merge.