Uh oh!
There was an error while loading. Please reload this page.
perf(spac): one indexed pass over s1_classification; default sec resolve's version - #297
Open
sroussey wants to merge 1 commit into
Open
perf(spac): one indexed pass over s1_classification; default sec resolve's version#297sroussey wants to merge 1 commit into
sec resolve's version#297sroussey wants to merge 1 commit into
Conversation
…s version
Two independent operability fixes.
**Scan.** `buildScanSql` read `s1_classification` twice per candidate entity —
a SELECT-list correlated scalar subquery and a WHERE `EXISTS` — against a table
whose only index is its `(extractor_id, accession_number)` primary key. Both
fragments collapse into one pre-aggregated `LEFT JOIN`, and the registry gains
an `(cik, sic)` index; `setupDatabase()` re-emits `CREATE INDEX IF NOT EXISTS`
every run, so existing databases pick it up with no migration. Semantics are
unchanged: no matching group yields NULL from the LEFT JOIN, which is the
documented "not asked yet" rather than false.
Moving the `BLANK_CHECK_SIC` bind out of the SELECT list surfaced a real
**pre-existing SQLite bug**: `bind()` appends in declaration order but SQLite
numbers `?` by POSITION, and `filedSic6770` was declared third while appearing
fourth (and `sinceClause` was declared before `filedSicMatch` while appearing
after). Every parameter from there on was shifted by one, so on SQLite
`filed_sic_6770` came back false for a filer whose registration really was
filed under a 6770 header — the one signal a completed de-SPAC cannot erase.
Postgres was unaffected (`$n` is numbered by bind order), so the two backends
disagreed. The new fixtures make the existing SQL-vs-repository-twin parity
assertions catch it, and a plan assertion pins the table to one materialized,
index-covered read.
**Resolve.** `--resolver-version` was a `requiredOption`, so commander rejected
`sec resolve --kind company --all --renormalize` — the re-key ceremony's step
3b, the one step the docs mark REQUIRED and silent-if-skipped. It now defaults
to the active slot ("next if a dev cycle exists, else current"), the same rule
`ResolverCoverageTask` and the role query read, and the resolved version is
echoed in the summary line. An explicitly supplied value still wins and is
still semver-validated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lgxtp7mQECdh7F2UT9CVwN
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two independent operability fixes.
(a)
spacCandidateScan: two correlated subqueries over an unindexed tablebuildScanSqlreads1_classificationtwice per candidate entity — a SELECT-list correlated scalar subquery (filed_sic_6770) and a WHEREEXISTS(filedSicMatch) — against a table whose only index is its(extractor_id, accession_number)primary key. Neither could use it.Both fixes land, because the index alone does not remove the per-row work:
indexes: [["cik", "sic"]]on theS1_CLASSIFICATION_REPOSITORY_TOKENentry instorageRegistry.ts. It reaches existing databases with no migration:setupDatabase()callscreateTableAndIndexes()on every run, emittingCREATE INDEX IF NOT EXISTSon both backends, and the PK-prefix redundancy check does not suppress it (cik≠extractor_id).LEFT JOIN:with
c.filed_sic_6770 = 1in the WHERE. Semantics preserved exactly: no matching group yields NULL from the LEFT JOIN, which is the documented "not asked yet" — different from false.The parameter-order trap is not hypothetical; main already trips it.
bind()appends in declaration order but SQLite numbers?by position in the statement, and on main:filedSic6770is declared third in the SELECT-list block but appears fourth in the emitted SELECT list (afterrenamedFrom); andsinceClauseis declared beforefiledSicMatchbut appears after it.So every parameter from that point on is shifted by one. Concretely, on SQLite
filed_sic_6770came back false for a filer whose registration really was filed under a 6770 header — the one signal a completed de-SPAC cannot erase, and the whole reason that column exists. Postgres was unaffected ($nis numbered by bind order), so the two backends disagreed silently.This PR fixes it as a side effect:
BLANK_CHECK_SICnow binds in the FROM clause andfiledSicMatchbinds nothing, so declaration order and statement order line up (SELECT-list subqueries → derived-table sic → WHERE predicates). ThebuildScanSqlJSDoc now spells the rule out and notes that a slip yields a silently wrong scan rather than an error, on one backend only.Tests
The existing SQL-vs-
scanRepositoryparity assertions are the correctness guard for the rewrite. The seed grows three CIKs that reach thes1_classificationfragments:Joby Aviation, Inc., SIC 3721) whose only signal is a 6770s1_classificationrow;filed_sic_6770: null, never false;CIKs 8 and 10 are what failed against main, with the repository twin reporting
trueand the SQL reportingfalse— the binding shift, caught by exactly the assertion that exists to catch it.A new plan assertion pins the access shape:
It asserts the count rather than the absence of the literal string
SCAN s1_classification, because the correct plan legitimately readsSCAN s1_classification USING COVERING INDEX s1_classification_cik_sicunderMATERIALIZE c— one pass for the whole scan. The two correlated fragments show up as two rows, so this fails both against current code and against a version that adds the index but keeps theEXISTScorrelated (which reads as twoSEARCH ... USING COVERING INDEX (cik=?)rows).buildScanSqlis exported for the assertion.(b)
sec resolve --kind company --all --renormalizecould not run--resolver-versionwas a.requiredOption(resolve.ts:24), so commander rejected the documented re-key ceremony's step 3b — the one stepCLAUDE.mdmarks REQUIRED and silent-if-skipped.The docs are right about intent, and asking an operator to look up a semver mid-ceremony is exactly the failure mode the doc warns about, so the flag is defaulted rather than the docs changed:
.requiredOption→.option("--resolver-version <semver>", "target resolver semver (default: the active slot)"), typedresolverVersion?: string.--kind/ batch-resolvable checks viagetActiveSlot(new VersionRegistry(globalServiceRegistry.get(COMPONENT_VERSION_REPOSITORY_TOKEN)), "resolver", kind)— "next if a dev cycle exists, else current", the same ruleResolverCoverageTaskandRoleQueryalready read. Absent, it throws inComputeFormsWorklistTask's wording:No active slot for resolver '<kind>'. Run 'sec db setup' to bootstrap.isValidSemvervalidation is kept for an explicitly supplied value; a registry-sourced one was already validated when written.Tests
resolve.test.tsgains a case running the documented command verbatim with no--resolver-versionthroughrunCliProcess, asserting exit 0 and that stdout names the bootstrapped slot (resolved 0 company observation(s) at v1.0.0). Confirmed failing first against main with commander's required-option error and exit 1. A sibling case asserts an explicit--resolver-version 2.3.4still wins; the three existing malformed-semver cases are unchanged and still error.Verification
Note:
version.test.tsspawns CLI subprocesses with 15 s per-case timeouts and times out when run concurrently with the other CLI suites on this machine. It passes when run alone, before and after this change — a test-harness load artifact, not a regression.Docs
CLAUDE.md's re-key ceremony notes that step 3b takes no--resolver-versionand defaults to the active slot.Generated by Claude Code