Put the Floors in the Contract Table, and Tie the Table to the Data - #646
Conversation
The table answers presence, and a tool below its floor answers --version like any other, so every column reported a host as fine while host_gate.py failed it. A host on 3.12 read the Python 3 row and stopped. That was equally true of gh at 2.46 before this change, so the fix is a Floor column covering both rather than a note on the one row the finding named. The kind is named beside the number, since a measured floor gives a failing host a defect to point at and a target floor does not. git-restore-mtime is deliberately absent from the table, carrying a floor while being optional, and the prose below already says why. Restating a number in prose creates the drift it was meant to prevent, so a test now reads the shipped docs/host-setup.md and asserts every required tool's declared floor appears in a table row. It was written first against the old table to confirm it fails, and the failure it produced was a real one: git-restore-mtime has a floor and no row, which is what scoped the assertion to required tools. Rewriting the table to 3.12 fails it with "python3 declares 3.13 and no table row states it". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR updates the host setup contract documentation to make minimum-version floors explicit in the contract table, and adds a test that ties those documented floors back to the authoritative spec/host-tools.json data to prevent drift.
Changes:
- Add a
Floorcolumn todocs/host-setup.md(including whether each floor is “measured” vs “target”) to distinguish tool presence from version sufficiency. - Add a unit test in
scripts/test_host_gate.pythat asserts every required tool with a declared floor has a corresponding contract-table row that states that floor.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| scripts/test_host_gate.py | Adds a regression test ensuring the host contract table restates required tool floors from spec/host-tools.json. |
| docs/host-setup.md | Expands the host contract table with a Floor column and explains why the column exists and how to interpret floor kinds. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Uh oh!
There was an error while loading. Please reload this page.
The row lookup searched the entire line, so `python3` matched the probe command in
the Present when column rather than the Tool label. The test passed for a reason it
did not intend, and would have broken had that column ever been reworded. The
rstrip('3') fallback was dead either way, since it produces `python`, which never
matches `Python 3`.
The row is now keyed on its Tool cell alone, with both sides normalized by
lowercasing and dropping anything not alphanumeric, so the prose label `Python 3`
keys as the identifier `python3` and the match is the one the test means.
Checked against three rewrites of the table rather than one. A drifted number and a
removed floor both fail it, and rewording Present when to "the interpreter answers
its version flag" leaves it passing, which is the brittleness the finding named.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.
Suppressed comments (2)
scripts/test_host_gate.py:439
- This failure message is misleading:
assertIn('python3', rows, ...)can fail because the table exists but the python3 row is missing. In that case the error is not 'no contract table found'. Adjust the message to reflect the actual condition being checked.
self.assertIn('python3', rows, 'no contract table found in docs/host-setup.md')
scripts/test_host_gate.py:449
- The assertion for the minimum version can still pass for the wrong reason because it searches all cells in the row. If the minimum appears in
Present when(e.g.,python3.13 --version) or another column, the test would pass even if the Floor column drifted or was removed, reintroducing the brittleness this test is meant to prevent. Assert the version specifically in the Floor cell (last column).
self.assertTrue(
any(t['minimum'] in c for c in rows[key]),
f'{t["name"]} declares {t["minimum"]} and no table row states it')
Two findings, and the second is the first one over again in the next column. The version assertion searched every cell in the row, so a table stating a version in Present when would satisfy it with the Floor column drifted or deleted, which is the failure this test exists to catch. My own three-case check missed it because Present when does not happen to contain 3.13 today, which is the same accidental pass in a different place. The Floor column is now located by its header rather than by index or by search, and the assertion reads that one cell. Locating it by header rather than as the last column also survives a column being appended. The other finding is a message that named the wrong condition: assertIn on the python3 row reported "no contract table found", which is untrue when the table is present and the row is not. Finding the table and finding a row are now separate assertions with separate messages, and the version failure prints the Floor cell it actually read. Checked against five table rewrites rather than three, including the case the finding named: a version moved into Present when with Floor blanked now fails, as does removing the Floor column outright, while rewording Present when still passes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ptr727
commented
Aug 9, 2026
Answering both suppressed findings from the round on The version assertion searched the whole row
Accepted, and it is the previous finding over again one column across. The row lookup was fixed to key on the Tool cell, and the version assertion was left searching every cell, so the same accidental match was still available one line down. Worth stating plainly: my own three-case check did not catch this, and passing it is what made me confident. It only passed because The Floor column is now located by its header and the assertion reads that one cell. By header rather than as the last column, so appending a column later does not silently move what is asserted. Re-checked against five rewrites, including the one you name:
The failure message named the wrong condition
Accepted. That assertion was doing two jobs, proving a table was parsed and proving one row existed, and reported only the first. A reader who deleted the They are now separate: |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.
Suppressed comments (1)
scripts/test_host_gate.py:445
- The contract-table parser keeps scanning every Markdown table in docs/host-setup.md after it finds the contract table header. Today that happens to be harmless, but it makes the test fragile (a later table could accidentally overwrite a tool key, or a future contract-table move could change which table is parsed). Consider stopping after the first contract table ends (first non-table line after the header) and only recording rows while inside that table.
rows, floor_col = {}, None
for ln in doc.splitlines():
if not ln.startswith('| '):
continue
cells = [c.strip() for c in ln.strip('|').split('|')]
The parser scanned every table in the document, so a later table's first column was a key like any other and would overwrite a tool row. The document carries a second table today, "Now possible", whose keys collide with nothing, and a test that depends on that is one the next table silently breaks. Reading now stops at the first non-table line after the contract table. The previous version, run against a document with a later table carrying a Python 3 row, fails with "2 not greater than 3 : the python3 row has no Floor cell", which is a real break rather than a hypothetical one. This version passes it, while the drift cases it must catch still fail. That is the third finding in a row of one shape on this test, each a lookup wider than the thing it meant to name: the row keyed on the whole line, the version searched across every cell, and now the table read to the end of the file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ptr727
commented
Aug 9, 2026
Answering the suppressed finding from the round on
Accepted. Measured the "today that happens to be harmless" part rather than taking it on trust, because it decides whether this is a latent break or a live one:
The second table's first-column keys normalize to Reading now stops at the first non-table line after the contract table. Confirmed the fix is load-bearing rather than defensive, by appending a later table carrying a previous test (f04ef1f) -> FAILS AssertionError: 2 not greater than 3 : the python3 row has no Floor cellthis commit -> passesSo the overwrite was real and the message it produced pointed at the contract table, which is the wrong place entirely. The drift cases still fail correctly: a floor drifting to 3.12 and a version hidden in On the shape of these three findingsThis is the third finding in a row on this one test, and all three are the same mistake: a lookup wider than the thing it meant to name. The row keyed on the whole line, then the version searched across every cell, then the table read to the end of the file. Each time I narrowed the one you named and left the next one open, and each time my own verification passed because the collision it needed did not exist yet in this document. That is worth recording as more than an apology, because it is the failure mode the test itself was written to prevent: a check that passes on today's data while asserting nothing about the property it claims. The cases I construct to verify a check are drawn from the same understanding that wrote the check, so they confirm it rather than test it. The three rewrites I ran after the first fix are a good illustration, since they passed while two independent holes remained. |
Uh oh!
There was an error while loading. Please reload this page.
Answers one of the two suppressed findings on the promotion pull request #643, which cannot take a fix of its own since its head is
develop.Accepted, and widened to the class. The table answers presence, and a tool below its floor answers
--versionexactly like one above it, so every column reported a host as fine whilehost_gate.pyfailed it. That was already true ofghbefore this session: a host on2.46reads thegh --versionrow, concludes it qualifies, and then fails the floor that has been declared since it was added. Patching only the row the finding named would have left the same trap one row up, so the fix is aFloorcolumn covering both.The kind is named beside the number, since the distinction decides what a failing host can do next. A measured floor gives them a defect to point at, and a target floor does not.
git-restore-mtimecarries a floor and is deliberately absent from the table, because it is optional and no procedure here needs it, which the prose below the table already explains.The number is now restated in two places, so a test holds them together
Copying a value out of
spec/host-tools.jsoninto prose creates exactly the drift this column exists to prevent: the data moves, the table does not, and a host reads the stale number and stops. A test now reads the shippeddocs/host-setup.mdand asserts every required tool's declared floor appears in a table row.It was written before the table was correct, to confirm it fails rather than assuming it would, and the first failure it produced was a real finding rather than a bug in the test:
git-restore-mtime declares a floor and has no row in the contract table. That is what scoped the assertion to required tools, and the reason is recorded in the test rather than left to the next reader.Confirmed it catches the case it exists for, by rewriting the table to
3.12and running it:AssertionError: False is not true : python3 declares 3.13 and no table row states itVerification
552 tests pass,
spec/validate.pyandscripts/host_gate.pyexit 0,prose_lint.pywith the CI check list exits 0,repo_gate.pyreports all three checks clean,editorconfig-checkerexits 0, andmarkdownlint-cli2reports 0 issues across 44 files. The hook caught a wrapped comment sentence in the new test before it landed.🤖 Generated with Claude Code