diff --git a/TODO.md b/TODO.md index 5eef2199..73f30764 100644 --- a/TODO.md +++ b/TODO.md @@ -22,38 +22,6 @@ The steps below are followed in order rather than sampled. ## Work Clusters -### The Prose Gate Scope Floor - -One pull request making [`prose_lint.py`][prose-lint] assert a floor on its own scope, which every verdict below it depends on, since a gate that finds nothing is indistinguishable from a gate with nothing to find. - -**State** `ready`. **Touches** [`scripts/prose_lint.py`][prose-lint] and its test file. **Cost** one hub edit, hub-only, no sweep. - -- **Assert a floor on what a `--diff` run actually scanned.** A run that resolves a non-empty diff and then matches zero files has almost certainly failed to scope rather than found a clean change, so it says so instead of exiting 0. - - **Blocked by** - Nothing. - - **Issue** - None filed. - - **Checked** - `develop` at `1ed0cc8` on 2026-08-03, where four separate guards each close one route to the same false clean. - - **Open** - Nothing. - - **Settled** - Four routes to the same false clean are on record from one session, an unresolvable base widening to a whole-tree scan, a multi-line `paths` input read only to its first newline, a diff taken in one repository while scanning another, and a path under no repository at all. - - **Settled** - Per-route guards are the wrong shape, because the fifth route needs a fifth guard and gets found by a reviewer rather than by the gate, which is what a floor assertion covers. - - **Settled** - The honest limit is that a change touching only files the gate does not read, an image or a lock file, legitimately scopes to zero, so the assertion compares against the diff's own file list rather than against zero alone. - - **Settled** - `LEAST_PLAUSIBLE` at 60 is the existing floor on a whole-tree sweep, so the shape is already in the file and the `--diff` path is what lacks it. - -### The Representative-Data Path Check - -One pull request gating the pattern-detectable half of the representative-data rule, which is worth having only once the gate can prove it read something. - -**State** `blocked` on "The Prose Gate Scope Floor". **Touches** [`scripts/prose_lint.py`][prose-lint] and its test file. **Cost** one hub edit, hub-only, no sweep. - -- **Flag an absolute home path or a bare drive letter in committed prose, a comment, or a fixture.** [`GOVERNANCE.md`][governance] "Representative Data in Agent-Authored Text" states the rule and says why a check is a floor rather than an answer. - - **Blocked by** - The Prose Gate Scope Floor. - - **Issue** - None filed. - - **Checked** - `develop` at `1ed0cc8` on 2026-08-03, where the rule is stated and no check reads for it. - - **Open** - Whether a home path in an operational repo's runbook is a finding, since it may be the literal path an operator types, which is the repo's own content rather than an agent quoting the maintainer's environment, so the answer is a scope by file, by repo type, or left to the author. - - **Settled** - The shapes are `/home/`, `/Users/`, `C:\Users\`, and a bare drive letter. - - **Settled** - The check is introduced as covering the easy half or it gets read as closing the rule, which is the specific way it would make things worse. - - **Settled** - The exemption carries the whole burden, since the rule's own wording, the [`host-setup/`][agent-safety] docs, and the audit's examples all quote path shapes in order to describe them, and a wrong exemption hands out a work list that damages correct documents. - - **Settled** - The leak that motivated the rule was in a pull request comment, which no committed-file linter reads, so the gate says what surface it covers rather than letting its name imply the rule. - ### The Prose Content Backlog One pull request clearing prose findings, leading with [`catalog/snippets/`][snippets] because a non-conformant snippet seeds its violations into every repo that adopts it and the downstream repo is then flagged for content it was handed. @@ -535,7 +503,6 @@ Each was checked against the tree and has nothing left to do anywhere. Closing i [pr-review]: ./scripts/pr_review.py [project-types]: ./spec/project-types.json [prose-gate]: ./.github/actions/prose-gate/action.yml -[prose-lint]: ./scripts/prose_lint.py [readme]: ./README.md [readme-structure]: ./spec/readme-structure.md [repo-gate]: ./scripts/repo_gate.py diff --git a/scripts/prose_lint.py b/scripts/prose_lint.py index 926338e2..5a04c6d3 100644 --- a/scripts/prose_lint.py +++ b/scripts/prose_lint.py @@ -12,6 +12,7 @@ dupword No duplicated consecutive word. sentence-split A sentence must not wrap across lines (one sentence per line). spelling No British spelling, the repo-wide convention being US English. + home-path No absolute home path naming a real account, per the representative-data rule. Exit 1 if any violation is found. Read-only, never edits. """ @@ -31,9 +32,10 @@ 'dupword': 'a duplicated consecutive word', 'sentence-split': 'a sentence wrapping across lines', 'spelling': 'a British spelling where the repo convention is US English', + 'home-path': 'an absolute home path naming a real account', } DEFAULT_RULES = frozenset({'charset', 'charset-unknown', 'semicolon', 'dash', 'dupword', - 'spelling', 'comment-wrap', 'comment-case'}) + 'spelling', 'comment-wrap', 'comment-case', 'home-path'}) # Trees this repo generates rather than authors, skipped when a wider scan expands into them. # The gate then measures hand-written prose. @@ -53,6 +55,24 @@ # A sweep that quietly stops finding files satisfies every rule by having nothing to read. LEAST_PLAUSIBLE = 60 +# The pattern-detectable half of the representative-data rule, and only that half. +# A real user segment is required, so a documented placeholder describes the shape unmatched. +# That is how the rule's own wording escapes its own gate, with no exemption naming files. +# A bare drive letter is deliberately not a shape here. +# Measured against this repo it matched 11 files and named a path in none of them. +# An escaped newline after a word ending in a letter and a colon reads as a drive letter. +# `Users` is matched case-insensitively on the Windows branch alone, since that filesystem is. +# The POSIX branches stay case-sensitive, since a lowercase `/users/` is a common REST path. +# An API route is not a home directory, and widening this would flag one in every doc. +HOME_PATH = re.compile( + r'(?:/home/|/Users/|[A-Za-z]:\\(?i:users)\\)(?P[A-Za-z][A-Za-z0-9._-]*)') + +# Accounts that belong to a container or a runner rather than to a person. +# Every one is a fixed name an image ships, so a path under it names no environment. +# `vscode` is the devcontainer user this repo's own snippets mount into. +# `runner` is the GitHub Actions user, and the rest are stock image accounts. +SERVICE_ACCOUNTS = frozenset({'vscode', 'runner', 'root', 'ubuntu', 'node', 'shared', 'public'}) + def rel(path: Path) -> str: """The repo-relative posix key a git diff uses for this path. @@ -128,6 +148,35 @@ def unread_diff_files(scope: dict[str, set[int]], paths: list[str], return out +def home_path_findings(lineno: int, line: str) -> list[tuple[int, str, str]]: + """Absolute home paths on this line that name a real account. + + The exposure this gates was a maintainer's own path reaching a public comment, so the unit + is the raw line rather than stripped prose. A path is the same exposure in a JSON config + value, in a fenced transcript pasted from a terminal, and in a sentence. + """ + out = [] + for m in HOME_PATH.finditer(line): + if m.group('user').lower() in SERVICE_ACCOUNTS: + continue + out.append((lineno, 'home-path', + f'absolute home path {m.group(0)!r} -> use a constructed path, not an ' + 'observed one')) + return out + + +def operational_checkout(root: Path) -> bool: + """Whether this checkout is an operational repository, read from what it carries. + + `spec/files.json` declares `repo-config/operational/develop.json` for the operational model + and `repo-config/develop.json` for the release one, so a repository states its own model and + nothing has to reach the hub registry to ask. The hub itself carries both payloads, being the + template for each, so carrying the release payload decides it. + """ + return ((root / 'repo-config' / 'operational' / 'develop.json').is_file() + and not (root / 'repo-config' / 'develop.json').is_file()) + + def repo_prefix(root: Path) -> str: """Where `root` sits inside its repository, as a posix prefix, or '' when git cannot say. @@ -934,6 +983,10 @@ def check_file(path: Path, rules: set[str]) -> list[tuple[int, str, str]]: prev_no = 0 for i, line in enumerate(lines, 1): line = line.rstrip('\r') + # Judged before the fence and inline-code handling below, deliberately. + # A path pasted inside a fenced transcript is the same exposure as one in a sentence. + if 'home-path' in rules: + out.extend(home_path_findings(i, line)) if CODE_FENCE.match(line): in_fence = not in_fence prev_txt = '' @@ -1022,6 +1075,15 @@ def main(argv: list[str] | None = None) -> int: rules = set(a.checks or DEFAULT_RULES) + # An operational repository's runbook carries the literal path an operator types. + # That is the repository's own content, not an agent quoting an environment it observed. + # The skip is announced, since a rule that silently stops running reads as one that passed. + # That is the same failure the diff-scope floor below exists to prevent. + if 'home-path' in rules and operational_checkout(Path(repo_root(Path('.')) or '.')): + rules.discard('home-path') + print('note: home-path is not checked in an operational repository, where an absolute ' + 'path is the operator instruction rather than observed data.', file=sys.stderr) + # Checked before discovery, which reads every tracked file to classify it as text. # A run this rejects would otherwise pay that cost and throw the result away. # `--list-files` is exempt, since it reports the scan scope and never consults the diff. diff --git a/scripts/test_prose_lint.py b/scripts/test_prose_lint.py index cc1d7a1f..ffbe2f32 100644 --- a/scripts/test_prose_lint.py +++ b/scripts/test_prose_lint.py @@ -1540,6 +1540,190 @@ def test_an_unreadable_file_is_skipped_rather_than_raising(self) -> None: self.assertEqual([], prose_lint.check_file(self.tmp / 'absent.md', {'dupword'})) +# The gate reads this file, so a bait path is assembled rather than written. +# A literal would be a finding in the very file that defines the rule. +# This file already takes that approach for non-ASCII, which it writes as escapes. +# `adalovelace` is a constructed account name, never an account on any machine here. +BAIT_USER = 'adalovelace' +NIX_HOME = f'/home/{BAIT_USER}' +MAC_HOME = f'/Users/{BAIT_USER}' +WIN_HOME = f'C:\\Users\\{BAIT_USER}' + + +class TestHomePath(BaitCase): + """The pattern-detectable half of the representative-data rule, and only that half. + + Every fixture below uses a constructed account name. Writing the maintainer's own path into + a committed test would be the exact exposure the rule exists to prevent, in the file that + implements the rule. + + The shapes were chosen against the corpus rather than from the list the backlog recorded. A + bare drive letter matched 11 files here and named a path in none of them, because an escaped + newline after any word ending in a letter and a colon reads as one, so `jobs:\\n` inside a + fixture is a drive letter. The shape kept is a drive letter followed by `Users`. + """ + + def test_a_home_path_naming_a_person_is_flagged(self) -> None: + for text in (f'See {NIX_HOME}/notes.txt for the log.\n', + f'See {MAC_HOME}/notes.txt for the log.\n', + f'See {WIN_HOME}\\notes.txt for the log.\n'): + with self.subTest(text=text.strip()): + self.assertIn('home-path', self.kinds(text, {'home-path'})) + + def test_the_documentation_placeholder_is_not_a_finding(self) -> None: + """The rule's own wording quotes these shapes to describe them, and must survive its gate. + + A real user segment is required, so the placeholder form does not match and no exemption + list has to carry the files that describe the rule. A stale exemption list is what turns + a gate into a work list that damages correct documents. + """ + for text in ('The shapes are `/home/` and `C:\\Users\\`.\n', + 'A path under `/Users/` is the macOS form.\n'): + with self.subTest(text=text.strip()): + self.assertNotIn('home-path', self.kinds(text, {'home-path'})) + + def test_a_container_account_is_not_a_personal_home(self) -> None: + """`/home/vscode` is a fixed name a devcontainer image ships, so it names no environment. + + Every one of this repository's 15 home-path hits is this shape, from the devcontainer + snippets and the doc describing them, so a rule without this exemption would open with a + work list of 15 findings and no true positive among them. + """ + for account in ('vscode', 'runner', 'root', 'ubuntu', 'node'): + with self.subTest(account=account): + self.assertNotIn('home-path', + self.kinds(f'Mounted at /home/{account}/.ssh here.\n', + {'home-path'})) + + def test_a_path_inside_a_fenced_block_is_still_a_finding(self) -> None: + """A transcript pasted from a terminal is the exposure, and it arrives fenced. + + Every other prose rule skips a fenced block because it holds code rather than prose. This + one reads it, since the rule gates a literal path rather than a sentence. + """ + self.assertIn('home-path', + self.kinds(f'```text\n$ ls {NIX_HOME}/keys\n```\n', + {'home-path'})) + + def test_a_path_in_a_config_value_is_still_a_finding(self) -> None: + """A bind mount naming a real home is the exposure in its most consequential form.""" + self.assertIn('home-path', + self.kinds(f'{{ "target": "{NIX_HOME}/.ssh/id_ed25519.pub" }}\n', + {'home-path'}, 'a.json')) + + def test_a_relative_or_tilde_path_is_not_a_finding(self) -> None: + """`~/.ssh` names no account, which is the form the docs are supposed to use.""" + for text in ('Copy `~/.ssh/id_ed25519.pub` into place.\n', + f'The path `home/{BAIT_USER}` is relative.\n'): + with self.subTest(text=text.strip()): + self.assertNotIn('home-path', self.kinds(text, {'home-path'})) + + def test_the_windows_branch_is_case_insensitive(self) -> None: + """Windows filesystems are case-insensitive, so a pasted path may be any casing.""" + for form in (f'C:\\users\\{BAIT_USER}', f'c:\\USERS\\{BAIT_USER}'): + with self.subTest(form=form): + self.assertIn('home-path', self.kinds(f'See {form} here.\n', {'home-path'})) + + def test_the_posix_branch_stays_case_sensitive(self) -> None: + """A lowercase `/users/` is the commonest REST path there is, and names no home. + + This is why the case-insensitive fix is scoped to the drive-letter branch rather than + applied to the whole pattern: widening it would flag every API route in every doc. + """ + for text in (f'GET https://api.example.com/users/{BAIT_USER} returns the record.\n', + f'The route is `/users/{BAIT_USER}` in the API.\n'): + with self.subTest(text=text.strip()): + self.assertNotIn('home-path', self.kinds(text, {'home-path'})) + + def test_a_bare_drive_letter_is_not_a_shape(self) -> None: + """Kept as a case because the backlog recorded it as one and the corpus rejected it. + + These are the two forms that made it unworkable: an escaped newline in a fixture, and a + temporary directory in a string literal. + """ + for text in ("key: 'jobs:\\n a:'\n", 'var p = "C:\\tmp\\out";\n'): + with self.subTest(text=text.strip()): + self.assertNotIn('home-path', self.kinds(text, {'home-path'})) + + def test_the_rule_runs_by_default(self) -> None: + """A rule outside DEFAULT_RULES reads as enforced while nothing runs it.""" + self.assertIn('home-path', prose_lint.DEFAULT_RULES) + self.assertIn('home-path', prose_lint.RULES) + + def test_neither_source_file_carries_a_literal_home_path(self) -> None: + """Both files are read by the rule they implement, so the bait is assembled. + + This is the same guarantee the tier tables carry for non-ASCII, and it is the reason the + constants above exist. Written as literals, the gate would report its own definition and + its own cases, and the only repair would be an exemption naming these files, which is the + stale-exemption shape that hands out a work list damaging correct documents. + """ + for name in ('prose_lint.py', 'test_prose_lint.py'): + with self.subTest(source=name): + found = prose_lint.check_file(Path(__file__).with_name(name), {'home-path'}) + self.assertEqual([], found) + + +class TestOperationalExemption(unittest.TestCase): + """An operational repository's runbook carries the path an operator types. + + That is the repository's own content rather than an agent quoting an environment it observed, + which is the distinction the rule is about. + """ + + def setUp(self) -> None: + self.tmp = Path(self.enterContext(tempfile.TemporaryDirectory())) + self.enterContext(contextlib.redirect_stdout(io.StringIO())) + self.err = self.enterContext(contextlib.redirect_stderr(io.StringIO())) + + def _payload(self, *parts: str) -> None: + target = self.tmp.joinpath(*parts) + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text('{}\n', encoding='utf-8') + + def test_an_operational_checkout_is_read_from_what_it_carries(self) -> None: + """`spec/files.json` declares the payload per model, so the repository states its own.""" + self._payload('repo-config', 'operational', 'develop.json') + self.assertTrue(prose_lint.operational_checkout(self.tmp)) + + def test_a_release_checkout_is_not_operational(self) -> None: + self._payload('repo-config', 'develop.json') + self.assertFalse(prose_lint.operational_checkout(self.tmp)) + + def test_the_hub_carrying_both_payloads_is_not_operational(self) -> None: + """The hub is the template for each model, so carrying the release payload decides it. + + Read as operational, the hub would exempt itself from a rule it authors, which is the + one repository where that matters most. + """ + self._payload('repo-config', 'develop.json') + self._payload('repo-config', 'operational', 'develop.json') + self.assertFalse(prose_lint.operational_checkout(self.tmp)) + + def test_a_checkout_carrying_neither_payload_is_not_operational(self) -> None: + """An unknown model is gated rather than exempted, since exempting on doubt is the risk.""" + self.assertFalse(prose_lint.operational_checkout(self.tmp)) + + def test_the_skip_is_announced_rather_than_silent(self) -> None: + """A rule that stops running without saying so reads as a rule that passed.""" + self._payload('repo-config', 'operational', 'develop.json') + bait = self.tmp / 'runbook.md' + bait.write_text(f'Deploy into {NIX_HOME}/stack here.\n', encoding='utf-8') + with mock.patch.object(prose_lint, 'repo_root', return_value=str(self.tmp)), \ + mock.patch.object(prose_lint, 'discover', return_value=[bait]): + self.assertEqual(0, prose_lint.main(['--check', 'home-path'])) + self.assertIn('operational repository', self.err.getvalue()) + + def test_a_release_repository_still_reports_the_finding(self) -> None: + """The exemption must not be the whole rule.""" + self._payload('repo-config', 'develop.json') + bait = self.tmp / 'runbook.md' + bait.write_text(f'Deploy into {NIX_HOME}/stack here.\n', encoding='utf-8') + with mock.patch.object(prose_lint, 'repo_root', return_value=str(self.tmp)), \ + mock.patch.object(prose_lint, 'discover', return_value=[bait]): + self.assertEqual(1, prose_lint.main(['--check', 'home-path'])) + + class TestDiffScopeFloor(unittest.TestCase): """The assertion every `--diff` verdict rests on, that the run matched something it was given.