From aafd46cb2329b8a896d4f11e1be2d52f7a0e1e0c Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Sat, 8 Aug 2026 18:11:10 -0700 Subject: [PATCH 1/2] Pin the one unpinned shebang script, and gate the pin list both ways `ops/vps-backup-pull` is extensionless, so no `*.sh` or `*.py` rule reached it, and `* -text` keeps git passive by default. A CRLF checkout would hand systemd a broken interpreter line for a script it runs unattended. The reason it was missed is the more interesting half. `.gitattributes` already pinned `deploy/blog-deploy-shell` and `deploy/authorized_keys`, and this repository has never carried either file, in any commit: they live on the server, described in OPERATIONS.md "Server Hardening". A pin for a file that does not exist binds nothing, and the comment above those two claimed the extensionless shebang case was handled. That is what made the real one invisible. Both pins are dropped, with the reason recorded where they were. `deploy/bootstrap.Caddyfile` gets a pin it never had. It is read by Caddy from the container's config directory and is the only Caddy file outside the release bundle, so it is the same daemon-parsed class as `deploy/Caddyfile` beside it, which was pinned. `checks/check-eol-pins.py` reads the list back, in both directions, because a hand-maintained list that nothing verifies is how this got here: unpinned a tracked file opening with `#!` whose resolved eol is not lf. The shebang is the test rather than the mode bit, because the two move independently and it is the interpreter line a CRLF breaks. dead a pattern matching no tracked file, which reads as coverage while binding nothing. It asks `git check-attr` for the resolved attribute instead of re-implementing the match rules, so it cannot disagree with what git actually applies on checkout, and it fails rather than passing if it finds no shebang files at all. Verified by reintroducing each defect against the gate: dropping the `ops/vps-backup-pull` line reports it unpinned, and re-adding `deploy/blog-deploy-shell` reports it dead. It also caught its own file before that was staged, which is the third direction working. `git add --renormalize .` after the change stages nothing beyond these three files, so the pins bind future checkouts and rewrite no bytes now. systemd units are deliberately not pinned. Measured rather than assumed: `systemd-analyze verify` on a CRLF copy of `vps-backup-pull.service` reports the same clean result as the LF original, and the same tool does flag an unreachable `ExecStart`, so the negative means something. Found by Copilot review on #68. Co-Authored-By: Claude Opus 5 (1M context) --- .gitattributes | 24 ++++- .github/workflows/validate-task.yml | 5 + checks/check-eol-pins.py | 138 ++++++++++++++++++++++++++++ 3 files changed, 162 insertions(+), 5 deletions(-) create mode 100755 checks/check-eol-pins.py diff --git a/.gitattributes b/.gitattributes index 12b533e..190c347 100644 --- a/.gitattributes +++ b/.gitattributes @@ -15,6 +15,12 @@ # A CRLF shebang breaks execution, so scripts stay LF regardless of the default. *.sh text eol=lf +# Shebang-executable files carrying no extension, which every rule above matches by +# extension and therefore misses. `ops/vps-backup-pull` is run by systemd on the backup +# host. A named line per file, as with the Python rules below, and `check-eol-pins.py` +# fails if a tracked shebang file ever lands without one. +ops/vps-backup-pull text eol=lf + # Shebang-executable Python, where a CRLF on line one is a broken interpreter line rather # than a cosmetic difference. `capture/` is pinned whole, because everything there is a # script; under `checks/` the two executables are named, because that directory also holds @@ -23,13 +29,21 @@ capture/*.py text eol=lf checks/check-url-parity.py text eol=lf checks/check-env-docs.py text eol=lf +checks/check-eol-pins.py text eol=lf -# These formats are parsed line by line by a daemon rather than by a shell. -# Caddy and OpenSSH both reject or silently mis-parse a CRLF file. -# The deploy shell is an extensionless shebang script that matches no rule above. +# Caddy config is parsed line by line by a daemon rather than by a shell, and a CRLF file +# is rejected or silently mis-parsed. Both files are named: the bundle's `Caddyfile`, and +# the bootstrap that is installed into the container's config directory and is the only +# Caddy file outside the release. +# +# The restricted `authorized_keys` and the forced-command deploy shell were pinned here +# too, and this repository has never carried either: they live on the server, described in +# OPERATIONS.md "Server Hardening". A pin binds nothing for a file that does not exist, +# and the comment claiming to cover "the extensionless shebang script" is what let the +# real one, `ops/vps-backup-pull`, sit unpinned above. `check-eol-pins.py` now fails on a +# pattern matching no tracked file, so neither can come back silently. deploy/Caddyfile text eol=lf -deploy/blog-deploy-shell text eol=lf -deploy/authorized_keys text eol=lf +deploy/bootstrap.Caddyfile text eol=lf # Caddy map files are tabular data read by `map` directives. # They stay LF for the same reason as the Caddyfile. diff --git a/.github/workflows/validate-task.yml b/.github/workflows/validate-task.yml index b4dab3a..33bc8f2 100644 --- a/.github/workflows/validate-task.yml +++ b/.github/workflows/validate-task.yml @@ -79,6 +79,11 @@ jobs: - name: Check environment docs step run: python3 checks/check-env-docs.py + # The .gitattributes line-ending pins are a hand-maintained list that nothing read back. + # Fails on a tracked shebang file with no LF pin, and on a pin naming no tracked file. + - name: Check line-ending pins step + run: python3 checks/check-eol-pins.py + # The pin lives in the action, so validation and the deploy cannot install different generators. - name: Install Hugo step uses: ./.github/actions/install-hugo diff --git a/checks/check-eol-pins.py b/checks/check-eol-pins.py new file mode 100755 index 0000000..d204aee --- /dev/null +++ b/checks/check-eol-pins.py @@ -0,0 +1,138 @@ +#!/usr/bin/env python3 +"""Fail if a tracked shebang file has no LF pin, or a .gitattributes pattern matches nothing. + +`.gitattributes` keeps git passive with `* -text` and then names the files whose line +endings are load-bearing. That design is right and it has one weakness: the pins are a +hand-maintained list, so a new execution-sensitive file is pinned only if its author +remembers, and nothing reads the list back. Both directions have already failed here. + + unpinned `ops/vps-backup-pull` is extensionless, so no `*.sh` or `*.py` rule reached + it, and a CRLF checkout would hand systemd a broken interpreter line. + dead `deploy/blog-deploy-shell` and `deploy/authorized_keys` were pinned and have + never been tracked in this repository. That is the worse half: a pin for a + file that does not exist binds nothing while reading as coverage, and its + comment claimed the extensionless case was handled, which is why the file + that actually needed it went unnoticed. + +So this gate reads both directions, and neither is a style rule. A shebang on line one is +the test for the first, because that is exactly the property a CRLF breaks. Executability +is deliberately not the test: the mode bit and the interpreter line move independently, +and it is the interpreter line that fails. + +Text-format files a daemon parses, the Caddy configs and the map tables, are pinned for +the same reason and are not detectable by any property of their contents, so they stay a +named list and only the dead-pattern direction covers them. + +Read-only. Exit 1 on any finding. +""" +from __future__ import annotations + +import subprocess +import sys +from pathlib import Path + +REPO = Path(__file__).resolve().parent.parent +ATTRIBUTES = REPO / ".gitattributes" + +# `* -text` is the passive default the pins sit on top of, and it is expected to match +# every tracked file. Reporting it as a pattern that "matches nothing" is impossible, but +# excluding it keeps the dead-pattern check about the named pins. +BASELINE = {"*"} + + +def git(*args: str) -> str: + """Run git in the repository and return stdout, failing loudly rather than silently.""" + result = subprocess.run( + ["git", "-C", str(REPO), *args], + capture_output=True, + text=True, + check=True, + ) + return result.stdout + + +def tracked_files() -> list[str]: + return [line for line in git("ls-files", "-z").split("\0") if line] + + +def patterns() -> list[tuple[int, str]]: + """The pattern from every non-comment, non-blank line, with its line number.""" + found = [] + for number, raw in enumerate(ATTRIBUTES.read_text(encoding="utf-8").splitlines(), 1): + line = raw.strip() + if not line or line.startswith("#"): + continue + found.append((number, line.split()[0])) + return found + + +def has_shebang(path: Path) -> bool: + """True if the file opens `#!`, read as bytes so a binary file cannot raise.""" + try: + with path.open("rb") as handle: + return handle.read(2) == b"#!" + except OSError: + return False + + +def eol_attribute(paths: list[str]) -> dict[str, str]: + """The resolved `eol` attribute per path, from git rather than by re-implementing the + match rules, because a hand-rolled matcher is a second source of truth that can differ + from the one git actually applies on checkout.""" + if not paths: + return {} + payload = "\0".join(paths) + "\0" + result = subprocess.run( + ["git", "-C", str(REPO), "check-attr", "--stdin", "-z", "eol"], + input=payload, + capture_output=True, + text=True, + check=True, + ) + # -z emits a flat NUL-separated stream of path, attribute, value triples. + fields = [field for field in result.stdout.split("\0") if field != ""] + return {fields[i]: fields[i + 2] for i in range(0, len(fields) - 2, 3)} + + +def main() -> int: + findings: list[str] = [] + files = tracked_files() + + # Direction one: a tracked shebang file whose resolved eol is not lf. + shebangs = sorted(f for f in files if has_shebang(REPO / f)) + if not shebangs: + print("error: no tracked shebang files found, so this gate checked nothing") + return 1 + resolved = eol_attribute(shebangs) + for path in shebangs: + if resolved.get(path) != "lf": + findings.append( + f"unpinned: {path} opens with a shebang and resolves to " + f"eol={resolved.get(path, 'unspecified')}. Add a line to .gitattributes." + ) + + # Direction two: a pin naming a file the repository does not carry. + for number, pattern in patterns(): + if pattern in BASELINE: + continue + if not git("ls-files", "--", pattern).strip(): + findings.append( + f"dead: .gitattributes:{number} pattern {pattern!r} matches no tracked " + f"file, so it binds nothing while reading as coverage." + ) + + if findings: + for finding in findings: + print(f"error: {finding}") + print(f"\nFAIL - {len(findings)} line-ending pin finding(s)") + return 1 + + print( + f"PASS - {len(shebangs)} shebang files pinned to LF, " + f"{len(patterns()) - len(BASELINE)} patterns all matching tracked files" + ) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) From c48b9d768e973defd9df3b5c5dfdb127049b47b5 Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Sat, 8 Aug 2026 18:17:42 -0700 Subject: [PATCH 2/2] Correct the tense in two places that outlived the change making them false The workflow comment read "a hand-maintained list that nothing read back", which is both ungrammatical and, as of the step it introduces, no longer true. The script's own docstring had the same problem in the present tense: it asserted that nothing reads the list back, which its own existence disproves. Found by Copilot review on #69, which caught the workflow half. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/validate-task.yml | 5 +++-- checks/check-eol-pins.py | 3 ++- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/.github/workflows/validate-task.yml b/.github/workflows/validate-task.yml index 33bc8f2..49b2fba 100644 --- a/.github/workflows/validate-task.yml +++ b/.github/workflows/validate-task.yml @@ -79,8 +79,9 @@ jobs: - name: Check environment docs step run: python3 checks/check-env-docs.py - # The .gitattributes line-ending pins are a hand-maintained list that nothing read back. - # Fails on a tracked shebang file with no LF pin, and on a pin naming no tracked file. + # The .gitattributes line-ending pins are hand-maintained, and nothing read them back. + # This is what reads them back: it fails on a tracked shebang file with no LF pin, + # and on a pin naming no tracked file. - name: Check line-ending pins step run: python3 checks/check-eol-pins.py diff --git a/checks/check-eol-pins.py b/checks/check-eol-pins.py index d204aee..e329b52 100755 --- a/checks/check-eol-pins.py +++ b/checks/check-eol-pins.py @@ -4,7 +4,8 @@ `.gitattributes` keeps git passive with `* -text` and then names the files whose line endings are load-bearing. That design is right and it has one weakness: the pins are a hand-maintained list, so a new execution-sensitive file is pinned only if its author -remembers, and nothing reads the list back. Both directions have already failed here. +remembers, and until this gate nothing read the list back. Both directions had already +failed by the time it was written. unpinned `ops/vps-backup-pull` is extensionless, so no `*.sh` or `*.py` rule reached it, and a CRLF checkout would hand systemd a broken interpreter line.