Skip to content

Define walk in the filter rather than calling it, for jq 1.5 - #553

Merged
ptr727 merged 2 commits into
developfrom
configure-jq15-portable
Aug 3, 2026
Merged

Define walk in the filter rather than calling it, for jq 1.5#553
ptr727 merged 2 commits into
developfrom
configure-jq15-portable

Conversation

@ptr727

Copy link
Copy Markdown
Owner

Closes#549, raised by the Blog agent from a Copilot review on a downstream re-vendor and declined there because configure.sh is carried verbatim.

The defect

The payload-driven ruleset comparison added in #540 built its normalizer on walk/1, which arrived in jq 1.6. On a host carrying jq 1.5 the filter does not degrade — it fails to compile, so check_ruleset reports drift on every parameterized rule it never actually compared.

That is the inverse of the false clean #540 was written to close, and arguably worse: a false failure teaches an operator to distrust the tool, where a false pass merely fails to warn them.

Verified on a real jq 1.5, not argued

jq version: jq-1.5-1-a5b5cbe--- OLD filter (calls walk):def n: walk(if type=="array" then sort else . end); njq: 1 compile error--- NEW filter (defines its own):{"allowed_merge_methods":["merge","squash"],"nested":{"deep":[1,2,3]},"required_status_checks":[{"context":"a"},{"context":"b"}]}

The reported failure reproduces exactly, and the fix compiles and returns the sorted document. Output is byte-identical to jq 1.7 on the same input, including a deliberately nested array to confirm the local definition still recurses.

Finding a genuine jq 1.5 took three attempts — imega/jq:1.5 does not exist, Debian buster's archives are gone, and Alpine 3.8 ships a master build that already has walk. Ubuntu 18.04 has it. Worth recording, because "I could not reproduce it" would have been the wrong conclusion from the first two.

The choice

walk is defined inside the filter, so the script calls nothing jq 1.5 lacks and keeps one code path across versions.

The alternative in the issue — assert a jq version up front and fail with a message — was considered and declined for the reason the issue itself gives: check is the read-only mode, and refusing to run at all is a worse outcome than running. The issue stated that as a preference rather than a decision, and I agree with it.

Verification

bash -n clean, diff-scoped prose_lint clean, and a live check against this repo still passes on all three parameterized rules of both rulesets.

Downstream

configure.sh is carried verbatim with appliesTo: "*", so this joins the re-vendor the ledger already lists. It is the same file that generated a re-vendor three times today, which is precisely the evidence behind the vendored-tooling entry added in #546.

ptr727and others added 2 commits August 3, 2026 16:03
The payload-driven ruleset comparison added in #540 built its normalizer on
walk/1, which arrived in jq 1.6. On a host carrying jq 1.5 the filter does not
degrade, it fails to compile, so check_ruleset reports drift on every
parameterized rule it never compared. That is the inverse of the false clean
the comparison was written to close, and it is worse, since a false failure
teaches an operator to distrust the tool.
walk is now defined inside the filter, so the script calls nothing that jq 1.5
lacks and keeps one code path across versions. The alternative considered and
declined was asserting a jq version up front: check is the read-only mode, and
refusing to run at all is a worse outcome than running.
Reproduced and verified on a real jq 1.5 rather than argued. On jq-1.5-1-a5b5cbe
the old filter gives "jq: 1 compile error" and the new one returns the sorted
document. Output is byte-identical to jq 1.7 on the same input, and a live check
against this repo still passes on all three parameterized rules of both rulesets.
Raised by the Blog agent in #549, found by a Copilot review on a downstream
re-vendor of this canonical, and declined there because the file is carried
verbatim.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both started on a lowercase identifier, which the comment rules read as a
sentence that failed to start. Same content, named as a builtin and a function.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CopilotAI review requested due to automatic review settings August 3, 2026 23:03

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request updates repo-config/configure.sh so its check_ruleset normalization filter no longer depends on jq's walk/1 builtin (introduced in jq 1.6). By defining an equivalent walk-style helper inside the jq program, the ruleset-parameter comparison can run on hosts with jq 1.5 without failing to compile.

Changes:

  • Replace the jq walk/1 usage in the norm filter with an in-filter recursive helper (w/1) so jq 1.5 can compile the program.
  • Add explanatory comments documenting the jq 1.5 failure mode and why the longer portable filter is justified.

@ptr727
ptr727 merged commit fe01060 into developAug 3, 2026
7 checks passed
@ptr727
ptr727 deleted the configure-jq15-portable branch August 3, 2026 23:07
ptr727 added a commit that referenced this pull request Aug 3, 2026
…555)
Three commits, and the first two are repairs to defects this hub shipped
earlier today. Merged as a **merge commit**, never squashed.
| Commit | PR | What |
| --- | --- | --- |
| `71b7691` | #547 | Catalogs Blog and records its first audit |
| `82e3658` | #551 | Stops the audit asserting a bypass list the config
no longer manages |
| `fe01060` | #553 | Defines `walk` in the filter rather than calling
it, for jq 1.5 |
## Why this one should not wait
**`main` is currently wrong about every repository in the fleet.** The
previous promotion took `bypass_actors` out of the ruleset payloads, but
`spec/audit.py` compares live-against-payload over a subset that still
included that field. A payload deliberately declaring no bypass, against
a live ruleset that has one, reads as a normalized diff, so every repo
audits as two ruleset DEFECTs:
```text
== Utilities (csharp, nuget; release) @ main@e179288 ==
DEFECT ruleset: develop diverges from repo-config/develop.json (normalized diff)
DEFECT ruleset: main diverges from repo-config/main.json (normalized diff)
```
Anyone auditing against `main` today gets a false failure on every
repository. #551 removes the field from the compared subset, because the
defect was **two tools comparing one field under opposite policies**,
not the field's value: after the bypass change, `apply` writes the live
list back unchanged and `check` reports it without asserting, so the
audit asserting it contradicted the config directly.
Verified on `develop`: Utilities and PlexCleaner went from two ruleset
DEFECTs each to none, and Utilities' remaining two findings are LETTER
results for absent `GOVERNANCE.md` and `OPERATIONS.md`, which is the
real propagation gap.
## The second repair
#553closes#549. The payload-driven comparison was built on `walk/1`,
which arrived in **jq 1.6**. On jq 1.5 the filter does not degrade, it
fails to compile, so `check_ruleset` would report drift on every
parameterized rule it never compared. Reproduced on a genuine
`jq-1.5-1-a5b5cbe` and confirmed fixed, with output byte-identical to jq
1.7.
Both defects were found by review rather than by a gate: #551 while
reviewing the Blog registration, #553 by a Copilot review on a
downstream re-vendor of the same canonical.
## Blog
#547 catalogs Blog, taking the registry to 22 repositories. Its
committed report claims zero defects, which was true when written,
briefly false while the audit regression stood, and is true again on
this branch. Confirmed rather than assumed: `spec/audit.py Blog` reports
**0 defect/letter/error** here.
## Verification
`spec/validate.py` OK at 22 cataloged, diff-scoped `prose_lint` clean,
editorconfig clean, live read-only `configure.sh check` against this
repo passes, and the audit reports the corrected verdicts above.
ptr727 added a commit that referenced this pull request Aug 7, 2026
Clears the `A Home for a Disproved Finding` cluster from `TODO.md`. Its
`Checked` anchor was re-verified against `develop` at `756a53e` before
anything was written: the file still says the reviewer is sometimes
factually wrong, still requires a decline to carry evidence, and still
keeps its list of known non-working request paths, with nowhere to put
the proof itself.
## The gap
A decline that carries proof proves something about this tree, and
[GOVERNANCE.md "Every Finding Ends in an
Action"](https://github.com/ptr727/ProjectTemplate/blob/develop/GOVERNANCE.md#every-finding-ends-in-an-action)
is right that the thread is where it belongs while the pull request is
open. Afterwards the thread is the wrong place. The pull request merges,
the next round starts with no memory of the last, and the second
occurrence of the same claim reaches a maintainer with no way to tell it
from a first.
`Disproved Claims` is a new `###` under the runbook, so the three
declared `##` sections in `spec/files.json` are unchanged. An entry
names the claim, what was run or read to disprove it, the revision it
was proved against, and what ends it. **It is deliberately not a list to
append to**: an entry outliving the code it was proved against becomes a
reason not to check, which is strictly worse than proving the claim
again, so an entry whose subject moves is deleted by the change that
moves it rather than edited to look current. Two guards ride with it.
The record answers a repeated claim and never dismisses a new one, so a
finding is judged on its merits first and matched second, and a reply
carries the proof re-read rather than a pointer to a file the reviewer
cannot open. And the entries are this repository's own, so a repository
holding a copy carries the shape and deletes an entry whose subject it
does not carry.
## The three worked examples
**`keys_unsorted` requires jq 1.6.** A suppressed finding on #555
against the normalizer in `repo-config/configure.sh`, reasoning by
analogy from the `walk/1` failure #553 fixed. Re-run in this session
rather than quoted from the thread, on `jq-1.5-1-a5b5cbe` in
`ubuntu:18.04`, the build that reproduces `walk/1`:
```console
jq-1.5-1-a5b5cbe
$ echo '{"b":1,"a":2}' | jq -c 'keys_unsorted'
["b","a"]
$ echo '{"b":1}' | jq -c 'walk(.)'
jq: error: walk/1 is not defined at <top-level>, line 1:
jq: 1 compile error
```
**The write-guard's fallback parse, from #601.** Declined on the ground
that the arm cannot execute, since `punctuation_chars` arrived in Python
3.6, the module uses f-strings throughout, and `install.py` refuses
below 3.7. That is exactly the kind of disproof that expires, which is
why the entry names the floor as what ends it. It also records that the
finding earned a test case rather than a change, since only `ValueError`
from unbalanced quoting reaches that path in practice and nothing
covered it.
**The bare-SHA design, from #602.** This one came from this repository's
own backlog rather than from a reviewer, and it is here because a
rejected method costs the same to re-propose as a declined finding costs
to re-derive, while a backlog has a place for a claim the tree
contradicts and none for a method a measurement rejects. Over the 25
most recent merged pull requests the bare-SHA arm raised four references
and all four were correct prose, and a path arm flagged 54 of 215
backticked candidates.
Both #601 and #602 are folded under disposition **Amends "Record what
was tested and against which revision, and delete an entry whose subject
changes."** They land as the second and third worked examples in the
shipped section rather than as `Settled` bullets, because the entry
ships in the same change.
## Verification
- `python3 scripts/prose_lint.py . --diff develop` clean,
`markdownlint-cli2` clean on both files.
- `python3 -m unittest discover -s scripts` reports 372 tests OK and
`python3 spec/audit.py --selftest` passes, neither touched by this
change but both run because the file is carried.
- Every factual claim in the new section was read against the tree at
`756a53e`: the `keys_unsorted` call in `repo-config/configure.sh`,
`_git_subcommand_arglists` and `_PUNCTUATION_CHARS` in
`gh-write-guard.py`, the `(3, 7)` floor in `install.py`, and the verb
alternation in `pr_review.py`.
## TODO.md
The cluster is deleted per the file's own step 9. What this change does
not carry becomes a new `ready` cluster: `GOVERNANCE.md` outcome 2 ends
at the thread and nothing agent-agnostic points at the record, so an
agent that never opens the provider runbook posts a decline and has
nowhere to put the proof. That is a byte-locked carried section and a
fleet re-vendor, which is why it is a separate change rather than a
second surface in this one.
## Downstream
`.github/copilot-instructions.md` is carried `whole` at `intent` with
`appliesTo: "*"`, so this joins the re-vendor the ledger already lists.
A repository taking it carries the section and its rules, and starts its
own entries empty.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@ptr727