From d1bff509f58a8abc5e83ef91e6e05826cc2eed25 Mon Sep 17 00:00:00 2001 From: choiyounggi <74581798+choiyounggi@users.noreply.github.com> Date: Thu, 3 Sep 2026 21:57:44 +0900 Subject: [PATCH] =?UTF-8?q?knowledge:=20ingest=207=20verified=20insight(s)?= =?UTF-8?q?=20=E2=80=94=204=20new=20pages,=203=20merges;=204=20folded=20in?= =?UTF-8?q?to=20open=20PRs,=201=20dropped=20duplicate?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .dev-loop/INGEST_REPORT.md | 276 ++++-------------- log.md | 8 + .../common/llm/context-window-budget.md | 2 +- ...vendor-benchmark-claims-for-an-llm-tool.md | 64 ++++ wiki/backend/index.md | 1 + ...idation-ownership-in-task-decomposition.md | 72 +++++ ...ial-identifiers-across-parallel-workers.md | 81 +++++ wiki/infrastructure/index.md | 2 + wiki/platforms/index.md | 2 +- .../tools/harness-mediated-tool-results.md | 18 +- wiki/qa/bug-reports/reproducible-reports.md | 2 +- .../browser-console-capture-gaps.md | 1 + .../environments/element-crop-screenshots.md | 71 +++++ wiki/qa/index.md | 3 +- wiki/qa/process/adversarial-change-review.md | 2 +- wiki/qa/process/evaluating-review-feedback.md | 7 +- wiki/qa/process/llm-review-pipelines.md | 2 +- wiki/testing/index.md | 2 +- wiki/testing/quality/minimum-case-set.md | 6 +- 19 files changed, 390 insertions(+), 232 deletions(-) create mode 100644 wiki/backend/common/llm/vendor-benchmark-claims-for-an-llm-tool.md create mode 100644 wiki/infrastructure/agent-orchestration/inbound-validation-ownership-in-task-decomposition.md create mode 100644 wiki/infrastructure/agent-orchestration/sequential-identifiers-across-parallel-workers.md create mode 100644 wiki/qa/environments/element-crop-screenshots.md diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 3a965e5..bb2b9ac 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,237 +1,69 @@ -# Knowledge flush — 13 insight(s) ingested (21 claimed, 2 dropped, 6 released) +# Knowledge flush — 12 insight(s) -Cross-Check: 1× independent adversarial `claude` CLI headless pass over the 5 new pages — it refuted the changed-files gate page's "prettier exits 0 on an empty match set" claim; re-measured against Prettier 3.7.4, confirmed the reviewer was right (unmatched operand exits **2**), and rewrote the page, report rows 3/5 and `log.md`. Other 5 claim groups verdicted sound. Limits: the reviewer's sandbox blocked repo reads, so source-quote-supports-directive and self-contradiction dimensions went unaudited (details in `## Cross-Check`). - -Queue drained under run id `20260827-125731-38371` (this session is the detached -`hooks/auto-flush.sh` run; its step-0 acquire resolved re-entrantly to -`already-owned`, not to a competing holder). 21 rows were claimed; 13 are -ingested below, 2 are retired as out-of-layer, and 6 are released back to -`pending` for a later flush because each needs its own page rather than a row, -and rushing six more pages in one pass would have lowered the bar on all of them. +Flush run `20260903-213946-4161` (headless auto-flush, lock re-entered under the parent hook's run id). Queue rows are keyed by `hash`. Outcome: 4 new pages, 3 merges into existing pages, 4 folds pushed to open knowledge PRs, 1 dropped as a pending duplicate. ## Verified best-practice -Every external claim below was live-fetched this session and quoted in the page's -`Sources` block. Field evidence carries the repo, date, and the measured numbers. - -| # | Claim | Sources checked | How verified | Confidence | -|---|-------|-----------------|--------------|------------| -| 1 | `now()` is `transaction_timestamp()` (fixed at transaction start) while `clock_timestamp()` "changes even within a single SQL statement"; `RETURNING` yields computed defaults "without needing a separate database query" | postgresql.org `functions-datetime`, `dml-returning`, `transaction-iso` | Fetched; both key sentences quoted verbatim into the page | verified | -| 2 | A boundary recomputed in a follow-up step is a *second, later* `now`, widening a `<= boundary` set | Field: `rtb-unified` `packages/orpc/src/routers/batch.ts` — codifies "one `now` per decision" and passes `now` into the boundary helper; its result type omits the boundary, which is the shape that invites recomputation | Read the invariant and the signature in the cited file | field-tested | -| 3 | **[CORRECTED BY CROSS-CHECK]** The vacuous-pass shapes for `prettier --check` are: no operands (rc **0**), all operands ignore-filtered (rc **0**), and unsupported extensions with `--ignore-unknown` (rc **0**). A pattern/operand matching nothing exits **2** — it prints the success sentence *and* an unmatched-pattern error | prettier.io CLI + ignore docs; local measurement, Prettier 3.7.4 | The first draft generalised "empty match set ⇒ exit 0" from a field log where both messages appeared together. The independent reviewer flagged it; I then ran all seven cases against a real binary and rewrote the page around the measured table | verified (re-measured) | -| 4 | zsh does not word-split unquoted parameter expansions by default, so `cmd $FILES` arrives as **one** operand | zsh FAQ ch. 3 (`SH_WORD_SPLIT`) | Fetched; quoted ("By default, zsh does not have that behaviour: the variable remains intact") | verified | -| 5 | The zsh word-split operand exits **2**, but its log still carries the success sentence — so the log misleads even though the exit code does not | Field 2026-08-24 (`rtb-unified`, zsh) + local measurement 2026-08-27 | Field log showed both messages together; the local run reproduced it as `rc=2`. The page now says explicitly that this row fails loudly *unless* `--no-error-on-unmatched-pattern` is set. Probe placement re-confirmed: `.claude/tmp/` is `.gitignore`d, so a probe there passes at rc 0 | verified (re-measured) | -| 6 | TypeScript applies excess-property/contextual typing to fresh object literals, so a value of a type can be constructed with the type's name absent from the text | typescriptlang.org handbook, *Object Types* | Fetched; confirmed the check follows from the contextual type, not from a written annotation | verified | -| 7 | `tsc`'s program is `files` ∪ `include` ∪ transitive imports; `exclude` "only changes which files are included as a result of the `include` setting" and does not stop an imported file entering the program | typescriptlang.org TSConfig `#include`, `#exclude` | Fetched; the `exclude` sentence quoted (it sharpens the rule to "in the program", not "in `include`") | verified | -| 8 | Consequence of 6+7 measured | Field 2026-08-24/25 (`rtb-unified`): `grep "DealViewer"` reported 3 construction sites, actual 8 — the missed set included production wiring `routers/deal.ts:38`; `ContractScopeActor` 7→~22. Separately, `packages/orpc/tsconfig.json` `include: ["src/**/*"]` produced 3 production + 13 api-test errors and **zero** for `__tests__/routers/deal.test.ts`, whose 6 sites appeared only as 6 failing tests | Counts recorded from the cited runs | verified | -| 9 | cgroup v2: `memory.peak` is max usage since creation/reset; at `memory.max` "the OOM killer is invoked in the cgroup"; in `memory.events`, `max` counts times usage "was about to go over the max boundary" — **distinct** from `oom_kill` | docs.kernel.org cgroup-v2 admin guide | Fetched; all four quoted. This corrected the candidate, which had read a non-zero `max` as a kill; the page now states the distinction explicitly | verified | -| 10 | An `exec`'d process joins the container's cgroup and is invisible to the application's own semaphore | kubernetes.io `manage-resources-containers`, `assign-memory-resource`, `kubectl exec` reference + field 2026-08-26 (review-bot pod, `limits.memory: 3Gi`): `memory.current` 2.54 GiB, `memory.peak` 3.0 GiB (at the limit), `memory.events: max 5`, while `maxConcurrentAgents: 20` reported free slots | Docs fetched; pod numbers from the cited measurement | verified | -| 11 | Basename-keyed mutation backups collide across directories and restore cross-writes; an untracked file's `git diff` is empty whether restored or destroyed | Field 2026-08-21 (`rtb-unified`, NEWRTB-2936): restore wrote `schemas/deal.ts` into `routers/deal.ts` → `Cannot find module './common.js'`, `grep -c dealRouter` = 0; **both files were 154 lines**, so a line-count check passed; after re-keying, M9/M10 flipped SURVIVED→KILLED. Plus stryker mutant-states / pitest for the verdict vocabulary | Reproduced end to end in the cited run | field-tested | -| 12 | A negative assertion is vacuous when the fixture never supplies the triggering input | Field 2026-08-25 (`rtb-unified`): with `staleQueuedJobIds: []` the code early-returned; the widening the assertion claimed to catch survived 116/116 green | Mutation applied and observed | field-tested | -| 13 | A body-level (non-inline) review finding cites no file, so rebutting from an assumed file rejects real defects | Field 2026-08-19 (PR #327 r16): quote matched `report.py:393/416/425`, not the assumed `fill_plan.py:307` — sibling modules, one already fixed | Grep resolved the quote to the real site | field-tested | -| 14 | Unifying two duplicate allowlists defaults to the union and silently widens each side | Field 2026-08-25 (`rtb-unified` PR #965): folding `DISPLAYABLE_ERROR_CODES` into `USER_FACING_ERROR_CODES` would have added `UNAUTHORIZED` + `VALIDATION_ERROR`, exposing raw server messages as inline UI errors; caught only by computing the difference first | Difference computed before the merge | field-tested | - -Not upgraded: nothing was marked `verified` on field evidence alone. Two pages -carry `confidence: field-tested` (`mutation-harness-file-custody`, -plus the pre-existing `evaluating-review-feedback`), and no candidate was -recorded as `verified` without a fetched primary source. +Every external quote below was re-checked against the raw page with `curl -sL … | grep` on 2026-09-03 (not only through a summarizing fetch), except where noted. + +1. **520214f2e52d14c4 — sequential artifact numbers across parallel workers** → `confidence: verified`. Claim: the coordinator assigns RFC/ADR/migration numbers at dispatch; a worker's branch point cannot see a sibling's unmerged number and distinct filenames merge without conflict, so the uniqueness lint runs on the merged tree. Sources: Django migrations topic ("two migrations with the same number"), Django `makemigrations --merge` ("Enables fixing of migration conflicts"), Rails 3.2 migrations guide (creation-time timestamps to avoid clashes), git-merge ("incorporated in the final result verbatim"), adr-tools issue #102 (two devs both denote ADR 6). Field evidence: linkly t112/t119 both created RFC-0034. +2. **3fb0fd84e5ec5aa4 — inbound validation ownership when a plan splits a process boundary across tasks** → `confidence: verified`. Source: OWASP Input Validation Cheat Sheet ("as early as possible in the data flow, preferably as soon as the data is received from the external party"). Field evidence: agent-crew M2 `handle_envelope` missing `validate()` caught only by integration review. +3. **e5bc5ce7cb8508b4 — vendor benchmark claims for a memory/RAG tool** → `confidence: verified`. Sources: LoCoMo paper (arXiv 2402.17753), Zheng et al. LLM-as-a-judge (arXiv 2306.05685, "over 80% agreement"), Zep blog disputing Mem0's LoCoMo SOTA claim, Mem0's counter-reply (getzep/zep-papers#5, "58.44%"), OpenViking `stat_judge_result.py` (QA and Import token usage counted separately) and `judge.py` ("be generous with your grading"). The candidate's `judge.py:239` line reference no longer matches the 203-line file; the leniency instruction itself is confirmed and the page says so. +4. **b5e0cd6b60242fb6 — element crop screenshots (`clip` from `boundingBox()`)** → `confidence: field-tested`. Playwright semantics verified (element screenshot via `locator.screenshot()`, `clip` option, `boundingBox()` is viewport-relative and scroll-dependent); the Aside CLI clip misbehaviour itself is single-session field evidence, not reproduced here (no browser run). Directive generalised to: element-screenshot primitive first, read back the first crop before a batch, fall back to full-page capture on a persistent wrong-region clip. +5. **a019efb64f7bf316 — WebFetch summary vs raw page** → `confidence: verified`. Source: Claude Code tools reference ("runs the prompt against the content using a small, fast model. For most fetches, Claude receives that model's answer, not the raw page"; "use curl via Bash for the unprocessed page"). Field evidence re-confirmed: the tmap-skopenapi `routeSequential30` page contains the exact string "경유지는 최대 30개까지 설정할 수 있습니다." in the raw response. +6. **c2665906bb4df3a9 — Steps-prose resilience guarantee needs its own test** → `confidence: verified` (merged into an already-verified page). Source: SWE book ch12 ("A behavior is any guarantee that a system makes…"). Field evidence: wt-t4-event-push task 03 auditor FAIL→PASS after one added test. +7. **ee33bdf217330afe — split a CI fact-checker's "fabricated" verdict** → `confidence: verified` for the evidence (dev-loop PR #164 is public and merged; commit `f5d2395` message confirmed via `gh api`), directive itself field-tested; merged into a `field-tested` page. +8. **e040b9a62688a56e — Depends-on table vs Steps prose** → fold (see Open-PR check); field evidence only, no new external source claimed. +9. **a3560e8f5bd8f249 — brief workers to write measured counts** → fold; field evidence only (linkly t119 vs t112/t115/t117). +10. **0fa9c12c34ec038c — doc-currency gates inside the doc task** → fold; field evidence only (linkly enf0829, 10 integration failures). +11. **4977fec4fae1c1db — route worker edits through Bash when the guard is Bash-only** → fold; field evidence only. The claim that Edit/Write ignore cwd could not be confirmed in the Claude Code docs (only the Read tool section says "always pass absolute paths"), so the inserted row relies on the page's existing verified evidence that Edit/Write bypass a Bash-matched hook and does not state the cwd claim. +12. **a2023caa7c6da204 — multi-name `command -v`** → dropped, pending duplicate: PR #181's `path-resolution.md` already carries this exact edge case, instead-of row, POSIX synopsis source and the same local reproduction. ## Existing-layer check -Method: routed via `INDEX.md` → domain `index.md`; then built a full id+title -index of all 265 pre-existing pages and probed it with concept greps -(`clock_timestamp|clock skew`, `changed[- ]files|--ignore-unknown`, `tsconfig`, -`contextual typ|excess property`, `set difference|allowlist`, `cgroup`, -`basename|backup.*restore`, `2>&1`, `delta|baseline`) before deciding new vs merge. - -Pages read: testing-quality-source-text-wiring-assertions, testing-quality-tests-that-cannot-fail, backend-common-change-impact-call-site-enumeration, backend-common-change-impact-widening-a-closed-value-table, qa-process-evaluating-review-feedback, infrastructure-containers-host-cgroup-visibility, testing-quality-behavior-not-implementation +Pages read: platforms-environment-path-resolution, security-input-validation-at-trust-boundaries, qa-document-verification-spec-document-gates, qa-deliverables-quantitative-claims-in-a-published-document, infrastructure-agent-orchestration-worktree-isolated-workers, qa-process-completion-claims, platforms-tools-harness-mediated-tool-results, testing-quality-minimum-case-set, qa-process-evaluating-review-feedback, qa-process-llm-review-pipelines, qa-document-verification-generated-reference-drift-gates, backend-common-llm-context-window-budget, qa-process-adversarial-change-review, qa-bug-reports-reproducible-reports, qa-environments-browser-console-capture-gaps -Findings: +Also read on open-PR heads (not on this checkout): checkable-claims-in-an-adopted-plan, sibling-validators-on-a-shared-node (#181); semantic-conflicts-after-parallel-merge, verify-command-in-a-worker-brief (#179); ours-resolution-on-a-mixed-content-conflict, forward-references-in-a-numbered-protocol (#180). -- **Zero coverage** (→ new pages): changed-files-only gates, tsconfig/contextual - typing, allowlist set-difference, app-clock-vs-DB-timestamp, exec-into-a-running-container. - The concept greps returned no hits for these; the clock hits were incidental - (offline sync, token handling) and none compared an app clock to a DB column. -- **Already covered — one candidate all but retired.** The comment-stripping - insight is `source-text-wiring-assertions` step 2 verbatim ("Make the - assertion's subject the file with comments removed"), and its false-RED and - negative/count false-GREEN shapes are already edge rows. Only the *empty-slice* - consequence was new, so that alone was merged. -- **Line-cap conflict handled without breaking the invariant.** - `source-text-wiring-assertions` sits at exactly **120** body lines (the - documented cap). Rather than add a row and violate maintenance invariant 5, the - new nuance and the new field evidence were merged **in place** into an existing - edge row and an existing source bullet. Body count re-measured after editing: - still 120. -- **No conflicts found.** Nothing ingested contradicts an existing directive. - The one correction made was to a *candidate*, not to the wiki (item 9: the - `memory.events` `max` counter is approaches-to-limit, not kills). -- **Related links added both ways**: `tests-that-cannot-fail` ↔ - `mutation-harness-file-custody`; `widening-a-closed-value-table` ↔ - `compiler-as-call-site-inventory` (+ `errors-diagnostics-from-a-shared-code-path`); - `host-cgroup-visibility` → `exec-added-processes-and-the-memory-budget` - (from its existing self-monitoring row). -- **Indexes/log updated**: 4 domain indexes (+5 "load when" rows), `log.md` - appended. Root `INDEX.md` unchanged — no new domain. - -Gates run (the exact CI commands from `.github/workflows/test.yml`): -`node scripts/wiki-structure-checks.js wiki` → **pages: 270, indexes: 13, -findings: 0**; `node scripts/wiki-lint-prohibitions.js wiki` → **directives 72, -compliant 72, violations 0** (the 1 `info` is pre-existing in -`config/keys-ahead-of-their-consumer.md`, untouched); `bash scripts/check-versions.sh` -→ `ok: dev-loop 1.11.2`. The `bats tests/` job was **not** run — bats is not -installed on this machine, and this change touches only wiki markdown (no -scripts or hooks), so that suite's subject is unchanged. +- **Merged (3):** WebFetch-summary case → `harness-mediated-tool-results` (new when-this-applies sentence, edge-case row, instead-of row, source, field context; index cell widened). Steps-prose guarantee → `minimum-case-set` (edge-case row, instead-of row, SWE-book quote + field evidence; index cell widened). Split verdict → `evaluating-review-feedback` (edge-case row, instead-of row, PR #164 source; index cell widened). None of these three pages is touched by an open knowledge PR. +- **Created (4):** `infrastructure/agent-orchestration/sequential-identifiers-across-parallel-workers`, `infrastructure/agent-orchestration/inbound-validation-ownership-in-task-decomposition`, `backend/common/llm/vendor-benchmark-claims-for-an-llm-tool`, `qa/environments/element-crop-screenshots`. Each has an index row and a log line. +- **Conflicts flagged:** none. The inbound-validation page agrees with `security-input-validation-at-trust-boundaries` ("validate at the consumer boundary anyway") and adds the task-decomposition angle. +- **Related links:** new pages link to existing ones; back-links added on `qa-process-adversarial-change-review`, `backend-common-llm-context-window-budget`, `qa-process-llm-review-pipelines`, `qa-environments-browser-console-capture-gaps`, `qa-bug-reports-reproducible-reports`. Back-links deliberately NOT added on `worktree-isolated-workers`, `shared-run-state`, `spec-document-gates`, `validation-at-trust-boundaries`, `quantitative-claims-in-a-published-document`, `completion-claims`: their `related:`/frontmatter lines are rewritten by PR #179/#180/#181 and a second edit would conflict at merge. Owner may add them after those PRs land. +- Lint on this branch: `wiki-structure-checks` 279 pages / 0 findings, `wiki-lint-prohibitions` 0 violations, all touched pages ≤ 120 body lines. ## Open-PR check -`gh pr list --repo choiyounggi/dev-loop --state open --search "head:knowledge/"` -returned **no open PRs**, and a second unfiltered `gh pr list --state open` -returned none either — the repository has zero open PRs at flush time. There -were therefore no in-flight sibling branches to diff against, and no -`git fetch origin ` / `git diff origin/main origin/ -- wiki/` -comparisons to run. - -Per-candidate verdict: **all 21 = `new`.** No `fold`, no `drop-as-pending-duplicate`. -(The 2 drops recorded below are out-of-layer drops, not pending-duplicate drops.) +Open `knowledge/*` heads listed via `gh pr list --search "head:knowledge/"`: #179 `knowledge/choiyounggi-20260903-172728`, #180 `knowledge/choiyounggi-20260903-184706`, #181 `knowledge/choiyounggi-20260903-203836`. Each was fetched and diffed against `origin/main -- wiki/`. + +| Candidate | Overlapping open head | Verdict | +|-----------|-----------------------|---------| +| a2023caa7c6da204 multi-name `command -v` | #181 `path-resolution.md` (identical edge case + reproduction) | **drop** (pending duplicate) | +| e040b9a62688a56e Depends-on table vs Steps prose | #181 `checkable-claims-in-an-adopted-plan.md` (same trigger: checking an adopted plan) | **fold** → pushed as `4bc6de8` on #181 + PR comment | +| a3560e8f5bd8f249 workers write measured counts | #180 `ours-resolution-on-a-mixed-content-conflict.md` (merge-time count reconciliation) | **fold** → pushed as `3b78273` on #180 + PR comment | +| 0fa9c12c34ec038c doc-currency gates in the doc task | #179 `verify-command-in-a-worker-brief.md` (what the brief's verify line names) | **fold** → pushed as `e242b2c` on #179 + PR comment | +| 4977fec4fae1c1db Bash-routed edits under a Bash-only guard | #179 `worktree-isolated-workers.md` (Edit/Write bypass + matcher widening) | **fold** → same commit `e242b2c` on #179 | +| 520214f2e52d14c4 sequential numbers | #179 `semantic-conflicts-after-parallel-merge.md` (enum/match semantic conflicts), #180 `ours-resolution` (count conflicts) — adjacent, different trigger (distinct new files, no conflict at all) | **new** | +| 3fb0fd84e5ec5aa4 inbound validation ownership | #181 `validation-at-trust-boundaries.md` edit (spatial-value clamping) — different trigger | **new** (separate page; no edit to the security page to avoid conflicting with #181) | +| e5bc5ce7cb8508b4 vendor benchmark claims | #181 `synthetic-corpus-measurement-floor.md` (measuring on your own corpus) — different trigger | **new** | +| b5e0cd6b60242fb6 element crop screenshots | none | **new** | +| a019efb64f7bf316 WebFetch summary | #181 touched `quantitative-claims-in-a-published-document.md` related line only | **new** (merged into `harness-mediated-tool-results`, untouched by open PRs) | +| c2665906bb4df3a9 Steps-prose guarantee | none (`minimum-case-set.md` untouched) | **new** (merge) | +| ee33bdf217330afe split verdict | #179 touched `completion-claims.md`, not `evaluating-review-feedback.md` | **new** (merge) | + +Lint (`wiki-structure-checks`, `wiki-lint-prohibitions`) was run on each fold branch after the edit: 0 findings, 0 violations; fold pages remain ≤ 120 body lines (83/55/73/93 for #181/#180/#179 verify/#179 worktree). ## Routing decision -**New pages (5)** - -| Page | Domain/category | From | Why not an existing page | -|------|-----------------|------|--------------------------| -| `application-clock-vs-database-timestamps` | databases / transactions | `2b27d15d` + `bea92fdd` | No page compares an app clock to a DB column. `transactions` chosen over `schema-design` because the decisive content is transaction-time semantics (`now()` = transaction start ⇒ stamp order ≠ commit order) and the fix is a lock/isolation choice | -| `changed-files-only-gates` | infrastructure / ci-cd | `ff041061` + `4b9af3a0` | Zero grep hits. Both candidates are the same defect (a gate green with an empty subject) from two directions, so they became one page rather than two | -| `compiler-as-call-site-inventory` | backend / common / change-impact | `702dcf4e` + `94d55f2f` | `call-site-enumeration` is the sibling case (callers of a changed signature, Python positional-vs-keyword) and is at 80 body lines; the TS mechanism is *constructors of a type* with its own workflow, so per "one case per page" it is a separate page, cross-linked | -| `mutation-harness-file-custody` | testing / quality | `6a9de235` + `41fa1c87` | `harness-reverse-controls` covers scoring a harness; nothing covers the harness's custody of the tree. Both candidates are that one case (keying, and the read window) | -| `exec-added-processes-and-the-memory-budget` | infrastructure / containers | `7b9e8788` | `host-cgroup-visibility` is cross-pod read mechanics and explicitly routes self-monitoring elsewhere; `resource-limits-and-probes` is manifest authoring. This is a runtime preflight before adding load | - -No new category was created — all five landed in existing categories. - -**Merged into existing pages (5 candidates)** - -| Candidate | Merged into | Shape | -|-----------|-------------|-------| -| `91ef5d53` | `testing-quality-tests-that-cannot-fail` | +1 never-fails row, +1 Instead-of row, +1 source | -| `f189f423` | `testing-quality-source-text-wiring-assertions` | In-place extension of 1 edge row + 1 source bullet (page at the 120-line cap) | -| `bb6d8539` | `backend-common-change-impact-widening-a-closed-value-table` | +Do-this 6 & 7 (incl. a set-difference ruling table), +1 Instead-of row, +1 source | -| `60a817ee` | `qa-process-evaluating-review-feedback` | +2 edge rows, +1 Instead-of row, +1 source | -| `7b9e8788` | `infrastructure-containers-host-cgroup-visibility` | Cross-link from its self-monitoring row to the new page | - -**Dropped — out of layer (2, retired)** - -- `094dedf3` — a Figma MCP `inspect_node` → `get_dev_ready` children-fetch - workaround. The server is a private, org-internal MCP plugin; the behavior is - not publicly verifiable and the directive does not transfer to any other reader. -- `e165a365` — an `/rtb:review` remote-fallback runbook naming - `~/.claude/tools/rtb-remote-review.sh` and an internal pod. The transferable - kernel ("a two-provider review gate degraded to one provider is not a passed - gate") is already the subject of `qa-process-llm-review-pipelines`; what remains - is machine-specific paths. - -**Released back to `pending` (6)** — each needs its own page, not a row, and is -better served by a dedicated pass than by being appended here: -`81dc1f98` (naming the carrier field/type when a plan says "wire A to B"), -`b9ae304a` (`VAR="$(cmd 2>&1)"` mixing stderr into a value used as a path), -`fdd0b3c6` (monitor markers anchored at line start; delta rather than absolute -state; first cycle records a baseline), -`c2adb2be` (positional-order assertions on rendered SQL predicates), -`815e8cb9` (grep only *active* `DATABASE_URL` assignments, and confirm which -dotenv file the tool loads, before a destructive DB command), -`f1146adb` (CI ticket-key extraction scoped by changed-file intersection rather -than by mention). - -## Decision Log - -**Intent.** Drain the harvested `★ Insight` queue into reviewable wiki knowledge -without lowering the wiki's evidence bar. The queue held 21 rows accumulated over -several days; the goal was correct routing and real verification, not a high -ingest count. - -**Alternatives considered and rejected.** - -- *Ingest all 21 in this pass.* Rejected: six of them each need their own page, - and writing six more pages in one pass would have produced thin, weakly-sourced - entries. They are released to `pending`, not dropped, so the next flush takes - them with a full budget. -- *Append the two TypeScript candidates to `call-site-enumeration`.* Rejected: - that page is the sibling case (callers of a changed signature, Python - positional-vs-keyword). AGENTS.md requires one case per page, so the - constructor-enumeration case became its own page, cross-linked both ways. -- *Add a row to `source-text-wiring-assertions` for the empty-slice nuance.* - Rejected: that page is at exactly the documented 120-line body cap, so adding a - line would violate maintenance invariant 5. The nuance was merged **in place** - into an existing edge row instead; body re-measured at 120. -- *Drop the comment-stripping candidate entirely as a duplicate.* Rejected: its - directive is already the page's step 2, but the empty-slice consequence - (vacuous **green**, not the documented noisy red) was genuinely absent. -- *Claim a cross-check exemption because this PR cannot merge itself.* Rejected — - see below; the check found a real error, which is the argument against exempting. -- *Push to `origin`* as the skill's snippet does. Not available: this contributor - has no write access to `choiyounggi/dev-loop` (403). Used the pre-existing - `fork` remote, which is how every prior knowledge branch here was published. -- *Branch name from `git config user.name`.* The skill's ASCII sanitisation of a - Korean name yields an empty string → `anon`, defeating the attribution the - branch name exists for. Used the gh login, matching existing branch names. - -**Where reviewers should look hardest.** - -1. `infrastructure/ci-cd/changed-files-only-gates.md` — rewritten after the - cross-check. The measured table is the load-bearing part; please sanity-check - it against your own Prettier version, since the exit codes are version-visible - behaviour rather than a documented contract. -2. `databases/transactions/application-clock-vs-database-timestamps.md` step 5–6 — - the claim that timestamp order is not commit order, and that the remedy is a - lock/isolation level rather than finer clock resolution. `[추정]` on the MySQL - `NOW()`/`SYSDATE()` row: taken from general MySQL semantics, not fetched this - session like the PostgreSQL pages were. -3. `widening-a-closed-value-table.md` Do-this 6–7 — this inserts a security-shaped - concern (allowlist widening) into a page whose original subject was value - tables. If that reads as two cases, it should be split. -4. The 2 dropped candidates — if you consider private-tooling runbooks in scope - for this wiki, they should be restored rather than retired. - -## Cross-Check - -Independent adversarial pass via `claude` CLI headless (separate process, no -shared context), prompted to refute rather than confirm, over the five new pages' -technical claims. - -**It found a real error, and the page was rewritten because of it.** The reviewer -challenged the claim that `prettier --check` exits 0 on an empty match set, -arguing an unmatched pattern errors by default and that exit-0 belongs to the -ignore-filtered case. I resolved it by measurement rather than by argument — -running all seven cases against Prettier 3.7.4 — and the reviewer was right: -an unmatched operand exits **2** (while still printing the success sentence), -whereas the genuine silent vacuous passes are no-operands, all-ignore-filtered, -and `--ignore-unknown`-with-unsupported-extensions. The page, this report's -rows 3 and 5, and the `log.md` entry were all corrected. - -Verdicts on the other five claim groups: **sound** (PostgreSQL clock semantics — -noted as if anything *understated*; zsh word-splitting; TS contextual typing; -`tsc` program membership incl. `exclude`-does-not-stop-imports; cgroup v2 -`max` vs `oom_kill` and `kubectl exec` cgroup placement). - -Stated limits of the check: the reviewer's sandbox denied it read access to -`~/.dev-loop/repo/wiki`, so it adjudicated the six claims as quoted in its prompt -and could **not** audit (b) whether each `Sources` quote supports the directive it -is cited for, or (c) whether any page contradicts its own edge-case rows. Those -two dimensions remain unreviewed by an independent party and are the residual -risk in this PR. A first attempt also returned only the session's Stop-hook -output rather than a verdict; that run was discarded rather than read as -"no findings". - -## Review notes - -- PR-only, as required: no merge, no push to `main`. -- Commit is under the contributor's own ambient git identity - (`최영기 `, gh `dch0202-rsquare`); no assistant identity - and no `Co-Authored-By` trailer. The branch uses the gh login because - sanitizing the Korean `user.name` to ASCII yields an empty string, which the - skill's snippet would have turned into `anon` — that would have defeated the - attribution the branch name exists for. -- Scope purity: only `wiki/**`, four domain indexes, `log.md`, and this report. - Two untracked leftovers from earlier flushes - (`.dev-loop/CROSSCHECK_FINDINGS.md`, `.dev-loop/fold-note-73.md`) were left - untouched and unstaged. +| Insight | Target | +|---------|--------| +| 520214f2 sequential numbers | `infrastructure/agent-orchestration/sequential-identifiers-across-parallel-workers` — NEW page; agent-orchestration already owns worker briefs and shared run state | +| 3fb0fd84 inbound validation ownership | `infrastructure/agent-orchestration/inbound-validation-ownership-in-task-decomposition` — NEW page; the lesson is about which task's brief carries the decision, so orchestration rather than security (linked to the security page) | +| e5bc5ce7 vendor benchmark claims | `backend/common/llm/vendor-benchmark-claims-for-an-llm-tool` — NEW page; backend/common/llm owns consuming LLM tooling; no new category needed | +| b5e0cd6b element crop screenshots | `qa/environments/element-crop-screenshots` — NEW page; qa/environments already holds browser-tooling gaps (console capture, bot blocking) | +| a019efb6 WebFetch summary | merged into `platforms/tools/harness-mediated-tool-results` — same class (a tool result mediated before the agent sees it) | +| c2665906 Steps-prose guarantee | merged into `testing/quality/minimum-case-set` — it is a "which cases are required" rule | +| ee33bdf2 split verdict | merged into `qa/process/evaluating-review-feedback` — it is a response-to-review-finding rule | +| e040b9a6 / a3560e8f / 0fa9c12c / 4977fec4 | folded into PR #181 / #180 / #179 / #179 pages respectively (see Open-PR check) | +| a2023caa multi-name `command -v` | dropped — already on #181 `platforms/environment/path-resolution` | + +No new category was added; every insight fit an existing domain/category. diff --git a/log.md b/log.md index 3f8265b..d8b3a57 100644 --- a/log.md +++ b/log.md @@ -103,3 +103,11 @@ Append-only. Format: `## [YYYY-MM-DD] .md`) rather than asking the worker to compute it. +2. **Replace any "check existing numbers first" instruction with the assigned + number.** A worker confined to its own worktree/branch point can only see + artifacts that existed at that branch point — a sibling worker's + not-yet-merged number is invisible to it, so the instruction is + unexecutable, not merely risky. +3. **When numbers cannot be pre-assigned (a worker discovers the need for one + mid-task, not named in its brief), use a collision-proof identifier + instead of a sequential one** — a timestamp (Rails switched migration + filenames to creation-time timestamps for exactly this reason) or a UUID — + and leave renumbering to a single later pass on the merged tree, not to + the worker. +4. **Put the numbering check in the merge-time lint, and make it a hard + failure**, not a warning: `git merge` combines two branches' distinct new + files without any conflict — non-overlapping additions "are incorporated + in the final result verbatim" — so two same-numbered artifacts merge + silently and only a post-merge lint over the combined tree catches the + duplicate. + +| Case | Do | +|------|----| +| Coordinator dispatches N parallel tasks that each produce one numbered artifact | Assign each task's number in the dispatch brief before the worker starts | +| A worker's brief did not anticipate the need for a number (discovered mid-task) | Generate a timestamp or UUID identifier instead of guessing a sequential number | +| The artifact scheme is inherently timestamp/UUID-based (Rails-style migrations) | No coordinator assignment needed — collision is already structurally prevented | +| The numbering/uniqueness lint runs per-worker inside each worktree | Move it to run once, after merge, over the integrated tree — a per-worktree run cannot see sibling numbers either | + +## Edge cases + +| Case | Then | +|------|------| +| Two workers are dispatched from the same branch point and neither's task depends on the other's artifact | Still assign both numbers at dispatch — same-branch-point siblings collide exactly like sequential ones, per the adr-tools numbering-conflict report | +| A worker's task is later dropped or fails and its assigned number is never used | Leave the gap and give any later task the next fresh number; recycling the unused number into another worker's task reintroduces the same race | +| The coordinator itself cannot see a number some other, unrelated run already claimed | Track assigned numbers in the shared run-state directory ([infrastructure-agent-orchestration-shared-run-state]), not by scanning worker worktrees | +| The project's existing convention is "developer picks the next number by hand" (pre-dating agent workers) | Migrate to coordinator-assigned numbers; a note saying "resolve manually on conflict" covers nothing, because distinct files raise no merge conflict to resolve | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Tell each worker "look at the current highest number in the repo and use the next one" | Assign the number in the dispatch brief | The worker's worktree/branch point cannot see a sibling's number until after merge — the check is unexecutable, not merely racy | +| Rely on the artifact's per-worktree review/lint to catch duplicate numbers | Add the same check as a merge-time gate over the integrated tree | A per-worktree lint sees only that worker's own artifact; the duplicate exists only once both are present, i.e. after merge | +| Expect `git merge` to flag two artifacts that claim the same number | Run the numbering lint on the merged tree as a gate | `git merge` conflicts on overlapping changes to the same path, not on two different files whose content happens to claim the same logical number | + +## Sources + +- https://docs.djangoproject.com/en/5.1/topics/migrations/ — "you and another developer have both committed a migration to the same app at the same time, resulting in two migrations with the same number"; Django prompts to linearize them +- https://docs.djangoproject.com/en/5.1/ref/django-admin/#makemigrations — `--merge`: "Enables fixing of migration conflicts" +- https://guides.rubyonrails.org/v3.2/migrations.html — "With multiple developers it was easy for these to clash requiring you to rollback migrations and renumber them. With Rails 2.1+ this is largely avoided by using the creation time of the migration to identify them" +- https://git-scm.com/docs/git-merge — non-overlapping changes "are incorporated in the final result verbatim"; conflicts are raised only when both sides changed the same area +- https://github.com/npryce/adr-tools/issues/102 — "ADR numbering sequence may break when merging multiple PRs": "Dev A creates a PR with an ADR (he/she denotes that ADR with number 6). Dev B creates a PR with an ADR (he/she denotes that ADR with number 6)" — merged at different times, two ADRs share a number +- Field evidence 2026-08-25 (linkly repo, worker t119 branched from da1256e): sibling worker t112 had already created RFC-0034 on its own branch; t119, unable to see it, created a different RFC-0034 too. Both merged without a git conflict (distinct filenames); `rfc_lint.py`'s `check_numbering` (§3, no duplicate numbers) only caught the duplicate after the merge diff --git a/wiki/infrastructure/index.md b/wiki/infrastructure/index.md index f781a71..0cf2db7 100644 --- a/wiki/infrastructure/index.md +++ b/wiki/infrastructure/index.md @@ -22,6 +22,8 @@ Match your situation to a "load when" line; load only matching pages. | [worktree-isolated-workers](agent-orchestration/worktree-isolated-workers.md) | Authoring the brief/output contract for parallel workers each confined to its own git worktree; workers stall at the same phase with no task-level error; deciding where shared or produced artifacts live and which direction (read vs write) a worktree guardrail stops; a guardrail escalates on read-only access to another worktree; the isolation guard is a Bash-command hook while workers also edit files with native Edit/Write tools | | [autonomous-decision-rulings](agent-orchestration/autonomous-decision-rulings.md) | An unattended agent hits a decision its plan does not answer and must choose between stopping to ask and proceeding; a run stalls on questions no human needed to see; deciding which decision categories require a human; recording autonomous decisions for audit; resuming after interruption/compaction without re-dispatching completed work | | [session-context-token-budget](agent-orchestration/session-context-token-budget.md) | Planning or running long-lived coordinator/worker agent sessions and deciding when to compact or clear context; a run's cost is dominated by cache reads; screenshots or large file reads are entering a long-lived session; choosing slot counts / per-phase token budgets for an orchestrated run | +| [sequential-identifiers-across-parallel-workers](agent-orchestration/sequential-identifiers-across-parallel-workers.md) | Several parallel workers, each confined to its own worktree/branch, each independently create a new sequentially-numbered artifact (RFC, ADR, migration, ticket number); reviewing a brief that tells a worker to check the next free number before creating one; designing the merge-time lint that must catch duplicate numbers | +| [inbound-validation-ownership-in-task-decomposition](agent-orchestration/inbound-validation-ownership-in-task-decomposition.md) | A wiki-plan or task decomposition splits a message's producer and receiver (WS/IPC/HTTP) across separate tasks; deciding whether the receiver task's brief must carry its own inbound-validation decision; a plan's decision-to-page map cites a trust-boundary page only under the producer task | ## ci-cd diff --git a/wiki/platforms/index.md b/wiki/platforms/index.md index 8200fae..84b1383 100644 --- a/wiki/platforms/index.md +++ b/wiki/platforms/index.md @@ -28,7 +28,7 @@ Match your situation to a "load when" line; load only matching pages. |------|-----------| | [agent-permission-classifier-denials](tools/agent-permission-classifier-denials.md) | A model-based permission gate (Claude Code auto mode) denied an action that is correct for the task — a push, a config write, a deploy command; writing the permission or `autoMode` config itself is refused as self-modification; choosing the settings scope a classifier actually reads; designing an automation whose steps a classifier will judge, and where the consent sentence has to sit | | [bsd-vs-gnu-cli](tools/bsd-vs-gnu-cli.md) | A command works on Linux but fails on macOS or vice versa (`date`, `sed -i`, `timeout`, `seq`, `grep -P`, `readlink`, `stat`); writing a script or CI step that must run on both userlands; deciding whether to install GNU coreutils on macOS or write POSIX-only | -| [harness-mediated-tool-results](tools/harness-mediated-tool-results.md) | A plugin or hook in your agent harness returned substitute content for a built-in tool (truncated read, redaction, a note telling you to call something else); deciding whether a short result is interception or a genuinely small/empty file; the hook's suggested workaround failed too; briefing spawned worker sessions about a known-degraded tool | +| [harness-mediated-tool-results](tools/harness-mediated-tool-results.md) | A plugin or hook in your agent harness returned substitute content for a built-in tool (truncated read, redaction, a note telling you to call something else); deciding whether a short result is interception or a genuinely small/empty file; the hook's suggested workaround failed too; briefing spawned worker sessions about a known-degraded tool; WebFetch (or any fetch tool) returns a small-model summary you are about to cite as a verified quote or number | | [plugin-mcp-server-registration](tools/plugin-mcp-server-registration.md) | A plugin-bundled MCP server is missing from `/mcp` or its tools are uncallable while the plugin's skills work; deciding whether the fault is registration, config shape, or the server itself; about to change a plugin `.mcp.json` (bare map vs `mcpServers` wrapper, `command`/`args`/`env`) to make a server appear; relaying a shell variable into a server with `${VAR}` and the server reads the wrong value; a plugin updated mid-session and its server still runs the old path | | [version-keyed-artifact-cache](tools/version-keyed-artifact-cache.md) | Shipping a code update to a distribution system that caches artifacts by a version string (a Claude Code marketplace plugin, or any tag-pinned cache) and the update runs but the old behavior persists; deciding why `/plugin update` reports "at latest" yet new code never runs; locating and clearing a stale `~/.claude/plugins/cache////` | | [unpacked-extension-source-reload](tools/unpacked-extension-source-reload.md) | You edited a file of an unpacked Chromium extension (content script, service worker, manifest) and are about to judge the change in the browser, or the edit appears to have no effect; deciding which of extension-reload and host-page-reload a given file requires; adding a running-build version marker; automating the `chrome://extensions` reload control from an agent or CDP script | diff --git a/wiki/platforms/tools/harness-mediated-tool-results.md b/wiki/platforms/tools/harness-mediated-tool-results.md index db81be7..12d8e91 100644 --- a/wiki/platforms/tools/harness-mediated-tool-results.md +++ b/wiki/platforms/tools/harness-mediated-tool-results.md @@ -6,7 +6,8 @@ applies_to: [claude-code, agent-harness] confidence: verified sources: - https://code.claude.com/docs/en/hooks -last_verified: 2026-08-05 + - https://code.claude.com/docs/en/tools-reference +last_verified: 2026-09-03 related: [platforms-shells-command-text-inspected-before-execution, platforms-processes-non-interactive-cli-invocation, infrastructure-agent-orchestration-control-signals-vs-primary-artifacts, platforms-tools-agent-permission-classifier-denials] --- @@ -18,6 +19,9 @@ A file-reading or search tool returns content that does not match the file on disk — truncated to the first line, summarized, or replaced by a note telling you to call something else; a plugin/hook is installed in the session; or you are about to brief worker sessions that will read files in the same repo. +Also when a fetch tool (`WebFetch`) returns a summary written by a small, +fast intermediary model rather than the page's raw text, and you are about +to record a quoted sentence or number from that summary as verified. ## Do this @@ -66,6 +70,7 @@ about to brief worker sessions that will read files in the same repo. | A second agent reports the tool working normally | Hook config is per settings scope (user/project/local); confirm which scope each session loaded before concluding the hook was removed | | The tool is mediated but writes still land | `PostToolUse` runs after execution, so write tools take effect even when their reported result is rewritten — verify the write on disk, not from the returned text | | The wrapping hook also intercepts your shell fallback | Read through a different mechanism (an editor/Write-tool round trip, `base64` of a byte range) and escalate the harness configuration to the human — a harness that blocks every read path is a configuration fault, not a puzzle to route around | +| The mediating tool is a fetch/summarize call (`WebFetch`) rather than a file read, and its result is about to be recorded as a verified quote or number | Fetch the raw page yourself (`curl -sL `) and grep for the literal string before recording it — treat the tool's summary as a lead, not proof | ## Instead of @@ -75,10 +80,12 @@ about to brief worker sessions that will read files in the same repo. | Conclude the file is empty, one line, or missing | Check `wc -l` on the path | The one-line result is the hook's message, not the file's length | | Let each spawned worker discover the mediation itself | Name the mediated tool and the fallback command in the brief | The cost is per agent otherwise, and each pays it before doing any real work | | Disable the plugin to get a clean read | Use the unmediated tool for the read you need | The plugin serves the session's other work; a per-read fallback is reversible and scoped | +| Cite a WebFetch summary's quoted sentence or number as a verified fact | Fetch the raw page with `curl -sL ` and grep for the literal string before recording it | The summary is produced by a small, fast intermediary model that can paraphrase or fabricate specifics even when the general gist is accurate | ## Sources - https://code.claude.com/docs/en/hooks — `PostToolUse` `hookSpecificOutput.updatedToolOutput` "replaces the tool's result"; `PreToolUse` `hookSpecificOutput.updatedInput` "replaces a tool's arguments before it runs"; "For redaction or transformation use cases, intercept at `PreToolUse` for outbound tool inputs and `PostToolUse` for inbound tool results"; `PostToolUse` fires after the tool has executed +- https://code.claude.com/docs/en/tools-reference — "WebFetch takes a URL and a prompt describing what to extract. It fetches the page, converts the response to Markdown when the server returns HTML, and runs the prompt against the content using a small, fast model. For most fetches, Claude receives that model's answer, not the raw page"; the docs' own remedy: "use curl via Bash for the unprocessed page" (raw page grep 2026-09-03) ## Field context @@ -88,3 +95,12 @@ the suggested retry (`offset=151, limit=120`) returned line 1 again. Two orchestrated worker sessions independently logged the same interception and each fell back to `cat -n`/`sed` on its own, having spent three to four tool calls apiece rediscovering it. + +Observed 2026-09-02 (repo wt-v3-docs): WebFetch on tmap-skopenapi.readme.io's +`routeSequential30` page reported the quoted sentence "경유지는 최대 30개까지 +설정할 수 있습니다." (waypoints up to a maximum of 30); a direct `curl` + grep +against the raw page confirmed the exact string inside the `viaPoints` field's +description (re-confirmed 2026-09-03). The same check on the `matrix` page +showed WebFetch's negative claim (no limit stated) was also accurate — the +summary is not always wrong, which is why a spot-check, not blanket distrust, +is the right response. diff --git a/wiki/qa/bug-reports/reproducible-reports.md b/wiki/qa/bug-reports/reproducible-reports.md index 0899058..49d8951 100644 --- a/wiki/qa/bug-reports/reproducible-reports.md +++ b/wiki/qa/bug-reports/reproducible-reports.md @@ -7,7 +7,7 @@ confidence: verified sources: - https://bugzilla.mozilla.org/page.cgi?id=bug-writing.html last_verified: 2026-07-10 -related: [qa-exploratory-exploratory-sessions, qa-process-severity-and-priority] +related: [qa-exploratory-exploratory-sessions, qa-process-severity-and-priority, qa-environments-element-crop-screenshots] --- # Writing a Bug Report a Stranger Can Reproduce diff --git a/wiki/qa/environments/browser-console-capture-gaps.md b/wiki/qa/environments/browser-console-capture-gaps.md index 63ba1d1..a46cfef 100644 --- a/wiki/qa/environments/browser-console-capture-gaps.md +++ b/wiki/qa/environments/browser-console-capture-gaps.md @@ -14,6 +14,7 @@ related: platforms-tools-unpacked-extension-source-reload, platforms-processes-tool-diagnostics-without-a-failing-exit-code, testing-quality-tests-that-cannot-fail, + qa-environments-element-crop-screenshots, ] --- diff --git a/wiki/qa/environments/element-crop-screenshots.md b/wiki/qa/environments/element-crop-screenshots.md new file mode 100644 index 0000000..199a31d --- /dev/null +++ b/wiki/qa/environments/element-crop-screenshots.md @@ -0,0 +1,71 @@ +--- +id: qa-environments-element-crop-screenshots +domain: qa +category: environments +applies_to: [general, playwright] +confidence: field-tested +sources: + - https://playwright.dev/docs/screenshots#element-screenshot + - https://playwright.dev/docs/api/class-page + - https://playwright.dev/docs/api/class-locator +last_verified: 2026-09-03 +related: [qa-environments-browser-console-capture-gaps, qa-process-completion-claims, qa-bug-reports-reproducible-reports] +--- + +# Cropping a Screenshot to a Single Page Element + +## When this applies + +A browser-automation tool (Playwright, an `aside repl` session, a CDP script) +needs a close-up image of one element for a bug report, QA screenshot, or +visual diff — not the full page. Also when composing `page.screenshot({ +clip })` by hand from a `boundingBox()` reading, or when a saved crop shows +the wrong region of the page even though the coordinates you passed look +correct. + +## Do this + +1. **Reach for the element-screenshot primitive first**: `locator.screenshot({ + path })`. The tool computes and applies the clip itself from the element's + own geometry — fewer moving parts than composing coordinates by hand. +2. **When you must compose `clip` from `boundingBox()` yourself**, take the + `boundingBox()` reading immediately before the screenshot call, on the same + scroll position: the box is relative to the main frame viewport and + scrolling changes it (`x`/`y` can go negative). For a `fullPage: true` + capture, the clip is measured on the full scrollable page, so add the + current scroll offset (`window.scrollX`/`scrollY`) to a viewport-relative + box. +3. **Read back the first crop before producing a batch.** Open the saved PNG + and confirm it shows the intended element before looping the same pattern + over several more elements in one session — a systematic clip bug shows + the same wrong region on every subsequent crop, so the first check catches + the whole batch's defect at the cost of one look. +4. **When a crop shows the wrong region regardless of which coordinates you + pass**, stop varying the coordinates and take a `fullPage: true` (or plain + viewport) screenshot instead, then reference the element's region by its + `boundingBox()` coordinates in the report text — the clip pipeline itself + is the broken part, not the numbers fed into it. + +## Edge cases + +| Case | Then | +|------|------| +| Element requires scrolling into view first | Scroll it into view, then read `boundingBox()` — a reading taken before the scroll describes a viewport position that no longer matches | +| Element is a scrollable container | `locator.screenshot()` captures only the currently-scrolled content inside it, not the container's full scrollable content — state that scope next to the image | +| The automation tool wraps Playwright without exposing `locator.screenshot()` (only a generic `page.screenshot({ clip })`) | Confirm the wrapper's `clip` behaves like Playwright's on one known element before trusting a batch of crops through it | +| Several clip calls in one session all land on the same wrong region (near the page's top-left) | Treat it as a tool-side coordinate bug, not a per-call mistake — switch every remaining crop in that session to full-page capture | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Compute `page.screenshot({ clip: boundingBox })` for one element and trust the first result | Call `locator.screenshot({ path })` on that element directly | The tool derives and applies the clip itself instead of you composing raw viewport coordinates by hand | +| Produce a batch of clip screenshots without opening any of them | Open the first saved PNG before producing the rest | A systematic clip bug repeats identically on every later crop; one check catches it before the batch is wasted | +| Keep retrying with new coordinates when a clip result shows the wrong region every time | Fall back to a full-page or viewport screenshot and cite the element's region by its `boundingBox()` coordinates in the report | Coordinate correctness does not fix the output when the clip pipeline itself is the broken part | + +## Sources + +- https://playwright.dev/docs/screenshots#element-screenshot — element screenshots use `await page.locator('.header').screenshot({ path: 'screenshot.png' });` +- https://playwright.dev/docs/api/class-page — `clip` is "An object which specifies clipping of the resulting image," with `x`/`y` as "top-left corner of clip area"; `fullPage`: "When true, takes a screenshot of the full scrollable page, instead of the currently visible viewport. Defaults to false." +- https://playwright.dev/docs/api/class-locator — `locator.boundingBox()`: "The bounding box is calculated relative to the main frame viewport - which is usually the same as the browser window" and "Scrolling affects the returned bounding box... x and/or y may be negative"; `locator.screenshot()`: "captures a screenshot of the page, clipped to the size and position of a particular element matching the locator... If the element is a scrollable container, only the currently scrolled content will be visible on the screenshot." +- Field evidence 2026-09-01 (repo t1-visual, `aside repl`, Aside CLI 1.26.831.1513 / 1.26.810.1915): composing `page.screenshot({ path, clip: boundingBox })` from a locator's `boundingBox()` reading, 5 separate clip calls in one session (rail/roster/matrix/thread/right-column crops) each saved an image of the wrong region — `.panel--roster` at `{x:1081,y:139,w:359,h:761}` produced the rail+artifacts region instead — while every `fullPage: true` capture in the same session was correct; two subagents and the author read the files back independently. Not reproduced outside that session diff --git a/wiki/qa/index.md b/wiki/qa/index.md index 18a4e9a..2727ac0 100644 --- a/wiki/qa/index.md +++ b/wiki/qa/index.md @@ -21,7 +21,7 @@ Match your situation to a "load when" line; load only matching pages. | [scope-purity-checks](process/scope-purity-checks.md) | Proving a change/session/agent run touched nothing outside an allowed path set by filtering `git status --porcelain` output; a purity gate flags `?? dir/` for a directory that is wholly in scope; writing such a gate for an orchestration/CI workflow; a purity check in a permanent test suite fails on unrelated uncommitted files | | [defect-class-resweep-after-review](process/defect-class-resweep-after-review.md) | Addressing review findings when the remediation itself adds code (a new function, branch, call site, formatter); about to hand that diff to a verifier, an audit, or CI as "review comments addressed"; deciding whether a bot's acknowledged/resolved thread means the class is closed; reporting fix completeness as a class-with-method count rather than a per-finding list | | [completion-claims](process/completion-claims.md) | About to report work as done, fixed, or passing — to a human, a coordinator, a PR, or a commit message; about to write "should work" or "tests pass" without a fresh run; deciding what evidence a completion claim requires; relaying a subagent's success report | -| [evaluating-review-feedback](process/evaluating-review-feedback.md) | Review findings arrived (human, bot, or reviewer agent) and you are deciding what to implement; a finding is unclear; a reviewer proposes robustness or features nothing uses; you disagree with a finding and are deciding how to respond | +| [evaluating-review-feedback](process/evaluating-review-feedback.md) | Review findings arrived (human, bot, or reviewer agent) and you are deciding what to implement; a finding is unclear; a reviewer proposes robustness or features nothing uses; you disagree with a finding and are deciding how to respond; a CI fact-checking reviewer rules a claim "fabricated" because it cites a preview-gated or environment-local tool the runner cannot see | | [adversarial-change-review](process/adversarial-change-review.md) | Reviewing a diff ≥50 changed lines or one touching auth/payments/migrations/external APIs; a checklist review found nothing on a high-blast-radius change; deciding review depth for a risky diff; constructing failure scenarios (assumption violation, composition, cascade, abuse) | | [llm-review-pipelines](process/llm-review-pipelines.md) | Building or configuring an automated LLM code-review pipeline (CI review bot, review skill, PR-reviewer agent); review token cost grows with changeset size; deciding which stages run as deterministic code vs model judgment; keeping review rules as matchable data not prompt prose; model-written comments land on wrong lines; measuring a precision-vs-recall threshold change | @@ -50,6 +50,7 @@ Match your situation to a "load when" line; load only matching pages. | [test-environment-parity](environments/test-environment-parity.md) | A bug reproduces only in production; planning what a staging environment must mirror; deciding whether a staging pass clears a release | | [headless-browser-bot-blocking](environments/headless-browser-bot-blocking.md) | QA/dogfooding an external production site through a headless browser shows an intact page shell but empty lists/data with generic "temporary delay" toasts; deciding between "their server is down" and "our client is classified as a bot"; data APIs alone return 4xx while static assets load | | [browser-console-capture-gaps](environments/browser-console-capture-gaps.md) | About to read a browser-automation tool's console output as a QA verdict ("no errors", "the script never ran"); the collected list is empty for a page known to log on load; deciding when the collector attaches and whether to clear+reload; judging whether a browser extension's content script executed when its logs never appear | +| [element-crop-screenshots](environments/element-crop-screenshots.md) | About to save a close-up screenshot of one element for a bug report or visual diff; composing `page.screenshot({ clip })` by hand from a `boundingBox()` reading; a saved crop shows the wrong region of the page even though the coordinates look correct; deciding whether to batch several element crops in one session | ## bug-reports diff --git a/wiki/qa/process/adversarial-change-review.md b/wiki/qa/process/adversarial-change-review.md index b4c5a59..5f137a0 100644 --- a/wiki/qa/process/adversarial-change-review.md +++ b/wiki/qa/process/adversarial-change-review.md @@ -7,7 +7,7 @@ confidence: field-tested sources: - https://github.com/EveryInc/compound-engineering-plugin last_verified: 2026-08-22 -related: [qa-process-evaluating-review-feedback, qa-process-regression-scope, qa-process-post-release-verification] +related: [qa-process-evaluating-review-feedback, qa-process-regression-scope, qa-process-post-release-verification, infrastructure-agent-orchestration-inbound-validation-ownership-in-task-decomposition] --- # Constructing Failure Scenarios for a High-Risk Diff diff --git a/wiki/qa/process/evaluating-review-feedback.md b/wiki/qa/process/evaluating-review-feedback.md index 9fc15b1..bada0fd 100644 --- a/wiki/qa/process/evaluating-review-feedback.md +++ b/wiki/qa/process/evaluating-review-feedback.md @@ -8,7 +8,8 @@ sources: - https://github.com/obra/superpowers - https://google.github.io/eng-practices/review/reviewer/standard.html - https://google.github.io/eng-practices/review/reviewer/looking-for.html -last_verified: 2026-08-27 + - https://github.com/choiyounggi/dev-loop/pull/164 +last_verified: 2026-09-03 related: [qa-process-defect-class-resweep-after-review, qa-process-adversarial-change-review, qa-process-llm-review-pipelines] --- @@ -56,6 +57,7 @@ robustness or supporting features; you disagree with a finding. | A reviewer agent flagged pre-existing code outside the diff | Verify it, then file it as separate work — expanding the current change silently mixes concerns for every later reader | | The finding sits in the review **body** rather than an inline comment, so it quotes code without naming a file | Step 2 has no cited lines to open: grep the quoted string across the whole changed set before ruling on it, and rule only against the file the grep resolves it to | | The quoted code does not match the file you assumed, and sibling files implement the same contract | Read it as "not yet located", not as a false positive — the usual shape is that one sibling was already fixed and another still carries the defect, so the quote matches the file you did not check | +| A CI fact-checking review agent rules a claim "fabricated" because it cites a preview-gated or environment-local tool the runner cannot see | Split the verdict: accept the verifiability half (downgrade confidence to the experience tier, add public fetchable URLs, condition the directive on the tool being present in the session's roster) and refute the existence half with ground-truth evidence (roster listing, on-disk payload) in a PR comment | ## Instead of @@ -65,6 +67,7 @@ robustness or supporting features; you disagree with a finding. | Reply "You're absolutely right!" and start editing | Verify the claim against the code, then let the fix speak | Performative agreement commits you before verification, and adds noise for the next reader | | Build the "proper" version a reviewer sketched | Grep for real usage first; propose removal when unused | Unused robustness is dead weight that still has to be maintained and reviewed | | Rebut an unanchored finding because the file you assumed does not contain the quoted code | Grep the quoted string across the changed set, then rule against the file it resolves to | A body-level finding names no file; rebutting from the wrong file rejects a real defect with an argument that looks verified | +| Fight a CI reviewer's whole "fabricated" verdict, or weaken true content to satisfy it | Accept the verifiability half and refute only the existence half with ground-truth evidence in the reply | The reviewer's environment is a different trust domain — absence there is not absence everywhere, but an un-fetchable source still fails a sourcing gate on its own terms | ## Sources @@ -72,3 +75,5 @@ robustness or supporting features; you disagree with a finding. - https://google.github.io/eng-practices/review/reviewer/standard.html — technical facts and data overrule opinions and personal preferences - https://google.github.io/eng-practices/review/reviewer/looking-for.html — reviewers guard against over-engineering: solve the known problem, not the speculated future one - Field measurement 2026-08-19 (PR #327, round 16): a bot's body-level finding quoted `label = r.get("key") if _nonempty_str(...)` with no file. The assumed file, `fill_plan.py:307`, already wrapped the call as `_label(x.get("key"))`, which read as a false positive. Grepping the quoted shape across the sibling modules resolved it to `report.py:393/416/425`, an exact match and a real defect — the two files implement the same contract and only one had been fixed +- https://github.com/choiyounggi/dev-loop/pull/164 — "docs(wiki): mandatory design-skill routing for visual-design deliverables" (merged, public); the PR whose agent gate flipped fail→pass after the split-verdict remediation +- Field evidence 2026-08-30 (dev-loop PR #164, commit `f5d2395` "fix(wiki): remediate agent-gate Check 3 on design-canvas-workflow"): the gate's "skill does not exist" finding was refuted with the authoring session's skill roster (the reviewer had conflated the preview `design` skill with `/design-sync`) while the un-fetchable-source half was accepted by downgrading confidence and adding public URLs; the gate passed on the next run diff --git a/wiki/qa/process/llm-review-pipelines.md b/wiki/qa/process/llm-review-pipelines.md index 82bb5ce..ffd598e 100644 --- a/wiki/qa/process/llm-review-pipelines.md +++ b/wiki/qa/process/llm-review-pipelines.md @@ -8,7 +8,7 @@ sources: - https://github.com/alibaba/open-code-review - https://github.com/alibaba/open-code-review/blob/main/skills/open-code-review/SKILL.md last_verified: 2026-08-24 -related: [qa-process-evaluating-review-feedback] +related: [qa-process-evaluating-review-feedback, backend-common-llm-vendor-benchmark-claims-for-an-llm-tool] --- # Structuring an Automated LLM Code-Review Pipeline diff --git a/wiki/testing/index.md b/wiki/testing/index.md index 4e65ac9..540ccdd 100644 --- a/wiki/testing/index.md +++ b/wiki/testing/index.md @@ -36,7 +36,7 @@ Match your situation to a "load when" line; load only matching pages. | [write-path-assertions](quality/write-path-assertions.md) | Writing an HTTP-level test for an endpoint that persists something (form submit, create/update, onboarding step) and choosing what to assert beyond the status code; such a test is green while the records are empty or defaulted; sending repeated form fields from a client (httpx/TestClient) and deciding the `data=` shape | | [default-values-under-test](quality/default-values-under-test.md) | A constructor/factory/config default is named as a number in a spec, plan, or measurement record (`ttl_s=600`, `max_tokens=256`, `retries=3`) and you are judging whether it is guarded; a mutation of that default left the suite green; deciding between a mechanism test that passes the value in and a test of the shipped default; choosing the boundary cases and the two mutation directions for a default | | [signed-link-verification-assertions](quality/signed-link-verification-assertions.md) | Choosing the assertions for code that builds a URL carrying its own credential (approval or magic link with `?t=`, object-storage presigned URL, webhook callback, unsubscribe link); such a test is green while the link the product sends is rejected by its receiver; deciding between asserting the parameter's presence, running the token through the production verifier, and issuing the real request; pinning a token to both its key and its subject | -| [minimum-case-set](quality/minimum-case-set.md) | Writing tests for a function/endpoint/change and choosing which cases to cover; reviewing whether coverage suffices; picking boundary values by input type; adding a regression test for a bug fix | +| [minimum-case-set](quality/minimum-case-set.md) | Writing tests for a function/endpoint/change and choosing which cases to cover; reviewing whether coverage suffices; picking boundary values by input type; adding a regression test for a bug fix; a task's Steps describe a resilience/edge-case guarantee that is not in the plan's enumerated Verify list | | [behavior-not-implementation](quality/behavior-not-implementation.md) | Deciding what a test should assert; a behavior-preserving refactor broke tests; tempted to expose privates for testing; deciding whether a snapshot test is appropriate; a value-preserving move of a literal into config needs a test that detects re-inlining without asserting an internal | | [tests-that-cannot-fail](quality/tests-that-cannot-fail.md) | Reviewing tests that always pass; a bug shipped through an area the suite reported as covered; auditing a suspiciously green suite; judging whether an assertion, error-path test, or mock-based test can actually detect a defect; a shell-test (bats) assertion passes mid-test on code it should fail | | [checks-that-cannot-pass](quality/checks-that-cannot-pass.md) | Authoring a check whose target does not exist yet (grep/regex gate on an unwritten file or doc section, lint/scan rule, schema assertion on an unbuilt endpoint, a plan's verification command) and it has only ever been observed failing; reviewing a plan's gates before adopting them; separating "target missing" from "content missing" in a gate's exit status. This owns unwritten-target gate patterns for spec/RFC documents too — qa/document-verification does not | diff --git a/wiki/testing/quality/minimum-case-set.md b/wiki/testing/quality/minimum-case-set.md index 5492ee6..c382c61 100644 --- a/wiki/testing/quality/minimum-case-set.md +++ b/wiki/testing/quality/minimum-case-set.md @@ -8,7 +8,7 @@ sources: - https://martinfowler.com/articles/practical-test-pyramid.html - https://abseil.io/resources/swe-book/html/ch12.html - https://martinfowler.com/bliki/TestDrivenDevelopment.html -last_verified: 2026-07-10 +last_verified: 2026-09-03 related: [testing-strategy-test-level-choice, testing-quality-behavior-not-implementation, testing-quality-checks-that-cannot-pass, testing-quality-unasserted-return-fields, testing-quality-value-preserving-refactor-assertions, qa-exploratory-guard-true-path-coverage, testing-quality-default-values-under-test] --- @@ -58,6 +58,7 @@ sufficient. | The boundary is unreachable through the public interface (guarded upstream) | Test at the level that owns the guard; do not force unreachable inputs through the unit ([testing-quality-behavior-not-implementation]) | | Reviewing a change that only adds a case to existing behavior | Require the new case's test; the existing normal/error/boundary set stands | | The observable outcome is a composite of several computed fields | "The return value" is then a set of fields plus the invariants binding them — cover the fields and assert the relations ([testing-quality-unasserted-return-fields]) | +| A task's Steps prose states a resilience guarantee ("one item's failure must not halt the batch") that is not among the plan's enumerated Verify cases | Write it as its own behavior — one normal case, one error case — before treating the enumerated list as complete; the list is a floor, not a ceiling | ## Instead of @@ -67,9 +68,12 @@ sufficient. | Assert only `toThrow()` with no error type | Assert the error type and the message/code callers depend on | The test passes when the wrong error fires for the wrong reason | | Fix a bug and add the test after the fix, never seeing it fail | Write the reproducing test first, watch it fail, then fix | A test that never failed can pass vacuously and guard nothing | | Chase a coverage percentage by touching lines without assertions | Add normal/error/boundary cases per behavior | Line coverage without outcome assertions detects nothing | +| Treat a plan's enumerated Verify cases as the full required coverage | Test every behavior the task's Steps describe, named or not, before calling a test-quality audit | An auditor grades against everything the diff changes and the Steps describe, not only the plan's named case list | ## Sources - https://martinfowler.com/articles/practical-test-pyramid.html — test one condition per test; cover happy path and edge cases - https://abseil.io/resources/swe-book/html/ch12.html — test behaviors (guarantees), not methods; clear assertions per behavior - https://martinfowler.com/bliki/TestDrivenDevelopment.html — write the failing test first (red–green–refactor) +- https://abseil.io/resources/swe-book/html/ch12.html — "A behavior is any guarantee that a system makes about how it will respond to a series of inputs while in a particular state" — a guarantee stated in Steps prose is a behavior whether or not the Verify list names it +- Field evidence 2026-08-29 (repo wt-t4-event-push, task 03 reminder-sweep): the task's Steps stated "전체 try/catch — 한 사용자의 실패가 스윕 전체를 멈추지 않게 할 것" (one user's failure must not stop the whole sweep), not among the plan's 6 enumerated Verify cases; the first test-quality-auditor call returned VERDICT: FAIL for exactly this missing case, and adding one test (two members, one's `prefs.get` throws) then re-running the auditor returned VERDICT: PASS