Skip to content

fix(sync): close docs drift review cluster - #3167

Merged
stranske merged 4 commits into
mainfrom
automation/docs-drift-followup-2
Aug 22, 2026
Merged

stranske merged 4 commits into
mainfrom
automation/docs-drift-followup-2

Conversation

@stranske

@stranske stranske commented Aug 22, 2026

Copy link
Copy Markdown
Owner

Why

Portable canary PR #2233 exposed four source-owned review findings after the prior docs-drift repair. This successor fixes that cluster in Workflows source. Fresh exact-head review then found three additional migration and ignore-semantics defects; the current head includes those corrections too.

Scope

  • keep deterministic findings for the same target atomic across repair batches
  • deduplicate repair issues with stable per-finding identities, exclude already-covered findings, and recognize exact markers from the pre-atomic batch layout
  • ignore ordinary node_modules trees at every depth while re-including only the deliberately vendored .github/scripts/node_modules tree
  • safely replace a .gitignore containing only legacy node_modules rules
  • sync scripts/sync_status_file_ignores.py into consumers so the managed-block validation command exists
  • regenerate Renovate managed-path ownership for the manifest addition
  • keep root/template script copies exact and document the propagation contract

Source review threads

  • Portable#2233 PRRT_kwDOO15QxM6baR8L
  • Portable#2233 PRRT_kwDOO15QxM6baR8N
  • Portable#2233 PRRT_kwDOO15QxM6baR8P
  • Portable#2233 PRRT_kwDOO15QxM6baR8Q
  • Workflows#3167 PRRT_kwDOQprj9M6baV3E
  • Workflows#3167 PRRT_kwDOQprj9M6baYtY
  • Workflows#3167 PRRT_kwDOQprj9M6baYtZ

Validation

  • full suite: 4944 passed, 3 skipped, 3 expected xfails
  • focused source/template/manifest/docs surface: 196 passed
  • Ruff and Black pass
  • template completeness and exact-sync validators pass
  • generated Renovate ownership preset check passes
  • root/template script pairs are byte-identical
  • direct git check-ignore proof preserves the vendored tree and ignores ordinary root/nested installs
  • git diff --check passes

Rollout

This changes the sync manifest and coalesces into the unresolved full source rollout. After merge: no-filter Maint 68 full canary, exact-plan Maint 71 evidence, promotion, delivery reconciliation, Health 83, and fresh zero-owned inventory.

@stranske stranske added agent:codex Agent-created issues from Codex agent-high-privilege Grants elevated permissions for agent workflows codex codex-automation labels Aug 22, 2026
@coderabbitai

coderabbitai Bot commented Aug 22, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your current included review allowance is based on your included PR review attempts over the past 7 days.

Next review available in: 10 minutes

Limit details: You’ve used the included review currently available. Your 66 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 88406da2-e302-40b7-b3df-67f94f4c16a5

📥 Commits

Reviewing files that changed from the base of the PR and between 2192493 and 1a67501.

📒 Files selected for processing (11)
  • .github/sync-manifest.yml
  • .gitignore
  • docs/ops/LOCAL_LANES.md
  • renovate-presets/consumer-managed-paths.json
  • scripts/docs_drift_fix_agent.py
  • scripts/sync_status_file_ignores.py
  • templates/consumer-repo/.gitignore
  • templates/consumer-repo/scripts/docs_drift_fix_agent.py
  • templates/consumer-repo/scripts/sync_status_file_ignores.py
  • tests/scripts/test_docs_drift_fix_agent.py
  • tests/scripts/test_sync_status_file_ignores.py

Comment @coderabbitai help to get the list of available commands.

@stranske

Copy link
Copy Markdown
Owner Author

@codex review

Exact-head review requested for 7981324. Please verify all four Portable#2233 source findings are covered: legacy node_modules conflict removal, atomic duplicate-target batching, stable per-finding issue dedupe with partial coverage, and consumer availability of sync_status_file_ignores.py.

@stranske
stranske temporarily deployed to agent-high-privilege August 22, 2026 17:48 — with GitHub Actions Inactive
@agents-workflows-bot

Copy link
Copy Markdown
Contributor

🤖 Keepalive Loop Status

PR #3167 | Agent: Codex | Iteration 0/12

Current State

Metric Value
Iteration progress [----------] 0/12
Action stop (no-checklists)
Gate unknown
Tasks 0/0 complete
Timeout 45 min (default)
Timeout usage 0m elapsed (1%, 45m remaining)
Keepalive ✅ enabled
Autofix ❌ disabled

🔍 Failure Classification

| Error type | infrastructure |
| Error category | unknown |
| Suggested recovery | Capture logs and context; retry once and escalate if the issue persists. |

@agents-workflows-bot

agents-workflows-bot Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor
Keepalive Work Log (click to expand)
# Time (UTC) Agent Action Result Files Tasks Progress Commit Gate
0 2026-08-22 17:49:12 Codex stop (no-checklists) skipped 0 0/0
0 2026-08-22 17:49:48 Codex stop (no-checklists) skipped 0 0/0 cancelled
0 2026-08-22 17:52:31 Codex stop (no-checklists) skipped 0 0/0 cancelled
0 2026-08-22 17:59:43 Codex stop (no-checklists) skipped 0 0/0 cancelled
0 2026-08-22 18:06:08 Codex stop (no-checklists) skipped 0 0/0 failure
1 2026-08-22 18:26:12 Codex run (ready) success 32 file(s) 0 0/15 success
1 2026-08-22 18:43:53 Codex wait (gate-pending-transient) skipped 0 0/15
1 2026-08-22 18:53:07 Codex run (agent-run-failed) failure 1 file(s) 0 0/15 success

@agents-workflows-bot

Copy link
Copy Markdown
Contributor

🤖 Keepalive Loop Status

PR #3167 | Agent: Codex | Iteration 0/12

Current State

Metric Value
Iteration progress [----------] 0/12
Action stop (no-checklists)
Gate cancelled
Tasks 0/0 complete
Timeout 45 min (default)
Timeout usage 1m elapsed (3%, 44m remaining)
Keepalive ✅ enabled
Autofix ❌ disabled

🔍 Failure Classification

| Error type | infrastructure |
| Error category | unknown |
| Suggested recovery | Capture logs and context; retry once and escalate if the issue persists. |

@stranske

Copy link
Copy Markdown
Owner Author

@codex review

Fresh exact-head review requested after the Python 3.9/Ruff compatibility correction. Please review the complete cluster on the new head.

@agents-workflows-bot

Copy link
Copy Markdown
Contributor

🤖 Keepalive Loop Status

PR #3167 | Agent: Codex | Iteration 0/12

Current State

Metric Value
Iteration progress [----------] 0/12
Action stop (no-checklists)
Gate cancelled
Tasks 0/0 complete
Timeout 45 min (default)
Timeout usage 3m elapsed (9%, 42m remaining)
Keepalive ✅ enabled
Autofix ❌ disabled

🔍 Failure Classification

| Error type | infrastructure |
| Error category | unknown |
| Suggested recovery | Capture logs and context; retry once and escalate if the issue persists. |

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5cad59b6bc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/sync_status_file_ignores.py Outdated
Comment on lines +305 to +306
repo_lines = without_conflicts(lines)
merged = repo_lines + ([""] if repo_lines[-1].strip() else []) + block.splitlines()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Handle an empty list after removing legacy rules

When a consumer .gitignore contains only node_modules/ (or only repeated copies of that legacy rule), lines is nonempty but without_conflicts(lines) returns an empty list, so indexing repo_lines[-1] raises IndexError. The full-plan Maint 68 sync calls this function directly for each consumer, causing reconciliation to fail instead of replacing the legacy content with the managed block; choose the separator without indexing when repo_lines is empty.

Useful? React with 👍 / 👎.

@stranske

Copy link
Copy Markdown
Owner Author

@codex review

Fresh exact-head review requested after aligning the consumer fallback block version with template v3. The bounded fallback-review advisory is now covered by a regression assertion.

@agents-workflows-bot

Copy link
Copy Markdown
Contributor

🤖 Keepalive Loop Status

PR #3167 | Agent: Codex | Iteration 0/12

Current State

Metric Value
Iteration progress [----------] 0/12
Action stop (no-checklists)
Gate cancelled
Tasks 0/0 complete
Timeout 45 min (default)
Timeout usage 9m elapsed (21%, 36m remaining)
Keepalive ✅ enabled
Autofix ❌ disabled

🔍 Failure Classification

| Error type | infrastructure |
| Error category | unknown |
| Suggested recovery | Capture logs and context; retry once and escalate if the issue persists. |

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 46a9c24803

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .gitignore Outdated

# Node.js
node_modules/
/node_modules/

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve ignores for nested dependency installations

When dependencies are installed below the repository root, this anchored rule no longer ignores them; for example, .github/actions/verifier-codex-cli/package.json produces .github/actions/verifier-codex-cli/node_modules/, which now appears as a large untracked tree. The same regression reaches consumers because the full Maint 68 reconciliation deletes their exact node_modules/ rule and leaves only /node_modules/. Preserve the all-depth ignore and add a targeted exception for the deliberately vendored .github/scripts/node_modules subtree instead of narrowing every consumer's dependency ignore.

Useful? React with 👍 / 👎.

Comment thread scripts/docs_drift_fix_agent.py Outdated
Comment on lines +601 to +604
legacy_digest = hashlib.sha256(
f"{batch['issue_title']}\0{legacy_issue_body}".encode()
).hexdigest()[:16]
legacy_marker = f"<!-- docs-drift-fix-agent:{legacy_digest} -->"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Migrate legacy markers before regrouping findings

When open issues were created by the previous implementation and a repeated deterministic target crossed a batch boundary, the new atomic grouping changes the issue body. This legacy digest is calculated from that newly combined body, so it cannot match either old per-batch marker, while those old issues do not contain the new per-finding markers; every lookup therefore misses and --apply opens a duplicate issue for findings that are already covered. Recover the legacy markers from the previous batch composition or otherwise inspect legacy-marked open issues during this migration.

Useful? React with 👍 / 👎.

@stranske-keepalive

stranske-keepalive Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

🤖 Keepalive Loop Status

PR #3167 | Agent: Codex | Iteration 1/12

Current State

Metric Value
Iteration progress [#---------] 1/12
Action run (agent-run-failed)
Agent status ❌ AGENT FAILED
Gate success
Tasks 0/15 complete
Timeout 45 min (default)
Timeout usage 10m elapsed (23%, 35m remaining)
Keepalive ✅ enabled
Autofix ❌ disabled

Last Codex Run

Result Value
Status ❌ AGENT FAILED
Reason agent-run-failed
Exit code unknown
Failures 1/3 before pause

To retry immediately:

  • Add the agent:retry label to this PR

Or wait for the next successful Gate run to automatically retry.

🔍 Failure Classification

| Error type | infrastructure |
| Error category | transient |
| Suggested recovery | Capture logs and context; retry once and escalate if the issue persists. |

⚠️ Task Analysis Unavailable

Warning: LLM-powered task completion analysis did not run for this iteration.
This may be due to:

  • Missing analysis dependencies or scripts
  • Failed PR body fetch
  • Session data not captured

Task checkboxes may not be automatically updated. Please review manually.

⚠️ Failure Tracking

| Consecutive failures | 1/3 |
| Reason | agent-run-failed |

@stranske

Copy link
Copy Markdown
Owner Author

@codex review

Fresh exact-head review requested for 1a67501 after implementing all three active acceptance criteria: empty legacy-only .gitignore handling, all-depth node_modules ignores with a targeted vendored-tree exception, and reconstruction of pre-atomic legacy batch markers. Full local suite: 4944 passed, 3 skipped, 3 expected xfails.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: 1a675019d6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@agents-workflows-bot

Copy link
Copy Markdown
Contributor

🤖 Bot Comment Handler

  • Agent: codex
  • Bot comments to address: 3

The agent has been assigned to this PR to address the bot review comments.

Instructions for agent

  1. Implement suggested fixes that improve the code
  2. Skip suggestions that don't apply (note why in your response)

The bot comment handler workflow has prepared context in the artifacts.

@stranske
stranske merged commit f49a694 into main Aug 22, 2026
53 checks passed
@stranske
stranske deleted the automation/docs-drift-followup-2 branch August 22, 2026 18:26
@stranske stranske added the verify:compare Compare multiple LLM evaluations label Aug 22, 2026
@stranske
stranske temporarily deployed to agent-high-privilege August 22, 2026 18:43 — with GitHub Actions Inactive
@github-actions

Copy link
Copy Markdown
Contributor

Provider Comparison Report

Provider Summary

Provider Model Verdict Confidence Summary
openai gpt-5.6-terra PASS 90% The changes address the documented repair cluster: docs-drift findings gain stable deterministic identity and pre-existing marker recognition for idempotent issue batching; the status-file ignore s...
anthropic claude-sonnet-5 PASS 62% The PR's file-level changes map cleanly onto the stated task list: sync-manifest gains an exact-sync entry for scripts/sync_status_file_ignores.py, the Renovate consumer-managed-paths preset is reg...
📋 Full Provider Details (click to expand)

openai

  • Model: gpt-5.6-terra
  • Verdict: PASS
  • Confidence: 90%
  • Scores:
    • Correctness: 9.0/10
    • Completeness: 9.0/10
    • Quality: 9.0/10
    • Testing: 8.0/10
    • Risks: 9.0/10
  • Summary: The changes address the documented repair cluster: docs-drift findings gain stable deterministic identity and pre-existing marker recognition for idempotent issue batching; the status-file ignore synchronizer is added to the manifest and consumer template; Renovate ownership is regenerated for that newly managed script; and documentation records the propagation contract. The node_modules ignore rules use an all-depth ignore pattern with explicit root-anchored re-inclusions for only .github/scripts/node_modules and its contents, which preserves the intended vendored dependency tree while ignoring ordinary root and nested installations. The managed ignore-block logic is expanded to replace legacy node_modules-only configurations safely. Root/template copies are represented as exact-sync managed assets, and focused tests were added for both docs-drift behavior and ignore-block migration behavior. The implementation is readable, bounded in scope, and carries low compatibility or security risk.

anthropic

  • Model: claude-sonnet-5
  • Verdict: PASS
  • Confidence: 62%
  • Scores:
    • Correctness: 7.0/10
    • Completeness: 7.0/10
    • Quality: 7.0/10
    • Testing: 7.0/10
    • Risks: 6.0/10
  • Summary: The PR's file-level changes map cleanly onto the stated task list: sync-manifest gains an exact-sync entry for scripts/sync_status_file_ignores.py, the Renovate consumer-managed-paths preset is regenerated (214→215, path added), .gitignore is updated with negation rules to re-include the vendored .github/scripts/node_modules tree while ignoring node_modules at every other depth, the template .gitignore is rewritten to replace legacy-only rules, and the template copies of docs_drift_fix_agent.py and sync_status_file_ignores.py mirror the source exactly (identical +175/-40 diff stats for the shared script, indicating an exact-sync contract is honored). New tests were added for both test_docs_drift_fix_agent.py and test_sync_status_file_ignores.py, aligning with the described dedup/marker-recognition and ignore-semantics fixes. Given the diff was truncated for the core logic file and the full CI test-count assertions from acceptance criteria weren't independently verifiable in this context, some correctness details rest on trust in the reported CI success (selftest-ci and pr-11-ci-smoke both green). No structural gaps or missing task areas were identified from the diff summary, so this merits a PASS with moderate confidence, acknowledging documentation/testing verification limits.
  • Concerns:
    • The full diff for scripts/docs_drift_fix_agent.py (the core logic implementing deterministic findings, dedup, and marker recognition) is truncated in the review context, so the correctness of the dedup/identity logic itself could not be directly verified from the diff alone.
    • Cannot independently confirm the specific acceptance-criteria numeric claims (e.g., '4944 passed, 3 skipped, 3 xfails', '196 passed focused suite') from the diff; these rely on CI logs not fully inspectable here, though CI workflows referenced did report success.
    • The .gitignore rewrite relies on negation patterns (!/.github/scripts/node_modules/ and /**) which is a known-correct git idiom for re-including a nested ignored path, but interacts with parent-directory ignore rules in subtle ways; the PR states a 'direct git check-ignore proof' was used, which is not visible in this diff excerpt.
    • Template .gitignore diff (+8/-10) differs in shape from root .gitignore diff (+3/-1), consistent with 'safely replacing legacy-only node_modules rules' but not independently verified line-by-line here.

Agreement

  • Verdict: PASS (all providers)
  • Testing: scores within 1 point (avg 7.5/10, range 7.0-8.0)

Disagreement

Dimension openai anthropic
Correctness 9.0/10 7.0/10
Completeness 9.0/10 7.0/10
Quality 9.0/10 7.0/10
Risks 9.0/10 6.0/10

Unique Insights

  • openai: The changes address the documented repair cluster: docs-drift findings gain stable deterministic identity and pre-existing marker recognition for idempotent issue batching; the status-file ignore synchronizer is added to the manifest and consumer template; Renovate ownership is regenerated for th...
  • anthropic: The full diff for scripts/docs_drift_fix_agent.py (the core logic implementing deterministic findings, dedup, and marker recognition) is truncated in the review context, so the correctness of the dedup/identity logic itself could not be directly verified from the diff alone.; Cannot independently confirm the specific acceptance-criteria numeric claims (e.g., '4944 passed, 3 skipped, 3 xfails', '196 passed focused suite') from the diff; these rely on CI logs not fully inspectable here, though CI workflows referenced did report success.; The .gitignore rewrite relies on negation patterns (!/.github/scripts/node_modules/ and /**) which is a known-correct git idiom for re-including a nested ignored path, but interacts with parent-directory ignore rules in subtle ways; the PR states a 'direct git check-ignore proof' was used, which is not visible in this diff excerpt.; Template .gitignore diff (+8/-10) differs in shape from root .gitignore diff (+3/-1), consistent with 'safely replacing legacy-only node_modules rules' but not independently verified line-by-line here.

🔍 LangSmith Traces

@github-actions

github-actions Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Runner dispatch state for codex on PR #3167. Do not edit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent:codex Agent-created issues from Codex agent-high-privilege Grants elevated permissions for agent workflows codex codex-automation verify:compare Compare multiple LLM evaluations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants