governance: flaky_governance 参数落地 + 隔离到期回炉 sweep（P2-9，ADR-0043） - #122
Merged
Conversation
|
Warning Review limit reached
Next review available in: 4 minutes Limit details: You’ve used all 10 included reviews currently available. Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
Comment |
PR Summary by Qodo落地 flaky 测试治理参数与隔离到期扫描
AI Description
Diagram
High-Level Assessment
Files changed (3)
|
Code Review by Qodo
1. Manifest content never parsed
|
Comment on lines
+49
to
+50
| api -X POST "https://api.github.com/repos/$ORG/$r/issues" -d "$(jq -n --arg t "$TITLE" --arg b "$BODY" '{title:$t,body:$b,labels:["flaky-quarantine"]}')" >/dev/null 2>&1 || \ | ||
| api -X POST "https://api.github.com/repos/$ORG/$r/issues" -d "$(jq -n --arg t "$TITLE" --arg b "$BODY" '{title:$t,body:$b}')" >/dev/null |
There was a problem hiding this comment.
2. flaky-sweep opens forbidden issues 📘 Rule violation § Compliance
The new automation reports quarantine violations by creating repository issues, but the automation standard permits machine feedback only through failed check runs or ordinary PR comments. This introduces a feedback channel outside the documented bot standards.
Agent Prompt
## Issue description
The sweep creates standalone GitHub issues even though the documented automation feedback standard limits bots to failed check runs or ordinary PR comments.
## Issue Context
Replace automatic issue creation with an approved feedback mechanism, or update the automation standard through the required governance process before using standalone issues as an escalation channel.
## Fix Focus Areas
- governance/flaky-sweep.sh[47-51]
- standards/automation/bot-channels.md[3-18]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+22
to
+25
| OUT=$(python3 - "$r" "$MAX_DAYS" <<'PYEOF' | ||
| import sys, yaml, datetime | ||
| repo, max_days = sys.argv[1], int(sys.argv[2]) | ||
| d = yaml.safe_load(sys.stdin) or {} |
There was a problem hiding this comment.
3. Manifest content never parsed 🐞 Bug ≡ Correctness
python3 - consumes the heredoc as its program, leaving sys.stdin unavailable for yaml.safe_load; the outer here-string does not provide a separate data stream to the Python program. Consequently every valid quarantine manifest is treated as empty, so expired and malformed entries are never detected.
Agent Prompt
## Issue description
The embedded Python program and quarantine YAML both attempt to use stdin. Ensure Python receives the program separately from the manifest data, and propagate parser failures.
## Issue Context
`python3 -` reads the heredoc from stdin as source code, so `yaml.safe_load(sys.stdin)` cannot then read the quarantine manifest supplied outside the command substitution.
## Fix Focus Areas
- governance/flaky-sweep.sh[22-43]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+30
to
+33
| if not e.get(k): | ||
| malformed.append(f"{e} 缺 {k}") | ||
| try: | ||
| exp = datetime.date.fromisoformat(str(e["expires"])) |
There was a problem hiding this comment.
4. Malformed manifests falsely pass 🐞 Bug ☼ Reliability
The parser assumes both the document and every quarantined entry are mappings, then accesses missing required fields after merely recording them. Scalar/list documents, non-mapping entries, or entries missing expires raise uncaught exceptions; without set -e or an explicit status check, the shell continues with empty output and can exit successfully.
Agent Prompt
## Issue description
Validate the top-level YAML type, the `quarantined` collection type, and each entry before accessing fields. Treat every parsing or schema exception as a counted fail-closed violation.
## Issue Context
The current code calls mapping methods and indexes required keys without validating types or stopping after a missing-key finding. Command-substitution failure is not checked because the script omits `set -e`.
## Fix Focus Areas
- governance/flaky-sweep.sh[8-8]
- governance/flaky-sweep.sh[22-43]
- governance/flaky-sweep.sh[55-56]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+51
to
+52
| echo "ISSUE repo '$r': 过期 $(grep -c . <<<"$EXP_LIST" || true) / 违规 $(grep -c . <<<"$MAL_LIST" || true)" | ||
| ISSUES=$((ISSUES+1)); EXPIRED=$((EXPIRED+$(grep -c . <<<"$EXP_LIST" || true))) |
There was a problem hiding this comment.
5. Structural violations exit green 🐞 Bug ≡ Correctness
Parsed structural violations populate MAL_LIST, but line 52 only increments EXPIRED; MALFORMED is reserved for content-fetch failures. A repository containing only invalid quarantine entries is therefore reported as 违规仓=0 and the workflow exits successfully.
Agent Prompt
## Issue description
Increment the malformed repository counter whenever `MAL_LIST` is non-empty and ensure the final status fails for structural violations.
## Issue Context
The parser emits structural findings into `MAL_LIST`, but only fetch failures currently increment the counter checked at script exit.
## Fix Focus Areas
- governance/flaky-sweep.sh[44-56]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+49
to
+50
| api -X POST "https://api.github.com/repos/$ORG/$r/issues" -d "$(jq -n --arg t "$TITLE" --arg b "$BODY" '{title:$t,body:$b,labels:["flaky-quarantine"]}')" >/dev/null 2>&1 || \ | ||
| api -X POST "https://api.github.com/repos/$ORG/$r/issues" -d "$(jq -n --arg t "$TITLE" --arg b "$BODY" '{title:$t,body:$b}')" >/dev/null |
There was a problem hiding this comment.
6. Failed issue posts look successful 🐞 Bug ☼ Reliability
The API wrapper uses curl without --fail-with-body, so HTTP 4xx/5xx responses return success and the unlabeled fallback is not attempted; transport failures from both attempts are also ignored because set -e is disabled. The script then unconditionally prints ISSUE and increments ISSUES, even when no escalation issue exists.
Agent Prompt
## Issue description
Make the API helper fail on non-2xx responses, validate the issue response, and only increment `ISSUES` after confirmed creation. If both labeled and unlabeled creation fail, record a failure that makes the sweep exit nonzero.
## Issue Context
Plain `curl -sS` treats GitHub HTTP errors as successful transfers, defeating the `||` fallback and allowing issue-creation failures to be reported as successes.
## Fix Focus Areas
- governance/flaky-sweep.sh[12-12]
- governance/flaky-sweep.sh[49-56]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+46
to
+49
| if [[ -n "$EXP_LIST" || -n "$MAL_LIST" ]]; then | ||
| TITLE="[flaky] $r 隔离清单待处置（过期回炉/结构违规，ADR-0043）" | ||
| BODY="flaky-sweep 每日检测（ADR-0043）：\n\n## 过期条目（已自动回炉——过期隔离不豁免，须修复测试或走 ADR 重新隔离）\n${EXP_LIST:--}\n\n## 结构违规\n${MAL_LIST:--}\n\n处置：修复测试后经 PR 移除条目（引用 ADR）；或新 ADR 重新隔离。" | ||
| api -X POST "https://api.github.com/repos/$ORG/$r/issues" -d "$(jq -n --arg t "$TITLE" --arg b "$BODY" '{title:$t,body:$b,labels:["flaky-quarantine"]}')" >/dev/null 2>&1 || \ |
There was a problem hiding this comment.
7. Daily sweep duplicates issues 🐞 Bug ☼ Reliability
Every run with a persistent expired or malformed entry unconditionally creates another issue without looking for an existing sweep-owned issue. Because the workflow runs daily, unresolved violations will produce one duplicate issue per repository per day.
Agent Prompt
## Issue description
Use a dedicated label or marker to find an existing open flaky-sweep issue and update or comment on it instead of creating a duplicate. Optionally close the owned issue when the repository becomes healthy.
## Issue Context
The scheduled workflow runs every day, while the script's only issue lifecycle operation is an unconditional POST whenever findings remain.
## Fix Focus Areas
- .github/workflows/flaky-sweep.yml[4-7]
- governance/flaky-sweep.sh[46-50]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
摘要
P2-9（#94 / ADR-0043）治理侧：
testing.yaml：T-08 flaky_governance 工具接入（flaky-retry.sh + flaky-sweep）+flaky_governance参数段（retry_max=2 / 窗口 30 天 / 阈值 3 次 / 隔离 ≤30 天）——CI-Workflows flaky-retry.sh 的拉取真源governance/flaky-sweep.sh：每日扫描受管仓tests/quarantine.yaml——结构校验（test/owner/expires/adr + ≤30 天）+ 过期条目开升级 issue（过期隔离不豁免=自动回炉执法）.github/workflows/flaky-sweep.yml：每日 02:30（GOVERNANCE_TOKEN，fail-closed）验证
C1：governance/ + .github/ + policy 路径，ADR-0043 背书。