Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .github/workflows/flaky-sweep.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: flaky-sweep
# P2-9(ADR-0043):隔离清单到期回炉检测——过期隔离不豁免 + 升级 issue。
# 回炉执法在 check 侧(过期条目不参与豁免)+ 本 sweep 提示人工按 ADR 移除条目。
on:
schedule:
- cron: "30 2 * * *" # 每日(错开整点 drift 洪峰)
workflow_dispatch:

permissions: {}

concurrency:
group: flaky-sweep
cancel-in-progress: false

jobs:
sweep:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: 隔离清单到期扫描
env:
GH_TOKEN: ${{ secrets.GOVERNANCE_TOKEN }}
run: |
if [[ -z "$GH_TOKEN" ]]; then
echo "::error::缺 org secret GOVERNANCE_TOKEN" >&2; exit 2
fi
set -o pipefail
bash governance/flaky-sweep.sh | tee sweep-report.txt
56 changes: 56 additions & 0 deletions governance/flaky-sweep.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
#!/usr/bin/env bash
# flaky-sweep.sh —— 隔离清单到期回炉检测(P2-9,ADR-0043)
# 每日扫描全部受管仓的 tests/quarantine.yaml:
# - 条目结构校验(test/owner/expires/adr 齐备;expires ≤ quarantine_max_days)
# - expires 已过 → 该条目自动回炉(过期隔离不豁免)+ 开升级 issue(人工按 ADR 移除条目)
# - 清单拉取失败 = fail-closed
# 用法: GH_TOKEN=<org admin> bash flaky-sweep.sh(CI 由 flaky-sweep.yml 调度)
set -uo pipefail
ORG="${ORG:-Cloudbird-Software}"
DIR="$(cd "$(dirname "$0")" && pwd)"
MAX_DAYS=$(python3 -c "import yaml;print(yaml.safe_load(open('$DIR/policy/testing.yaml',encoding='utf-8'))['flaky_governance']['quarantine_max_days'])")
api() { curl -sS -H "Authorization: Bearer ${GH_TOKEN:?}" -H "Accept: application/vnd.github+json" "$@"; }
REPOS=$(python3 -c "import yaml;print(' '.join(r['name'] for r in yaml.safe_load(open('$DIR/REPOS.yaml',encoding='utf-8'))['repos'] if r.get('status')=='active'))")
ISSUES=0; EXPIRED=0; MALFORMED=0
for r in $REPOS; do
RESP=$(api "https://api.github.com/repos/$ORG/$r/contents/tests/quarantine.yaml")
CONTENT=$(jq -r '.content // empty' <<<"$RESP" | base64 -d 2>/dev/null)
if [[ -z "$CONTENT" ]]; then
jq -e '.message == "Not Found"' <<<"$RESP" >/dev/null 2>&1 && continue
echo "DRIFT repo '$r' tests/quarantine.yaml 读取失败(fail-closed,ADR-0043)"; MALFORMED=$((MALFORMED+1)); continue
fi
OUT=$(python3 - "$r" "$MAX_DAYS" <<'PYEOF'
import sys, yaml, datetime
repo, max_days = sys.argv[1], int(sys.argv[2])
d = yaml.safe_load(sys.stdin) or {}
Comment on lines +22 to +25

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

3. Manifest content never parsed 🐞 Bug ≡ Correctness

python3 - consumes the heredoc as its program, leaving sys.stdin unavailable for
yaml.safe_load; the outer here-string does not provide a separate data stream to the Python
program. Consequently every valid quarantine manifest is treated as empty, so expired and malformed
entries are never detected.
Agent Prompt
## Issue description
The embedded Python program and quarantine YAML both attempt to use stdin. Ensure Python receives the program separately from the manifest data, and propagate parser failures.

## Issue Context
`python3 -` reads the heredoc from stdin as source code, so `yaml.safe_load(sys.stdin)` cannot then read the quarantine manifest supplied outside the command substitution.

## Fix Focus Areas
- governance/flaky-sweep.sh[22-43]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

today = datetime.date.today()
expired, malformed = [], []
for e in d.get("quarantined", []):
for k in ("test", "owner", "expires", "adr"):
if not e.get(k):
malformed.append(f"{e} 缺 {k}")
try:
exp = datetime.date.fromisoformat(str(e["expires"]))
Comment on lines +30 to +33

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

4. Malformed manifests falsely pass 🐞 Bug ☼ Reliability

The parser assumes both the document and every quarantined entry are mappings, then accesses
missing required fields after merely recording them. Scalar/list documents, non-mapping entries, or
entries missing expires raise uncaught exceptions; without set -e or an explicit status check,
the shell continues with empty output and can exit successfully.
Agent Prompt
## Issue description
Validate the top-level YAML type, the `quarantined` collection type, and each entry before accessing fields. Treat every parsing or schema exception as a counted fail-closed violation.

## Issue Context
The current code calls mapping methods and indexes required keys without validating types or stopping after a missing-key finding. Command-substitution failure is not checked because the script omits `set -e`.

## Fix Focus Areas
- governance/flaky-sweep.sh[8-8]
- governance/flaky-sweep.sh[22-43]
- governance/flaky-sweep.sh[55-56]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

if (exp - today).days > max_days:
malformed.append(f"{e['test']} expires 距今超 quarantine_max_days={max_days}")
elif exp < today:
expired.append(f"{e['test']}(owner={e['owner']},过期于 {e['expires']},adr={e['adr']})")
except ValueError:
malformed.append(f"{e} expires 非法")
print("EXPIRED\n" + "\n".join(expired) if expired else "EXPIRED\n-")
print("MALFORMED\n" + "\n".join(malformed) if malformed else "MALFORMED\n-")
PYEOF
) <<<"$CONTENT"
EXP_LIST=$(sed -n '/^EXPIRED$/,/^MALFORMED$/p' <<<"$OUT" | sed '1d;$d' | grep -v '^-$' || true)
MAL_LIST=$(sed -n '/^MALFORMED$/,$p' <<<"$OUT" | sed '1d' | grep -v '^-$' || true)
if [[ -n "$EXP_LIST" || -n "$MAL_LIST" ]]; then
TITLE="[flaky] $r 隔离清单待处置(过期回炉/结构违规,ADR-0043)"
BODY="flaky-sweep 每日检测(ADR-0043):\n\n## 过期条目(已自动回炉——过期隔离不豁免,须修复测试或走 ADR 重新隔离)\n${EXP_LIST:--}\n\n## 结构违规\n${MAL_LIST:--}\n\n处置:修复测试后经 PR 移除条目(引用 ADR);或新 ADR 重新隔离。"
api -X POST "https://api.github.com/repos/$ORG/$r/issues" -d "$(jq -n --arg t "$TITLE" --arg b "$BODY" '{title:$t,body:$b,labels:["flaky-quarantine"]}')" >/dev/null 2>&1 || \
Comment on lines +46 to +49

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

7. Daily sweep duplicates issues 🐞 Bug ☼ Reliability

Every run with a persistent expired or malformed entry unconditionally creates another issue without
looking for an existing sweep-owned issue. Because the workflow runs daily, unresolved violations
will produce one duplicate issue per repository per day.
Agent Prompt
## Issue description
Use a dedicated label or marker to find an existing open flaky-sweep issue and update or comment on it instead of creating a duplicate. Optionally close the owned issue when the repository becomes healthy.

## Issue Context
The scheduled workflow runs every day, while the script's only issue lifecycle operation is an unconditional POST whenever findings remain.

## Fix Focus Areas
- .github/workflows/flaky-sweep.yml[4-7]
- governance/flaky-sweep.sh[46-50]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

api -X POST "https://api.github.com/repos/$ORG/$r/issues" -d "$(jq -n --arg t "$TITLE" --arg b "$BODY" '{title:$t,body:$b}')" >/dev/null
Comment on lines +49 to +50

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

2. flaky-sweep opens forbidden issues 📘 Rule violation § Compliance

The new automation reports quarantine violations by creating repository issues, but the automation
standard permits machine feedback only through failed check runs or ordinary PR comments. This
introduces a feedback channel outside the documented bot standards.
Agent Prompt
## Issue description
The sweep creates standalone GitHub issues even though the documented automation feedback standard limits bots to failed check runs or ordinary PR comments.

## Issue Context
Replace automatic issue creation with an approved feedback mechanism, or update the automation standard through the required governance process before using standalone issues as an escalation channel.

## Fix Focus Areas
- governance/flaky-sweep.sh[47-51]
- standards/automation/bot-channels.md[3-18]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment on lines +49 to +50

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

6. Failed issue posts look successful 🐞 Bug ☼ Reliability

The API wrapper uses curl without --fail-with-body, so HTTP 4xx/5xx responses return success and
the unlabeled fallback is not attempted; transport failures from both attempts are also ignored
because set -e is disabled. The script then unconditionally prints ISSUE and increments
ISSUES, even when no escalation issue exists.
Agent Prompt
## Issue description
Make the API helper fail on non-2xx responses, validate the issue response, and only increment `ISSUES` after confirmed creation. If both labeled and unlabeled creation fail, record a failure that makes the sweep exit nonzero.

## Issue Context
Plain `curl -sS` treats GitHub HTTP errors as successful transfers, defeating the `||` fallback and allowing issue-creation failures to be reported as successes.

## Fix Focus Areas
- governance/flaky-sweep.sh[12-12]
- governance/flaky-sweep.sh[49-56]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

echo "ISSUE repo '$r': 过期 $(grep -c . <<<"$EXP_LIST" || true) / 违规 $(grep -c . <<<"$MAL_LIST" || true)"
ISSUES=$((ISSUES+1)); EXPIRED=$((EXPIRED+$(grep -c . <<<"$EXP_LIST" || true)))
Comment on lines +51 to +52

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

5. Structural violations exit green 🐞 Bug ≡ Correctness

Parsed structural violations populate MAL_LIST, but line 52 only increments EXPIRED; MALFORMED
is reserved for content-fetch failures. A repository containing only invalid quarantine entries is
therefore reported as 违规仓=0 and the workflow exits successfully.
Agent Prompt
## Issue description
Increment the malformed repository counter whenever `MAL_LIST` is non-empty and ensure the final status fails for structural violations.

## Issue Context
The parser emits structural findings into `MAL_LIST`, but only fetch failures currently increment the counter checked at script exit.

## Fix Focus Areas
- governance/flaky-sweep.sh[44-56]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

fi
done
echo "结果: 开 issue=$ISSUES 过期条目=$EXPIRED 违规仓=$MALFORMED(无输出=全部健康)"
[[ $EXPIRED -eq 0 && $MALFORMED -eq 0 ]]
17 changes: 15 additions & 2 deletions governance/policy/testing.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,9 @@ active_now:
- {id: T-05, name: doc_examples, tool: "go Example functions", placement: gate, applies: go}
- {id: T-06, name: license_scan, tool: license-checker, placement: gate}
- {id: T-07, name: sbom, tool: syft, placement: release}
- {id: T-08, name: flaky_governance, tool: none,
rule: "重跑一次过≠通过;同测试两次飘→隔离+issue,修复后回归"}
- {id: T-08, name: flaky_governance, tool: "CI-Workflows flaky-retry.sh + .github flaky-sweep",
rule: "重试≤retry_max 且入账;失败→通过转移记 flaky 事件;窗口内≥阈值入隔离候选;
隔离带过期自动回炉(ADR-0043 参数真源=本文件 flaky_governance 段)"}
- {id: T-09, name: differential, tool: "golden fixtures + 双实现回放",
placement: gate, note: "重写项目 = gate 必选项"}
- {id: T-10, name: mutation, tools: "stryker | go-mutesting", placement: weekly,
Expand Down Expand Up @@ -113,3 +114,15 @@ test_integrity:
suppression: '(^|[^A-Za-z_])([Ss]kip|[Xx][Ff][Aa][Ii][Ll])[[:space:]]*\(|\.skip\b|\.only\b|t\.Skip|mark\.skip|mark\.xfail|@Ignore|@Disabled|\[ignore|\.todo\(|unittest\.skip'
non_source: '^(\.github/|docs/|governance/)|(^|/)([^/]*\.md)$|(^|/)(LICENSE|CODEOWNERS|NOTICE|\.gitignore|\.gitattributes)$'
fail_closed: true

# ---- flaky_governance 参数(ADR-0043 / .github #94,P2-9)----------------
# 机器可执行真源:CI-Workflows scripts/flaky-retry.sh 拉取本节(fail-closed)。
# 语义约束:重试必须入账(无声重试=作弊);仅"失败→重试通过"转移记 flaky 事件
# (确定性失败重试全败仍红、不产生记录——真回归不误放);隔离(tests/quarantine.yaml)
# 条目含 test/owner/expires/adr,过期自动回炉(governance/flaky-sweep.sh 每日
# 检测 + 升级 issue);清单变更走 ADR(adr-required 路径覆盖)。
flaky_governance:
retry_max: 2 # 自动重试上限(总运行 ≤3)
flaky_window_days: 30 # flaky 事件统计窗口
flaky_threshold: 3 # 窗口内事件 ≥ 此值 → 隔离候选(自动开 issue 人确认)
quarantine_max_days: 30 # 隔离条目最长有效期(到期回炉)