Uh oh!
There was an error while loading. Please reload this page.
fix(ci): Improve flaky test issue deduplication - #22042
Conversation
Strip more matrix/variant noise from job and test names so the same flaky test dedupes to a single issue instead of one per variant. Job names now also collapse (Node xx) and (n/m) shard groups, fold all Playwright bundle/esm build configs into one bucket, and drop E2E "-node-xx" app-name suffixes. Test names now also strip the [chromium] browser prefix, Playwright line:column locations (which drift on every file edit and reopened issues), the trailing [esm]/[cjs] module-format suffix, and the bare "esm/cjs >" describe segment so the suffix and describe-block forms converge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 6d78642. Configure here.
| .replace(/(\.[cm]?[jt]sx?):\d+:\d+/gi, '$1') | ||
| .replace(/esm\/cjs\s*>\s*(?:esm|cjs)\b/gi, 'esm/cjs') | ||
| .replace(/\besm\/cjs\s*>\s*/gi, '') | ||
| .replace(/\s*\[(?:esm|cjs)\]/gi, '') |
There was a problem hiding this comment.
Missing normalization regression tests
Low Severity
This fix expands normalizeJobName and normalizeTestName, but the PR adds no unit tests for those cases. Without table-driven examples (Playwright job folding, [esm]/[cjs] titles, line-number stripping), future regex edits can silently break flaky-issue deduplication again.
Triggered by project rule: PR Review Guidelines for Cursor Bot
Reviewed by Cursor Bugbot for commit 6d78642. Configure here.
report-ci-failures.mjsopens one GitHub issue per failing test, keyed on a normalized job + test name. Several variant patterns weren't being stripped, so the same flaky test kept opening duplicate issues. This widens both normalizers.normalizeJobNamenow also collapses:(Node 22)and(n/m)shard groups (in addition to the existing(22)/(TS 3.8))Playwright Testsbucket (e.g.bundle_tracing,bundle_tracing_replay,esm (1/4)) — the same profiling flake was filing an issue per config-node-20-18app-name version suffixesNon-version parentheticals such as
(Orchestrion)are preserved.normalizeTestNamenow also strips:[chromium] ›browser project prefixtest.ts:33:11line/column locations — these drift whenever the test file is edited, silently reopening the same flaky issue under a new title[esm]/[cjs]module-format suffix (the main gap — the old describe-block form was handled but the newer suffix form was not) and the bareesm/cjs >describe segment, so both forms convergeVerified against current open
Flaky Testissues: the[esm]/[cjs]postgres/mysql pairs collapse to one, the profiling test acrossbundle_tracing/bundle_tracing_replay/esm (1/4)collapses to one, and line-drift variants merge.Note: dedup still matches on exact issue title, so existing issues (which carry the old normalized titles) won't retroactively merge — the next run files them once under the new canonical titles. Bulk-closing the current flaky issues after merge is recommended.
🤖 Generated with Claude Code