Skip to content

feat(test-runner): add --merge-strategy flag to merge-reports - #42287

Open
areebmohammed wants to merge 2 commits into
microsoft:mainfrom
areebmohammed:feature/merge-strategy-flag
Open

feat(test-runner): add --merge-strategy flag to merge-reports#42287
areebmohammed wants to merge 2 commits into
microsoft:mainfrom
areebmohammed:feature/merge-strategy-flag

Conversation

@areebmohammed

Copy link
Copy Markdown

Summary

  • Adds --merge-strategy <separate|overwrite|as-retry> to merge-reports, so a test that appears in multiple blobs with the same id (e.g. a full run merged with a --last-failed rerun) can be reconciled instead of always showing up as a duplicate entry.
  • separate (default) keeps existing behavior. overwrite keeps only the chronologically later result. as-retry renumbers the later result to continue the original's retry sequence, so it renders like a normal Playwright retry.
  • Blobs are reconciled by each blob's own recorded run time, not by report filename, so ordering is correct regardless of how blob files happen to be named.

Fixes#33094

Adds `--merge-strategy <separate|overwrite|as-retry>` to `merge-reports`
so tests that appear in multiple blobs with the same id (e.g. a full run
merged with a --last-failed rerun) can be reconciled instead of always
showing up as duplicate entries. Default stays "separate" for backward
compatibility.
Fixes: microsoft#33094
@areebmohammed

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

workerIndex: number;
parallelIndex: number;
startTime: number;
// Set by report merging to discard previously-merged results for this test id

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove the comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

// Set by report merging to discard previously-merged results for this test id
// right before this result is added, without affecting every onTestBegin globally
// (see clearPreviousResultsWhenTestBegins, which applies to all of them).
discardPreviousResults?: boolean;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to repeat it on each event.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right — dropped the once-per-blob "already discarded" tracking entirely, see the MergeStrategyPatcher change below.

const globalTestIdSet = new Set<string>();
// Shared across all blobs so that "as-retry" keeps numbering retries
// consecutively for a given test id, instead of each blob restarting at 0.
const retryOffsets = new Map<string, number>();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we do away without the map?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced with a single shared counter map, per your suggestion below.

if (this._mergeStrategy === 'as-retry') {
if (isCollision)
this._offsetRetry(params.testId, params.result);
this._trackRetryIndex(params.testId, params.result.retry);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IdsPatcher is and odd place for code like this.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, pulled it out into a new MergeStrategyPatcher class that runs right after IdsPatcher in the chain. IdsPatcher no longer knows anything about merge strategy beyond id mapping.

private _globalTestIdSet: Set<string>;
private _mergeStrategy: MergeStrategy;
private _retryOffsets: Map<string, number>;
// Test ids that collided with an earlier blob (populated while patching this blob's

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't need such long comments, let's drop. If we cannot make sense of the code without the comments, it's a good signal that the code has to be simplified (and it feels so).

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed — _collidingTestIds and the comment both went away with the rest of the collision-tracking state.

this._discardedResultsForTestIds.add(params.testId);
}
if (this._mergeStrategy === 'as-retry') {
if (isCollision)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For as-retry, the merged retry index is just the number of results merged so far for that test id. A single shared Map<testId, number> replaces _retryOffsets + _retryOffsetForTestId + _offsetRetry + _trackRetryIndex.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done - one map, incremented per merged result, in MergeStrategyPatcher

case 'onTestBegin': {
params.testId = this._mapTestId(params.testId);
const isCollision = this._collidingTestIds.has(params.testId);
if (isCollision && this._mergeStrategy === 'overwrite' && !this._discardedResultsForTestIds.has(params.testId)) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For overwrite, discardPreviousResults = (result.retry === 0) is enough — clearing an empty result list is a no-op, so there's no need to track which ids collided or which were already discarded. _collidingTestIds and _discardedResultsForTestIds both go away.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — implemented as MergeStrategyPatcher's overwrite branch, and _collidingTestIds/_discardedResultsForTestIds are both removed.

Replaces IdsPatcher's per-test collision tracking (_collidingTestIds,
_discardedResultsForTestIds, _retryOffsetForTestId, _retryOffsets) with
a standalone MergeStrategyPatcher backed by a single shared counter map.
"overwrite" now discards on result.retry === 0 (a no-op when there's no
prior result to discard); "as-retry" renumbers retries by counting
merged results per test id. Neither needs to know which ids collided.
Addresses review feedback on PR microsoft#42287.
@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

1 failed
❌ [firefox] › mcp/annotate.spec.ts:446 › should switch screencast to -s session on show --annotate @mcp-windows-latest-firefox

8141 passed, 1361 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

1 failed
❌ [playwright-test] › hooks.spec.ts:652 › should not report error from test fixture teardown when beforeAll times out @ubuntu-latest-node24

21 flaky⚠️ [chromium-library] › library/global-fetch.spec.ts:293 › should return security details from response `@frozen-time-library-chromium-linux`
⚠️ [chromium-library] › library/har.spec.ts:639 › should have security details `@frozen-time-library-chromium-linux`
⚠️ [chromium-library] › library/video.spec.ts:294 › screencast › should capture navigation `@frozen-time-library-chromium-linux`
⚠️ [chromium-library] › library/video.spec.ts:736 › screencast › should work with video+trace `@frozen-time-library-chromium-linux`
⚠️ [chromium-library] › library/global-fetch.spec.ts:293 › should return security details from response `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/har.spec.ts:639 › should have security details `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/beforeunload.spec.ts:130 › should support dismissing the dialog multiple times `@realtime-time-library-chromium-linux`
⚠️ [chromium-library] › library/global-fetch.spec.ts:293 › should return security details from response `@realtime-time-library-chromium-linux`
⚠️ [chromium-library] › library/har.spec.ts:639 › should have security details `@realtime-time-library-chromium-linux`
⚠️ [chromium-library] › library/global-fetch.spec.ts:293 › should return security details from response `@chromium-ubuntu-22.04-node24`
⚠️ [chromium-library] › library/har.spec.ts:639 › should have security details `@chromium-ubuntu-22.04-node24`
⚠️ [chromium-library] › library/global-fetch.spec.ts:293 › should return security details from response `@chromium-ubuntu-22.04-node20`
⚠️ [chromium-library] › library/har.spec.ts:639 › should have security details `@chromium-ubuntu-22.04-node20`
⚠️ [chromium-library] › library/global-fetch.spec.ts:293 › should return security details from response `@chromium-ubuntu-22.04-node22`
⚠️ [chromium-library] › library/har.spec.ts:639 › should have security details `@chromium-ubuntu-22.04-node22`
⚠️ [firefox-library] › library/global-fetch.spec.ts:293 › should return security details from response `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/har.spec.ts:639 › should have security details `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/heap.spec.ts:223 › should not leak workers `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-page] › page/page-goto.spec.ts:90 › should work with Cross-Origin-Opener-Policy `@firefox-ubuntu-22.04-node20`
⚠️ [webkit-library] › library/global-fetch.spec.ts:293 › should return security details from response `@webkit-ubuntu-22.04-node20`
⚠️ [webkit-library] › library/har.spec.ts:639 › should have security details `@webkit-ubuntu-22.04-node20`

51306 passed, 1235 skipped


Merge workflow run.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Ability to merge Playwright reports from the main run and --last-failed flag

2 participants

@areebmohammed@yury-s