Skip to content

flaky(e2e): transcript-scroll:498 asserts the tail before a viewport change re-pins it #4648

Description

@Astro-Han

What happened

The Desktop E2E test a nested scroller near the history boundary does not request an earlier range (apps/desktop/e2e/transcript-scroll.spec.ts:498) fails intermittently on CI, before it exercises anything it is about.

The test resizes the viewport, waits six painted frames, and then asserts its two preconditions in one shot:

awaitpage.setViewportSize({width: 900,height: 1500});awaitwaitForPaintedFrames(page,6);constmetrics=awaitscrollMetrics(page);expect(metrics.scrollTop).toBeLessThanOrEqual(Math.max(640,metrics.clientHeight*2));expect(metrics.distance).toBeLessThanOrEqual(4);// <- fails here

Observed on CI:

Error: expect(received).toBeLessThanOrEqual(expected)
Expected: <= 4
Received: 58
at apps/desktop/e2e/transcript-scroll.spec.ts:505:28

A viewport change makes the transcript re-pin to the tail. Six painted frames is a guess at how long that takes, not a wait for it, so under CI load the assertion samples the scroller mid-pin and reads a distance of 58px. Nothing about the nested-scroller behaviour under test has run yet.

This is the same failure mode as #4619 (fixed in #4620) — asserting on transcript geometry before tail positioning settles — in a different test in the same file.

How to reproduce

Intermittent; it does not reproduce reliably on a developer machine (11/11 green locally, that test in 528ms). It appears under concurrent CI load, where the Desktop E2E job runs 109 tests across 4 workers on Xvfb.

  1. Push any branch and let the Desktop e2e job run.
  2. Observed twice today: apache/maka on main at 148f8eb (run 33737060147) and on PR refactor(runtime): converge AgentRun metadata into the RuntimeInvocation event spine #4631 (run 33741800324), both with Expected: <= 4 / Received: 58.

Rate over the last 20 CI runs on main: 13 completed (7 cancelled by newer pushes), 3 red, all in the scroll family — code-scroll.spec.ts:22 twice (fixed since, in #4632), this test once, prompt-rail.spec.ts:370 once. So this one is roughly 1 in 13 runs, and the job as a whole about 1 in 4.

Environment

Logs, screenshots, or additional context

Suggested fix. Replace the one-shot sample with a poll for the state the test needs, the way #4620 did for its neighbour a few lines below. The precondition is two facts at once — the reader sits near the history boundary and the transcript is pinned to the tail — so poll them together rather than asserting them separately after a fixed number of frames:

awaitpage.setViewportSize({width: 900,height: 1500});awaitexpect.poll(async()=>{constmetrics=awaitscrollMetrics(page);return{nearHistoryBoundary: metrics.scrollTop<=Math.max(640,metrics.clientHeight*2),settledAtTail: metrics.distance<=4,};},{message: 'the transcript re-pins to the tail after the viewport change'}).toMatchObject({nearHistoryBoundary: true,settledAtTail: true});

Two things worth keeping in mind while fixing it:

  • Do not relax the <= 4 tolerance. The number is the point of the test: the whole file exists to prove the transcript stays under the reader, and a looser bound would let a real release of the tail pass.
  • The same one-shot pattern appears elsewhere in this file after actions that move the scroller. Converting only this call site is a complete fix for this issue; converting the others is welcome but belongs in its own commit, and each conversion should say which action it is waiting out.

Verify with npm exec -w @maka/desktop -- playwright test --config e2e/playwright.config.ts e2e/transcript-scroll.spec.ts. Because the failure is load-dependent, a useful check is to run that file repeatedly with --repeat-each and several workers rather than once.

This is a self-contained change in one test file with a worked example in the tree, so it is a reasonable entry point for anyone who already has the Desktop E2E setup running. To claim it, comment take (see CONTRIBUTING.md).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workinghelp wantedExtra attention is needed

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions