[Flaky CI]: Node (16) Unit Tests `@sentry/core` `offline.test.ts`

[AbhiPrasad]

Flakiness Type

Timeout

Name of Job

Node (16) Unit Tests

Name of Test

@sentry/core test/lib/transports/offline.test.ts

Link to Test Run

https://github.com/getsentry/sentry-javascript/actions/runs/4115185807/jobs/7104054722

Details

> @sentry/core:test

$ jest
 PASS test/lib/base.test.ts (9.164 s)
 PASS test/lib/transports/base.test.ts
 PASS test/lib/integration.test.ts
 PASS test/lib/integrations/inboundfilters.test.ts
 FAIL test/lib/transports/offline.test.ts (17.526 s)
 ● makeOfflineTransport › Follows the Retry-After header

 expect(received).toEqual(expected) // deep equality

 Expected: 2
 Received: 1

 310 | await delay(4_000);
 311 |
 > 312 | expect(getSendCount()).toEqual(2);
 | ^
 313 | expect(queuedCount).toEqual(0);
 314 | expect(getCalls()).toEqual(['pop', 'pop']);
 315 | }, 7_000);

 at Object.<anonymous> (test/lib/transports/offline.test.ts:312:28)
 at fulfilled (node_modules/tslib/tslib.js:112:62)

[timfish]

This is still occasionally reoccurring.

[timfish]

Looks like this fails around 10% of the time in CI and much less than 1% on my machine.

Since I can get the tests passing by adding a few console.log throughout the code, I suspect this could be caused by a race in the implementation rather than a poorly designed test.

[AbhiPrasad]

Maybe there is an issue from popping from the store inside the timeout? That causes a stale retry header somehow?

[timfish]

I've been left a little stumped by this. It's not helped by the fact that it only really reproduces occasionally in CI.

I played CI whack a mole on and off for most of yesterday and it looks like the 2nd envelope is always popped from the store but it appears sometimes it's not counted at sent.