Uh oh!
There was an error while loading. Please reload this page.
test(e2e): improve integration test reliability - #8422
Merged
Conversation
🦋 Changeset detectedLatest commit: 6cf9d6c The changes in this PR will be included in the next version bump. This PR includes changesets to release 0 packagesWhen changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
The latest updates on your projects. Learn more about Vercel for GitHub.
|
@clerk/astro@clerk/backend@clerk/chrome-extension@clerk/clerk-js@clerk/dev-cli@clerk/expo@clerk/expo-passkeys@clerk/express@clerk/fastify@clerk/hono@clerk/localizations@clerk/nextjs@clerk/nuxt@clerk/react@clerk/react-router@clerk/shared@clerk/tanstack-react-start@clerk/testing@clerk/ui@clerk/upgrade@clerk/vuecommit: |
Contributor
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughA fix for e2e test reliability is introduced by adding a new Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
This was referenced Apr 30, 2026
…ent token-cache tests
Three small E2E reliability fixes uncovered while investigating
consistent CI failures across multiple suites.
1. Strong random fake-user passwords. createFakeUser built passwords
as `${email}${randomHash}`, which FAPI's compromised-password check
rejected with `form_password_compromised` (HTTP 422). Replaced with
a `fakerPassword()` helper that emits a high-entropy base64url
string with a fixed `Aa1!` prefix to satisfy default Clerk password
complexity rules.
2. Defensive afterAll cleanup. When a suite's beforeAll timed out or
threw, afterAll crashed with a TypeError on un-assigned fakeUser /
fakeOrganization / network / fakeOAuth / fakeAPIKey, masking the
real failure. Optional-chained the cleanup calls in:
- integration/tests/components.test.ts
- integration/testUtils/machineAuthHelpers.ts (apiKey, m2m, oauth)
3. Drop timer-dependent token-cache assertions. Two MemoryTokenCache
cross-tab tests assumed JWT TTL = 60s and waited 50s of wall-clock
time for the proactive-refresh setTimeout to fire. The dev test
instance now issues 300s tokens, so the timer fires at ~283s and
the tests never reached it.
- Deleted `multi-tab scheduled refreshes are deduped to a single
request` from single-session.test.ts. The same broadcast-dedup
path is exercised by the existing line-49 test via
`getToken({ skipCache: true })`.
- Rewrote multi-session.test.ts:249 (now `cross-session token
refreshes do not deduplicate`) to manually trigger
`getToken({ skipCache: true })` on each tab. Different sessions
have different tokenIds, so BroadcastChannel does NOT deduplicate
across them — unique coverage the single-session test doesn't
provide.
Proactive-refresh timer scheduling math is best validated by unit
tests that mock setTimeout, not by real-time integration tests
against a shared dev instance.
Scoped to integration/ only, no published package affected — empty
changeset.jacekradkoforce-pushed
the
jacek/fix-e2e-compromised-password
branch
from
April 30, 2026 21:29
6eebdd2 to
ef5dd73Comparewobsoriano
approved these changes
May 1, 2026
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three small E2E reliability fixes uncovered while investigating consistent CI failures across multiple suites.
1. Strong random fake-user passwords
createFakeUserbuilt passwords as${email}${randomHash}(e.g.76557538ea+clerk_test@clerkcookie.com76557538ea). FAPI's compromised-password check rejects these withform_password_compromised(HTTP 422), which broke any sign-in flow that wasn't intentionally testing the compromised-password path. Replaced with afakerPassword()helper that emits a high-entropybase64urlstring with a fixedAa1!prefix to satisfy default Clerk complexity rules (upper, lower, digit, symbol).2. Defensive
afterAllcleanupWhen a test suite's
beforeAlltimed out or threw (BAPI hiccup, dev server slow to come up, etc.),afterAllwould crash with aTypeErroron the un-assignedfakeUser/fakeOrganization/network/fakeOAuth/fakeAPIKey, masking the real failure. Optional-chain the cleanup calls in:integration/tests/components.test.tsintegration/testUtils/machineAuthHelpers.ts(apiKey, m2m, oauth describes)3. Drop timer-dependent token-cache assertions
Two
MemoryTokenCachecross-tab tests assumed JWT TTL = 60s and waited 50s of wall-clock time for the proactive-refreshsetTimeoutto fire. The dev test instance now issues 300s tokens, so the refresh fires at ~283s and the tests never reached it.The BroadcastChannel-based deduplication these tests were checking is already covered elsewhere; the proactive-refresh timer scheduling itself is best validated by unit tests that mock
setTimeout, not by real-time integration tests.integration/tests/session-token-cache/single-session.test.ts"multi-tab scheduled refreshes are deduped to a single request". The same broadcast-dedup path is exercised by the existing line-49 test ("multi-tab token sharing works when clearing the cache") viagetToken({ skipCache: true }).integration/tests/session-token-cache/multi-session.test.ts:249(now "cross-session token refreshes do not deduplicate") to manually triggergetToken({ skipCache: true })on each tab. Different sessions have different tokenIds, so BroadcastChannel doesn't dedupe — each tab is expected to make its own request. Unique coverage the single-session test doesn't provide.Scoped to
integration/only and doesn't affect any published package — empty changeset.Test plan
genericcomponents-component-smoke— realbeforeAllfailure surfaces instead ofTypeError: ... 'delete'machinetests-with-invalid-M2M-tokens/local-verification— realClerkAPIResponseErrorsurfaces instead ofTypeError: ... 'cleanup'genericmulti-tab token sharing works when clearing the cache— still green (covers the broadcast-dedup path the deleted timer test was duplicating)nextjscross-session token refreshes do not deduplicate— green; runs in seconds and asserts the cross-session non-dedup casesession-tasks-sign-in-reset-password(test still callssetPasswordCompromisedon purpose)