fix(e2e): more robust user deletion#9375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dstaley merged 8 commits into from
Aug 11, 2026
Merged

fix(e2e): more robust user deletion #9375

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
6b841fc
fix(e2e): delete all matching users
dstaley Aug 10, 2026
d76132e
fix(e2e): mark created users, always cleanup
dstaley Aug 10, 2026
2bd619c
fix: don't retry or fail on cleanup
dstaley Aug 10, 2026
a326a24
fix(e2e): use underscore
dstaley Aug 10, 2026
7513812
Merge branch 'main' into ds.fix/delete-all-match
dstaley Aug 10, 2026
3c096a3
Merge branch 'main' into ds.fix/delete-all-match
dstaley Aug 10, 2026
ad2f98a
Merge branch 'main' into ds.fix/delete-all-match
dstaley Aug 11, 2026
417bb3b
fix(repo): rm extra changeset
dstaley Aug 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .changeset/silver-lands-cut.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
---
---
17 changes: 17 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -504,6 +504,15 @@ jobs:
sudo apt-get install -y xvfb
fi

- name: Configure test user cleanup
run: echo "INTEGRATION_TEST_RUN_KEY=${RUN_ID}-${RUN_ATTEMPT}-${TEST_NAME}-${TEST_PROJECT}-${NEXT_VERSION:-default}" >> "$GITHUB_ENV"
env:
RUN_ID: ${{ github.run_id }}
RUN_ATTEMPT: ${{ github.run_attempt }}
TEST_NAME: ${{ matrix.test-name }}
TEST_PROJECT: ${{ matrix.test-project }}
NEXT_VERSION: ${{ matrix.next-version }}

- name: Run Integration Tests
id: integration-tests
timeout-minutes: 25
Expand All@@ -525,6 +534,14 @@ jobs:
NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem
VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}

- name: Delete integration-test users
if: ${{ always() && steps.integration-tests.outcome != 'skipped' }}
timeout-minutes: 4
run: pnpm test:integration:cleanup
env:
INTEGRATION_INSTANCE_KEYS: ${{ secrets.INTEGRATION_INSTANCE_KEYS }}
NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem

- name: Sanitize artifact name
if: ${{ cancelled() || failure() }}
id: sanitize
Expand Down
73 changes: 48 additions & 25 deletions integration/cleanup/cleanup.setup.ts
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -5,8 +5,11 @@ import { isStaging } from '@clerk/shared/utils';
import { test as setup } from '@playwright/test';

import { appConfigs } from '../presets/';
import { findE2ERunUsers, getE2ERunMarker } from '../testUtils/e2eRun';
import { withRetry } from '../testUtils/retryableClerkClient';

setup('cleanup instances ', async () => {
const runMarker = getE2ERunMarker();
const entries = Array.from(appConfigs.secrets.instanceKeys.values())
.map(({ pk, sk }) => {
const secretKey = sk;
Expand All@@ -32,6 +35,9 @@ setup('cleanup instances ', async () => {
}> = [];

console.log('🧹 Starting E2E Test Cleanup Process...\n');
if (runMarker) {
console.log(`Cleaning users for run marker ${runMarker}\n`);
}

for (const entry of entries) {
const instanceSummary = {
Expand All@@ -43,29 +49,32 @@ setup('cleanup instances ', async () => {
};

try {
const clerkClient = createClerkClient({ secretKey: entry.secretKey, apiUrl: entry.apiUrl });
const clerkClient = withRetry(createClerkClient({ secretKey: entry.secretKey, apiUrl: entry.apiUrl }));

// Get users with error handling
let users: any[] = [];
try {
const { data: usersWithEmail } = await clerkClient.users.getUserList({
orderBy: '-created_at',
query: 'clerkcookie',
limit: 500,
});

const { data: usersWithPhoneNumber } = await clerkClient.users.getUserList({
orderBy: '-created_at',
query: '55501',
limit: 500,
});

// Deduplicate users by ID
const allUsersMap = new Map();
[...usersWithEmail, ...usersWithPhoneNumber].forEach(user => {
allUsersMap.set(user.id, user);
});
users = Array.from(allUsersMap.values());
if (runMarker) {
users = await findE2ERunUsers(clerkClient, runMarker);
} else {
const { data: usersWithEmail } = await clerkClient.users.getUserList({
orderBy: '-created_at',
query: 'clerkcookie',
limit: 500,
});

const { data: usersWithPhoneNumber } = await clerkClient.users.getUserList({
orderBy: '-created_at',
query: '55501',
limit: 500,
});

const allUsersMap = new Map();
[...usersWithEmail, ...usersWithPhoneNumber].forEach(user => {
allUsersMap.set(user.id, user);
});
users = Array.from(allUsersMap.values());
}
} catch (error) {
instanceSummary.errors.push(`Failed to get users: ${error.message}`);
console.error(`Error getting users for ${entry.instanceName}:`, error);
Expand All@@ -75,10 +84,14 @@ setup('cleanup instances ', async () => {
// Get organizations with error handling
let orgs: any[] = [];
try {
const { data: orgsData } = await clerkClient.organizations.getOrganizationList({
limit: 500,
});
orgs = orgsData;
if (runMarker) {
orgs = [];
} else {
const { data: orgsData } = await clerkClient.organizations.getOrganizationList({
limit: 500,
});
orgs = orgsData;
}
} catch (error) {
// Treat 404 (not found) and 403 (forbidden) as "no orgs"
// 404 = no organizations exist, 403 = no permission to access organizations
Expand All@@ -91,8 +104,11 @@ setup('cleanup instances ', async () => {
}
}

const usersToDelete = batchElements(skipObjectsThatWereCreatedWithinTheLast10Minutes(users), 5);
const orgsToDelete = batchElements(skipObjectsThatWereCreatedWithinTheLast10Minutes(orgs), 5);
const usersToDelete = batchElements(
runMarker ? users : skipObjectsThatWereCreatedWithinTheLast10Minutes(users),
5,
);
const orgsToDelete = batchElements(runMarker ? [] : skipObjectsThatWereCreatedWithinTheLast10Minutes(orgs), 5);

// Delete users with tracking
for (const batch of usersToDelete) {
Expand DownExpand Up@@ -142,6 +158,13 @@ setup('cleanup instances ', async () => {
await new Promise(r => setTimeout(r, 1000));
}

if (runMarker) {
const remainingUsers = await findE2ERunUsers(clerkClient, runMarker);
if (remainingUsers.length > 0) {
instanceSummary.errors.push(`Users remain after cleanup: ${remainingUsers.map(user => user.id).join(', ')}`);
}
}

// Report instance results
const maskedKey = entry.secretKey.replace(/(sk_(test|live)_)(.+)(...)/, '$1***$4');
if (instanceSummary.usersDeleted > 0 || instanceSummary.orgsDeleted > 0) {
Expand Down
1 change: 1 addition & 0 deletions integration/playwright.cleanup.config.ts
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -9,6 +9,7 @@ config({ path: path.resolve(__dirname, '.env.local') });
export default defineConfig({
...common,
testDir: './cleanup',
retries: 0,
projects: [
{
name: 'setup',
Expand Down
42 changes: 42 additions & 0 deletions integration/testUtils/e2eRun.ts
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
import { createHash } from 'node:crypto';

import type { ClerkClient, User } from '@clerk/backend';

type E2EUserRecord = {
username: string | null;
emailAddresses: Array<{ emailAddress: string }>;
privateMetadata: Record<string, unknown>;
};

export const getE2ERunMarker = (runKey = process.env.INTEGRATION_TEST_RUN_KEY): string | undefined => {
if (!runKey) {
return;
}

const digest = createHash('sha256').update(runKey).digest('hex').slice(0, 20);
return `e2e_${digest}`;
};

export const userMatchesE2ERun = (user: E2EUserRecord, marker: string): boolean =>
Boolean(
user.username?.includes(marker) ||
user.emailAddresses.some(email => email.emailAddress.includes(marker)) ||
user.privateMetadata.e2eRunMarker === marker,
);

export const findE2ERunUsers = async (clerkClient: ClerkClient, marker: string): Promise<User[]> => {
const usersById = new Map<string, User>();
let offset = 0;

while (true) {
const { data } = await clerkClient.users.getUserList({ query: marker, limit: 100, offset });
data.filter(user => userMatchesE2ERun(user, marker)).forEach(user => usersById.set(user.id, user));

if (data.length < 100) {
break;
}
offset += data.length;
}

return Array.from(usersById.values());
};
51 changes: 34 additions & 17 deletions integration/testUtils/usersService.ts
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,7 @@ import { faker } from '@faker-js/faker';
import type { TestInfo } from '@playwright/test';

import { fakerPassword, hash } from '../models/helpers';
import { getE2ERunMarker } from './e2eRun';

async function withErrorLogging<T>(operation: string, fn: () => Promise<T>): Promise<T> {
try {
Expand DownExpand Up@@ -129,24 +130,29 @@ export const createUserService = (clerkClient: ClerkClient) => {
withUsername = false,
} = options || {};
const randomHash = hash();
const runMarker = getE2ERunMarker();
const markedHash = runMarker ? `${runMarker}_${randomHash}` : randomHash;
const email = fictionalEmail
? `${randomHash}+clerk_test@clerkcookie.com`
: `clerkcookie+${randomHash}@mailsac.com`;
? `${markedHash}+clerk_test@clerkcookie.com`
: `clerkcookie+${markedHash}@mailsac.com`;
const phoneNumber = fakerPhoneNumber();
const { file, line, title, titlePath } = test.info();
const fakeUserEmail = withEmail ? email : undefined;
const fakeUserPhoneNumber = withPhoneNumber ? phoneNumber : undefined;

return {
firstName: faker.person.firstName(),
lastName: faker.person.lastName(),
email: withEmail ? email : undefined,
username: withUsername ? `${randomHash}_clerk_cookie` : undefined,
email: fakeUserEmail,
username: withUsername ? `${markedHash}_clerk_cookie` : undefined,
password: withPassword ? fakerPassword() : undefined,
phoneNumber: withPhoneNumber ? phoneNumber : undefined,
phoneNumber: fakeUserPhoneNumber,
privateMetadata: {
title,
titlePath,
file,
line,
...(runMarker ? { e2eRunMarker: runMarker } : {}),
},
deleteIfExists: () => self.deleteIfExists({ email, phoneNumber }),
};
Expand All@@ -173,24 +179,35 @@ export const createUserService = (clerkClient: ClerkClient) => {
return await self.createBapiUser(fakeUser);
},
deleteIfExists: async (opts: { id?: string; email?: string; phoneNumber?: string }) => {
let id = opts.id;
const [usersByEmail, usersByPhoneNumber] = await Promise.all([
opts.email
? withErrorLogging('getUserList', () =>
clerkClient.users.getUserList({
emailAddress: [opts.email],
}),
)
: undefined,
opts.phoneNumber
? withErrorLogging('getUserList', () =>
clerkClient.users.getUserList({
phoneNumber: [opts.phoneNumber],
}),
)
: undefined,
]);

if (!id) {
const { data: users } = await withErrorLogging('getUserList', () =>
clerkClient.users.getUserList({
emailAddress: [opts.email],
phoneNumber: [opts.phoneNumber],
}),
);
id = users[0]?.id;
}
const ids = new Set([
...(opts.id ? [opts.id] : []),
...(usersByEmail?.data.map(user => user.id) ?? []),
...(usersByPhoneNumber?.data.map(user => user.id) ?? []),
]);

if (!id) {
if (ids.size === 0) {
console.log(`User "${opts.email || opts.phoneNumber}" does not exist!`);
return;
Comment on lines +205 to 207

@coderabbitaicoderabbitaiBotAug 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

Do not log the supplied email address or phone number.

Line 206 writes a user identifier to CI logs. Log a generic cleanup message instead.

Proposed fix
- console.log(`User "${opts.email || opts.phoneNumber}" does not exist!`);+ console.log('No user exists for the supplied cleanup criteria.');
📝 Committable suggestion

‼️IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if(ids.size===0){
console.log(`User "${opts.email||opts.phoneNumber}" does not exist!`);
return;
if(ids.size===0){
console.log('No user exists for the supplied cleanup criteria.');
return;
🧰 Tools
🪛 ast-grep (0.45.0)

[warning] 205-205: Avoid logging sensitive data
Context: console.log(User "${opts.email || opts.phoneNumber}" does not exist!)
Note: [CWE-532] Insertion of Sensitive Information into Log File.

(log-sensitive-data-typescript)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@integration/testUtils/usersService.ts` around lines 205 - 207, Update the
empty-user branch in the user cleanup flow to replace the interpolated
opts.email and opts.phoneNumber values with a generic cleanup message, ensuring
no supplied user identifier is written to logs while preserving the early
return.

Sources: Coding guidelines, Linters/SAST tools

}

await withErrorLogging('deleteUser', () => clerkClient.users.deleteUser(id));
await Promise.all(Array.from(ids, id => withErrorLogging('deleteUser', () => clerkClient.users.deleteUser(id))));
},
getUser: async (opts: { id?: string; email?: string }) => {
if (opts.id) {
Expand Down
1 change: 1 addition & 0 deletions turbo.json
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,7 @@
"globalPassThroughEnv": [
"AWS_SECRET_KEY",
"GITHUB_TOKEN",
"INTEGRATION_TEST_RUN_KEY",

@coderabbitaicoderabbitaiBotAug 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/usr/bin/env bashset -euo pipefail
# Every integration task must report cache=false.
jq -r ' (.tasks // {}) | to_entries[] | select(.key | test("test:integration")) | "\(.key): cache=\(.value.cache // true)"' turbo.json

Repository: clerk/javascript

Length of output: 1212

🏁 Script executed:

#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- turbo.json ---'
cat -n turbo.json | sed -n '1,90p'printf'%s\n''--- Turbo version and integration scripts ---'
rg -n '"turbo"|test:integration|INTEGRATION_TEST_RUN_KEY' package.json pnpm-lock.yaml packages apps examples .github 2>/dev/null | head -240

Repository: clerk/javascript

Length of output: 7965

🌐 Web query:

Turborepo globalPassThroughEnv cache key behavior passThroughEnv cacheable task

💡 Result:

In Turborepo, both globalPassThroughEnv and passThroughEnv serve to expose environment variables to task runtimes without including them in the task's cache key [1][2][3]. Because these variables are intentionally omitted from the hashing process, changes to their values will not trigger a cache miss, even if the change would logically affect the output of the task [1][4]. The fundamental behaviors for these configurations are: 1. Cache Key Exclusion: Variables listed in either globalPassThroughEnv or passThroughEnv do not contribute to the task hash [2][3]. Consequently, Turborepo may serve a cached result even if the underlying environment variable has changed [1][4]. This is why these keys should only be used for variables that do not affect the build output, such as authentication tokens (e.g., SENTRY_AUTH_TOKEN), CI metadata, or credentials consumed at runtime after the build process [2][4]. 2. Runtime Exposure: By including a variable in either key, you satisfy Turborepo's Strict Environment Mode, ensuring the task has access to that variable at runtime [1][5][3]. If a task attempts to access an environment variable that is not accounted for in env, globalEnv, passThroughEnv, or globalPassThroughEnv while in Strict Mode, the task may fail [1][5]. 3. Scope Differences: - globalPassThroughEnv (or global.passThroughEnv when using the global configuration flag) applies the passthrough behavior to all tasks within the repository [2][6]. - passThroughEnv applies only to the specific task where it is configured [3]. If you need a change in an environment variable to invalidate the cache and trigger a rebuild, you must include that variable in env or globalEnv instead of the passthrough keys [1][3].

Citations:

🏁 Script executed:

#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- integration task definitions ---'
rg -n -C 3 '"[`#/`]?test:integration|test:integration' turbo.json
printf'%s\n''--- CI Turbo arguments and integration invocation ---'
sed -n '400,535p' .github/workflows/ci.yml
sed -n '260,300p' .github/workflows/e2e-staging.yml
printf'%s\n''--- exact task keys parsed from turbo.json ---'
jq -r '(.tasks // {}) | to_entries[] | select(.key | contains("test:integration")) | [.key, (.value.cache // true | tostring)] | `@tsv`' turbo.json

Repository: clerk/javascript

Length of output: 16346

🏁 Script executed:

#!/usr/bin/env bashset -euo pipefail
python3 - <<'PY'import jsonfrom pathlib import Pathconfig = json.loads(Path("turbo.json").read_text())tasks = config.get("tasks", {})matches = [(name, task.get("cache", True)) for name, task in tasks.items() if "test:integration" in name]print("matching task count:", len(matches))for name, cache in matches: print(f"{name}: cache={cache}")PY

Repository: clerk/javascript

Length of output: 1236

🏁 Script executed:

#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- all Turbo force/argument settings ---'
rg -n -C 4 'TURBO_(ARGS|FORCE)|pnpm turbo test:integration|pnpm turbo run test:integration' .github/workflows package.json
printf'%s\n''--- CI workflow environment and matrix context ---'
sed -n '1,120p' .github/workflows/ci.yml
sed -n '380,445p' .github/workflows/ci.yml
sed -n '495,535p' .github/workflows/ci.yml
printf'%s\n''--- staging workflow context ---'
sed -n '1,110p' .github/workflows/e2e-staging.yml
sed -n '270,320p' .github/workflows/e2e-staging.yml

Repository: clerk/javascript

Length of output: 30087

Disable caching for CI integration tasks.

The CI integration job does not set TURBO_FORCE, and all 25 integration tasks inherit cache: true. globalPassThroughEnv does not affect the cache key. Set cache: false for these tasks, or move INTEGRATION_TEST_RUN_KEY to globalEnv.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@turbo.json` at line 37, Update the Turbo integration task configuration
containing INTEGRATION_TEST_RUN_KEY so CI integration tasks do not reuse cached
results: set cache to false for those tasks, or move INTEGRATION_TEST_RUN_KEY
into globalEnv to include it in the cache key.

"ACTIONS_RUNNER_DEBUG",
"ACTIONS_STEP_DEBUG",
"VERCEL_AUTOMATION_BYPASS_SECRET",
Expand Down
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
fix(e2e): more robust user deletion#9375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

dstaley
merged 8 commits into

from

Aug 11, 2026
 Merged


fix(e2e): more robust user deletion
#9375
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
6b841fc

fix(e2e): delete all matching users


dstaley Aug 10, 2026
d76132e

fix(e2e): mark created users, always cleanup


dstaley Aug 10, 2026
2bd619c

fix: don't retry or fail on cleanup


dstaley Aug 10, 2026
a326a24

fix(e2e): use underscore


dstaley Aug 10, 2026
7513812

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 10, 2026
3c096a3

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 10, 2026
ad2f98a

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 11, 2026
417bb3b

fix(repo): rm extra changeset


dstaley Aug 11, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 2 additions & 0 deletions
.changeset/silver-lands-cut.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
---
---

17 changes: 17 additions & 0 deletions
.github/workflows/ci.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -504,6 +504,15 @@ jobs:
 sudo apt-get install -y xvfb
 fi

 - name: Configure test user cleanup
 run: echo "INTEGRATION_TEST_RUN_KEY=${RUN_ID}-${RUN_ATTEMPT}-${TEST_NAME}-${TEST_PROJECT}-${NEXT_VERSION:-default}" >> "$GITHUB_ENV"
 env:
 RUN_ID: ${{ github.run_id }}
 RUN_ATTEMPT: ${{ github.run_attempt }}
 TEST_NAME: ${{ matrix.test-name }}
 TEST_PROJECT: ${{ matrix.test-project }}
 NEXT_VERSION: ${{ matrix.next-version }}

 - name: Run Integration Tests
 id: integration-tests
 timeout-minutes: 25
Expand All@@ -525,6 +534,14 @@ jobs:
 NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem
 VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}

 - name: Delete integration-test users
 if: ${{ always() && steps.integration-tests.outcome != 'skipped' }}
 timeout-minutes: 4
 run: pnpm test:integration:cleanup
 env:
 INTEGRATION_INSTANCE_KEYS: ${{ secrets.INTEGRATION_INSTANCE_KEYS }}
 NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem

 - name: Sanitize artifact name
 if: ${{ cancelled() || failure() }}
 id: sanitize
Expand Down

73 changes: 48 additions & 25 deletions
integration/cleanup/cleanup.setup.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -5,8 +5,11 @@ import { isStaging } from '@clerk/shared/utils';
import { test as setup } from '@playwright/test';

import { appConfigs } from '../presets/';
import { findE2ERunUsers, getE2ERunMarker } from '../testUtils/e2eRun';
import { withRetry } from '../testUtils/retryableClerkClient';

setup('cleanup instances ', async () => {
 const runMarker = getE2ERunMarker();
 const entries = Array.from(appConfigs.secrets.instanceKeys.values())
 .map(({ pk, sk }) => {
 const secretKey = sk;
Expand All@@ -32,6 +35,9 @@ setup('cleanup instances ', async () => {
 }> = [];

 console.log('🧹 Starting E2E Test Cleanup Process...\n');
 if (runMarker) {
 console.log(`Cleaning users for run marker ${runMarker}\n`);
 }

 for (const entry of entries) {
 const instanceSummary = {
Expand All@@ -43,29 +49,32 @@ setup('cleanup instances ', async () => {
 };

 try {
 const clerkClient = createClerkClient({ secretKey: entry.secretKey, apiUrl: entry.apiUrl });
 const clerkClient = withRetry(createClerkClient({ secretKey: entry.secretKey, apiUrl: entry.apiUrl }));

 // Get users with error handling
 let users: any[] = [];
 try {
 const { data: usersWithEmail } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: 'clerkcookie',
 limit: 500,
 });

 const { data: usersWithPhoneNumber } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: '55501',
 limit: 500,
 });

 // Deduplicate users by ID
 const allUsersMap = new Map();
 [...usersWithEmail, ...usersWithPhoneNumber].forEach(user => {
 allUsersMap.set(user.id, user);
 });
 users = Array.from(allUsersMap.values());
 if (runMarker) {
 users = await findE2ERunUsers(clerkClient, runMarker);
 } else {
 const { data: usersWithEmail } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: 'clerkcookie',
 limit: 500,
 });

 const { data: usersWithPhoneNumber } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: '55501',
 limit: 500,
 });

 const allUsersMap = new Map();
 [...usersWithEmail, ...usersWithPhoneNumber].forEach(user => {
 allUsersMap.set(user.id, user);
 });
 users = Array.from(allUsersMap.values());
 }
 } catch (error) {
 instanceSummary.errors.push(`Failed to get users: ${error.message}`);
 console.error(`Error getting users for ${entry.instanceName}:`, error);
Expand All@@ -75,10 +84,14 @@ setup('cleanup instances ', async () => {
 // Get organizations with error handling
 let orgs: any[] = [];
 try {
 const { data: orgsData } = await clerkClient.organizations.getOrganizationList({
 limit: 500,
 });
 orgs = orgsData;
 if (runMarker) {
 orgs = [];
 } else {
 const { data: orgsData } = await clerkClient.organizations.getOrganizationList({
 limit: 500,
 });
 orgs = orgsData;
 }
 } catch (error) {
 // Treat 404 (not found) and 403 (forbidden) as "no orgs"
 // 404 = no organizations exist, 403 = no permission to access organizations
Expand All@@ -91,8 +104,11 @@ setup('cleanup instances ', async () => {
 }
 }

 const usersToDelete = batchElements(skipObjectsThatWereCreatedWithinTheLast10Minutes(users), 5);
 const orgsToDelete = batchElements(skipObjectsThatWereCreatedWithinTheLast10Minutes(orgs), 5);
 const usersToDelete = batchElements(
 runMarker ? users : skipObjectsThatWereCreatedWithinTheLast10Minutes(users),
 5,
 );
 const orgsToDelete = batchElements(runMarker ? [] : skipObjectsThatWereCreatedWithinTheLast10Minutes(orgs), 5);

 // Delete users with tracking
 for (const batch of usersToDelete) {
Expand DownExpand Up@@ -142,6 +158,13 @@ setup('cleanup instances ', async () => {
 await new Promise(r => setTimeout(r, 1000));
 }

 if (runMarker) {
 const remainingUsers = await findE2ERunUsers(clerkClient, runMarker);
 if (remainingUsers.length > 0) {
 instanceSummary.errors.push(`Users remain after cleanup: ${remainingUsers.map(user => user.id).join(', ')}`);
 }
 }

 // Report instance results
 const maskedKey = entry.secretKey.replace(/(sk_(test|live)_)(.+)(...)/, '$1***$4');
 if (instanceSummary.usersDeleted > 0 || instanceSummary.orgsDeleted > 0) {
Expand Down

1 change: 1 addition & 0 deletions
integration/playwright.cleanup.config.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -9,6 +9,7 @@ config({ path: path.resolve(__dirname, '.env.local') });
export default defineConfig({
 ...common,
 testDir: './cleanup',
 retries: 0,
 projects: [
 {
 name: 'setup',
Expand Down

42 changes: 42 additions & 0 deletions
integration/testUtils/e2eRun.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
import { createHash } from 'node:crypto';

import type { ClerkClient, User } from '@clerk/backend';

type E2EUserRecord = {
 username: string | null;
 emailAddresses: Array<{ emailAddress: string }>;
 privateMetadata: Record<string, unknown>;
};

export const getE2ERunMarker = (runKey = process.env.INTEGRATION_TEST_RUN_KEY): string | undefined => {
 if (!runKey) {
 return;
 }

 const digest = createHash('sha256').update(runKey).digest('hex').slice(0, 20);
 return `e2e_${digest}`;
};

export const userMatchesE2ERun = (user: E2EUserRecord, marker: string): boolean =>
 Boolean(
 user.username?.includes(marker) ||
 user.emailAddresses.some(email => email.emailAddress.includes(marker)) ||
 user.privateMetadata.e2eRunMarker === marker,
 );

export const findE2ERunUsers = async (clerkClient: ClerkClient, marker: string): Promise<User[]> => {
 const usersById = new Map<string, User>();
 let offset = 0;

 while (true) {
 const { data } = await clerkClient.users.getUserList({ query: marker, limit: 100, offset });
 data.filter(user => userMatchesE2ERun(user, marker)).forEach(user => usersById.set(user.id, user));

 if (data.length < 100) {
 break;
 }
 offset += data.length;
 }

 return Array.from(usersById.values());
};

51 changes: 34 additions & 17 deletions
integration/testUtils/usersService.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,7 @@ import { faker } from '@faker-js/faker';
import type { TestInfo } from '@playwright/test';

import { fakerPassword, hash } from '../models/helpers';
import { getE2ERunMarker } from './e2eRun';

async function withErrorLogging<T>(operation: string, fn: () => Promise<T>): Promise<T> {
 try {
Expand DownExpand Up@@ -129,24 +130,29 @@ export const createUserService = (clerkClient: ClerkClient) => {
 withUsername = false,
 } = options || {};
 const randomHash = hash();
 const runMarker = getE2ERunMarker();
 const markedHash = runMarker ? `${runMarker}_${randomHash}` : randomHash;
 const email = fictionalEmail
 ? `${randomHash}+clerk_test@clerkcookie.com`
 : `clerkcookie+${randomHash}@mailsac.com`;
 ? `${markedHash}+clerk_test@clerkcookie.com`
 : `clerkcookie+${markedHash}@mailsac.com`;
 const phoneNumber = fakerPhoneNumber();
 const { file, line, title, titlePath } = test.info();
 const fakeUserEmail = withEmail ? email : undefined;
 const fakeUserPhoneNumber = withPhoneNumber ? phoneNumber : undefined;

 return {
 firstName: faker.person.firstName(),
 lastName: faker.person.lastName(),
 email: withEmail ? email : undefined,
 username: withUsername ? `${randomHash}_clerk_cookie` : undefined,
 email: fakeUserEmail,
 username: withUsername ? `${markedHash}_clerk_cookie` : undefined,
 password: withPassword ? fakerPassword() : undefined,
 phoneNumber: withPhoneNumber ? phoneNumber : undefined,
 phoneNumber: fakeUserPhoneNumber,
 privateMetadata: {
 title,
 titlePath,
 file,
 line,
 ...(runMarker ? { e2eRunMarker: runMarker } : {}),
 },
 deleteIfExists: () => self.deleteIfExists({ email, phoneNumber }),
 };
Expand All@@ -173,24 +179,35 @@ export const createUserService = (clerkClient: ClerkClient) => {
 return await self.createBapiUser(fakeUser);
 },
 deleteIfExists: async (opts: { id?: string; email?: string; phoneNumber?: string }) => {
 let id = opts.id;
 const [usersByEmail, usersByPhoneNumber] = await Promise.all([
 opts.email
 ? withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 emailAddress: [opts.email],
 }),
 )
 : undefined,
 opts.phoneNumber
 ? withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 phoneNumber: [opts.phoneNumber],
 }),
 )
 : undefined,
 ]);

 if (!id) {
 const { data: users } = await withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 emailAddress: [opts.email],
 phoneNumber: [opts.phoneNumber],
 }),
 );
 id = users[0]?.id;
 }
 const ids = new Set([
 ...(opts.id ? [opts.id] : []),
 ...(usersByEmail?.data.map(user => user.id) ?? []),
 ...(usersByPhoneNumber?.data.map(user => user.id) ?? []),
 ]);

 if (!id) {
 if (ids.size === 0) {
 console.log(`User "${opts.email || opts.phoneNumber}" does not exist!`);
 return;

Comment on lines

+205
 to 
207

@coderabbitaicoderabbitaiBotAug 10, 2026

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
Do not log the supplied email address or phone number.
Line 206 writes a user identifier to CI logs. Log a generic cleanup message instead.
Proposed fix
- console.log(`User "${opts.email || opts.phoneNumber}" does not exist!`);+ console.log('No user exists for the supplied cleanup criteria.');
📝 Committable suggestion
‼️IMPORTANT

Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

if(ids.size===0){
console.log(`User "${opts.email||opts.phoneNumber}" does not exist!`);
return;
if(ids.size===0){
console.log('No user exists for the supplied cleanup criteria.');
return;
🧰 Tools
🪛 ast-grep (0.45.0)
[warning] 205-205: Avoid logging sensitive data

Context: console.log(User "${opts.email || opts.phoneNumber}" does not exist!)

Note: [CWE-532] Insertion of Sensitive Information into Log File.
(log-sensitive-data-typescript)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@integration/testUtils/usersService.ts` around lines 205 - 207, Update the
empty-user branch in the user cleanup flow to replace the interpolated
opts.email and opts.phoneNumber values with a generic cleanup message, ensuring
no supplied user identifier is written to logs while preserving the early
return.

Sources: Coding guidelines, Linters/SAST tools
 }

 await withErrorLogging('deleteUser', () => clerkClient.users.deleteUser(id));
 await Promise.all(Array.from(ids, id => withErrorLogging('deleteUser', () => clerkClient.users.deleteUser(id))));
 },
 getUser: async (opts: { id?: string; email?: string }) => {
 if (opts.id) {
Expand Down

1 change: 1 addition & 0 deletions
turbo.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,7 @@
 "globalPassThroughEnv": [
 "AWS_SECRET_KEY",
 "GITHUB_TOKEN",
 "INTEGRATION_TEST_RUN_KEY",
@coderabbitaicoderabbitaiBotAug 10, 2026

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
# Every integration task must report cache=false.
jq -r ' (.tasks // {}) | to_entries[] | select(.key | test("test:integration")) | "\(.key): cache=\(.value.cache // true)"' turbo.json
Repository: clerk/javascript
Length of output: 1212
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- turbo.json ---'
cat -n turbo.json | sed -n '1,90p'printf'%s\n''--- Turbo version and integration scripts ---'
rg -n '"turbo"|test:integration|INTEGRATION_TEST_RUN_KEY' package.json pnpm-lock.yaml packages apps examples .github 2>/dev/null | head -240
Repository: clerk/javascript
Length of output: 7965
🌐 Web query:
Turborepo globalPassThroughEnv cache key behavior passThroughEnv cacheable task
💡 Result:
In Turborepo, both globalPassThroughEnv and passThroughEnv serve to expose environment variables to task runtimes without including them in the task's cache key [1][2][3]. Because these variables are intentionally omitted from the hashing process, changes to their values will not trigger a cache miss, even if the change would logically affect the output of the task [1][4]. The fundamental behaviors for these configurations are: 1. Cache Key Exclusion: Variables listed in either globalPassThroughEnv or passThroughEnv do not contribute to the task hash [2][3]. Consequently, Turborepo may serve a cached result even if the underlying environment variable has changed [1][4]. This is why these keys should only be used for variables that do not affect the build output, such as authentication tokens (e.g., SENTRY_AUTH_TOKEN), CI metadata, or credentials consumed at runtime after the build process [2][4]. 2. Runtime Exposure: By including a variable in either key, you satisfy Turborepo's Strict Environment Mode, ensuring the task has access to that variable at runtime [1][5][3]. If a task attempts to access an environment variable that is not accounted for in env, globalEnv, passThroughEnv, or globalPassThroughEnv while in Strict Mode, the task may fail [1][5]. 3. Scope Differences: - globalPassThroughEnv (or global.passThroughEnv when using the global configuration flag) applies the passthrough behavior to all tasks within the repository [2][6]. - passThroughEnv applies only to the specific task where it is configured [3]. If you need a change in an environment variable to invalidate the cache and trigger a rebuild, you must include that variable in env or globalEnv instead of the passthrough keys [1][3].
Citations:
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- integration task definitions ---'
rg -n -C 3 '"[`#/`]?test:integration|test:integration' turbo.json
printf'%s\n''--- CI Turbo arguments and integration invocation ---'
sed -n '400,535p' .github/workflows/ci.yml
sed -n '260,300p' .github/workflows/e2e-staging.yml
printf'%s\n''--- exact task keys parsed from turbo.json ---'
jq -r '(.tasks // {}) | to_entries[] | select(.key | contains("test:integration")) | [.key, (.value.cache // true | tostring)] | `@tsv`' turbo.json
Repository: clerk/javascript
Length of output: 16346
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
python3 - <<'PY'import jsonfrom pathlib import Pathconfig = json.loads(Path("turbo.json").read_text())tasks = config.get("tasks", {})matches = [(name, task.get("cache", True)) for name, task in tasks.items() if "test:integration" in name]print("matching task count:", len(matches))for name, cache in matches: print(f"{name}: cache={cache}")PY
Repository: clerk/javascript
Length of output: 1236
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- all Turbo force/argument settings ---'
rg -n -C 4 'TURBO_(ARGS|FORCE)|pnpm turbo test:integration|pnpm turbo run test:integration' .github/workflows package.json
printf'%s\n''--- CI workflow environment and matrix context ---'
sed -n '1,120p' .github/workflows/ci.yml
sed -n '380,445p' .github/workflows/ci.yml
sed -n '495,535p' .github/workflows/ci.yml
printf'%s\n''--- staging workflow context ---'
sed -n '1,110p' .github/workflows/e2e-staging.yml
sed -n '270,320p' .github/workflows/e2e-staging.yml
Repository: clerk/javascript
Length of output: 30087
Disable caching for CI integration tasks.
The CI integration job does not set TURBO_FORCE, and all 25 integration tasks inherit cache: true. globalPassThroughEnv does not affect the cache key. Set cache: false for these tasks, or move INTEGRATION_TEST_RUN_KEY to globalEnv.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@turbo.json` at line 37, Update the Turbo integration task configuration
containing INTEGRATION_TEST_RUN_KEY so CI integration tasks do not reuse cached
results: set cache to false for those tasks, or move INTEGRATION_TEST_RUN_KEY
into globalEnv to include it in the cache key.

 "ACTIONS_RUNNER_DEBUG",
 "ACTIONS_STEP_DEBUG",
 "VERCEL_AUTOMATION_BYPASS_SECRET",
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix(e2e): more robust user deletion#9375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

dstaley
merged 8 commits into

from

Aug 11, 2026
 Merged


fix(e2e): more robust user deletion
#9375
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
6b841fc

fix(e2e): delete all matching users


dstaley Aug 10, 2026
d76132e

fix(e2e): mark created users, always cleanup


dstaley Aug 10, 2026
2bd619c

fix: don't retry or fail on cleanup


dstaley Aug 10, 2026
a326a24

fix(e2e): use underscore


dstaley Aug 10, 2026
7513812

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 10, 2026
3c096a3

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 10, 2026
ad2f98a

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 11, 2026
417bb3b

fix(repo): rm extra changeset


dstaley Aug 11, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 2 additions & 0 deletions
.changeset/silver-lands-cut.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
---
---

17 changes: 17 additions & 0 deletions
.github/workflows/ci.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -504,6 +504,15 @@ jobs:
 sudo apt-get install -y xvfb
 fi

 - name: Configure test user cleanup
 run: echo "INTEGRATION_TEST_RUN_KEY=${RUN_ID}-${RUN_ATTEMPT}-${TEST_NAME}-${TEST_PROJECT}-${NEXT_VERSION:-default}" >> "$GITHUB_ENV"
 env:
 RUN_ID: ${{ github.run_id }}
 RUN_ATTEMPT: ${{ github.run_attempt }}
 TEST_NAME: ${{ matrix.test-name }}
 TEST_PROJECT: ${{ matrix.test-project }}
 NEXT_VERSION: ${{ matrix.next-version }}

 - name: Run Integration Tests
 id: integration-tests
 timeout-minutes: 25
Expand All@@ -525,6 +534,14 @@ jobs:
 NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem
 VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}

 - name: Delete integration-test users
 if: ${{ always() && steps.integration-tests.outcome != 'skipped' }}
 timeout-minutes: 4
 run: pnpm test:integration:cleanup
 env:
 INTEGRATION_INSTANCE_KEYS: ${{ secrets.INTEGRATION_INSTANCE_KEYS }}
 NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem

 - name: Sanitize artifact name
 if: ${{ cancelled() || failure() }}
 id: sanitize
Expand Down

73 changes: 48 additions & 25 deletions
integration/cleanup/cleanup.setup.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -5,8 +5,11 @@ import { isStaging } from '@clerk/shared/utils';
import { test as setup } from '@playwright/test';

import { appConfigs } from '../presets/';
import { findE2ERunUsers, getE2ERunMarker } from '../testUtils/e2eRun';
import { withRetry } from '../testUtils/retryableClerkClient';

setup('cleanup instances ', async () => {
 const runMarker = getE2ERunMarker();
 const entries = Array.from(appConfigs.secrets.instanceKeys.values())
 .map(({ pk, sk }) => {
 const secretKey = sk;
Expand All@@ -32,6 +35,9 @@ setup('cleanup instances ', async () => {
 }> = [];

 console.log('🧹 Starting E2E Test Cleanup Process...\n');
 if (runMarker) {
 console.log(`Cleaning users for run marker ${runMarker}\n`);
 }

 for (const entry of entries) {
 const instanceSummary = {
Expand All@@ -43,29 +49,32 @@ setup('cleanup instances ', async () => {
 };

 try {
 const clerkClient = createClerkClient({ secretKey: entry.secretKey, apiUrl: entry.apiUrl });
 const clerkClient = withRetry(createClerkClient({ secretKey: entry.secretKey, apiUrl: entry.apiUrl }));

 // Get users with error handling
 let users: any[] = [];
 try {
 const { data: usersWithEmail } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: 'clerkcookie',
 limit: 500,
 });

 const { data: usersWithPhoneNumber } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: '55501',
 limit: 500,
 });

 // Deduplicate users by ID
 const allUsersMap = new Map();
 [...usersWithEmail, ...usersWithPhoneNumber].forEach(user => {
 allUsersMap.set(user.id, user);
 });
 users = Array.from(allUsersMap.values());
 if (runMarker) {
 users = await findE2ERunUsers(clerkClient, runMarker);
 } else {
 const { data: usersWithEmail } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: 'clerkcookie',
 limit: 500,
 });

 const { data: usersWithPhoneNumber } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: '55501',
 limit: 500,
 });

 const allUsersMap = new Map();
 [...usersWithEmail, ...usersWithPhoneNumber].forEach(user => {
 allUsersMap.set(user.id, user);
 });
 users = Array.from(allUsersMap.values());
 }
 } catch (error) {
 instanceSummary.errors.push(`Failed to get users: ${error.message}`);
 console.error(`Error getting users for ${entry.instanceName}:`, error);
Expand All@@ -75,10 +84,14 @@ setup('cleanup instances ', async () => {
 // Get organizations with error handling
 let orgs: any[] = [];
 try {
 const { data: orgsData } = await clerkClient.organizations.getOrganizationList({
 limit: 500,
 });
 orgs = orgsData;
 if (runMarker) {
 orgs = [];
 } else {
 const { data: orgsData } = await clerkClient.organizations.getOrganizationList({
 limit: 500,
 });
 orgs = orgsData;
 }
 } catch (error) {
 // Treat 404 (not found) and 403 (forbidden) as "no orgs"
 // 404 = no organizations exist, 403 = no permission to access organizations
Expand All@@ -91,8 +104,11 @@ setup('cleanup instances ', async () => {
 }
 }

 const usersToDelete = batchElements(skipObjectsThatWereCreatedWithinTheLast10Minutes(users), 5);
 const orgsToDelete = batchElements(skipObjectsThatWereCreatedWithinTheLast10Minutes(orgs), 5);
 const usersToDelete = batchElements(
 runMarker ? users : skipObjectsThatWereCreatedWithinTheLast10Minutes(users),
 5,
 );
 const orgsToDelete = batchElements(runMarker ? [] : skipObjectsThatWereCreatedWithinTheLast10Minutes(orgs), 5);

 // Delete users with tracking
 for (const batch of usersToDelete) {
Expand DownExpand Up@@ -142,6 +158,13 @@ setup('cleanup instances ', async () => {
 await new Promise(r => setTimeout(r, 1000));
 }

 if (runMarker) {
 const remainingUsers = await findE2ERunUsers(clerkClient, runMarker);
 if (remainingUsers.length > 0) {
 instanceSummary.errors.push(`Users remain after cleanup: ${remainingUsers.map(user => user.id).join(', ')}`);
 }
 }

 // Report instance results
 const maskedKey = entry.secretKey.replace(/(sk_(test|live)_)(.+)(...)/, '$1***$4');
 if (instanceSummary.usersDeleted > 0 || instanceSummary.orgsDeleted > 0) {
Expand Down

1 change: 1 addition & 0 deletions
integration/playwright.cleanup.config.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -9,6 +9,7 @@ config({ path: path.resolve(__dirname, '.env.local') });
export default defineConfig({
 ...common,
 testDir: './cleanup',
 retries: 0,
 projects: [
 {
 name: 'setup',
Expand Down

42 changes: 42 additions & 0 deletions
integration/testUtils/e2eRun.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
import { createHash } from 'node:crypto';

import type { ClerkClient, User } from '@clerk/backend';

type E2EUserRecord = {
 username: string | null;
 emailAddresses: Array<{ emailAddress: string }>;
 privateMetadata: Record<string, unknown>;
};

export const getE2ERunMarker = (runKey = process.env.INTEGRATION_TEST_RUN_KEY): string | undefined => {
 if (!runKey) {
 return;
 }

 const digest = createHash('sha256').update(runKey).digest('hex').slice(0, 20);
 return `e2e_${digest}`;
};

export const userMatchesE2ERun = (user: E2EUserRecord, marker: string): boolean =>
 Boolean(
 user.username?.includes(marker) ||
 user.emailAddresses.some(email => email.emailAddress.includes(marker)) ||
 user.privateMetadata.e2eRunMarker === marker,
 );

export const findE2ERunUsers = async (clerkClient: ClerkClient, marker: string): Promise<User[]> => {
 const usersById = new Map<string, User>();
 let offset = 0;

 while (true) {
 const { data } = await clerkClient.users.getUserList({ query: marker, limit: 100, offset });
 data.filter(user => userMatchesE2ERun(user, marker)).forEach(user => usersById.set(user.id, user));

 if (data.length < 100) {
 break;
 }
 offset += data.length;
 }

 return Array.from(usersById.values());
};

51 changes: 34 additions & 17 deletions
integration/testUtils/usersService.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,7 @@ import { faker } from '@faker-js/faker';
import type { TestInfo } from '@playwright/test';

import { fakerPassword, hash } from '../models/helpers';
import { getE2ERunMarker } from './e2eRun';

async function withErrorLogging<T>(operation: string, fn: () => Promise<T>): Promise<T> {
 try {
Expand DownExpand Up@@ -129,24 +130,29 @@ export const createUserService = (clerkClient: ClerkClient) => {
 withUsername = false,
 } = options || {};
 const randomHash = hash();
 const runMarker = getE2ERunMarker();
 const markedHash = runMarker ? `${runMarker}_${randomHash}` : randomHash;
 const email = fictionalEmail
 ? `${randomHash}+clerk_test@clerkcookie.com`
 : `clerkcookie+${randomHash}@mailsac.com`;
 ? `${markedHash}+clerk_test@clerkcookie.com`
 : `clerkcookie+${markedHash}@mailsac.com`;
 const phoneNumber = fakerPhoneNumber();
 const { file, line, title, titlePath } = test.info();
 const fakeUserEmail = withEmail ? email : undefined;
 const fakeUserPhoneNumber = withPhoneNumber ? phoneNumber : undefined;

 return {
 firstName: faker.person.firstName(),
 lastName: faker.person.lastName(),
 email: withEmail ? email : undefined,
 username: withUsername ? `${randomHash}_clerk_cookie` : undefined,
 email: fakeUserEmail,
 username: withUsername ? `${markedHash}_clerk_cookie` : undefined,
 password: withPassword ? fakerPassword() : undefined,
 phoneNumber: withPhoneNumber ? phoneNumber : undefined,
 phoneNumber: fakeUserPhoneNumber,
 privateMetadata: {
 title,
 titlePath,
 file,
 line,
 ...(runMarker ? { e2eRunMarker: runMarker } : {}),
 },
 deleteIfExists: () => self.deleteIfExists({ email, phoneNumber }),
 };
Expand All@@ -173,24 +179,35 @@ export const createUserService = (clerkClient: ClerkClient) => {
 return await self.createBapiUser(fakeUser);
 },
 deleteIfExists: async (opts: { id?: string; email?: string; phoneNumber?: string }) => {
 let id = opts.id;
 const [usersByEmail, usersByPhoneNumber] = await Promise.all([
 opts.email
 ? withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 emailAddress: [opts.email],
 }),
 )
 : undefined,
 opts.phoneNumber
 ? withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 phoneNumber: [opts.phoneNumber],
 }),
 )
 : undefined,
 ]);

 if (!id) {
 const { data: users } = await withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 emailAddress: [opts.email],
 phoneNumber: [opts.phoneNumber],
 }),
 );
 id = users[0]?.id;
 }
 const ids = new Set([
 ...(opts.id ? [opts.id] : []),
 ...(usersByEmail?.data.map(user => user.id) ?? []),
 ...(usersByPhoneNumber?.data.map(user => user.id) ?? []),
 ]);

 if (!id) {
 if (ids.size === 0) {
 console.log(`User "${opts.email || opts.phoneNumber}" does not exist!`);
 return;

Comment on lines

+205
 to 
207

@coderabbitaicoderabbitaiBotAug 10, 2026

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
Do not log the supplied email address or phone number.
Line 206 writes a user identifier to CI logs. Log a generic cleanup message instead.
Proposed fix
- console.log(`User "${opts.email || opts.phoneNumber}" does not exist!`);+ console.log('No user exists for the supplied cleanup criteria.');
📝 Committable suggestion
‼️IMPORTANT

Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

if(ids.size===0){
console.log(`User "${opts.email||opts.phoneNumber}" does not exist!`);
return;
if(ids.size===0){
console.log('No user exists for the supplied cleanup criteria.');
return;
🧰 Tools
🪛 ast-grep (0.45.0)
[warning] 205-205: Avoid logging sensitive data

Context: console.log(User "${opts.email || opts.phoneNumber}" does not exist!)

Note: [CWE-532] Insertion of Sensitive Information into Log File.
(log-sensitive-data-typescript)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@integration/testUtils/usersService.ts` around lines 205 - 207, Update the
empty-user branch in the user cleanup flow to replace the interpolated
opts.email and opts.phoneNumber values with a generic cleanup message, ensuring
no supplied user identifier is written to logs while preserving the early
return.

Sources: Coding guidelines, Linters/SAST tools
 }

 await withErrorLogging('deleteUser', () => clerkClient.users.deleteUser(id));
 await Promise.all(Array.from(ids, id => withErrorLogging('deleteUser', () => clerkClient.users.deleteUser(id))));
 },
 getUser: async (opts: { id?: string; email?: string }) => {
 if (opts.id) {
Expand Down

1 change: 1 addition & 0 deletions
turbo.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,7 @@
 "globalPassThroughEnv": [
 "AWS_SECRET_KEY",
 "GITHUB_TOKEN",
 "INTEGRATION_TEST_RUN_KEY",
@coderabbitaicoderabbitaiBotAug 10, 2026

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
# Every integration task must report cache=false.
jq -r ' (.tasks // {}) | to_entries[] | select(.key | test("test:integration")) | "\(.key): cache=\(.value.cache // true)"' turbo.json
Repository: clerk/javascript
Length of output: 1212
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- turbo.json ---'
cat -n turbo.json | sed -n '1,90p'printf'%s\n''--- Turbo version and integration scripts ---'
rg -n '"turbo"|test:integration|INTEGRATION_TEST_RUN_KEY' package.json pnpm-lock.yaml packages apps examples .github 2>/dev/null | head -240
Repository: clerk/javascript
Length of output: 7965
🌐 Web query:
Turborepo globalPassThroughEnv cache key behavior passThroughEnv cacheable task
💡 Result:
In Turborepo, both globalPassThroughEnv and passThroughEnv serve to expose environment variables to task runtimes without including them in the task's cache key [1][2][3]. Because these variables are intentionally omitted from the hashing process, changes to their values will not trigger a cache miss, even if the change would logically affect the output of the task [1][4]. The fundamental behaviors for these configurations are: 1. Cache Key Exclusion: Variables listed in either globalPassThroughEnv or passThroughEnv do not contribute to the task hash [2][3]. Consequently, Turborepo may serve a cached result even if the underlying environment variable has changed [1][4]. This is why these keys should only be used for variables that do not affect the build output, such as authentication tokens (e.g., SENTRY_AUTH_TOKEN), CI metadata, or credentials consumed at runtime after the build process [2][4]. 2. Runtime Exposure: By including a variable in either key, you satisfy Turborepo's Strict Environment Mode, ensuring the task has access to that variable at runtime [1][5][3]. If a task attempts to access an environment variable that is not accounted for in env, globalEnv, passThroughEnv, or globalPassThroughEnv while in Strict Mode, the task may fail [1][5]. 3. Scope Differences: - globalPassThroughEnv (or global.passThroughEnv when using the global configuration flag) applies the passthrough behavior to all tasks within the repository [2][6]. - passThroughEnv applies only to the specific task where it is configured [3]. If you need a change in an environment variable to invalidate the cache and trigger a rebuild, you must include that variable in env or globalEnv instead of the passthrough keys [1][3].
Citations:
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- integration task definitions ---'
rg -n -C 3 '"[`#/`]?test:integration|test:integration' turbo.json
printf'%s\n''--- CI Turbo arguments and integration invocation ---'
sed -n '400,535p' .github/workflows/ci.yml
sed -n '260,300p' .github/workflows/e2e-staging.yml
printf'%s\n''--- exact task keys parsed from turbo.json ---'
jq -r '(.tasks // {}) | to_entries[] | select(.key | contains("test:integration")) | [.key, (.value.cache // true | tostring)] | `@tsv`' turbo.json
Repository: clerk/javascript
Length of output: 16346
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
python3 - <<'PY'import jsonfrom pathlib import Pathconfig = json.loads(Path("turbo.json").read_text())tasks = config.get("tasks", {})matches = [(name, task.get("cache", True)) for name, task in tasks.items() if "test:integration" in name]print("matching task count:", len(matches))for name, cache in matches: print(f"{name}: cache={cache}")PY
Repository: clerk/javascript
Length of output: 1236
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- all Turbo force/argument settings ---'
rg -n -C 4 'TURBO_(ARGS|FORCE)|pnpm turbo test:integration|pnpm turbo run test:integration' .github/workflows package.json
printf'%s\n''--- CI workflow environment and matrix context ---'
sed -n '1,120p' .github/workflows/ci.yml
sed -n '380,445p' .github/workflows/ci.yml
sed -n '495,535p' .github/workflows/ci.yml
printf'%s\n''--- staging workflow context ---'
sed -n '1,110p' .github/workflows/e2e-staging.yml
sed -n '270,320p' .github/workflows/e2e-staging.yml
Repository: clerk/javascript
Length of output: 30087
Disable caching for CI integration tasks.
The CI integration job does not set TURBO_FORCE, and all 25 integration tasks inherit cache: true. globalPassThroughEnv does not affect the cache key. Set cache: false for these tasks, or move INTEGRATION_TEST_RUN_KEY to globalEnv.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@turbo.json` at line 37, Update the Turbo integration task configuration
containing INTEGRATION_TEST_RUN_KEY so CI integration tasks do not reuse cached
results: set cache to false for those tasks, or move INTEGRATION_TEST_RUN_KEY
into globalEnv to include it in the cache key.

 "ACTIONS_RUNNER_DEBUG",
 "ACTIONS_STEP_DEBUG",
 "VERCEL_AUTOMATION_BYPASS_SECRET",
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix(e2e): more robust user deletion#9375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

dstaley
merged 8 commits into

from

Aug 11, 2026
 Merged


fix(e2e): more robust user deletion
#9375
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
6b841fc

fix(e2e): delete all matching users


dstaley Aug 10, 2026
d76132e

fix(e2e): mark created users, always cleanup


dstaley Aug 10, 2026
2bd619c

fix: don't retry or fail on cleanup


dstaley Aug 10, 2026
a326a24

fix(e2e): use underscore


dstaley Aug 10, 2026
7513812

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 10, 2026
3c096a3

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 10, 2026
ad2f98a

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 11, 2026
417bb3b

fix(repo): rm extra changeset


dstaley Aug 11, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 2 additions & 0 deletions
.changeset/silver-lands-cut.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
---
---

17 changes: 17 additions & 0 deletions
.github/workflows/ci.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -504,6 +504,15 @@ jobs:
 sudo apt-get install -y xvfb
 fi

 - name: Configure test user cleanup
 run: echo "INTEGRATION_TEST_RUN_KEY=${RUN_ID}-${RUN_ATTEMPT}-${TEST_NAME}-${TEST_PROJECT}-${NEXT_VERSION:-default}" >> "$GITHUB_ENV"
 env:
 RUN_ID: ${{ github.run_id }}
 RUN_ATTEMPT: ${{ github.run_attempt }}
 TEST_NAME: ${{ matrix.test-name }}
 TEST_PROJECT: ${{ matrix.test-project }}
 NEXT_VERSION: ${{ matrix.next-version }}

 - name: Run Integration Tests
 id: integration-tests
 timeout-minutes: 25
Expand All@@ -525,6 +534,14 @@ jobs:
 NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem
 VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}

 - name: Delete integration-test users
 if: ${{ always() && steps.integration-tests.outcome != 'skipped' }}
 timeout-minutes: 4
 run: pnpm test:integration:cleanup
 env:
 INTEGRATION_INSTANCE_KEYS: ${{ secrets.INTEGRATION_INSTANCE_KEYS }}
 NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem

 - name: Sanitize artifact name
 if: ${{ cancelled() || failure() }}
 id: sanitize
Expand Down

73 changes: 48 additions & 25 deletions
integration/cleanup/cleanup.setup.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -5,8 +5,11 @@ import { isStaging } from '@clerk/shared/utils';
import { test as setup } from '@playwright/test';

import { appConfigs } from '../presets/';
import { findE2ERunUsers, getE2ERunMarker } from '../testUtils/e2eRun';
import { withRetry } from '../testUtils/retryableClerkClient';

setup('cleanup instances ', async () => {
 const runMarker = getE2ERunMarker();
 const entries = Array.from(appConfigs.secrets.instanceKeys.values())
 .map(({ pk, sk }) => {
 const secretKey = sk;
Expand All@@ -32,6 +35,9 @@ setup('cleanup instances ', async () => {
 }> = [];

 console.log('🧹 Starting E2E Test Cleanup Process...\n');
 if (runMarker) {
 console.log(`Cleaning users for run marker ${runMarker}\n`);
 }

 for (const entry of entries) {
 const instanceSummary = {
Expand All@@ -43,29 +49,32 @@ setup('cleanup instances ', async () => {
 };

 try {
 const clerkClient = createClerkClient({ secretKey: entry.secretKey, apiUrl: entry.apiUrl });
 const clerkClient = withRetry(createClerkClient({ secretKey: entry.secretKey, apiUrl: entry.apiUrl }));

 // Get users with error handling
 let users: any[] = [];
 try {
 const { data: usersWithEmail } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: 'clerkcookie',
 limit: 500,
 });

 const { data: usersWithPhoneNumber } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: '55501',
 limit: 500,
 });

 // Deduplicate users by ID
 const allUsersMap = new Map();
 [...usersWithEmail, ...usersWithPhoneNumber].forEach(user => {
 allUsersMap.set(user.id, user);
 });
 users = Array.from(allUsersMap.values());
 if (runMarker) {
 users = await findE2ERunUsers(clerkClient, runMarker);
 } else {
 const { data: usersWithEmail } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: 'clerkcookie',
 limit: 500,
 });

 const { data: usersWithPhoneNumber } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: '55501',
 limit: 500,
 });

 const allUsersMap = new Map();
 [...usersWithEmail, ...usersWithPhoneNumber].forEach(user => {
 allUsersMap.set(user.id, user);
 });
 users = Array.from(allUsersMap.values());
 }
 } catch (error) {
 instanceSummary.errors.push(`Failed to get users: ${error.message}`);
 console.error(`Error getting users for ${entry.instanceName}:`, error);
Expand All@@ -75,10 +84,14 @@ setup('cleanup instances ', async () => {
 // Get organizations with error handling
 let orgs: any[] = [];
 try {
 const { data: orgsData } = await clerkClient.organizations.getOrganizationList({
 limit: 500,
 });
 orgs = orgsData;
 if (runMarker) {
 orgs = [];
 } else {
 const { data: orgsData } = await clerkClient.organizations.getOrganizationList({
 limit: 500,
 });
 orgs = orgsData;
 }
 } catch (error) {
 // Treat 404 (not found) and 403 (forbidden) as "no orgs"
 // 404 = no organizations exist, 403 = no permission to access organizations
Expand All@@ -91,8 +104,11 @@ setup('cleanup instances ', async () => {
 }
 }

 const usersToDelete = batchElements(skipObjectsThatWereCreatedWithinTheLast10Minutes(users), 5);
 const orgsToDelete = batchElements(skipObjectsThatWereCreatedWithinTheLast10Minutes(orgs), 5);
 const usersToDelete = batchElements(
 runMarker ? users : skipObjectsThatWereCreatedWithinTheLast10Minutes(users),
 5,
 );
 const orgsToDelete = batchElements(runMarker ? [] : skipObjectsThatWereCreatedWithinTheLast10Minutes(orgs), 5);

 // Delete users with tracking
 for (const batch of usersToDelete) {
Expand DownExpand Up@@ -142,6 +158,13 @@ setup('cleanup instances ', async () => {
 await new Promise(r => setTimeout(r, 1000));
 }

 if (runMarker) {
 const remainingUsers = await findE2ERunUsers(clerkClient, runMarker);
 if (remainingUsers.length > 0) {
 instanceSummary.errors.push(`Users remain after cleanup: ${remainingUsers.map(user => user.id).join(', ')}`);
 }
 }

 // Report instance results
 const maskedKey = entry.secretKey.replace(/(sk_(test|live)_)(.+)(...)/, '$1***$4');
 if (instanceSummary.usersDeleted > 0 || instanceSummary.orgsDeleted > 0) {
Expand Down

1 change: 1 addition & 0 deletions
integration/playwright.cleanup.config.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -9,6 +9,7 @@ config({ path: path.resolve(__dirname, '.env.local') });
export default defineConfig({
 ...common,
 testDir: './cleanup',
 retries: 0,
 projects: [
 {
 name: 'setup',
Expand Down

42 changes: 42 additions & 0 deletions
integration/testUtils/e2eRun.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
import { createHash } from 'node:crypto';

import type { ClerkClient, User } from '@clerk/backend';

type E2EUserRecord = {
 username: string | null;
 emailAddresses: Array<{ emailAddress: string }>;
 privateMetadata: Record<string, unknown>;
};

export const getE2ERunMarker = (runKey = process.env.INTEGRATION_TEST_RUN_KEY): string | undefined => {
 if (!runKey) {
 return;
 }

 const digest = createHash('sha256').update(runKey).digest('hex').slice(0, 20);
 return `e2e_${digest}`;
};

export const userMatchesE2ERun = (user: E2EUserRecord, marker: string): boolean =>
 Boolean(
 user.username?.includes(marker) ||
 user.emailAddresses.some(email => email.emailAddress.includes(marker)) ||
 user.privateMetadata.e2eRunMarker === marker,
 );

export const findE2ERunUsers = async (clerkClient: ClerkClient, marker: string): Promise<User[]> => {
 const usersById = new Map<string, User>();
 let offset = 0;

 while (true) {
 const { data } = await clerkClient.users.getUserList({ query: marker, limit: 100, offset });
 data.filter(user => userMatchesE2ERun(user, marker)).forEach(user => usersById.set(user.id, user));

 if (data.length < 100) {
 break;
 }
 offset += data.length;
 }

 return Array.from(usersById.values());
};

51 changes: 34 additions & 17 deletions
integration/testUtils/usersService.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,7 @@ import { faker } from '@faker-js/faker';
import type { TestInfo } from '@playwright/test';

import { fakerPassword, hash } from '../models/helpers';
import { getE2ERunMarker } from './e2eRun';

async function withErrorLogging<T>(operation: string, fn: () => Promise<T>): Promise<T> {
 try {
Expand DownExpand Up@@ -129,24 +130,29 @@ export const createUserService = (clerkClient: ClerkClient) => {
 withUsername = false,
 } = options || {};
 const randomHash = hash();
 const runMarker = getE2ERunMarker();
 const markedHash = runMarker ? `${runMarker}_${randomHash}` : randomHash;
 const email = fictionalEmail
 ? `${randomHash}+clerk_test@clerkcookie.com`
 : `clerkcookie+${randomHash}@mailsac.com`;
 ? `${markedHash}+clerk_test@clerkcookie.com`
 : `clerkcookie+${markedHash}@mailsac.com`;
 const phoneNumber = fakerPhoneNumber();
 const { file, line, title, titlePath } = test.info();
 const fakeUserEmail = withEmail ? email : undefined;
 const fakeUserPhoneNumber = withPhoneNumber ? phoneNumber : undefined;

 return {
 firstName: faker.person.firstName(),
 lastName: faker.person.lastName(),
 email: withEmail ? email : undefined,
 username: withUsername ? `${randomHash}_clerk_cookie` : undefined,
 email: fakeUserEmail,
 username: withUsername ? `${markedHash}_clerk_cookie` : undefined,
 password: withPassword ? fakerPassword() : undefined,
 phoneNumber: withPhoneNumber ? phoneNumber : undefined,
 phoneNumber: fakeUserPhoneNumber,
 privateMetadata: {
 title,
 titlePath,
 file,
 line,
 ...(runMarker ? { e2eRunMarker: runMarker } : {}),
 },
 deleteIfExists: () => self.deleteIfExists({ email, phoneNumber }),
 };
Expand All@@ -173,24 +179,35 @@ export const createUserService = (clerkClient: ClerkClient) => {
 return await self.createBapiUser(fakeUser);
 },
 deleteIfExists: async (opts: { id?: string; email?: string; phoneNumber?: string }) => {
 let id = opts.id;
 const [usersByEmail, usersByPhoneNumber] = await Promise.all([
 opts.email
 ? withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 emailAddress: [opts.email],
 }),
 )
 : undefined,
 opts.phoneNumber
 ? withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 phoneNumber: [opts.phoneNumber],
 }),
 )
 : undefined,
 ]);

 if (!id) {
 const { data: users } = await withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 emailAddress: [opts.email],
 phoneNumber: [opts.phoneNumber],
 }),
 );
 id = users[0]?.id;
 }
 const ids = new Set([
 ...(opts.id ? [opts.id] : []),
 ...(usersByEmail?.data.map(user => user.id) ?? []),
 ...(usersByPhoneNumber?.data.map(user => user.id) ?? []),
 ]);

 if (!id) {
 if (ids.size === 0) {
 console.log(`User "${opts.email || opts.phoneNumber}" does not exist!`);
 return;

Comment on lines

+205
 to 
207

@coderabbitaicoderabbitaiBotAug 10, 2026

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
Do not log the supplied email address or phone number.
Line 206 writes a user identifier to CI logs. Log a generic cleanup message instead.
Proposed fix
- console.log(`User "${opts.email || opts.phoneNumber}" does not exist!`);+ console.log('No user exists for the supplied cleanup criteria.');
📝 Committable suggestion
‼️IMPORTANT

Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

if(ids.size===0){
console.log(`User "${opts.email||opts.phoneNumber}" does not exist!`);
return;
if(ids.size===0){
console.log('No user exists for the supplied cleanup criteria.');
return;
🧰 Tools
🪛 ast-grep (0.45.0)
[warning] 205-205: Avoid logging sensitive data

Context: console.log(User "${opts.email || opts.phoneNumber}" does not exist!)

Note: [CWE-532] Insertion of Sensitive Information into Log File.
(log-sensitive-data-typescript)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@integration/testUtils/usersService.ts` around lines 205 - 207, Update the
empty-user branch in the user cleanup flow to replace the interpolated
opts.email and opts.phoneNumber values with a generic cleanup message, ensuring
no supplied user identifier is written to logs while preserving the early
return.

Sources: Coding guidelines, Linters/SAST tools
 }

 await withErrorLogging('deleteUser', () => clerkClient.users.deleteUser(id));
 await Promise.all(Array.from(ids, id => withErrorLogging('deleteUser', () => clerkClient.users.deleteUser(id))));
 },
 getUser: async (opts: { id?: string; email?: string }) => {
 if (opts.id) {
Expand Down

1 change: 1 addition & 0 deletions
turbo.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,7 @@
 "globalPassThroughEnv": [
 "AWS_SECRET_KEY",
 "GITHUB_TOKEN",
 "INTEGRATION_TEST_RUN_KEY",
@coderabbitaicoderabbitaiBotAug 10, 2026

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
# Every integration task must report cache=false.
jq -r ' (.tasks // {}) | to_entries[] | select(.key | test("test:integration")) | "\(.key): cache=\(.value.cache // true)"' turbo.json
Repository: clerk/javascript
Length of output: 1212
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- turbo.json ---'
cat -n turbo.json | sed -n '1,90p'printf'%s\n''--- Turbo version and integration scripts ---'
rg -n '"turbo"|test:integration|INTEGRATION_TEST_RUN_KEY' package.json pnpm-lock.yaml packages apps examples .github 2>/dev/null | head -240
Repository: clerk/javascript
Length of output: 7965
🌐 Web query:
Turborepo globalPassThroughEnv cache key behavior passThroughEnv cacheable task
💡 Result:
In Turborepo, both globalPassThroughEnv and passThroughEnv serve to expose environment variables to task runtimes without including them in the task's cache key [1][2][3]. Because these variables are intentionally omitted from the hashing process, changes to their values will not trigger a cache miss, even if the change would logically affect the output of the task [1][4]. The fundamental behaviors for these configurations are: 1. Cache Key Exclusion: Variables listed in either globalPassThroughEnv or passThroughEnv do not contribute to the task hash [2][3]. Consequently, Turborepo may serve a cached result even if the underlying environment variable has changed [1][4]. This is why these keys should only be used for variables that do not affect the build output, such as authentication tokens (e.g., SENTRY_AUTH_TOKEN), CI metadata, or credentials consumed at runtime after the build process [2][4]. 2. Runtime Exposure: By including a variable in either key, you satisfy Turborepo's Strict Environment Mode, ensuring the task has access to that variable at runtime [1][5][3]. If a task attempts to access an environment variable that is not accounted for in env, globalEnv, passThroughEnv, or globalPassThroughEnv while in Strict Mode, the task may fail [1][5]. 3. Scope Differences: - globalPassThroughEnv (or global.passThroughEnv when using the global configuration flag) applies the passthrough behavior to all tasks within the repository [2][6]. - passThroughEnv applies only to the specific task where it is configured [3]. If you need a change in an environment variable to invalidate the cache and trigger a rebuild, you must include that variable in env or globalEnv instead of the passthrough keys [1][3].
Citations:
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- integration task definitions ---'
rg -n -C 3 '"[`#/`]?test:integration|test:integration' turbo.json
printf'%s\n''--- CI Turbo arguments and integration invocation ---'
sed -n '400,535p' .github/workflows/ci.yml
sed -n '260,300p' .github/workflows/e2e-staging.yml
printf'%s\n''--- exact task keys parsed from turbo.json ---'
jq -r '(.tasks // {}) | to_entries[] | select(.key | contains("test:integration")) | [.key, (.value.cache // true | tostring)] | `@tsv`' turbo.json
Repository: clerk/javascript
Length of output: 16346
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
python3 - <<'PY'import jsonfrom pathlib import Pathconfig = json.loads(Path("turbo.json").read_text())tasks = config.get("tasks", {})matches = [(name, task.get("cache", True)) for name, task in tasks.items() if "test:integration" in name]print("matching task count:", len(matches))for name, cache in matches: print(f"{name}: cache={cache}")PY
Repository: clerk/javascript
Length of output: 1236
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- all Turbo force/argument settings ---'
rg -n -C 4 'TURBO_(ARGS|FORCE)|pnpm turbo test:integration|pnpm turbo run test:integration' .github/workflows package.json
printf'%s\n''--- CI workflow environment and matrix context ---'
sed -n '1,120p' .github/workflows/ci.yml
sed -n '380,445p' .github/workflows/ci.yml
sed -n '495,535p' .github/workflows/ci.yml
printf'%s\n''--- staging workflow context ---'
sed -n '1,110p' .github/workflows/e2e-staging.yml
sed -n '270,320p' .github/workflows/e2e-staging.yml
Repository: clerk/javascript
Length of output: 30087
Disable caching for CI integration tasks.
The CI integration job does not set TURBO_FORCE, and all 25 integration tasks inherit cache: true. globalPassThroughEnv does not affect the cache key. Set cache: false for these tasks, or move INTEGRATION_TEST_RUN_KEY to globalEnv.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@turbo.json` at line 37, Update the Turbo integration task configuration
containing INTEGRATION_TEST_RUN_KEY so CI integration tasks do not reuse cached
results: set cache to false for those tasks, or move INTEGRATION_TEST_RUN_KEY
into globalEnv to include it in the cache key.

 "ACTIONS_RUNNER_DEBUG",
 "ACTIONS_STEP_DEBUG",
 "VERCEL_AUTOMATION_BYPASS_SECRET",
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
fix(e2e): more robust user deletion#9375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

dstaley
merged 8 commits into

from

Aug 11, 2026
 Merged


fix(e2e): more robust user deletion
#9375
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
6b841fc

fix(e2e): delete all matching users


dstaley Aug 10, 2026
d76132e

fix(e2e): mark created users, always cleanup


dstaley Aug 10, 2026
2bd619c

fix: don't retry or fail on cleanup


dstaley Aug 10, 2026
a326a24

fix(e2e): use underscore


dstaley Aug 10, 2026
7513812

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 10, 2026
3c096a3

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 10, 2026
ad2f98a

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 11, 2026
417bb3b

fix(repo): rm extra changeset


dstaley Aug 11, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 2 additions & 0 deletions
.changeset/silver-lands-cut.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
---
---

17 changes: 17 additions & 0 deletions
.github/workflows/ci.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -504,6 +504,15 @@ jobs:
 sudo apt-get install -y xvfb
 fi

 - name: Configure test user cleanup
 run: echo "INTEGRATION_TEST_RUN_KEY=${RUN_ID}-${RUN_ATTEMPT}-${TEST_NAME}-${TEST_PROJECT}-${NEXT_VERSION:-default}" >> "$GITHUB_ENV"
 env:
 RUN_ID: ${{ github.run_id }}
 RUN_ATTEMPT: ${{ github.run_attempt }}
 TEST_NAME: ${{ matrix.test-name }}
 TEST_PROJECT: ${{ matrix.test-project }}
 NEXT_VERSION: ${{ matrix.next-version }}

 - name: Run Integration Tests
 id: integration-tests
 timeout-minutes: 25
Expand All@@ -525,6 +534,14 @@ jobs:
 NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem
 VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}

 - name: Delete integration-test users
 if: ${{ always() && steps.integration-tests.outcome != 'skipped' }}
 timeout-minutes: 4
 run: pnpm test:integration:cleanup
 env:
 INTEGRATION_INSTANCE_KEYS: ${{ secrets.INTEGRATION_INSTANCE_KEYS }}
 NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem

 - name: Sanitize artifact name
 if: ${{ cancelled() || failure() }}
 id: sanitize
Expand Down

73 changes: 48 additions & 25 deletions
integration/cleanup/cleanup.setup.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -5,8 +5,11 @@ import { isStaging } from '@clerk/shared/utils';
import { test as setup } from '@playwright/test';

import { appConfigs } from '../presets/';
import { findE2ERunUsers, getE2ERunMarker } from '../testUtils/e2eRun';
import { withRetry } from '../testUtils/retryableClerkClient';

setup('cleanup instances ', async () => {
 const runMarker = getE2ERunMarker();
 const entries = Array.from(appConfigs.secrets.instanceKeys.values())
 .map(({ pk, sk }) => {
 const secretKey = sk;
Expand All@@ -32,6 +35,9 @@ setup('cleanup instances ', async () => {
 }> = [];

 console.log('🧹 Starting E2E Test Cleanup Process...\n');
 if (runMarker) {
 console.log(`Cleaning users for run marker ${runMarker}\n`);
 }

 for (const entry of entries) {
 const instanceSummary = {
Expand All@@ -43,29 +49,32 @@ setup('cleanup instances ', async () => {
 };

 try {
 const clerkClient = createClerkClient({ secretKey: entry.secretKey, apiUrl: entry.apiUrl });
 const clerkClient = withRetry(createClerkClient({ secretKey: entry.secretKey, apiUrl: entry.apiUrl }));

 // Get users with error handling
 let users: any[] = [];
 try {
 const { data: usersWithEmail } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: 'clerkcookie',
 limit: 500,
 });

 const { data: usersWithPhoneNumber } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: '55501',
 limit: 500,
 });

 // Deduplicate users by ID
 const allUsersMap = new Map();
 [...usersWithEmail, ...usersWithPhoneNumber].forEach(user => {
 allUsersMap.set(user.id, user);
 });
 users = Array.from(allUsersMap.values());
 if (runMarker) {
 users = await findE2ERunUsers(clerkClient, runMarker);
 } else {
 const { data: usersWithEmail } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: 'clerkcookie',
 limit: 500,
 });

 const { data: usersWithPhoneNumber } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: '55501',
 limit: 500,
 });

 const allUsersMap = new Map();
 [...usersWithEmail, ...usersWithPhoneNumber].forEach(user => {
 allUsersMap.set(user.id, user);
 });
 users = Array.from(allUsersMap.values());
 }
 } catch (error) {
 instanceSummary.errors.push(`Failed to get users: ${error.message}`);
 console.error(`Error getting users for ${entry.instanceName}:`, error);
Expand All@@ -75,10 +84,14 @@ setup('cleanup instances ', async () => {
 // Get organizations with error handling
 let orgs: any[] = [];
 try {
 const { data: orgsData } = await clerkClient.organizations.getOrganizationList({
 limit: 500,
 });
 orgs = orgsData;
 if (runMarker) {
 orgs = [];
 } else {
 const { data: orgsData } = await clerkClient.organizations.getOrganizationList({
 limit: 500,
 });
 orgs = orgsData;
 }
 } catch (error) {
 // Treat 404 (not found) and 403 (forbidden) as "no orgs"
 // 404 = no organizations exist, 403 = no permission to access organizations
Expand All@@ -91,8 +104,11 @@ setup('cleanup instances ', async () => {
 }
 }

 const usersToDelete = batchElements(skipObjectsThatWereCreatedWithinTheLast10Minutes(users), 5);
 const orgsToDelete = batchElements(skipObjectsThatWereCreatedWithinTheLast10Minutes(orgs), 5);
 const usersToDelete = batchElements(
 runMarker ? users : skipObjectsThatWereCreatedWithinTheLast10Minutes(users),
 5,
 );
 const orgsToDelete = batchElements(runMarker ? [] : skipObjectsThatWereCreatedWithinTheLast10Minutes(orgs), 5);

 // Delete users with tracking
 for (const batch of usersToDelete) {
Expand DownExpand Up@@ -142,6 +158,13 @@ setup('cleanup instances ', async () => {
 await new Promise(r => setTimeout(r, 1000));
 }

 if (runMarker) {
 const remainingUsers = await findE2ERunUsers(clerkClient, runMarker);
 if (remainingUsers.length > 0) {
 instanceSummary.errors.push(`Users remain after cleanup: ${remainingUsers.map(user => user.id).join(', ')}`);
 }
 }

 // Report instance results
 const maskedKey = entry.secretKey.replace(/(sk_(test|live)_)(.+)(...)/, '$1***$4');
 if (instanceSummary.usersDeleted > 0 || instanceSummary.orgsDeleted > 0) {
Expand Down

1 change: 1 addition & 0 deletions
integration/playwright.cleanup.config.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -9,6 +9,7 @@ config({ path: path.resolve(__dirname, '.env.local') });
export default defineConfig({
 ...common,
 testDir: './cleanup',
 retries: 0,
 projects: [
 {
 name: 'setup',
Expand Down

42 changes: 42 additions & 0 deletions
integration/testUtils/e2eRun.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
import { createHash } from 'node:crypto';

import type { ClerkClient, User } from '@clerk/backend';

type E2EUserRecord = {
 username: string | null;
 emailAddresses: Array<{ emailAddress: string }>;
 privateMetadata: Record<string, unknown>;
};

export const getE2ERunMarker = (runKey = process.env.INTEGRATION_TEST_RUN_KEY): string | undefined => {
 if (!runKey) {
 return;
 }

 const digest = createHash('sha256').update(runKey).digest('hex').slice(0, 20);
 return `e2e_${digest}`;
};

export const userMatchesE2ERun = (user: E2EUserRecord, marker: string): boolean =>
 Boolean(
 user.username?.includes(marker) ||
 user.emailAddresses.some(email => email.emailAddress.includes(marker)) ||
 user.privateMetadata.e2eRunMarker === marker,
 );

export const findE2ERunUsers = async (clerkClient: ClerkClient, marker: string): Promise<User[]> => {
 const usersById = new Map<string, User>();
 let offset = 0;

 while (true) {
 const { data } = await clerkClient.users.getUserList({ query: marker, limit: 100, offset });
 data.filter(user => userMatchesE2ERun(user, marker)).forEach(user => usersById.set(user.id, user));

 if (data.length < 100) {
 break;
 }
 offset += data.length;
 }

 return Array.from(usersById.values());
};

51 changes: 34 additions & 17 deletions
integration/testUtils/usersService.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,7 @@ import { faker } from '@faker-js/faker';
import type { TestInfo } from '@playwright/test';

import { fakerPassword, hash } from '../models/helpers';
import { getE2ERunMarker } from './e2eRun';

async function withErrorLogging<T>(operation: string, fn: () => Promise<T>): Promise<T> {
 try {
Expand DownExpand Up@@ -129,24 +130,29 @@ export const createUserService = (clerkClient: ClerkClient) => {
 withUsername = false,
 } = options || {};
 const randomHash = hash();
 const runMarker = getE2ERunMarker();
 const markedHash = runMarker ? `${runMarker}_${randomHash}` : randomHash;
 const email = fictionalEmail
 ? `${randomHash}+clerk_test@clerkcookie.com`
 : `clerkcookie+${randomHash}@mailsac.com`;
 ? `${markedHash}+clerk_test@clerkcookie.com`
 : `clerkcookie+${markedHash}@mailsac.com`;
 const phoneNumber = fakerPhoneNumber();
 const { file, line, title, titlePath } = test.info();
 const fakeUserEmail = withEmail ? email : undefined;
 const fakeUserPhoneNumber = withPhoneNumber ? phoneNumber : undefined;

 return {
 firstName: faker.person.firstName(),
 lastName: faker.person.lastName(),
 email: withEmail ? email : undefined,
 username: withUsername ? `${randomHash}_clerk_cookie` : undefined,
 email: fakeUserEmail,
 username: withUsername ? `${markedHash}_clerk_cookie` : undefined,
 password: withPassword ? fakerPassword() : undefined,
 phoneNumber: withPhoneNumber ? phoneNumber : undefined,
 phoneNumber: fakeUserPhoneNumber,
 privateMetadata: {
 title,
 titlePath,
 file,
 line,
 ...(runMarker ? { e2eRunMarker: runMarker } : {}),
 },
 deleteIfExists: () => self.deleteIfExists({ email, phoneNumber }),
 };
Expand All@@ -173,24 +179,35 @@ export const createUserService = (clerkClient: ClerkClient) => {
 return await self.createBapiUser(fakeUser);
 },
 deleteIfExists: async (opts: { id?: string; email?: string; phoneNumber?: string }) => {
 let id = opts.id;
 const [usersByEmail, usersByPhoneNumber] = await Promise.all([
 opts.email
 ? withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 emailAddress: [opts.email],
 }),
 )
 : undefined,
 opts.phoneNumber
 ? withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 phoneNumber: [opts.phoneNumber],
 }),
 )
 : undefined,
 ]);

 if (!id) {
 const { data: users } = await withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 emailAddress: [opts.email],
 phoneNumber: [opts.phoneNumber],
 }),
 );
 id = users[0]?.id;
 }
 const ids = new Set([
 ...(opts.id ? [opts.id] : []),
 ...(usersByEmail?.data.map(user => user.id) ?? []),
 ...(usersByPhoneNumber?.data.map(user => user.id) ?? []),
 ]);

 if (!id) {
 if (ids.size === 0) {
 console.log(`User "${opts.email || opts.phoneNumber}" does not exist!`);
 return;

Comment on lines

+205
 to 
207

@coderabbitaicoderabbitaiBotAug 10, 2026

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
Do not log the supplied email address or phone number.
Line 206 writes a user identifier to CI logs. Log a generic cleanup message instead.
Proposed fix
- console.log(`User "${opts.email || opts.phoneNumber}" does not exist!`);+ console.log('No user exists for the supplied cleanup criteria.');
📝 Committable suggestion
‼️IMPORTANT

Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

if(ids.size===0){
console.log(`User "${opts.email||opts.phoneNumber}" does not exist!`);
return;
if(ids.size===0){
console.log('No user exists for the supplied cleanup criteria.');
return;
🧰 Tools
🪛 ast-grep (0.45.0)
[warning] 205-205: Avoid logging sensitive data

Context: console.log(User "${opts.email || opts.phoneNumber}" does not exist!)

Note: [CWE-532] Insertion of Sensitive Information into Log File.
(log-sensitive-data-typescript)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@integration/testUtils/usersService.ts` around lines 205 - 207, Update the
empty-user branch in the user cleanup flow to replace the interpolated
opts.email and opts.phoneNumber values with a generic cleanup message, ensuring
no supplied user identifier is written to logs while preserving the early
return.

Sources: Coding guidelines, Linters/SAST tools
 }

 await withErrorLogging('deleteUser', () => clerkClient.users.deleteUser(id));
 await Promise.all(Array.from(ids, id => withErrorLogging('deleteUser', () => clerkClient.users.deleteUser(id))));
 },
 getUser: async (opts: { id?: string; email?: string }) => {
 if (opts.id) {
Expand Down

1 change: 1 addition & 0 deletions
turbo.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,7 @@
 "globalPassThroughEnv": [
 "AWS_SECRET_KEY",
 "GITHUB_TOKEN",
 "INTEGRATION_TEST_RUN_KEY",
@coderabbitaicoderabbitaiBotAug 10, 2026

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
# Every integration task must report cache=false.
jq -r ' (.tasks // {}) | to_entries[] | select(.key | test("test:integration")) | "\(.key): cache=\(.value.cache // true)"' turbo.json
Repository: clerk/javascript
Length of output: 1212
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- turbo.json ---'
cat -n turbo.json | sed -n '1,90p'printf'%s\n''--- Turbo version and integration scripts ---'
rg -n '"turbo"|test:integration|INTEGRATION_TEST_RUN_KEY' package.json pnpm-lock.yaml packages apps examples .github 2>/dev/null | head -240
Repository: clerk/javascript
Length of output: 7965
🌐 Web query:
Turborepo globalPassThroughEnv cache key behavior passThroughEnv cacheable task
💡 Result:
In Turborepo, both globalPassThroughEnv and passThroughEnv serve to expose environment variables to task runtimes without including them in the task's cache key [1][2][3]. Because these variables are intentionally omitted from the hashing process, changes to their values will not trigger a cache miss, even if the change would logically affect the output of the task [1][4]. The fundamental behaviors for these configurations are: 1. Cache Key Exclusion: Variables listed in either globalPassThroughEnv or passThroughEnv do not contribute to the task hash [2][3]. Consequently, Turborepo may serve a cached result even if the underlying environment variable has changed [1][4]. This is why these keys should only be used for variables that do not affect the build output, such as authentication tokens (e.g., SENTRY_AUTH_TOKEN), CI metadata, or credentials consumed at runtime after the build process [2][4]. 2. Runtime Exposure: By including a variable in either key, you satisfy Turborepo's Strict Environment Mode, ensuring the task has access to that variable at runtime [1][5][3]. If a task attempts to access an environment variable that is not accounted for in env, globalEnv, passThroughEnv, or globalPassThroughEnv while in Strict Mode, the task may fail [1][5]. 3. Scope Differences: - globalPassThroughEnv (or global.passThroughEnv when using the global configuration flag) applies the passthrough behavior to all tasks within the repository [2][6]. - passThroughEnv applies only to the specific task where it is configured [3]. If you need a change in an environment variable to invalidate the cache and trigger a rebuild, you must include that variable in env or globalEnv instead of the passthrough keys [1][3].
Citations:
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- integration task definitions ---'
rg -n -C 3 '"[`#/`]?test:integration|test:integration' turbo.json
printf'%s\n''--- CI Turbo arguments and integration invocation ---'
sed -n '400,535p' .github/workflows/ci.yml
sed -n '260,300p' .github/workflows/e2e-staging.yml
printf'%s\n''--- exact task keys parsed from turbo.json ---'
jq -r '(.tasks // {}) | to_entries[] | select(.key | contains("test:integration")) | [.key, (.value.cache // true | tostring)] | `@tsv`' turbo.json
Repository: clerk/javascript
Length of output: 16346
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
python3 - <<'PY'import jsonfrom pathlib import Pathconfig = json.loads(Path("turbo.json").read_text())tasks = config.get("tasks", {})matches = [(name, task.get("cache", True)) for name, task in tasks.items() if "test:integration" in name]print("matching task count:", len(matches))for name, cache in matches: print(f"{name}: cache={cache}")PY
Repository: clerk/javascript
Length of output: 1236
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- all Turbo force/argument settings ---'
rg -n -C 4 'TURBO_(ARGS|FORCE)|pnpm turbo test:integration|pnpm turbo run test:integration' .github/workflows package.json
printf'%s\n''--- CI workflow environment and matrix context ---'
sed -n '1,120p' .github/workflows/ci.yml
sed -n '380,445p' .github/workflows/ci.yml
sed -n '495,535p' .github/workflows/ci.yml
printf'%s\n''--- staging workflow context ---'
sed -n '1,110p' .github/workflows/e2e-staging.yml
sed -n '270,320p' .github/workflows/e2e-staging.yml
Repository: clerk/javascript
Length of output: 30087
Disable caching for CI integration tasks.
The CI integration job does not set TURBO_FORCE, and all 25 integration tasks inherit cache: true. globalPassThroughEnv does not affect the cache key. Set cache: false for these tasks, or move INTEGRATION_TEST_RUN_KEY to globalEnv.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@turbo.json` at line 37, Update the Turbo integration task configuration
containing INTEGRATION_TEST_RUN_KEY so CI integration tasks do not reuse cached
results: set cache to false for those tasks, or move INTEGRATION_TEST_RUN_KEY
into globalEnv to include it in the cache key.

 "ACTIONS_RUNNER_DEBUG",
 "ACTIONS_STEP_DEBUG",
 "VERCEL_AUTOMATION_BYPASS_SECRET",
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix(e2e): more robust user deletion#9375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

dstaley
merged 8 commits into

from

Aug 11, 2026
 Merged


fix(e2e): more robust user deletion
#9375
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
6b841fc

fix(e2e): delete all matching users


dstaley Aug 10, 2026
d76132e

fix(e2e): mark created users, always cleanup


dstaley Aug 10, 2026
2bd619c

fix: don't retry or fail on cleanup


dstaley Aug 10, 2026
a326a24

fix(e2e): use underscore


dstaley Aug 10, 2026
7513812

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 10, 2026
3c096a3

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 10, 2026
ad2f98a

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 11, 2026
417bb3b

fix(repo): rm extra changeset


dstaley Aug 11, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 2 additions & 0 deletions
.changeset/silver-lands-cut.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
---
---

17 changes: 17 additions & 0 deletions
.github/workflows/ci.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -504,6 +504,15 @@ jobs:
 sudo apt-get install -y xvfb
 fi

 - name: Configure test user cleanup
 run: echo "INTEGRATION_TEST_RUN_KEY=${RUN_ID}-${RUN_ATTEMPT}-${TEST_NAME}-${TEST_PROJECT}-${NEXT_VERSION:-default}" >> "$GITHUB_ENV"
 env:
 RUN_ID: ${{ github.run_id }}
 RUN_ATTEMPT: ${{ github.run_attempt }}
 TEST_NAME: ${{ matrix.test-name }}
 TEST_PROJECT: ${{ matrix.test-project }}
 NEXT_VERSION: ${{ matrix.next-version }}

 - name: Run Integration Tests
 id: integration-tests
 timeout-minutes: 25
Expand All@@ -525,6 +534,14 @@ jobs:
 NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem
 VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}

 - name: Delete integration-test users
 if: ${{ always() && steps.integration-tests.outcome != 'skipped' }}
 timeout-minutes: 4
 run: pnpm test:integration:cleanup
 env:
 INTEGRATION_INSTANCE_KEYS: ${{ secrets.INTEGRATION_INSTANCE_KEYS }}
 NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem

 - name: Sanitize artifact name
 if: ${{ cancelled() || failure() }}
 id: sanitize
Expand Down

73 changes: 48 additions & 25 deletions
integration/cleanup/cleanup.setup.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -5,8 +5,11 @@ import { isStaging } from '@clerk/shared/utils';
import { test as setup } from '@playwright/test';

import { appConfigs } from '../presets/';
import { findE2ERunUsers, getE2ERunMarker } from '../testUtils/e2eRun';
import { withRetry } from '../testUtils/retryableClerkClient';

setup('cleanup instances ', async () => {
 const runMarker = getE2ERunMarker();
 const entries = Array.from(appConfigs.secrets.instanceKeys.values())
 .map(({ pk, sk }) => {
 const secretKey = sk;
Expand All@@ -32,6 +35,9 @@ setup('cleanup instances ', async () => {
 }> = [];

 console.log('🧹 Starting E2E Test Cleanup Process...\n');
 if (runMarker) {
 console.log(`Cleaning users for run marker ${runMarker}\n`);
 }

 for (const entry of entries) {
 const instanceSummary = {
Expand All@@ -43,29 +49,32 @@ setup('cleanup instances ', async () => {
 };

 try {
 const clerkClient = createClerkClient({ secretKey: entry.secretKey, apiUrl: entry.apiUrl });
 const clerkClient = withRetry(createClerkClient({ secretKey: entry.secretKey, apiUrl: entry.apiUrl }));

 // Get users with error handling
 let users: any[] = [];
 try {
 const { data: usersWithEmail } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: 'clerkcookie',
 limit: 500,
 });

 const { data: usersWithPhoneNumber } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: '55501',
 limit: 500,
 });

 // Deduplicate users by ID
 const allUsersMap = new Map();
 [...usersWithEmail, ...usersWithPhoneNumber].forEach(user => {
 allUsersMap.set(user.id, user);
 });
 users = Array.from(allUsersMap.values());
 if (runMarker) {
 users = await findE2ERunUsers(clerkClient, runMarker);
 } else {
 const { data: usersWithEmail } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: 'clerkcookie',
 limit: 500,
 });

 const { data: usersWithPhoneNumber } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: '55501',
 limit: 500,
 });

 const allUsersMap = new Map();
 [...usersWithEmail, ...usersWithPhoneNumber].forEach(user => {
 allUsersMap.set(user.id, user);
 });
 users = Array.from(allUsersMap.values());
 }
 } catch (error) {
 instanceSummary.errors.push(`Failed to get users: ${error.message}`);
 console.error(`Error getting users for ${entry.instanceName}:`, error);
Expand All@@ -75,10 +84,14 @@ setup('cleanup instances ', async () => {
 // Get organizations with error handling
 let orgs: any[] = [];
 try {
 const { data: orgsData } = await clerkClient.organizations.getOrganizationList({
 limit: 500,
 });
 orgs = orgsData;
 if (runMarker) {
 orgs = [];
 } else {
 const { data: orgsData } = await clerkClient.organizations.getOrganizationList({
 limit: 500,
 });
 orgs = orgsData;
 }
 } catch (error) {
 // Treat 404 (not found) and 403 (forbidden) as "no orgs"
 // 404 = no organizations exist, 403 = no permission to access organizations
Expand All@@ -91,8 +104,11 @@ setup('cleanup instances ', async () => {
 }
 }

 const usersToDelete = batchElements(skipObjectsThatWereCreatedWithinTheLast10Minutes(users), 5);
 const orgsToDelete = batchElements(skipObjectsThatWereCreatedWithinTheLast10Minutes(orgs), 5);
 const usersToDelete = batchElements(
 runMarker ? users : skipObjectsThatWereCreatedWithinTheLast10Minutes(users),
 5,
 );
 const orgsToDelete = batchElements(runMarker ? [] : skipObjectsThatWereCreatedWithinTheLast10Minutes(orgs), 5);

 // Delete users with tracking
 for (const batch of usersToDelete) {
Expand DownExpand Up@@ -142,6 +158,13 @@ setup('cleanup instances ', async () => {
 await new Promise(r => setTimeout(r, 1000));
 }

 if (runMarker) {
 const remainingUsers = await findE2ERunUsers(clerkClient, runMarker);
 if (remainingUsers.length > 0) {
 instanceSummary.errors.push(`Users remain after cleanup: ${remainingUsers.map(user => user.id).join(', ')}`);
 }
 }

 // Report instance results
 const maskedKey = entry.secretKey.replace(/(sk_(test|live)_)(.+)(...)/, '$1***$4');
 if (instanceSummary.usersDeleted > 0 || instanceSummary.orgsDeleted > 0) {
Expand Down

1 change: 1 addition & 0 deletions
integration/playwright.cleanup.config.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -9,6 +9,7 @@ config({ path: path.resolve(__dirname, '.env.local') });
export default defineConfig({
 ...common,
 testDir: './cleanup',
 retries: 0,
 projects: [
 {
 name: 'setup',
Expand Down

42 changes: 42 additions & 0 deletions
integration/testUtils/e2eRun.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
import { createHash } from 'node:crypto';

import type { ClerkClient, User } from '@clerk/backend';

type E2EUserRecord = {
 username: string | null;
 emailAddresses: Array<{ emailAddress: string }>;
 privateMetadata: Record<string, unknown>;
};

export const getE2ERunMarker = (runKey = process.env.INTEGRATION_TEST_RUN_KEY): string | undefined => {
 if (!runKey) {
 return;
 }

 const digest = createHash('sha256').update(runKey).digest('hex').slice(0, 20);
 return `e2e_${digest}`;
};

export const userMatchesE2ERun = (user: E2EUserRecord, marker: string): boolean =>
 Boolean(
 user.username?.includes(marker) ||
 user.emailAddresses.some(email => email.emailAddress.includes(marker)) ||
 user.privateMetadata.e2eRunMarker === marker,
 );

export const findE2ERunUsers = async (clerkClient: ClerkClient, marker: string): Promise<User[]> => {
 const usersById = new Map<string, User>();
 let offset = 0;

 while (true) {
 const { data } = await clerkClient.users.getUserList({ query: marker, limit: 100, offset });
 data.filter(user => userMatchesE2ERun(user, marker)).forEach(user => usersById.set(user.id, user));

 if (data.length < 100) {
 break;
 }
 offset += data.length;
 }

 return Array.from(usersById.values());
};

51 changes: 34 additions & 17 deletions
integration/testUtils/usersService.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,7 @@ import { faker } from '@faker-js/faker';
import type { TestInfo } from '@playwright/test';

import { fakerPassword, hash } from '../models/helpers';
import { getE2ERunMarker } from './e2eRun';

async function withErrorLogging<T>(operation: string, fn: () => Promise<T>): Promise<T> {
 try {
Expand DownExpand Up@@ -129,24 +130,29 @@ export const createUserService = (clerkClient: ClerkClient) => {
 withUsername = false,
 } = options || {};
 const randomHash = hash();
 const runMarker = getE2ERunMarker();
 const markedHash = runMarker ? `${runMarker}_${randomHash}` : randomHash;
 const email = fictionalEmail
 ? `${randomHash}+clerk_test@clerkcookie.com`
 : `clerkcookie+${randomHash}@mailsac.com`;
 ? `${markedHash}+clerk_test@clerkcookie.com`
 : `clerkcookie+${markedHash}@mailsac.com`;
 const phoneNumber = fakerPhoneNumber();
 const { file, line, title, titlePath } = test.info();
 const fakeUserEmail = withEmail ? email : undefined;
 const fakeUserPhoneNumber = withPhoneNumber ? phoneNumber : undefined;

 return {
 firstName: faker.person.firstName(),
 lastName: faker.person.lastName(),
 email: withEmail ? email : undefined,
 username: withUsername ? `${randomHash}_clerk_cookie` : undefined,
 email: fakeUserEmail,
 username: withUsername ? `${markedHash}_clerk_cookie` : undefined,
 password: withPassword ? fakerPassword() : undefined,
 phoneNumber: withPhoneNumber ? phoneNumber : undefined,
 phoneNumber: fakeUserPhoneNumber,
 privateMetadata: {
 title,
 titlePath,
 file,
 line,
 ...(runMarker ? { e2eRunMarker: runMarker } : {}),
 },
 deleteIfExists: () => self.deleteIfExists({ email, phoneNumber }),
 };
Expand All@@ -173,24 +179,35 @@ export const createUserService = (clerkClient: ClerkClient) => {
 return await self.createBapiUser(fakeUser);
 },
 deleteIfExists: async (opts: { id?: string; email?: string; phoneNumber?: string }) => {
 let id = opts.id;
 const [usersByEmail, usersByPhoneNumber] = await Promise.all([
 opts.email
 ? withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 emailAddress: [opts.email],
 }),
 )
 : undefined,
 opts.phoneNumber
 ? withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 phoneNumber: [opts.phoneNumber],
 }),
 )
 : undefined,
 ]);

 if (!id) {
 const { data: users } = await withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 emailAddress: [opts.email],
 phoneNumber: [opts.phoneNumber],
 }),
 );
 id = users[0]?.id;
 }
 const ids = new Set([
 ...(opts.id ? [opts.id] : []),
 ...(usersByEmail?.data.map(user => user.id) ?? []),
 ...(usersByPhoneNumber?.data.map(user => user.id) ?? []),
 ]);

 if (!id) {
 if (ids.size === 0) {
 console.log(`User "${opts.email || opts.phoneNumber}" does not exist!`);
 return;

Comment on lines

+205
 to 
207

@coderabbitaicoderabbitaiBotAug 10, 2026

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
Do not log the supplied email address or phone number.
Line 206 writes a user identifier to CI logs. Log a generic cleanup message instead.
Proposed fix
- console.log(`User "${opts.email || opts.phoneNumber}" does not exist!`);+ console.log('No user exists for the supplied cleanup criteria.');
📝 Committable suggestion
‼️IMPORTANT

Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

if(ids.size===0){
console.log(`User "${opts.email||opts.phoneNumber}" does not exist!`);
return;
if(ids.size===0){
console.log('No user exists for the supplied cleanup criteria.');
return;
🧰 Tools
🪛 ast-grep (0.45.0)
[warning] 205-205: Avoid logging sensitive data

Context: console.log(User "${opts.email || opts.phoneNumber}" does not exist!)

Note: [CWE-532] Insertion of Sensitive Information into Log File.
(log-sensitive-data-typescript)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@integration/testUtils/usersService.ts` around lines 205 - 207, Update the
empty-user branch in the user cleanup flow to replace the interpolated
opts.email and opts.phoneNumber values with a generic cleanup message, ensuring
no supplied user identifier is written to logs while preserving the early
return.

Sources: Coding guidelines, Linters/SAST tools
 }

 await withErrorLogging('deleteUser', () => clerkClient.users.deleteUser(id));
 await Promise.all(Array.from(ids, id => withErrorLogging('deleteUser', () => clerkClient.users.deleteUser(id))));
 },
 getUser: async (opts: { id?: string; email?: string }) => {
 if (opts.id) {
Expand Down

1 change: 1 addition & 0 deletions
turbo.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,7 @@
 "globalPassThroughEnv": [
 "AWS_SECRET_KEY",
 "GITHUB_TOKEN",
 "INTEGRATION_TEST_RUN_KEY",
@coderabbitaicoderabbitaiBotAug 10, 2026

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
# Every integration task must report cache=false.
jq -r ' (.tasks // {}) | to_entries[] | select(.key | test("test:integration")) | "\(.key): cache=\(.value.cache // true)"' turbo.json
Repository: clerk/javascript
Length of output: 1212
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- turbo.json ---'
cat -n turbo.json | sed -n '1,90p'printf'%s\n''--- Turbo version and integration scripts ---'
rg -n '"turbo"|test:integration|INTEGRATION_TEST_RUN_KEY' package.json pnpm-lock.yaml packages apps examples .github 2>/dev/null | head -240
Repository: clerk/javascript
Length of output: 7965
🌐 Web query:
Turborepo globalPassThroughEnv cache key behavior passThroughEnv cacheable task
💡 Result:
In Turborepo, both globalPassThroughEnv and passThroughEnv serve to expose environment variables to task runtimes without including them in the task's cache key [1][2][3]. Because these variables are intentionally omitted from the hashing process, changes to their values will not trigger a cache miss, even if the change would logically affect the output of the task [1][4]. The fundamental behaviors for these configurations are: 1. Cache Key Exclusion: Variables listed in either globalPassThroughEnv or passThroughEnv do not contribute to the task hash [2][3]. Consequently, Turborepo may serve a cached result even if the underlying environment variable has changed [1][4]. This is why these keys should only be used for variables that do not affect the build output, such as authentication tokens (e.g., SENTRY_AUTH_TOKEN), CI metadata, or credentials consumed at runtime after the build process [2][4]. 2. Runtime Exposure: By including a variable in either key, you satisfy Turborepo's Strict Environment Mode, ensuring the task has access to that variable at runtime [1][5][3]. If a task attempts to access an environment variable that is not accounted for in env, globalEnv, passThroughEnv, or globalPassThroughEnv while in Strict Mode, the task may fail [1][5]. 3. Scope Differences: - globalPassThroughEnv (or global.passThroughEnv when using the global configuration flag) applies the passthrough behavior to all tasks within the repository [2][6]. - passThroughEnv applies only to the specific task where it is configured [3]. If you need a change in an environment variable to invalidate the cache and trigger a rebuild, you must include that variable in env or globalEnv instead of the passthrough keys [1][3].
Citations:
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- integration task definitions ---'
rg -n -C 3 '"[`#/`]?test:integration|test:integration' turbo.json
printf'%s\n''--- CI Turbo arguments and integration invocation ---'
sed -n '400,535p' .github/workflows/ci.yml
sed -n '260,300p' .github/workflows/e2e-staging.yml
printf'%s\n''--- exact task keys parsed from turbo.json ---'
jq -r '(.tasks // {}) | to_entries[] | select(.key | contains("test:integration")) | [.key, (.value.cache // true | tostring)] | `@tsv`' turbo.json
Repository: clerk/javascript
Length of output: 16346
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
python3 - <<'PY'import jsonfrom pathlib import Pathconfig = json.loads(Path("turbo.json").read_text())tasks = config.get("tasks", {})matches = [(name, task.get("cache", True)) for name, task in tasks.items() if "test:integration" in name]print("matching task count:", len(matches))for name, cache in matches: print(f"{name}: cache={cache}")PY
Repository: clerk/javascript
Length of output: 1236
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- all Turbo force/argument settings ---'
rg -n -C 4 'TURBO_(ARGS|FORCE)|pnpm turbo test:integration|pnpm turbo run test:integration' .github/workflows package.json
printf'%s\n''--- CI workflow environment and matrix context ---'
sed -n '1,120p' .github/workflows/ci.yml
sed -n '380,445p' .github/workflows/ci.yml
sed -n '495,535p' .github/workflows/ci.yml
printf'%s\n''--- staging workflow context ---'
sed -n '1,110p' .github/workflows/e2e-staging.yml
sed -n '270,320p' .github/workflows/e2e-staging.yml
Repository: clerk/javascript
Length of output: 30087
Disable caching for CI integration tasks.
The CI integration job does not set TURBO_FORCE, and all 25 integration tasks inherit cache: true. globalPassThroughEnv does not affect the cache key. Set cache: false for these tasks, or move INTEGRATION_TEST_RUN_KEY to globalEnv.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@turbo.json` at line 37, Update the Turbo integration task configuration
containing INTEGRATION_TEST_RUN_KEY so CI integration tasks do not reuse cached
results: set cache to false for those tasks, or move INTEGRATION_TEST_RUN_KEY
into globalEnv to include it in the cache key.

 "ACTIONS_RUNNER_DEBUG",
 "ACTIONS_STEP_DEBUG",
 "VERCEL_AUTOMATION_BYPASS_SECRET",
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix(e2e): more robust user deletion#9375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

dstaley
merged 8 commits into

from

Aug 11, 2026
 Merged


fix(e2e): more robust user deletion
#9375
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
6b841fc

fix(e2e): delete all matching users


dstaley Aug 10, 2026
d76132e

fix(e2e): mark created users, always cleanup


dstaley Aug 10, 2026
2bd619c

fix: don't retry or fail on cleanup


dstaley Aug 10, 2026
a326a24

fix(e2e): use underscore


dstaley Aug 10, 2026
7513812

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 10, 2026
3c096a3

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 10, 2026
ad2f98a

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 11, 2026
417bb3b

fix(repo): rm extra changeset


dstaley Aug 11, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 2 additions & 0 deletions
.changeset/silver-lands-cut.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
---
---

17 changes: 17 additions & 0 deletions
.github/workflows/ci.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -504,6 +504,15 @@ jobs:
 sudo apt-get install -y xvfb
 fi

 - name: Configure test user cleanup
 run: echo "INTEGRATION_TEST_RUN_KEY=${RUN_ID}-${RUN_ATTEMPT}-${TEST_NAME}-${TEST_PROJECT}-${NEXT_VERSION:-default}" >> "$GITHUB_ENV"
 env:
 RUN_ID: ${{ github.run_id }}
 RUN_ATTEMPT: ${{ github.run_attempt }}
 TEST_NAME: ${{ matrix.test-name }}
 TEST_PROJECT: ${{ matrix.test-project }}
 NEXT_VERSION: ${{ matrix.next-version }}

 - name: Run Integration Tests
 id: integration-tests
 timeout-minutes: 25
Expand All@@ -525,6 +534,14 @@ jobs:
 NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem
 VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}

 - name: Delete integration-test users
 if: ${{ always() && steps.integration-tests.outcome != 'skipped' }}
 timeout-minutes: 4
 run: pnpm test:integration:cleanup
 env:
 INTEGRATION_INSTANCE_KEYS: ${{ secrets.INTEGRATION_INSTANCE_KEYS }}
 NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem

 - name: Sanitize artifact name
 if: ${{ cancelled() || failure() }}
 id: sanitize
Expand Down

73 changes: 48 additions & 25 deletions
integration/cleanup/cleanup.setup.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -5,8 +5,11 @@ import { isStaging } from '@clerk/shared/utils';
import { test as setup } from '@playwright/test';

import { appConfigs } from '../presets/';
import { findE2ERunUsers, getE2ERunMarker } from '../testUtils/e2eRun';
import { withRetry } from '../testUtils/retryableClerkClient';

setup('cleanup instances ', async () => {
 const runMarker = getE2ERunMarker();
 const entries = Array.from(appConfigs.secrets.instanceKeys.values())
 .map(({ pk, sk }) => {
 const secretKey = sk;
Expand All@@ -32,6 +35,9 @@ setup('cleanup instances ', async () => {
 }> = [];

 console.log('🧹 Starting E2E Test Cleanup Process...\n');
 if (runMarker) {
 console.log(`Cleaning users for run marker ${runMarker}\n`);
 }

 for (const entry of entries) {
 const instanceSummary = {
Expand All@@ -43,29 +49,32 @@ setup('cleanup instances ', async () => {
 };

 try {
 const clerkClient = createClerkClient({ secretKey: entry.secretKey, apiUrl: entry.apiUrl });
 const clerkClient = withRetry(createClerkClient({ secretKey: entry.secretKey, apiUrl: entry.apiUrl }));

 // Get users with error handling
 let users: any[] = [];
 try {
 const { data: usersWithEmail } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: 'clerkcookie',
 limit: 500,
 });

 const { data: usersWithPhoneNumber } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: '55501',
 limit: 500,
 });

 // Deduplicate users by ID
 const allUsersMap = new Map();
 [...usersWithEmail, ...usersWithPhoneNumber].forEach(user => {
 allUsersMap.set(user.id, user);
 });
 users = Array.from(allUsersMap.values());
 if (runMarker) {
 users = await findE2ERunUsers(clerkClient, runMarker);
 } else {
 const { data: usersWithEmail } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: 'clerkcookie',
 limit: 500,
 });

 const { data: usersWithPhoneNumber } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: '55501',
 limit: 500,
 });

 const allUsersMap = new Map();
 [...usersWithEmail, ...usersWithPhoneNumber].forEach(user => {
 allUsersMap.set(user.id, user);
 });
 users = Array.from(allUsersMap.values());
 }
 } catch (error) {
 instanceSummary.errors.push(`Failed to get users: ${error.message}`);
 console.error(`Error getting users for ${entry.instanceName}:`, error);
Expand All@@ -75,10 +84,14 @@ setup('cleanup instances ', async () => {
 // Get organizations with error handling
 let orgs: any[] = [];
 try {
 const { data: orgsData } = await clerkClient.organizations.getOrganizationList({
 limit: 500,
 });
 orgs = orgsData;
 if (runMarker) {
 orgs = [];
 } else {
 const { data: orgsData } = await clerkClient.organizations.getOrganizationList({
 limit: 500,
 });
 orgs = orgsData;
 }
 } catch (error) {
 // Treat 404 (not found) and 403 (forbidden) as "no orgs"
 // 404 = no organizations exist, 403 = no permission to access organizations
Expand All@@ -91,8 +104,11 @@ setup('cleanup instances ', async () => {
 }
 }

 const usersToDelete = batchElements(skipObjectsThatWereCreatedWithinTheLast10Minutes(users), 5);
 const orgsToDelete = batchElements(skipObjectsThatWereCreatedWithinTheLast10Minutes(orgs), 5);
 const usersToDelete = batchElements(
 runMarker ? users : skipObjectsThatWereCreatedWithinTheLast10Minutes(users),
 5,
 );
 const orgsToDelete = batchElements(runMarker ? [] : skipObjectsThatWereCreatedWithinTheLast10Minutes(orgs), 5);

 // Delete users with tracking
 for (const batch of usersToDelete) {
Expand DownExpand Up@@ -142,6 +158,13 @@ setup('cleanup instances ', async () => {
 await new Promise(r => setTimeout(r, 1000));
 }

 if (runMarker) {
 const remainingUsers = await findE2ERunUsers(clerkClient, runMarker);
 if (remainingUsers.length > 0) {
 instanceSummary.errors.push(`Users remain after cleanup: ${remainingUsers.map(user => user.id).join(', ')}`);
 }
 }

 // Report instance results
 const maskedKey = entry.secretKey.replace(/(sk_(test|live)_)(.+)(...)/, '$1***$4');
 if (instanceSummary.usersDeleted > 0 || instanceSummary.orgsDeleted > 0) {
Expand Down

1 change: 1 addition & 0 deletions
integration/playwright.cleanup.config.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -9,6 +9,7 @@ config({ path: path.resolve(__dirname, '.env.local') });
export default defineConfig({
 ...common,
 testDir: './cleanup',
 retries: 0,
 projects: [
 {
 name: 'setup',
Expand Down

42 changes: 42 additions & 0 deletions
integration/testUtils/e2eRun.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
import { createHash } from 'node:crypto';

import type { ClerkClient, User } from '@clerk/backend';

type E2EUserRecord = {
 username: string | null;
 emailAddresses: Array<{ emailAddress: string }>;
 privateMetadata: Record<string, unknown>;
};

export const getE2ERunMarker = (runKey = process.env.INTEGRATION_TEST_RUN_KEY): string | undefined => {
 if (!runKey) {
 return;
 }

 const digest = createHash('sha256').update(runKey).digest('hex').slice(0, 20);
 return `e2e_${digest}`;
};

export const userMatchesE2ERun = (user: E2EUserRecord, marker: string): boolean =>
 Boolean(
 user.username?.includes(marker) ||
 user.emailAddresses.some(email => email.emailAddress.includes(marker)) ||
 user.privateMetadata.e2eRunMarker === marker,
 );

export const findE2ERunUsers = async (clerkClient: ClerkClient, marker: string): Promise<User[]> => {
 const usersById = new Map<string, User>();
 let offset = 0;

 while (true) {
 const { data } = await clerkClient.users.getUserList({ query: marker, limit: 100, offset });
 data.filter(user => userMatchesE2ERun(user, marker)).forEach(user => usersById.set(user.id, user));

 if (data.length < 100) {
 break;
 }
 offset += data.length;
 }

 return Array.from(usersById.values());
};

51 changes: 34 additions & 17 deletions
integration/testUtils/usersService.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,7 @@ import { faker } from '@faker-js/faker';
import type { TestInfo } from '@playwright/test';

import { fakerPassword, hash } from '../models/helpers';
import { getE2ERunMarker } from './e2eRun';

async function withErrorLogging<T>(operation: string, fn: () => Promise<T>): Promise<T> {
 try {
Expand DownExpand Up@@ -129,24 +130,29 @@ export const createUserService = (clerkClient: ClerkClient) => {
 withUsername = false,
 } = options || {};
 const randomHash = hash();
 const runMarker = getE2ERunMarker();
 const markedHash = runMarker ? `${runMarker}_${randomHash}` : randomHash;
 const email = fictionalEmail
 ? `${randomHash}+clerk_test@clerkcookie.com`
 : `clerkcookie+${randomHash}@mailsac.com`;
 ? `${markedHash}+clerk_test@clerkcookie.com`
 : `clerkcookie+${markedHash}@mailsac.com`;
 const phoneNumber = fakerPhoneNumber();
 const { file, line, title, titlePath } = test.info();
 const fakeUserEmail = withEmail ? email : undefined;
 const fakeUserPhoneNumber = withPhoneNumber ? phoneNumber : undefined;

 return {
 firstName: faker.person.firstName(),
 lastName: faker.person.lastName(),
 email: withEmail ? email : undefined,
 username: withUsername ? `${randomHash}_clerk_cookie` : undefined,
 email: fakeUserEmail,
 username: withUsername ? `${markedHash}_clerk_cookie` : undefined,
 password: withPassword ? fakerPassword() : undefined,
 phoneNumber: withPhoneNumber ? phoneNumber : undefined,
 phoneNumber: fakeUserPhoneNumber,
 privateMetadata: {
 title,
 titlePath,
 file,
 line,
 ...(runMarker ? { e2eRunMarker: runMarker } : {}),
 },
 deleteIfExists: () => self.deleteIfExists({ email, phoneNumber }),
 };
Expand All@@ -173,24 +179,35 @@ export const createUserService = (clerkClient: ClerkClient) => {
 return await self.createBapiUser(fakeUser);
 },
 deleteIfExists: async (opts: { id?: string; email?: string; phoneNumber?: string }) => {
 let id = opts.id;
 const [usersByEmail, usersByPhoneNumber] = await Promise.all([
 opts.email
 ? withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 emailAddress: [opts.email],
 }),
 )
 : undefined,
 opts.phoneNumber
 ? withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 phoneNumber: [opts.phoneNumber],
 }),
 )
 : undefined,
 ]);

 if (!id) {
 const { data: users } = await withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 emailAddress: [opts.email],
 phoneNumber: [opts.phoneNumber],
 }),
 );
 id = users[0]?.id;
 }
 const ids = new Set([
 ...(opts.id ? [opts.id] : []),
 ...(usersByEmail?.data.map(user => user.id) ?? []),
 ...(usersByPhoneNumber?.data.map(user => user.id) ?? []),
 ]);

 if (!id) {
 if (ids.size === 0) {
 console.log(`User "${opts.email || opts.phoneNumber}" does not exist!`);
 return;

Comment on lines

+205
 to 
207

@coderabbitaicoderabbitaiBotAug 10, 2026

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
Do not log the supplied email address or phone number.
Line 206 writes a user identifier to CI logs. Log a generic cleanup message instead.
Proposed fix
- console.log(`User "${opts.email || opts.phoneNumber}" does not exist!`);+ console.log('No user exists for the supplied cleanup criteria.');
📝 Committable suggestion
‼️IMPORTANT

Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

if(ids.size===0){
console.log(`User "${opts.email||opts.phoneNumber}" does not exist!`);
return;
if(ids.size===0){
console.log('No user exists for the supplied cleanup criteria.');
return;
🧰 Tools
🪛 ast-grep (0.45.0)
[warning] 205-205: Avoid logging sensitive data

Context: console.log(User "${opts.email || opts.phoneNumber}" does not exist!)

Note: [CWE-532] Insertion of Sensitive Information into Log File.
(log-sensitive-data-typescript)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@integration/testUtils/usersService.ts` around lines 205 - 207, Update the
empty-user branch in the user cleanup flow to replace the interpolated
opts.email and opts.phoneNumber values with a generic cleanup message, ensuring
no supplied user identifier is written to logs while preserving the early
return.

Sources: Coding guidelines, Linters/SAST tools
 }

 await withErrorLogging('deleteUser', () => clerkClient.users.deleteUser(id));
 await Promise.all(Array.from(ids, id => withErrorLogging('deleteUser', () => clerkClient.users.deleteUser(id))));
 },
 getUser: async (opts: { id?: string; email?: string }) => {
 if (opts.id) {
Expand Down

1 change: 1 addition & 0 deletions
turbo.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,7 @@
 "globalPassThroughEnv": [
 "AWS_SECRET_KEY",
 "GITHUB_TOKEN",
 "INTEGRATION_TEST_RUN_KEY",
@coderabbitaicoderabbitaiBotAug 10, 2026

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
# Every integration task must report cache=false.
jq -r ' (.tasks // {}) | to_entries[] | select(.key | test("test:integration")) | "\(.key): cache=\(.value.cache // true)"' turbo.json
Repository: clerk/javascript
Length of output: 1212
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- turbo.json ---'
cat -n turbo.json | sed -n '1,90p'printf'%s\n''--- Turbo version and integration scripts ---'
rg -n '"turbo"|test:integration|INTEGRATION_TEST_RUN_KEY' package.json pnpm-lock.yaml packages apps examples .github 2>/dev/null | head -240
Repository: clerk/javascript
Length of output: 7965
🌐 Web query:
Turborepo globalPassThroughEnv cache key behavior passThroughEnv cacheable task
💡 Result:
In Turborepo, both globalPassThroughEnv and passThroughEnv serve to expose environment variables to task runtimes without including them in the task's cache key [1][2][3]. Because these variables are intentionally omitted from the hashing process, changes to their values will not trigger a cache miss, even if the change would logically affect the output of the task [1][4]. The fundamental behaviors for these configurations are: 1. Cache Key Exclusion: Variables listed in either globalPassThroughEnv or passThroughEnv do not contribute to the task hash [2][3]. Consequently, Turborepo may serve a cached result even if the underlying environment variable has changed [1][4]. This is why these keys should only be used for variables that do not affect the build output, such as authentication tokens (e.g., SENTRY_AUTH_TOKEN), CI metadata, or credentials consumed at runtime after the build process [2][4]. 2. Runtime Exposure: By including a variable in either key, you satisfy Turborepo's Strict Environment Mode, ensuring the task has access to that variable at runtime [1][5][3]. If a task attempts to access an environment variable that is not accounted for in env, globalEnv, passThroughEnv, or globalPassThroughEnv while in Strict Mode, the task may fail [1][5]. 3. Scope Differences: - globalPassThroughEnv (or global.passThroughEnv when using the global configuration flag) applies the passthrough behavior to all tasks within the repository [2][6]. - passThroughEnv applies only to the specific task where it is configured [3]. If you need a change in an environment variable to invalidate the cache and trigger a rebuild, you must include that variable in env or globalEnv instead of the passthrough keys [1][3].
Citations:
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- integration task definitions ---'
rg -n -C 3 '"[`#/`]?test:integration|test:integration' turbo.json
printf'%s\n''--- CI Turbo arguments and integration invocation ---'
sed -n '400,535p' .github/workflows/ci.yml
sed -n '260,300p' .github/workflows/e2e-staging.yml
printf'%s\n''--- exact task keys parsed from turbo.json ---'
jq -r '(.tasks // {}) | to_entries[] | select(.key | contains("test:integration")) | [.key, (.value.cache // true | tostring)] | `@tsv`' turbo.json
Repository: clerk/javascript
Length of output: 16346
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
python3 - <<'PY'import jsonfrom pathlib import Pathconfig = json.loads(Path("turbo.json").read_text())tasks = config.get("tasks", {})matches = [(name, task.get("cache", True)) for name, task in tasks.items() if "test:integration" in name]print("matching task count:", len(matches))for name, cache in matches: print(f"{name}: cache={cache}")PY
Repository: clerk/javascript
Length of output: 1236
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- all Turbo force/argument settings ---'
rg -n -C 4 'TURBO_(ARGS|FORCE)|pnpm turbo test:integration|pnpm turbo run test:integration' .github/workflows package.json
printf'%s\n''--- CI workflow environment and matrix context ---'
sed -n '1,120p' .github/workflows/ci.yml
sed -n '380,445p' .github/workflows/ci.yml
sed -n '495,535p' .github/workflows/ci.yml
printf'%s\n''--- staging workflow context ---'
sed -n '1,110p' .github/workflows/e2e-staging.yml
sed -n '270,320p' .github/workflows/e2e-staging.yml
Repository: clerk/javascript
Length of output: 30087
Disable caching for CI integration tasks.
The CI integration job does not set TURBO_FORCE, and all 25 integration tasks inherit cache: true. globalPassThroughEnv does not affect the cache key. Set cache: false for these tasks, or move INTEGRATION_TEST_RUN_KEY to globalEnv.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@turbo.json` at line 37, Update the Turbo integration task configuration
containing INTEGRATION_TEST_RUN_KEY so CI integration tasks do not reuse cached
results: set cache to false for those tasks, or move INTEGRATION_TEST_RUN_KEY
into globalEnv to include it in the cache key.

 "ACTIONS_RUNNER_DEBUG",
 "ACTIONS_STEP_DEBUG",
 "VERCEL_AUTOMATION_BYPASS_SECRET",
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
fix(e2e): more robust user deletion#9375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

dstaley
merged 8 commits into

from

Aug 11, 2026
 Merged


fix(e2e): more robust user deletion
#9375
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
6b841fc

fix(e2e): delete all matching users


dstaley Aug 10, 2026
d76132e

fix(e2e): mark created users, always cleanup


dstaley Aug 10, 2026
2bd619c

fix: don't retry or fail on cleanup


dstaley Aug 10, 2026
a326a24

fix(e2e): use underscore


dstaley Aug 10, 2026
7513812

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 10, 2026
3c096a3

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 10, 2026
ad2f98a

Merge branch 'main' into ds.fix/delete-all-match


dstaley Aug 11, 2026
417bb3b

fix(repo): rm extra changeset


dstaley Aug 11, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


2 changes: 2 additions & 0 deletions
.changeset/silver-lands-cut.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
---
---

17 changes: 17 additions & 0 deletions
.github/workflows/ci.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -504,6 +504,15 @@ jobs:
 sudo apt-get install -y xvfb
 fi

 - name: Configure test user cleanup
 run: echo "INTEGRATION_TEST_RUN_KEY=${RUN_ID}-${RUN_ATTEMPT}-${TEST_NAME}-${TEST_PROJECT}-${NEXT_VERSION:-default}" >> "$GITHUB_ENV"
 env:
 RUN_ID: ${{ github.run_id }}
 RUN_ATTEMPT: ${{ github.run_attempt }}
 TEST_NAME: ${{ matrix.test-name }}
 TEST_PROJECT: ${{ matrix.test-project }}
 NEXT_VERSION: ${{ matrix.next-version }}

 - name: Run Integration Tests
 id: integration-tests
 timeout-minutes: 25
Expand All@@ -525,6 +534,14 @@ jobs:
 NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem
 VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}

 - name: Delete integration-test users
 if: ${{ always() && steps.integration-tests.outcome != 'skipped' }}
 timeout-minutes: 4
 run: pnpm test:integration:cleanup
 env:
 INTEGRATION_INSTANCE_KEYS: ${{ secrets.INTEGRATION_INSTANCE_KEYS }}
 NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem

 - name: Sanitize artifact name
 if: ${{ cancelled() || failure() }}
 id: sanitize
Expand Down

73 changes: 48 additions & 25 deletions
integration/cleanup/cleanup.setup.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -5,8 +5,11 @@ import { isStaging } from '@clerk/shared/utils';
import { test as setup } from '@playwright/test';

import { appConfigs } from '../presets/';
import { findE2ERunUsers, getE2ERunMarker } from '../testUtils/e2eRun';
import { withRetry } from '../testUtils/retryableClerkClient';

setup('cleanup instances ', async () => {
 const runMarker = getE2ERunMarker();
 const entries = Array.from(appConfigs.secrets.instanceKeys.values())
 .map(({ pk, sk }) => {
 const secretKey = sk;
Expand All@@ -32,6 +35,9 @@ setup('cleanup instances ', async () => {
 }> = [];

 console.log('🧹 Starting E2E Test Cleanup Process...\n');
 if (runMarker) {
 console.log(`Cleaning users for run marker ${runMarker}\n`);
 }

 for (const entry of entries) {
 const instanceSummary = {
Expand All@@ -43,29 +49,32 @@ setup('cleanup instances ', async () => {
 };

 try {
 const clerkClient = createClerkClient({ secretKey: entry.secretKey, apiUrl: entry.apiUrl });
 const clerkClient = withRetry(createClerkClient({ secretKey: entry.secretKey, apiUrl: entry.apiUrl }));

 // Get users with error handling
 let users: any[] = [];
 try {
 const { data: usersWithEmail } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: 'clerkcookie',
 limit: 500,
 });

 const { data: usersWithPhoneNumber } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: '55501',
 limit: 500,
 });

 // Deduplicate users by ID
 const allUsersMap = new Map();
 [...usersWithEmail, ...usersWithPhoneNumber].forEach(user => {
 allUsersMap.set(user.id, user);
 });
 users = Array.from(allUsersMap.values());
 if (runMarker) {
 users = await findE2ERunUsers(clerkClient, runMarker);
 } else {
 const { data: usersWithEmail } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: 'clerkcookie',
 limit: 500,
 });

 const { data: usersWithPhoneNumber } = await clerkClient.users.getUserList({
 orderBy: '-created_at',
 query: '55501',
 limit: 500,
 });

 const allUsersMap = new Map();
 [...usersWithEmail, ...usersWithPhoneNumber].forEach(user => {
 allUsersMap.set(user.id, user);
 });
 users = Array.from(allUsersMap.values());
 }
 } catch (error) {
 instanceSummary.errors.push(`Failed to get users: ${error.message}`);
 console.error(`Error getting users for ${entry.instanceName}:`, error);
Expand All@@ -75,10 +84,14 @@ setup('cleanup instances ', async () => {
 // Get organizations with error handling
 let orgs: any[] = [];
 try {
 const { data: orgsData } = await clerkClient.organizations.getOrganizationList({
 limit: 500,
 });
 orgs = orgsData;
 if (runMarker) {
 orgs = [];
 } else {
 const { data: orgsData } = await clerkClient.organizations.getOrganizationList({
 limit: 500,
 });
 orgs = orgsData;
 }
 } catch (error) {
 // Treat 404 (not found) and 403 (forbidden) as "no orgs"
 // 404 = no organizations exist, 403 = no permission to access organizations
Expand All@@ -91,8 +104,11 @@ setup('cleanup instances ', async () => {
 }
 }

 const usersToDelete = batchElements(skipObjectsThatWereCreatedWithinTheLast10Minutes(users), 5);
 const orgsToDelete = batchElements(skipObjectsThatWereCreatedWithinTheLast10Minutes(orgs), 5);
 const usersToDelete = batchElements(
 runMarker ? users : skipObjectsThatWereCreatedWithinTheLast10Minutes(users),
 5,
 );
 const orgsToDelete = batchElements(runMarker ? [] : skipObjectsThatWereCreatedWithinTheLast10Minutes(orgs), 5);

 // Delete users with tracking
 for (const batch of usersToDelete) {
Expand DownExpand Up@@ -142,6 +158,13 @@ setup('cleanup instances ', async () => {
 await new Promise(r => setTimeout(r, 1000));
 }

 if (runMarker) {
 const remainingUsers = await findE2ERunUsers(clerkClient, runMarker);
 if (remainingUsers.length > 0) {
 instanceSummary.errors.push(`Users remain after cleanup: ${remainingUsers.map(user => user.id).join(', ')}`);
 }
 }

 // Report instance results
 const maskedKey = entry.secretKey.replace(/(sk_(test|live)_)(.+)(...)/, '$1***$4');
 if (instanceSummary.usersDeleted > 0 || instanceSummary.orgsDeleted > 0) {
Expand Down

1 change: 1 addition & 0 deletions
integration/playwright.cleanup.config.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -9,6 +9,7 @@ config({ path: path.resolve(__dirname, '.env.local') });
export default defineConfig({
 ...common,
 testDir: './cleanup',
 retries: 0,
 projects: [
 {
 name: 'setup',
Expand Down

42 changes: 42 additions & 0 deletions
integration/testUtils/e2eRun.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
import { createHash } from 'node:crypto';

import type { ClerkClient, User } from '@clerk/backend';

type E2EUserRecord = {
 username: string | null;
 emailAddresses: Array<{ emailAddress: string }>;
 privateMetadata: Record<string, unknown>;
};

export const getE2ERunMarker = (runKey = process.env.INTEGRATION_TEST_RUN_KEY): string | undefined => {
 if (!runKey) {
 return;
 }

 const digest = createHash('sha256').update(runKey).digest('hex').slice(0, 20);
 return `e2e_${digest}`;
};

export const userMatchesE2ERun = (user: E2EUserRecord, marker: string): boolean =>
 Boolean(
 user.username?.includes(marker) ||
 user.emailAddresses.some(email => email.emailAddress.includes(marker)) ||
 user.privateMetadata.e2eRunMarker === marker,
 );

export const findE2ERunUsers = async (clerkClient: ClerkClient, marker: string): Promise<User[]> => {
 const usersById = new Map<string, User>();
 let offset = 0;

 while (true) {
 const { data } = await clerkClient.users.getUserList({ query: marker, limit: 100, offset });
 data.filter(user => userMatchesE2ERun(user, marker)).forEach(user => usersById.set(user.id, user));

 if (data.length < 100) {
 break;
 }
 offset += data.length;
 }

 return Array.from(usersById.values());
};

51 changes: 34 additions & 17 deletions
integration/testUtils/usersService.ts

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -3,6 +3,7 @@ import { faker } from '@faker-js/faker';
import type { TestInfo } from '@playwright/test';

import { fakerPassword, hash } from '../models/helpers';
import { getE2ERunMarker } from './e2eRun';

async function withErrorLogging<T>(operation: string, fn: () => Promise<T>): Promise<T> {
 try {
Expand DownExpand Up@@ -129,24 +130,29 @@ export const createUserService = (clerkClient: ClerkClient) => {
 withUsername = false,
 } = options || {};
 const randomHash = hash();
 const runMarker = getE2ERunMarker();
 const markedHash = runMarker ? `${runMarker}_${randomHash}` : randomHash;
 const email = fictionalEmail
 ? `${randomHash}+clerk_test@clerkcookie.com`
 : `clerkcookie+${randomHash}@mailsac.com`;
 ? `${markedHash}+clerk_test@clerkcookie.com`
 : `clerkcookie+${markedHash}@mailsac.com`;
 const phoneNumber = fakerPhoneNumber();
 const { file, line, title, titlePath } = test.info();
 const fakeUserEmail = withEmail ? email : undefined;
 const fakeUserPhoneNumber = withPhoneNumber ? phoneNumber : undefined;

 return {
 firstName: faker.person.firstName(),
 lastName: faker.person.lastName(),
 email: withEmail ? email : undefined,
 username: withUsername ? `${randomHash}_clerk_cookie` : undefined,
 email: fakeUserEmail,
 username: withUsername ? `${markedHash}_clerk_cookie` : undefined,
 password: withPassword ? fakerPassword() : undefined,
 phoneNumber: withPhoneNumber ? phoneNumber : undefined,
 phoneNumber: fakeUserPhoneNumber,
 privateMetadata: {
 title,
 titlePath,
 file,
 line,
 ...(runMarker ? { e2eRunMarker: runMarker } : {}),
 },
 deleteIfExists: () => self.deleteIfExists({ email, phoneNumber }),
 };
Expand All@@ -173,24 +179,35 @@ export const createUserService = (clerkClient: ClerkClient) => {
 return await self.createBapiUser(fakeUser);
 },
 deleteIfExists: async (opts: { id?: string; email?: string; phoneNumber?: string }) => {
 let id = opts.id;
 const [usersByEmail, usersByPhoneNumber] = await Promise.all([
 opts.email
 ? withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 emailAddress: [opts.email],
 }),
 )
 : undefined,
 opts.phoneNumber
 ? withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 phoneNumber: [opts.phoneNumber],
 }),
 )
 : undefined,
 ]);

 if (!id) {
 const { data: users } = await withErrorLogging('getUserList', () =>
 clerkClient.users.getUserList({
 emailAddress: [opts.email],
 phoneNumber: [opts.phoneNumber],
 }),
 );
 id = users[0]?.id;
 }
 const ids = new Set([
 ...(opts.id ? [opts.id] : []),
 ...(usersByEmail?.data.map(user => user.id) ?? []),
 ...(usersByPhoneNumber?.data.map(user => user.id) ?? []),
 ]);

 if (!id) {
 if (ids.size === 0) {
 console.log(`User "${opts.email || opts.phoneNumber}" does not exist!`);
 return;

Comment on lines

+205
 to 
207

@coderabbitaicoderabbitaiBotAug 10, 2026

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
Do not log the supplied email address or phone number.
Line 206 writes a user identifier to CI logs. Log a generic cleanup message instead.
Proposed fix
- console.log(`User "${opts.email || opts.phoneNumber}" does not exist!`);+ console.log('No user exists for the supplied cleanup criteria.');
📝 Committable suggestion
‼️IMPORTANT

Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

if(ids.size===0){
console.log(`User "${opts.email||opts.phoneNumber}" does not exist!`);
return;
if(ids.size===0){
console.log('No user exists for the supplied cleanup criteria.');
return;
🧰 Tools
🪛 ast-grep (0.45.0)
[warning] 205-205: Avoid logging sensitive data

Context: console.log(User "${opts.email || opts.phoneNumber}" does not exist!)

Note: [CWE-532] Insertion of Sensitive Information into Log File.
(log-sensitive-data-typescript)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@integration/testUtils/usersService.ts` around lines 205 - 207, Update the
empty-user branch in the user cleanup flow to replace the interpolated
opts.email and opts.phoneNumber values with a generic cleanup message, ensuring
no supplied user identifier is written to logs while preserving the early
return.

Sources: Coding guidelines, Linters/SAST tools
 }

 await withErrorLogging('deleteUser', () => clerkClient.users.deleteUser(id));
 await Promise.all(Array.from(ids, id => withErrorLogging('deleteUser', () => clerkClient.users.deleteUser(id))));
 },
 getUser: async (opts: { id?: string; email?: string }) => {
 if (opts.id) {
Expand Down

1 change: 1 addition & 0 deletions
turbo.json

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,7 @@
 "globalPassThroughEnv": [
 "AWS_SECRET_KEY",
 "GITHUB_TOKEN",
 "INTEGRATION_TEST_RUN_KEY",
@coderabbitaicoderabbitaiBotAug 10, 2026

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
# Every integration task must report cache=false.
jq -r ' (.tasks // {}) | to_entries[] | select(.key | test("test:integration")) | "\(.key): cache=\(.value.cache // true)"' turbo.json
Repository: clerk/javascript
Length of output: 1212
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- turbo.json ---'
cat -n turbo.json | sed -n '1,90p'printf'%s\n''--- Turbo version and integration scripts ---'
rg -n '"turbo"|test:integration|INTEGRATION_TEST_RUN_KEY' package.json pnpm-lock.yaml packages apps examples .github 2>/dev/null | head -240
Repository: clerk/javascript
Length of output: 7965
🌐 Web query:
Turborepo globalPassThroughEnv cache key behavior passThroughEnv cacheable task
💡 Result:
In Turborepo, both globalPassThroughEnv and passThroughEnv serve to expose environment variables to task runtimes without including them in the task's cache key [1][2][3]. Because these variables are intentionally omitted from the hashing process, changes to their values will not trigger a cache miss, even if the change would logically affect the output of the task [1][4]. The fundamental behaviors for these configurations are: 1. Cache Key Exclusion: Variables listed in either globalPassThroughEnv or passThroughEnv do not contribute to the task hash [2][3]. Consequently, Turborepo may serve a cached result even if the underlying environment variable has changed [1][4]. This is why these keys should only be used for variables that do not affect the build output, such as authentication tokens (e.g., SENTRY_AUTH_TOKEN), CI metadata, or credentials consumed at runtime after the build process [2][4]. 2. Runtime Exposure: By including a variable in either key, you satisfy Turborepo's Strict Environment Mode, ensuring the task has access to that variable at runtime [1][5][3]. If a task attempts to access an environment variable that is not accounted for in env, globalEnv, passThroughEnv, or globalPassThroughEnv while in Strict Mode, the task may fail [1][5]. 3. Scope Differences: - globalPassThroughEnv (or global.passThroughEnv when using the global configuration flag) applies the passthrough behavior to all tasks within the repository [2][6]. - passThroughEnv applies only to the specific task where it is configured [3]. If you need a change in an environment variable to invalidate the cache and trigger a rebuild, you must include that variable in env or globalEnv instead of the passthrough keys [1][3].
Citations:
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- integration task definitions ---'
rg -n -C 3 '"[`#/`]?test:integration|test:integration' turbo.json
printf'%s\n''--- CI Turbo arguments and integration invocation ---'
sed -n '400,535p' .github/workflows/ci.yml
sed -n '260,300p' .github/workflows/e2e-staging.yml
printf'%s\n''--- exact task keys parsed from turbo.json ---'
jq -r '(.tasks // {}) | to_entries[] | select(.key | contains("test:integration")) | [.key, (.value.cache // true | tostring)] | `@tsv`' turbo.json
Repository: clerk/javascript
Length of output: 16346
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
python3 - <<'PY'import jsonfrom pathlib import Pathconfig = json.loads(Path("turbo.json").read_text())tasks = config.get("tasks", {})matches = [(name, task.get("cache", True)) for name, task in tasks.items() if "test:integration" in name]print("matching task count:", len(matches))for name, cache in matches: print(f"{name}: cache={cache}")PY
Repository: clerk/javascript
Length of output: 1236
🏁 Script executed:
#!/usr/bin/env bashset -euo pipefail
printf'%s\n''--- all Turbo force/argument settings ---'
rg -n -C 4 'TURBO_(ARGS|FORCE)|pnpm turbo test:integration|pnpm turbo run test:integration' .github/workflows package.json
printf'%s\n''--- CI workflow environment and matrix context ---'
sed -n '1,120p' .github/workflows/ci.yml
sed -n '380,445p' .github/workflows/ci.yml
sed -n '495,535p' .github/workflows/ci.yml
printf'%s\n''--- staging workflow context ---'
sed -n '1,110p' .github/workflows/e2e-staging.yml
sed -n '270,320p' .github/workflows/e2e-staging.yml
Repository: clerk/javascript
Length of output: 30087
Disable caching for CI integration tasks.
The CI integration job does not set TURBO_FORCE, and all 25 integration tasks inherit cache: true. globalPassThroughEnv does not affect the cache key. Set cache: false for these tasks, or move INTEGRATION_TEST_RUN_KEY to globalEnv.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@turbo.json` at line 37, Update the Turbo integration task configuration
containing INTEGRATION_TEST_RUN_KEY so CI integration tasks do not reuse cached
results: set cache to false for those tasks, or move INTEGRATION_TEST_RUN_KEY
into globalEnv to include it in the cache key.

 "ACTIONS_RUNNER_DEBUG",
 "ACTIONS_STEP_DEBUG",
 "VERCEL_AUTOMATION_BYPASS_SECRET",
Expand Down
Loading