Skip to content

fix(test-env): scrub Sentry credentials from the offline test environment - #1622

Merged
BigSimmo merged 11 commits into
mainfrom
claude/sentry-offline-scrub
Aug 5, 2026
Merged

fix(test-env): scrub Sentry credentials from the offline test environment#1622
BigSimmo merged 11 commits into
mainfrom
claude/sentry-offline-scrub

Conversation

@BigSimmo

@BigSimmoBigSimmo commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Summary

  • Scrub Sentry credentials from the offline test environment used by Vitest, Playwright, Lighthouse, and CLS measurement.
  • Pin SENTRY_DSN and NEXT_PUBLIC_SENTRY_DSN to an explicit blank (not deleted) so Next/Vite cannot reload a live DSN from .env.local during offline next build / next start.
  • Coerce blank optional URL env values to unset via optionalUrlEnv in src/lib/env.ts so Zod still parses.
  • Force SENTRY_ENABLE_LOGS=false, SENTRY_SEND_TEST_LOG=false, and SENTRY_TRACES_SAMPLE_RATE=0 for every offline wrapper consumer.
  • Add SENTRY_AUTH_TOKEN, SENTRY_DSN, and NEXT_PUBLIC_SENTRY_DSN to the Codex Cloud credential inventory (setup profile, raw-env check, shell-policy excludes).

Test plan

  • npm run test -- tests/test-runner-safety.test.ts tests/sentry-logging.test.ts tests/codex-cloud-setup.test.ts tests/env-sentry.test.ts — 65 passed
  • npm run check:codex-cloudPASS: static Cloud contracts match.
  • Regression: offline wrapper overwrites a live inherited DSN to "" and keeps the key present
  • Bugbot on branch diff — no bugs found
  • Confirm CI static-pr / unit suite stays green on the head

Clinical Governance Preflight

Not applicable — offline test-environment / env-schema hygiene only; no clinical, ingestion, answer, retrieval, privacy, or production-env behaviour change for end users.

RAG impact: no retrieval behaviour change — test harness and Sentry env scrubbing only.

…ment
offlineTestEnvironment() scrubs OpenAI, Supabase, Postgres and E2E names but
never covered Sentry, so an allegedly offline run inherited them. Demonstrated
against main with sentinel values: SENTRY_DSN came through as
"https://REAL@o1.ingest.sentry.io/9" and SENTRY_AUTH_TOKEN as
"sntrys_REAL_TOKEN", straight into the child processes spawned by
run-vitest.mjs and the Playwright wrappers. That is enough to initialise Sentry
during a local "offline" check and to carry a source-map upload token into it.
SENTRY_DSN is deleted rather than blanked or faked, and the distinction cuts
both ways: a blank string fails the runtime schema's .url(), while an inert
loopback URL is truthy and would leave the app's Sentry gates ENABLED for the
whole run — the opposite of the intent. SENTRY_ENABLE_LOGS and
SENTRY_SEND_TEST_LOG are forced off as feature flags instead of joining the
credential inventory, since they never carry a secret.
Adding the two credentials to providerEnvironmentKeys also enrols them in
providerCredentialVariables, which check-codex-cloud-setup.mjs cross-checks
against Cloud setup and the raw-env probe. That coupling is deliberate — one
inventory, enforced in three places — so setup-codex-cloud.sh and
check-codex-cloud-raw-env.sh gain the same two names. The full unit suite is
what surfaced this; the narrower suites passed without it.
tests/sentry-logging.test.ts needed adjusting because offline Vitest now forces
SENTRY_ENABLE_LOGS=false, which is what isSentryLoggingEnabled's default
parameter reads. It stubs the flag empty and keeps passing undefined, so the
case still exercises the default-parameter path its title describes; passing ""
positionally would silently have tested the explicit-empty path instead.
Ported from the Codex Cloud MCP work in PR #1613, which bundles it with a
contested change to hosted MCP registration. The scrub is independent of that
dispute and should not wait on it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T97Kqdj9Xh1Cubv5ms3KVy
@supabase

supabaseBot commented Aug 5, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@coderabbitai

coderabbitaiBot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in:25 minutes

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 6cb460b2-a82c-4489-9517-3377819fa08d

📥 Commits

Reviewing files that changed from the base of the PR and between b03b51d and 41f9ad6.

📒 Files selected for processing (8)
  • scripts/check-codex-cloud-raw-env.sh
  • scripts/setup-codex-cloud.sh
  • scripts/test-environment.mjs
  • src/lib/env.ts
  • tests/env-sentry.test.ts
  • tests/error-tracking.test.ts
  • tests/sentry-logging.test.ts
  • tests/test-runner-safety.test.ts

Comment @coderabbitai help to get the list of available commands.

@BigSimmoBigSimmo mentioned this pull request Aug 5, 2026
10 tasks
@BigSimmo
BigSimmo marked this pull request as ready for review August 5, 2026 15:20
devin-ai-integration[bot]

This comment was marked as resolved.

chatgpt-codex-connector[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@BigSimmo
BigSimmo enabled auto-merge (squash) August 5, 2026 15:32
cursoragentand others added 6 commits August 5, 2026 16:31
…-scrub
Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>
…alues
Delete left SENTRY_DSN absent, allowing .env.local to repopulate during
Playwright/Lighthouse starts. Pin blank server and public DSNs instead, and
coerce empty optional URL env values to unset for Zod.
Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>
Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>
Deleting SENTRY_DSN let Next reload a real destination from .env.local
during Playwright/Lighthouse builds. Pin both server and public DSNs to
an inert loopback URL, force SENTRY_TRACES_SAMPLE_RATE=0, and keep the
names in the Codex credential inventory so env-file reload cannot restore
live monitoring.
Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>
Merge the blank-DSN + optionalUrlEnv approach with inventory coverage for
NEXT_PUBLIC_SENTRY_DSN and forced SENTRY_TRACES_SAMPLE_RATE=0. Explicit blank
pins prevent .env.local reload while keeping Sentry gates falsy; optionalUrlEnv
coerces "" to unset for Zod.
Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>
Annotate the env-file reload regression test as Record so tsc accepts
property access on the offlineTestEnvironment return value.
Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

CI triage

CI failed on this PR. Automated classification of the 2 failed job(s):

  • Unit coverageneeds investigation: inspect the failing step and uploaded diagnostics; rerun only after classifying the cause.
  • PR requiredneeds investigation: inspect the failing step and uploaded diagnostics; rerun only after classifying the cause.

Compared with main CI run #8204 (cancelled).

Classification is evidence routing, not permission to ignore a failure. Exact quarantined Playwright identities remain governed by the flake ledger.

Offline Vitest now forces SENTRY_TRACES_SAMPLE_RATE=0, and passing
undefined still applies that default parameter. Clear the ambient value
before asserting the unset → 0.1 path.
Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>
devin-ai-integration[bot]

This comment was marked as resolved.

Inline z.preprocess at the declaration site so parseEnvSchemaNames still
matches NEXT_PUBLIC_SENTRY_DSN and SENTRY_DSN after blank-coercion landed.
Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>

@devin-ai-integrationdevin-ai-integrationBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View 2 additional findings in Devin Review.

Open in Devin Review

Comment threadsrc/lib/env.ts
@BigSimmo
BigSimmo merged commit c675294 into mainAug 5, 2026
36 checks passed
@BigSimmo
BigSimmo deleted the claude/sentry-offline-scrub branch August 5, 2026 17:06
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@BigSimmo@claude@cursoragent