Fix Codex Cloud runtime profile handoff - #1577
Conversation
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe setup script resolves access and retrieval modes, writes them to the runtime profile, and manages ChangesCodex Cloud setup
Estimated code review effort: 3 (Moderate) | ~25 minutes Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
Comment |
This pull request has been ignored for the connected project Preview Branches by Supabase. |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:137b363b56
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Uh oh!
There was an error while loading. Please reload this page.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
…rub full inventory Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Note Autofix is a beta feature. Expect some limitations and changes as we gather feedback and continue to improve it. The branch was updated while autofix was in progress. Please try again. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:cb6af57df4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
BigSimmo
commented
Aug 2, 2026
Plese implement the following fixes Codex Cloud runtime profile handoff — review fixes for setup-codex-cloud.sh Apply/verify the following fixes in setup-codex-cloud.sh: Reject unmanaged [shell_environment_policy] tables before rewriting config (CodeRabbit, setup-codex-cloud.sh:144-165). The sed at setup-codex-cloud.sh:150 only strips the managed marker block. If config.toml already contains an unmanaged [shell_environment_policy] table, it survives into codex_config_preserved, and the block at setup-codex-cloud.sh:161-171 then appends a second [shell_environment_policy] header — invalid duplicate-table TOML that the Codex CLI parser rejects. Fix: after computing codex_config_preserved, fail if a [shell_environment_policy] header still remains (setup-codex-cloud.sh:151-157). Pin the connected-mode retrieval value at setup time (Devin, setup-codex-cloud.sh:89-96). The generated profile must not re-evaluate ${RAG_PROVIDER_MODE:-auto} at source time, because the agent shell doesn't inherit that variable, so a connected env configured for offline retrieval would silently come up as auto. Fix: resolve and validate rag_provider_mode against auto|openai|offline at setup time (setup-codex-cloud.sh:71-79) and interpolate it literally into the generated profile (setup-codex-cloud.sh:110). Preserve existing config.toml across re-runs (Devin / Codex). Do not truncate the file wholesale. Write the shell policy inside a managed marker block (# BEGIN/END clinical-kb-codex-cloud shell policy), stripping only the previous managed block and preserving all other sections (mcp_servers, model, profiles, notify) — see setup-codex-cloud.sh:147-171. Must be idempotent across repeated setup/maintenance runs. Exclude the full provider inventory in the shell policy (Devin). inherit = "all" with only the CLI's default name-based excludes leaks provider variables that don't look credential-like (e.g. SUPABASE_URL, SUPABASE_PROJECT_REF, NEXT_PUBLIC_SUPABASE_URL, DATABASE_URL) into shells that never source the runtime profile. Fix: enumerate the full provider inventory in the exclude list, matching the runtime profile's scrub — see setup-codex-cloud.sh:126-137. please also fix and resolve any comments not addressed by that fix and then mark everything resolved once done and ensure all CI pass |
All requested review fixes are on What landed
Proof
  |
Uh oh!
There was an error while loading. Please reload this page.
Reject incomplete managed marker blocks and unmanaged [shell_environment_policy] tables before rewriting config.toml, and pin the contract checks that prove profile handoff safety.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/check-codex-cloud-setup.mjs`:
- Around line 344-357: Add behavior-level tests in
scripts/check-codex-cloud-setup.mjs using temporary-HOME fixtures for both
offline and connected profiles: run policy generation, parse config.toml, and
verify the expected provider exclusions and connected retrieval setting. Also
test unmanaged and incomplete managed shell_environment_policy inputs, asserting
each setup attempt fails before modifying config.toml.
In `@tests/codex-cloud-setup.test.ts`:
- Around line 220-232: Replace the source-text-only assertions in the setup
tests with behavior-level coverage for scripts/setup-codex-cloud.sh. Execute
setup using a temporary HOME and stubbed external tools, then verify connected
and offline profile output, all required exclude entries, rejection of invalid
shell policies, and preservation of unrelated configuration. Keep the existing
configuration and environment-variable expectations where applicable.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: bc261650-9c4e-4e4f-a71d-13b3ccf573f0
📒 Files selected for processing (3)
scripts/check-codex-cloud-setup.mjsscripts/setup-codex-cloud.shtests/codex-cloud-setup.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- scripts/setup-codex-cloud.sh
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Expose a policy-only setup stop for temporary-HOME fixtures and assert managed config merge, retrieval pinning, and unsafe-policy rejection.
Review fixes are in on tip Covered:
Local proof: |
Pull request was converted to draft
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:1a8cf69c57
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
CI triageCI failed on this PR. Automated classification of the 2 failed job(s):
Compared with main CI run #7797 (cancelled). Classification is evidence routing, not permission to ignore a failure. Exact quarantined Playwright identities remain governed by the flake ledger. |
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
tests/codex-cloud-setup.test.ts (1)
316-385: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winSplit the combined test into focused
it()blocks.This single test verifies four distinct behaviors: connected retrieval-mode pinning, unmanaged-config rejection, three formatted-header variants of unmanaged detection, and incomplete-managed-block rejection. If an early assertion fails, Vitest stops running the rest of the test body, so the formatted-header loop and the incomplete-block scenario never execute. A regression in a later scenario is masked whenever an earlier one also fails.
Split this into separate
it()blocks, one per scenario, so CI output identifies exactly which scenario regressed.♻️ Suggested split
- it("pins connected retrieval mode and rejects unsafe shell-policy configs", () => {- const connectedHome = temporaryDirectory("codex-cloud-connected-");- const connected = runSetupPolicyOnly(connectedHome, {- CODEX_CLOUD_ACCESS_PROFILE: "connected",- RAG_PROVIDER_MODE: "offline",- });- expect(connected.status, connected.stderr || connected.stdout).toBe(0);- const connectedProfile = readRuntimeProfile(connectedHome);- expect(connectedProfile).toContain('export CODEX_CLOUD_ACCESS_PROFILE="connected"');- expect(connectedProfile).toContain('export RAG_PROVIDER_MODE="offline"');- expect(connectedProfile).not.toContain("${RAG_PROVIDER_MODE:-auto}");-- const unmanagedHome = temporaryDirectory("codex-cloud-unmanaged-");+ it("pins connected retrieval mode", () => {+ const connectedHome = temporaryDirectory("codex-cloud-connected-");+ const connected = runSetupPolicyOnly(connectedHome, {+ CODEX_CLOUD_ACCESS_PROFILE: "connected",+ RAG_PROVIDER_MODE: "offline",+ });+ expect(connected.status, connected.stderr || connected.stdout).toBe(0);+ const connectedProfile = readRuntimeProfile(connectedHome);+ expect(connectedProfile).toContain('export CODEX_CLOUD_ACCESS_PROFILE="connected"');+ expect(connectedProfile).toContain('export RAG_PROVIDER_MODE="offline"');+ expect(connectedProfile).not.toContain("${RAG_PROVIDER_MODE:-auto}");+ });++ it("rejects an unmanaged shell-policy table", () => {+ const unmanagedHome = temporaryDirectory("codex-cloud-unmanaged-"); mkdirSync(path.join(unmanagedHome, ".codex"), { recursive: true }); const unmanagedConfig = [ "[mcp_servers.keep]", 'command = "echo"', "", "[shell_environment_policy]", 'inherit = "all"', "exclude = []", "", ].join("\n"); const unmanagedPath = path.join(unmanagedHome, ".codex/config.toml"); writeFileSync(unmanagedPath, unmanagedConfig); const unmanaged = runSetupPolicyOnly(unmanagedHome, { CODEX_CLOUD_ACCESS_PROFILE: "offline", }); expect(unmanaged.status).not.toBe(0); expect(unmanaged.stderr).toContain("Unmanaged [shell_environment_policy] table found"); expect(readFileSync(unmanagedPath, "utf8")).toBe(unmanagedConfig); + });+ it("rejects alternate valid TOML formattings of the unmanaged table", () => { for (const tableHeader of [ " [shell_environment_policy] # valid TOML", '[ "shell_environment_policy" ]', "['shell_environment_policy'] # valid TOML", ]) { const formattedHome = temporaryDirectory("codex-cloud-formatted-"); mkdirSync(path.join(formattedHome, ".codex"), { recursive: true }); const formattedConfig = [tableHeader, 'inherit = "all"', "exclude = []", ""].join("\n"); const formattedPath = path.join(formattedHome, ".codex/config.toml"); writeFileSync(formattedPath, formattedConfig); const formatted = runSetupPolicyOnly(formattedHome, { CODEX_CLOUD_ACCESS_PROFILE: "offline", }); expect(formatted.status).not.toBe(0); expect(formatted.stderr).toContain("Unmanaged [shell_environment_policy] table found"); expect(readFileSync(formattedPath, "utf8")).toBe(formattedConfig); } + });+ it("rejects an incomplete managed shell policy block", () => { const incompleteHome = temporaryDirectory("codex-cloud-incomplete-"); mkdirSync(path.join(incompleteHome, ".codex"), { recursive: true }); const incompleteConfig = [ "[mcp_servers.keep]", 'command = "echo"', "", "# BEGIN clinical-kb-codex-cloud shell policy (managed by setup-codex-cloud.sh)", "[shell_environment_policy]", 'inherit = "all"', "", ].join("\n"); const incompletePath = path.join(incompleteHome, ".codex/config.toml"); writeFileSync(incompletePath, incompleteConfig); const incomplete = runSetupPolicyOnly(incompleteHome, { CODEX_CLOUD_ACCESS_PROFILE: "offline", }); expect(incomplete.status).not.toBe(0); expect(incomplete.stderr).toContain("Incomplete managed shell policy block"); expect(readFileSync(incompletePath, "utf8")).toBe(incompleteConfig); });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/codex-cloud-setup.test.ts` around lines 316 - 385, Split the combined test into focused it() blocks: one for connected retrieval-mode pinning, one for unmanaged configuration rejection, one for formatted shell-environment-policy headers (retaining all three header variants), and one for incomplete managed policy rejection. Preserve each scenario’s setup, assertions, and fixture cleanup so failures identify the specific regression.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/codex-cloud.md`:
- Line 130: Update the Cloud permissions documentation near the “BigSimmo”
connection to state that repository write access is the default for tasks
publishing branches or pull requests. Remove administrator access as the
default, and mention it only for separately approved operations.
In `@tests/codex-cloud-setup.test.ts`:
- Around line 34-40: Expand the requiredPolicyExcludes constant in the codex
cloud setup tests to contain all 34 entries defined by
codex_shell_policy_excludes, preserving the existing entries and matching the
policy’s complete exclusion set so every exclusion is validated.
---
Nitpick comments:
In `@tests/codex-cloud-setup.test.ts`:
- Around line 316-385: Split the combined test into focused it() blocks: one for
connected retrieval-mode pinning, one for unmanaged configuration rejection, one
for formatted shell-environment-policy headers (retaining all three header
variants), and one for incomplete managed policy rejection. Preserve each
scenario’s setup, assertions, and fixture cleanup so failures identify the
specific regression.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 82ebd4d6-db2d-4ff1-9474-f4e2358e8f76
📒 Files selected for processing (4)
docs/codex-cloud.mdscripts/check-codex-cloud-setup.mjsscripts/setup-codex-cloud.shtests/codex-cloud-setup.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- scripts/check-codex-cloud-setup.mjs
- scripts/setup-codex-cloud.sh
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:0843d0d5f8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Summary
Why
A fresh Cloud task saw
connectedduring setup but later receivedofflinein the agent shell. The generated profile evaluated fallbacks again at source time. In addition, raw agent commands could inherit provider variables when the runtime profile was not sourced.The Cloud GitHub connection is present and native Cloud task controls can produce a verifiable GitHub PR. A missing direct tool in an individual task limits that API surface; it does not by itself prove that the GitHub integration is disconnected. Shell Git and
ghremain intentionally unauthenticated.Validation
~/.codex/config.tomlblock is idempotent and preserves unrelated Codex settings.make_prresult remains insufficient.7904aaf482e74a58f773af05708c7a631c94be29.npm run check:codex-cloudandnpm run check:codex-cloud -- --runtimewithCODEX_CLOUD_ACCESS_PROFILE=connected,RAG_PROVIDER_MODE=offline, andOPENAI_BASE_URL.present=false.Risk and rollout
npm run check:codex-clouddirectly.Provider and production effects
Clinical governance preflight
Limit
This repository-side remediation fixes the runtime-profile, inherited-variable, and documentation defects. It does not add raw GitHub credentials or make every GitHub API appear as a direct Cloud agent tool. Cloud's native diff/PR controls remain the supported ordinary publication path; unavailable API-specific capabilities must be reported or handled through an approved GitHub-connected workflow.
Do not merge or clean up without explicit approval.
Summary by CodeRabbit
New Features
Documentation
Bug Fixes