fix(codex): recover turns after usage limits

[estensen]

Decision

Codex App Server keeps one account per process, so a T3 thread stops when that account runs out. This change restarts the configured executable, strictly resumes the same provider thread, and retries a safe failed turn once. Account-selecting launchers can then continue the conversation with available quota. Other providers, contracts, clients, and UI behavior do not change.

What Changed

• Retain each in-flight Codex turn's mapped input and launch settings in the adapter.
• Detect the structured usageLimitExceeded completion emitted by Codex App Server.
• Restart the configured executable and require the original provider thread. Never fall back to a fresh thread during recovery.
• Read the resumed thread. Roll back only a failed user-only turn, then retry the exact input once.
• Preserve normal event forwarding, including the original usage-limit failure.
• Document automatic Codex recovery for account-selecting executables.

Production Behavior And Risk

The recovery path runs only for the first usage-limit failure. It refuses to retry after assistant output, tool requests, commands, hooks, diffs, plans, collaboration activity, or queued turns. It verifies the resumed provider thread and requested model before retrying.

Manual session starts, stops, and interrupts cancel an in-flight recovery. A recovery process cannot overwrite a newer user-started session. A second usage-limit failure remains visible and does not launch a third process.

The worst case is a configured executable that restarts but cannot resume the original provider thread. T3 Code closes that recovery session and exposes the existing failure. It never creates a disconnected conversation. T3 Code adds no migrations or new persistent state. Its only provider mutation is rolling back the failed user-only turn before retrying it.

Reviewer Focus

• Verify event ordering when turn/completed arrives before the turn/start response.
• Verify that the side-effect guard is fail-closed for provider activity while allowing Codex's user-message lifecycle events.
• Verify cancellation when manual session replacement races automatic recovery.

Test Plan

• vp test run apps/server/src/provider/Layers/CodexAdapter.test.ts apps/server/src/provider/Layers/CodexSessionRuntime.test.ts — 57 tests passed.
• Targeted vp lint for the four changed TypeScript files.
• vp run --filter t3 typecheck — passed with existing suggestions in unrelated files.
• vp fmt --check for all five changed files.

Checklist

• This PR is small and focused
• I explained what changed and why

Built with GPT-5.6 Sol in Codex.

---

Note

Medium Risk
Changes Codex session lifecycle (process restart, thread resume, rollback, and async recovery) on a critical provider path, but scope is narrow with fail-closed guards and extensive tests.

Overview
Codex now handles usageLimitExceeded turn failures by giving account-selecting launchers one automatic retry without leaving the conversation.

When a turn fails with that error and produced no assistant/tool activity (only user-message lifecycle events are allowed), the adapter restarts the configured Codex process, opens the session with requireResume so thread/resume cannot fall back to a fresh thread, verifies the same provider thread and model, optionally rolls back a user-only failed turn, then re-sends the exact prompt once. The original failure still flows through the event stream; a second limit hit is not retried again.

Guards and races: pending-turn tracking handles turn/completed arriving before turn/start; unsafeTurnIds blocks retry after any real provider output (including items only on completion); queued turns, manual startSession/stopSession/interruptTurn, and in-flight recovery use recovery tokens so recovery cannot replace a newer user session. sendTurn is rejected while recovery is active.

Tests cover happy path, race with manual session start, no-retry with activity, and strict resume behavior. User docs describe when auto-recovery runs and when it stops.

^{Reviewed by Cursor Bugbot for commit 92a148c. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add automatic recovery from usage-limit failures in CodexAdapter

• When a Codex turn fails with usageLimitExceeded and no provider activity has occurred in that turn, the adapter automatically restarts the runtime, resumes the same thread, and retries the turn exactly once.
• A new requireResume flag in CodexSessionRuntimeOptions prevents fallback to a fresh thread when strict resume is required; recoverable resume errors are surfaced instead of silently starting over.
• sendTurn is blocked while recovery is in progress or a recovery token is active; tokens are cleared on explicit starts, interrupts, and stops.
• New tests in CodexAdapter.test.ts verify recovery semantics using observable TestQueue utilities for deterministic event ordering.
• Risk: Any provider activity (non-userMessage items, tool calls, hooks, diffs) disables retry for that turn, aborting recovery silently.

📊 Macroscope summarized 92a148c. 3 files reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5024b059-11d5-4ed6-974c-a7656c1888cb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

🟠 HighLayers/CodexAdapter.ts:1965

A stale usage-limit recovery can stop a newer manual session and leave the thread with no active session. recoverUsageLimitTurn calls startSessionInternal before validating that input.token is still current; if startSession has already deleted the token and installed a replacement, the recovery replaces and then stops that replacement, and finally closes its own runtime. Validate the token before replacing the registered session and atomically guard the replacement.

🤖 Copy this AI Prompt to have your agent fix this:

In file @apps/server/src/provider/Layers/CodexAdapter.ts around line 1965:
A stale usage-limit recovery can stop a newer manual session and leave the thread with no active session. `recoverUsageLimitTurn` calls `startSessionInternal` before validating that `input.token` is still current; if `startSession` has already deleted the token and installed a replacement, the recovery replaces and then stops that replacement, and finally closes its own runtime. Validate the token before replacing the registered session and atomically guard the replacement.

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 92a148c. Configure here.}

[cursor]

Aborted recovery leaves session open

Medium Severity

When post-resume readThread finds provider activity on the failed turn, recoverUsageLimitTurn returns without calling stopSessionInternal. By then startSessionInternal has already stopped the original session and registered the recovery process. Sibling abort paths for wrong thread or model do close that session. An account-selecting executable can therefore leave the thread on a replaced process after refusing to retry, instead of closing the recovery session and surfacing the original usage-limit failure.

Additional Locations (1)

• apps/server/src/provider/Layers/CodexAdapter.ts#L1995-L2007

^{Reviewed by Cursor Bugbot for commit 92a148c. Configure here.}

[macroscopeapp]

Approvability

Verdict: Needs human review

1 blocking correctness issue found. This PR introduces a new automatic recovery feature for usage-limit failures with significant state management complexity (recovery tokens, pending turn tracking, deferred completions). A High-severity finding identifies a potential race condition where stale recovery can interfere with newer sessions.

^{You can customize Macroscope's approvability policy. Learn more.}

[estensen]

Decision

Codex App Server keeps one account per process, so a T3 thread stops when that account runs out. This change restarts the configured executable, strictly resumes the same provider thread, and retries a safe failed turn once. Account-selecting launchers can then continue the conversation with available quota. Other providers, contracts, clients, and UI behavior do not change.

What Changed

• Retain each in-flight Codex turn's mapped input and launch settings in the adapter.
• Detect the structured usageLimitExceeded completion emitted by Codex App Server.
• Restart the configured executable and require the original provider thread. Never fall back to a fresh thread during recovery.
• Read the resumed thread. Roll back only a failed user-only turn, then retry the exact input once.
• Preserve normal event forwarding, including the original usage-limit failure.
• Document automatic Codex recovery for account-selecting executables.

Production Behavior And Risk

The recovery path runs only for the first usage-limit failure. It refuses to retry after assistant output, tool requests, commands, hooks, diffs, plans, collaboration activity, or queued turns. It verifies the resumed provider thread and requested model before retrying.

Manual session starts, stops, and interrupts cancel an in-flight recovery. A recovery process cannot overwrite a newer user-started session. A second usage-limit failure remains visible and does not launch a third process.

The worst case is a configured executable that restarts but cannot resume the original provider thread. T3 Code closes that recovery session and exposes the existing failure. It never creates a disconnected conversation. T3 Code adds no migrations or new persistent state. Its only provider mutation is rolling back the failed user-only turn before retrying it.

Reviewer Focus

• Verify event ordering when turn/completed arrives before the turn/start response.
• Verify that the side-effect guard is fail-closed for provider activity while allowing Codex's user-message lifecycle events.
• Verify cancellation when manual session replacement races automatic recovery.

Test Plan

• vp test run apps/server/src/provider/Layers/CodexAdapter.test.ts apps/server/src/provider/Layers/CodexSessionRuntime.test.ts — 57 tests passed.
• Targeted vp lint for the four changed TypeScript files.
• vp run --filter t3 typecheck — passed with existing suggestions in unrelated files.
• vp fmt --check for all five changed files.

Checklist

• This PR is small and focused
• I explained what changed and why

Built with GPT-5.6 Sol in Codex.

---

Note

Medium Risk
Changes Codex session lifecycle (process restart, thread resume, rollback, and async recovery) on a critical provider path, but scope is narrow with fail-closed guards and extensive tests.

Overview
Codex now handles usageLimitExceeded turn failures by giving account-selecting launchers one automatic retry without leaving the conversation.

When a turn fails with that error and produced no assistant/tool activity (only user-message lifecycle events are allowed), the adapter restarts the configured Codex process, opens the session with requireResume so thread/resume cannot fall back to a fresh thread, verifies the same provider thread and model, optionally rolls back a user-only failed turn, then re-sends the exact prompt once. The original failure still flows through the event stream; a second limit hit is not retried again.

Guards and races: pending-turn tracking handles turn/completed arriving before turn/start; unsafeTurnIds blocks retry after any real provider output (including items only on completion); queued turns, manual startSession/stopSession/interruptTurn, and in-flight recovery use recovery tokens so recovery cannot replace a newer user session. sendTurn is rejected while recovery is active.

Tests cover happy path, race with manual session start, no-retry with activity, and strict resume behavior. User docs describe when auto-recovery runs and when it stops.

^{Reviewed by Cursor Bugbot for commit 92a148c. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add automatic recovery from usage-limit failures in CodexAdapter

• When a Codex turn fails with usageLimitExceeded and no provider activity has occurred in that turn, the adapter automatically restarts the runtime, resumes the same thread, and retries the turn exactly once.
• A new requireResume flag in CodexSessionRuntimeOptions prevents fallback to a fresh thread when strict resume is required; recoverable resume errors are surfaced instead of silently starting over.
• sendTurn is blocked while recovery is in progress or a recovery token is active; tokens are cleared on explicit starts, interrupts, and stops.
• New tests in CodexAdapter.test.ts verify recovery semantics using observable TestQueue utilities for deterministic event ordering.
• Risk: Any provider activity (non-userMessage items, tool calls, hooks, diffs) disables retry for that turn, aborting recovery silently.

📊 Macroscope summarized 92a148c. 3 files reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5024b059-11d5-4ed6-974c-a7656c1888cb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

🟠 HighLayers/CodexAdapter.ts:1965

A stale usage-limit recovery can stop a newer manual session and leave the thread with no active session. recoverUsageLimitTurn calls startSessionInternal before validating that input.token is still current; if startSession has already deleted the token and installed a replacement, the recovery replaces and then stops that replacement, and finally closes its own runtime. Validate the token before replacing the registered session and atomically guard the replacement.

🤖 Copy this AI Prompt to have your agent fix this:

In file @apps/server/src/provider/Layers/CodexAdapter.ts around line 1965:
A stale usage-limit recovery can stop a newer manual session and leave the thread with no active session. `recoverUsageLimitTurn` calls `startSessionInternal` before validating that `input.token` is still current; if `startSession` has already deleted the token and installed a replacement, the recovery replaces and then stops that replacement, and finally closes its own runtime. Validate the token before replacing the registered session and atomically guard the replacement.

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 92a148c. Configure here.}

[cursor]

Aborted recovery leaves session open

Medium Severity

When post-resume readThread finds provider activity on the failed turn, recoverUsageLimitTurn returns without calling stopSessionInternal. By then startSessionInternal has already stopped the original session and registered the recovery process. Sibling abort paths for wrong thread or model do close that session. An account-selecting executable can therefore leave the thread on a replaced process after refusing to retry, instead of closing the recovery session and surfacing the original usage-limit failure.

Additional Locations (1)

• apps/server/src/provider/Layers/CodexAdapter.ts#L1995-L2007

^{Reviewed by Cursor Bugbot for commit 92a148c. Configure here.}

[macroscopeapp]

Approvability

Verdict: Needs human review

1 blocking correctness issue found. This PR introduces a new automatic recovery feature for usage-limit failures with significant state management complexity (recovery tokens, pending turn tracking, deferred completions). A High-severity finding identifies a potential race condition where stale recovery can interfere with newer sessions.

^{You can customize Macroscope's approvability policy. Learn more.}

[estensen]

Decision

Codex App Server keeps one account per process, so a T3 thread stops when that account runs out. This change restarts the configured executable, strictly resumes the same provider thread, and retries a safe failed turn once. Account-selecting launchers can then continue the conversation with available quota. Other providers, contracts, clients, and UI behavior do not change.

What Changed

• Retain each in-flight Codex turn's mapped input and launch settings in the adapter.
• Detect the structured usageLimitExceeded completion emitted by Codex App Server.
• Restart the configured executable and require the original provider thread. Never fall back to a fresh thread during recovery.
• Read the resumed thread. Roll back only a failed user-only turn, then retry the exact input once.
• Preserve normal event forwarding, including the original usage-limit failure.
• Document automatic Codex recovery for account-selecting executables.

Production Behavior And Risk

The recovery path runs only for the first usage-limit failure. It refuses to retry after assistant output, tool requests, commands, hooks, diffs, plans, collaboration activity, or queued turns. It verifies the resumed provider thread and requested model before retrying.

Manual session starts, stops, and interrupts cancel an in-flight recovery. A recovery process cannot overwrite a newer user-started session. A second usage-limit failure remains visible and does not launch a third process.

The worst case is a configured executable that restarts but cannot resume the original provider thread. T3 Code closes that recovery session and exposes the existing failure. It never creates a disconnected conversation. T3 Code adds no migrations or new persistent state. Its only provider mutation is rolling back the failed user-only turn before retrying it.

Reviewer Focus

• Verify event ordering when turn/completed arrives before the turn/start response.
• Verify that the side-effect guard is fail-closed for provider activity while allowing Codex's user-message lifecycle events.
• Verify cancellation when manual session replacement races automatic recovery.

Test Plan

• vp test run apps/server/src/provider/Layers/CodexAdapter.test.ts apps/server/src/provider/Layers/CodexSessionRuntime.test.ts — 57 tests passed.
• Targeted vp lint for the four changed TypeScript files.
• vp run --filter t3 typecheck — passed with existing suggestions in unrelated files.
• vp fmt --check for all five changed files.

Checklist

• This PR is small and focused
• I explained what changed and why

Built with GPT-5.6 Sol in Codex.

---

Note

Medium Risk
Changes Codex session lifecycle (process restart, thread resume, rollback, and async recovery) on a critical provider path, but scope is narrow with fail-closed guards and extensive tests.

Overview
Codex now handles usageLimitExceeded turn failures by giving account-selecting launchers one automatic retry without leaving the conversation.

When a turn fails with that error and produced no assistant/tool activity (only user-message lifecycle events are allowed), the adapter restarts the configured Codex process, opens the session with requireResume so thread/resume cannot fall back to a fresh thread, verifies the same provider thread and model, optionally rolls back a user-only failed turn, then re-sends the exact prompt once. The original failure still flows through the event stream; a second limit hit is not retried again.

Guards and races: pending-turn tracking handles turn/completed arriving before turn/start; unsafeTurnIds blocks retry after any real provider output (including items only on completion); queued turns, manual startSession/stopSession/interruptTurn, and in-flight recovery use recovery tokens so recovery cannot replace a newer user session. sendTurn is rejected while recovery is active.

Tests cover happy path, race with manual session start, no-retry with activity, and strict resume behavior. User docs describe when auto-recovery runs and when it stops.

^{Reviewed by Cursor Bugbot for commit 92a148c. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add automatic recovery from usage-limit failures in CodexAdapter

• When a Codex turn fails with usageLimitExceeded and no provider activity has occurred in that turn, the adapter automatically restarts the runtime, resumes the same thread, and retries the turn exactly once.
• A new requireResume flag in CodexSessionRuntimeOptions prevents fallback to a fresh thread when strict resume is required; recoverable resume errors are surfaced instead of silently starting over.
• sendTurn is blocked while recovery is in progress or a recovery token is active; tokens are cleared on explicit starts, interrupts, and stops.
• New tests in CodexAdapter.test.ts verify recovery semantics using observable TestQueue utilities for deterministic event ordering.
• Risk: Any provider activity (non-userMessage items, tool calls, hooks, diffs) disables retry for that turn, aborting recovery silently.

📊 Macroscope summarized 92a148c. 3 files reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5024b059-11d5-4ed6-974c-a7656c1888cb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

🟠 HighLayers/CodexAdapter.ts:1965

A stale usage-limit recovery can stop a newer manual session and leave the thread with no active session. recoverUsageLimitTurn calls startSessionInternal before validating that input.token is still current; if startSession has already deleted the token and installed a replacement, the recovery replaces and then stops that replacement, and finally closes its own runtime. Validate the token before replacing the registered session and atomically guard the replacement.

🤖 Copy this AI Prompt to have your agent fix this:

In file @apps/server/src/provider/Layers/CodexAdapter.ts around line 1965:
A stale usage-limit recovery can stop a newer manual session and leave the thread with no active session. `recoverUsageLimitTurn` calls `startSessionInternal` before validating that `input.token` is still current; if `startSession` has already deleted the token and installed a replacement, the recovery replaces and then stops that replacement, and finally closes its own runtime. Validate the token before replacing the registered session and atomically guard the replacement.

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 92a148c. Configure here.}

[cursor]

Aborted recovery leaves session open

Medium Severity

When post-resume readThread finds provider activity on the failed turn, recoverUsageLimitTurn returns without calling stopSessionInternal. By then startSessionInternal has already stopped the original session and registered the recovery process. Sibling abort paths for wrong thread or model do close that session. An account-selecting executable can therefore leave the thread on a replaced process after refusing to retry, instead of closing the recovery session and surfacing the original usage-limit failure.

Additional Locations (1)

• apps/server/src/provider/Layers/CodexAdapter.ts#L1995-L2007

^{Reviewed by Cursor Bugbot for commit 92a148c. Configure here.}

[macroscopeapp]

Approvability

Verdict: Needs human review

1 blocking correctness issue found. This PR introduces a new automatic recovery feature for usage-limit failures with significant state management complexity (recovery tokens, pending turn tracking, deferred completions). A High-severity finding identifies a potential race condition where stale recovery can interfere with newer sessions.

^{You can customize Macroscope's approvability policy. Learn more.}

[estensen]

Decision

Codex App Server keeps one account per process, so a T3 thread stops when that account runs out. This change restarts the configured executable, strictly resumes the same provider thread, and retries a safe failed turn once. Account-selecting launchers can then continue the conversation with available quota. Other providers, contracts, clients, and UI behavior do not change.

What Changed

• Retain each in-flight Codex turn's mapped input and launch settings in the adapter.
• Detect the structured usageLimitExceeded completion emitted by Codex App Server.
• Restart the configured executable and require the original provider thread. Never fall back to a fresh thread during recovery.
• Read the resumed thread. Roll back only a failed user-only turn, then retry the exact input once.
• Preserve normal event forwarding, including the original usage-limit failure.
• Document automatic Codex recovery for account-selecting executables.

Production Behavior And Risk

The recovery path runs only for the first usage-limit failure. It refuses to retry after assistant output, tool requests, commands, hooks, diffs, plans, collaboration activity, or queued turns. It verifies the resumed provider thread and requested model before retrying.

Manual session starts, stops, and interrupts cancel an in-flight recovery. A recovery process cannot overwrite a newer user-started session. A second usage-limit failure remains visible and does not launch a third process.

The worst case is a configured executable that restarts but cannot resume the original provider thread. T3 Code closes that recovery session and exposes the existing failure. It never creates a disconnected conversation. T3 Code adds no migrations or new persistent state. Its only provider mutation is rolling back the failed user-only turn before retrying it.

Reviewer Focus

• Verify event ordering when turn/completed arrives before the turn/start response.
• Verify that the side-effect guard is fail-closed for provider activity while allowing Codex's user-message lifecycle events.
• Verify cancellation when manual session replacement races automatic recovery.

Test Plan

• vp test run apps/server/src/provider/Layers/CodexAdapter.test.ts apps/server/src/provider/Layers/CodexSessionRuntime.test.ts — 57 tests passed.
• Targeted vp lint for the four changed TypeScript files.
• vp run --filter t3 typecheck — passed with existing suggestions in unrelated files.
• vp fmt --check for all five changed files.

Checklist

• This PR is small and focused
• I explained what changed and why

Built with GPT-5.6 Sol in Codex.

---

Note

Medium Risk
Changes Codex session lifecycle (process restart, thread resume, rollback, and async recovery) on a critical provider path, but scope is narrow with fail-closed guards and extensive tests.

Overview
Codex now handles usageLimitExceeded turn failures by giving account-selecting launchers one automatic retry without leaving the conversation.

When a turn fails with that error and produced no assistant/tool activity (only user-message lifecycle events are allowed), the adapter restarts the configured Codex process, opens the session with requireResume so thread/resume cannot fall back to a fresh thread, verifies the same provider thread and model, optionally rolls back a user-only failed turn, then re-sends the exact prompt once. The original failure still flows through the event stream; a second limit hit is not retried again.

Guards and races: pending-turn tracking handles turn/completed arriving before turn/start; unsafeTurnIds blocks retry after any real provider output (including items only on completion); queued turns, manual startSession/stopSession/interruptTurn, and in-flight recovery use recovery tokens so recovery cannot replace a newer user session. sendTurn is rejected while recovery is active.

Tests cover happy path, race with manual session start, no-retry with activity, and strict resume behavior. User docs describe when auto-recovery runs and when it stops.

^{Reviewed by Cursor Bugbot for commit 92a148c. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add automatic recovery from usage-limit failures in CodexAdapter

• When a Codex turn fails with usageLimitExceeded and no provider activity has occurred in that turn, the adapter automatically restarts the runtime, resumes the same thread, and retries the turn exactly once.
• A new requireResume flag in CodexSessionRuntimeOptions prevents fallback to a fresh thread when strict resume is required; recoverable resume errors are surfaced instead of silently starting over.
• sendTurn is blocked while recovery is in progress or a recovery token is active; tokens are cleared on explicit starts, interrupts, and stops.
• New tests in CodexAdapter.test.ts verify recovery semantics using observable TestQueue utilities for deterministic event ordering.
• Risk: Any provider activity (non-userMessage items, tool calls, hooks, diffs) disables retry for that turn, aborting recovery silently.

📊 Macroscope summarized 92a148c. 3 files reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5024b059-11d5-4ed6-974c-a7656c1888cb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

🟠 HighLayers/CodexAdapter.ts:1965

A stale usage-limit recovery can stop a newer manual session and leave the thread with no active session. recoverUsageLimitTurn calls startSessionInternal before validating that input.token is still current; if startSession has already deleted the token and installed a replacement, the recovery replaces and then stops that replacement, and finally closes its own runtime. Validate the token before replacing the registered session and atomically guard the replacement.

🤖 Copy this AI Prompt to have your agent fix this:

In file @apps/server/src/provider/Layers/CodexAdapter.ts around line 1965:
A stale usage-limit recovery can stop a newer manual session and leave the thread with no active session. `recoverUsageLimitTurn` calls `startSessionInternal` before validating that `input.token` is still current; if `startSession` has already deleted the token and installed a replacement, the recovery replaces and then stops that replacement, and finally closes its own runtime. Validate the token before replacing the registered session and atomically guard the replacement.

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 92a148c. Configure here.}

[cursor]

Aborted recovery leaves session open

Medium Severity

When post-resume readThread finds provider activity on the failed turn, recoverUsageLimitTurn returns without calling stopSessionInternal. By then startSessionInternal has already stopped the original session and registered the recovery process. Sibling abort paths for wrong thread or model do close that session. An account-selecting executable can therefore leave the thread on a replaced process after refusing to retry, instead of closing the recovery session and surfacing the original usage-limit failure.

Additional Locations (1)

• apps/server/src/provider/Layers/CodexAdapter.ts#L1995-L2007

^{Reviewed by Cursor Bugbot for commit 92a148c. Configure here.}

[macroscopeapp]

Approvability

Verdict: Needs human review

1 blocking correctness issue found. This PR introduces a new automatic recovery feature for usage-limit failures with significant state management complexity (recovery tokens, pending turn tracking, deferred completions). A High-severity finding identifies a potential race condition where stale recovery can interfere with newer sessions.

^{You can customize Macroscope's approvability policy. Learn more.}

[estensen]

Decision

Codex App Server keeps one account per process, so a T3 thread stops when that account runs out. This change restarts the configured executable, strictly resumes the same provider thread, and retries a safe failed turn once. Account-selecting launchers can then continue the conversation with available quota. Other providers, contracts, clients, and UI behavior do not change.

What Changed

• Retain each in-flight Codex turn's mapped input and launch settings in the adapter.
• Detect the structured usageLimitExceeded completion emitted by Codex App Server.
• Restart the configured executable and require the original provider thread. Never fall back to a fresh thread during recovery.
• Read the resumed thread. Roll back only a failed user-only turn, then retry the exact input once.
• Preserve normal event forwarding, including the original usage-limit failure.
• Document automatic Codex recovery for account-selecting executables.

Production Behavior And Risk

The recovery path runs only for the first usage-limit failure. It refuses to retry after assistant output, tool requests, commands, hooks, diffs, plans, collaboration activity, or queued turns. It verifies the resumed provider thread and requested model before retrying.

Manual session starts, stops, and interrupts cancel an in-flight recovery. A recovery process cannot overwrite a newer user-started session. A second usage-limit failure remains visible and does not launch a third process.

The worst case is a configured executable that restarts but cannot resume the original provider thread. T3 Code closes that recovery session and exposes the existing failure. It never creates a disconnected conversation. T3 Code adds no migrations or new persistent state. Its only provider mutation is rolling back the failed user-only turn before retrying it.

Reviewer Focus

• Verify event ordering when turn/completed arrives before the turn/start response.
• Verify that the side-effect guard is fail-closed for provider activity while allowing Codex's user-message lifecycle events.
• Verify cancellation when manual session replacement races automatic recovery.

Test Plan

• vp test run apps/server/src/provider/Layers/CodexAdapter.test.ts apps/server/src/provider/Layers/CodexSessionRuntime.test.ts — 57 tests passed.
• Targeted vp lint for the four changed TypeScript files.
• vp run --filter t3 typecheck — passed with existing suggestions in unrelated files.
• vp fmt --check for all five changed files.

Checklist

• This PR is small and focused
• I explained what changed and why

Built with GPT-5.6 Sol in Codex.

---

Note

Medium Risk
Changes Codex session lifecycle (process restart, thread resume, rollback, and async recovery) on a critical provider path, but scope is narrow with fail-closed guards and extensive tests.

Overview
Codex now handles usageLimitExceeded turn failures by giving account-selecting launchers one automatic retry without leaving the conversation.

When a turn fails with that error and produced no assistant/tool activity (only user-message lifecycle events are allowed), the adapter restarts the configured Codex process, opens the session with requireResume so thread/resume cannot fall back to a fresh thread, verifies the same provider thread and model, optionally rolls back a user-only failed turn, then re-sends the exact prompt once. The original failure still flows through the event stream; a second limit hit is not retried again.

Guards and races: pending-turn tracking handles turn/completed arriving before turn/start; unsafeTurnIds blocks retry after any real provider output (including items only on completion); queued turns, manual startSession/stopSession/interruptTurn, and in-flight recovery use recovery tokens so recovery cannot replace a newer user session. sendTurn is rejected while recovery is active.

Tests cover happy path, race with manual session start, no-retry with activity, and strict resume behavior. User docs describe when auto-recovery runs and when it stops.

^{Reviewed by Cursor Bugbot for commit 92a148c. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add automatic recovery from usage-limit failures in CodexAdapter

• When a Codex turn fails with usageLimitExceeded and no provider activity has occurred in that turn, the adapter automatically restarts the runtime, resumes the same thread, and retries the turn exactly once.
• A new requireResume flag in CodexSessionRuntimeOptions prevents fallback to a fresh thread when strict resume is required; recoverable resume errors are surfaced instead of silently starting over.
• sendTurn is blocked while recovery is in progress or a recovery token is active; tokens are cleared on explicit starts, interrupts, and stops.
• New tests in CodexAdapter.test.ts verify recovery semantics using observable TestQueue utilities for deterministic event ordering.
• Risk: Any provider activity (non-userMessage items, tool calls, hooks, diffs) disables retry for that turn, aborting recovery silently.

📊 Macroscope summarized 92a148c. 3 files reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5024b059-11d5-4ed6-974c-a7656c1888cb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

🟠 HighLayers/CodexAdapter.ts:1965

A stale usage-limit recovery can stop a newer manual session and leave the thread with no active session. recoverUsageLimitTurn calls startSessionInternal before validating that input.token is still current; if startSession has already deleted the token and installed a replacement, the recovery replaces and then stops that replacement, and finally closes its own runtime. Validate the token before replacing the registered session and atomically guard the replacement.

🤖 Copy this AI Prompt to have your agent fix this:

In file @apps/server/src/provider/Layers/CodexAdapter.ts around line 1965:
A stale usage-limit recovery can stop a newer manual session and leave the thread with no active session. `recoverUsageLimitTurn` calls `startSessionInternal` before validating that `input.token` is still current; if `startSession` has already deleted the token and installed a replacement, the recovery replaces and then stops that replacement, and finally closes its own runtime. Validate the token before replacing the registered session and atomically guard the replacement.

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 92a148c. Configure here.}

[cursor]

Aborted recovery leaves session open

Medium Severity

When post-resume readThread finds provider activity on the failed turn, recoverUsageLimitTurn returns without calling stopSessionInternal. By then startSessionInternal has already stopped the original session and registered the recovery process. Sibling abort paths for wrong thread or model do close that session. An account-selecting executable can therefore leave the thread on a replaced process after refusing to retry, instead of closing the recovery session and surfacing the original usage-limit failure.

Additional Locations (1)

• apps/server/src/provider/Layers/CodexAdapter.ts#L1995-L2007

^{Reviewed by Cursor Bugbot for commit 92a148c. Configure here.}

[macroscopeapp]

Approvability

Verdict: Needs human review

1 blocking correctness issue found. This PR introduces a new automatic recovery feature for usage-limit failures with significant state management complexity (recovery tokens, pending turn tracking, deferred completions). A High-severity finding identifies a potential race condition where stale recovery can interfere with newer sessions.

^{You can customize Macroscope's approvability policy. Learn more.}

[estensen]

Decision

Codex App Server keeps one account per process, so a T3 thread stops when that account runs out. This change restarts the configured executable, strictly resumes the same provider thread, and retries a safe failed turn once. Account-selecting launchers can then continue the conversation with available quota. Other providers, contracts, clients, and UI behavior do not change.

What Changed

• Retain each in-flight Codex turn's mapped input and launch settings in the adapter.
• Detect the structured usageLimitExceeded completion emitted by Codex App Server.
• Restart the configured executable and require the original provider thread. Never fall back to a fresh thread during recovery.
• Read the resumed thread. Roll back only a failed user-only turn, then retry the exact input once.
• Preserve normal event forwarding, including the original usage-limit failure.
• Document automatic Codex recovery for account-selecting executables.

Production Behavior And Risk

The recovery path runs only for the first usage-limit failure. It refuses to retry after assistant output, tool requests, commands, hooks, diffs, plans, collaboration activity, or queued turns. It verifies the resumed provider thread and requested model before retrying.

Manual session starts, stops, and interrupts cancel an in-flight recovery. A recovery process cannot overwrite a newer user-started session. A second usage-limit failure remains visible and does not launch a third process.

The worst case is a configured executable that restarts but cannot resume the original provider thread. T3 Code closes that recovery session and exposes the existing failure. It never creates a disconnected conversation. T3 Code adds no migrations or new persistent state. Its only provider mutation is rolling back the failed user-only turn before retrying it.

Reviewer Focus

• Verify event ordering when turn/completed arrives before the turn/start response.
• Verify that the side-effect guard is fail-closed for provider activity while allowing Codex's user-message lifecycle events.
• Verify cancellation when manual session replacement races automatic recovery.

Test Plan

• vp test run apps/server/src/provider/Layers/CodexAdapter.test.ts apps/server/src/provider/Layers/CodexSessionRuntime.test.ts — 57 tests passed.
• Targeted vp lint for the four changed TypeScript files.
• vp run --filter t3 typecheck — passed with existing suggestions in unrelated files.
• vp fmt --check for all five changed files.

Checklist

• This PR is small and focused
• I explained what changed and why

Built with GPT-5.6 Sol in Codex.

---

Note

Medium Risk
Changes Codex session lifecycle (process restart, thread resume, rollback, and async recovery) on a critical provider path, but scope is narrow with fail-closed guards and extensive tests.

Overview
Codex now handles usageLimitExceeded turn failures by giving account-selecting launchers one automatic retry without leaving the conversation.

When a turn fails with that error and produced no assistant/tool activity (only user-message lifecycle events are allowed), the adapter restarts the configured Codex process, opens the session with requireResume so thread/resume cannot fall back to a fresh thread, verifies the same provider thread and model, optionally rolls back a user-only failed turn, then re-sends the exact prompt once. The original failure still flows through the event stream; a second limit hit is not retried again.

Guards and races: pending-turn tracking handles turn/completed arriving before turn/start; unsafeTurnIds blocks retry after any real provider output (including items only on completion); queued turns, manual startSession/stopSession/interruptTurn, and in-flight recovery use recovery tokens so recovery cannot replace a newer user session. sendTurn is rejected while recovery is active.

Tests cover happy path, race with manual session start, no-retry with activity, and strict resume behavior. User docs describe when auto-recovery runs and when it stops.

^{Reviewed by Cursor Bugbot for commit 92a148c. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add automatic recovery from usage-limit failures in CodexAdapter

• When a Codex turn fails with usageLimitExceeded and no provider activity has occurred in that turn, the adapter automatically restarts the runtime, resumes the same thread, and retries the turn exactly once.
• A new requireResume flag in CodexSessionRuntimeOptions prevents fallback to a fresh thread when strict resume is required; recoverable resume errors are surfaced instead of silently starting over.
• sendTurn is blocked while recovery is in progress or a recovery token is active; tokens are cleared on explicit starts, interrupts, and stops.
• New tests in CodexAdapter.test.ts verify recovery semantics using observable TestQueue utilities for deterministic event ordering.
• Risk: Any provider activity (non-userMessage items, tool calls, hooks, diffs) disables retry for that turn, aborting recovery silently.

📊 Macroscope summarized 92a148c. 3 files reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5024b059-11d5-4ed6-974c-a7656c1888cb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

🟠 HighLayers/CodexAdapter.ts:1965

A stale usage-limit recovery can stop a newer manual session and leave the thread with no active session. recoverUsageLimitTurn calls startSessionInternal before validating that input.token is still current; if startSession has already deleted the token and installed a replacement, the recovery replaces and then stops that replacement, and finally closes its own runtime. Validate the token before replacing the registered session and atomically guard the replacement.

🤖 Copy this AI Prompt to have your agent fix this:

In file @apps/server/src/provider/Layers/CodexAdapter.ts around line 1965:
A stale usage-limit recovery can stop a newer manual session and leave the thread with no active session. `recoverUsageLimitTurn` calls `startSessionInternal` before validating that `input.token` is still current; if `startSession` has already deleted the token and installed a replacement, the recovery replaces and then stops that replacement, and finally closes its own runtime. Validate the token before replacing the registered session and atomically guard the replacement.

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 92a148c. Configure here.}

[cursor]

Aborted recovery leaves session open

Medium Severity

When post-resume readThread finds provider activity on the failed turn, recoverUsageLimitTurn returns without calling stopSessionInternal. By then startSessionInternal has already stopped the original session and registered the recovery process. Sibling abort paths for wrong thread or model do close that session. An account-selecting executable can therefore leave the thread on a replaced process after refusing to retry, instead of closing the recovery session and surfacing the original usage-limit failure.

Additional Locations (1)

• apps/server/src/provider/Layers/CodexAdapter.ts#L1995-L2007

^{Reviewed by Cursor Bugbot for commit 92a148c. Configure here.}

[macroscopeapp]

Approvability

Verdict: Needs human review

1 blocking correctness issue found. This PR introduces a new automatic recovery feature for usage-limit failures with significant state management complexity (recovery tokens, pending turn tracking, deferred completions). A High-severity finding identifies a potential race condition where stale recovery can interfere with newer sessions.

^{You can customize Macroscope's approvability policy. Learn more.}

[estensen]

Decision

Codex App Server keeps one account per process, so a T3 thread stops when that account runs out. This change restarts the configured executable, strictly resumes the same provider thread, and retries a safe failed turn once. Account-selecting launchers can then continue the conversation with available quota. Other providers, contracts, clients, and UI behavior do not change.

What Changed

• Retain each in-flight Codex turn's mapped input and launch settings in the adapter.
• Detect the structured usageLimitExceeded completion emitted by Codex App Server.
• Restart the configured executable and require the original provider thread. Never fall back to a fresh thread during recovery.
• Read the resumed thread. Roll back only a failed user-only turn, then retry the exact input once.
• Preserve normal event forwarding, including the original usage-limit failure.
• Document automatic Codex recovery for account-selecting executables.

Production Behavior And Risk

The recovery path runs only for the first usage-limit failure. It refuses to retry after assistant output, tool requests, commands, hooks, diffs, plans, collaboration activity, or queued turns. It verifies the resumed provider thread and requested model before retrying.

Manual session starts, stops, and interrupts cancel an in-flight recovery. A recovery process cannot overwrite a newer user-started session. A second usage-limit failure remains visible and does not launch a third process.

The worst case is a configured executable that restarts but cannot resume the original provider thread. T3 Code closes that recovery session and exposes the existing failure. It never creates a disconnected conversation. T3 Code adds no migrations or new persistent state. Its only provider mutation is rolling back the failed user-only turn before retrying it.

Reviewer Focus

• Verify event ordering when turn/completed arrives before the turn/start response.
• Verify that the side-effect guard is fail-closed for provider activity while allowing Codex's user-message lifecycle events.
• Verify cancellation when manual session replacement races automatic recovery.

Test Plan

• vp test run apps/server/src/provider/Layers/CodexAdapter.test.ts apps/server/src/provider/Layers/CodexSessionRuntime.test.ts — 57 tests passed.
• Targeted vp lint for the four changed TypeScript files.
• vp run --filter t3 typecheck — passed with existing suggestions in unrelated files.
• vp fmt --check for all five changed files.

Checklist

• This PR is small and focused
• I explained what changed and why

Built with GPT-5.6 Sol in Codex.

---

Note

Medium Risk
Changes Codex session lifecycle (process restart, thread resume, rollback, and async recovery) on a critical provider path, but scope is narrow with fail-closed guards and extensive tests.

Overview
Codex now handles usageLimitExceeded turn failures by giving account-selecting launchers one automatic retry without leaving the conversation.

When a turn fails with that error and produced no assistant/tool activity (only user-message lifecycle events are allowed), the adapter restarts the configured Codex process, opens the session with requireResume so thread/resume cannot fall back to a fresh thread, verifies the same provider thread and model, optionally rolls back a user-only failed turn, then re-sends the exact prompt once. The original failure still flows through the event stream; a second limit hit is not retried again.

Guards and races: pending-turn tracking handles turn/completed arriving before turn/start; unsafeTurnIds blocks retry after any real provider output (including items only on completion); queued turns, manual startSession/stopSession/interruptTurn, and in-flight recovery use recovery tokens so recovery cannot replace a newer user session. sendTurn is rejected while recovery is active.

Tests cover happy path, race with manual session start, no-retry with activity, and strict resume behavior. User docs describe when auto-recovery runs and when it stops.

^{Reviewed by Cursor Bugbot for commit 92a148c. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add automatic recovery from usage-limit failures in CodexAdapter

• When a Codex turn fails with usageLimitExceeded and no provider activity has occurred in that turn, the adapter automatically restarts the runtime, resumes the same thread, and retries the turn exactly once.
• A new requireResume flag in CodexSessionRuntimeOptions prevents fallback to a fresh thread when strict resume is required; recoverable resume errors are surfaced instead of silently starting over.
• sendTurn is blocked while recovery is in progress or a recovery token is active; tokens are cleared on explicit starts, interrupts, and stops.
• New tests in CodexAdapter.test.ts verify recovery semantics using observable TestQueue utilities for deterministic event ordering.
• Risk: Any provider activity (non-userMessage items, tool calls, hooks, diffs) disables retry for that turn, aborting recovery silently.

📊 Macroscope summarized 92a148c. 3 files reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5024b059-11d5-4ed6-974c-a7656c1888cb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

🟠 HighLayers/CodexAdapter.ts:1965

A stale usage-limit recovery can stop a newer manual session and leave the thread with no active session. recoverUsageLimitTurn calls startSessionInternal before validating that input.token is still current; if startSession has already deleted the token and installed a replacement, the recovery replaces and then stops that replacement, and finally closes its own runtime. Validate the token before replacing the registered session and atomically guard the replacement.

🤖 Copy this AI Prompt to have your agent fix this:

In file @apps/server/src/provider/Layers/CodexAdapter.ts around line 1965:
A stale usage-limit recovery can stop a newer manual session and leave the thread with no active session. `recoverUsageLimitTurn` calls `startSessionInternal` before validating that `input.token` is still current; if `startSession` has already deleted the token and installed a replacement, the recovery replaces and then stops that replacement, and finally closes its own runtime. Validate the token before replacing the registered session and atomically guard the replacement.

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 92a148c. Configure here.}

[cursor]

Aborted recovery leaves session open

Medium Severity

When post-resume readThread finds provider activity on the failed turn, recoverUsageLimitTurn returns without calling stopSessionInternal. By then startSessionInternal has already stopped the original session and registered the recovery process. Sibling abort paths for wrong thread or model do close that session. An account-selecting executable can therefore leave the thread on a replaced process after refusing to retry, instead of closing the recovery session and surfacing the original usage-limit failure.

Additional Locations (1)

• apps/server/src/provider/Layers/CodexAdapter.ts#L1995-L2007

^{Reviewed by Cursor Bugbot for commit 92a148c. Configure here.}

[macroscopeapp]

Approvability

Verdict: Needs human review

1 blocking correctness issue found. This PR introduces a new automatic recovery feature for usage-limit failures with significant state management complexity (recovery tokens, pending turn tracking, deferred completions). A High-severity finding identifies a potential race condition where stale recovery can interfere with newer sessions.

^{You can customize Macroscope's approvability policy. Learn more.}

[estensen]

Decision

Codex App Server keeps one account per process, so a T3 thread stops when that account runs out. This change restarts the configured executable, strictly resumes the same provider thread, and retries a safe failed turn once. Account-selecting launchers can then continue the conversation with available quota. Other providers, contracts, clients, and UI behavior do not change.

What Changed

• Retain each in-flight Codex turn's mapped input and launch settings in the adapter.
• Detect the structured usageLimitExceeded completion emitted by Codex App Server.
• Restart the configured executable and require the original provider thread. Never fall back to a fresh thread during recovery.
• Read the resumed thread. Roll back only a failed user-only turn, then retry the exact input once.
• Preserve normal event forwarding, including the original usage-limit failure.
• Document automatic Codex recovery for account-selecting executables.

Production Behavior And Risk

The recovery path runs only for the first usage-limit failure. It refuses to retry after assistant output, tool requests, commands, hooks, diffs, plans, collaboration activity, or queued turns. It verifies the resumed provider thread and requested model before retrying.

Manual session starts, stops, and interrupts cancel an in-flight recovery. A recovery process cannot overwrite a newer user-started session. A second usage-limit failure remains visible and does not launch a third process.

The worst case is a configured executable that restarts but cannot resume the original provider thread. T3 Code closes that recovery session and exposes the existing failure. It never creates a disconnected conversation. T3 Code adds no migrations or new persistent state. Its only provider mutation is rolling back the failed user-only turn before retrying it.

Reviewer Focus

• Verify event ordering when turn/completed arrives before the turn/start response.
• Verify that the side-effect guard is fail-closed for provider activity while allowing Codex's user-message lifecycle events.
• Verify cancellation when manual session replacement races automatic recovery.

Test Plan

• vp test run apps/server/src/provider/Layers/CodexAdapter.test.ts apps/server/src/provider/Layers/CodexSessionRuntime.test.ts — 57 tests passed.
• Targeted vp lint for the four changed TypeScript files.
• vp run --filter t3 typecheck — passed with existing suggestions in unrelated files.
• vp fmt --check for all five changed files.

Checklist

• This PR is small and focused
• I explained what changed and why

Built with GPT-5.6 Sol in Codex.

---

Note

Medium Risk
Changes Codex session lifecycle (process restart, thread resume, rollback, and async recovery) on a critical provider path, but scope is narrow with fail-closed guards and extensive tests.

Overview
Codex now handles usageLimitExceeded turn failures by giving account-selecting launchers one automatic retry without leaving the conversation.

When a turn fails with that error and produced no assistant/tool activity (only user-message lifecycle events are allowed), the adapter restarts the configured Codex process, opens the session with requireResume so thread/resume cannot fall back to a fresh thread, verifies the same provider thread and model, optionally rolls back a user-only failed turn, then re-sends the exact prompt once. The original failure still flows through the event stream; a second limit hit is not retried again.

Guards and races: pending-turn tracking handles turn/completed arriving before turn/start; unsafeTurnIds blocks retry after any real provider output (including items only on completion); queued turns, manual startSession/stopSession/interruptTurn, and in-flight recovery use recovery tokens so recovery cannot replace a newer user session. sendTurn is rejected while recovery is active.

Tests cover happy path, race with manual session start, no-retry with activity, and strict resume behavior. User docs describe when auto-recovery runs and when it stops.

^{Reviewed by Cursor Bugbot for commit 92a148c. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add automatic recovery from usage-limit failures in CodexAdapter

• When a Codex turn fails with usageLimitExceeded and no provider activity has occurred in that turn, the adapter automatically restarts the runtime, resumes the same thread, and retries the turn exactly once.
• A new requireResume flag in CodexSessionRuntimeOptions prevents fallback to a fresh thread when strict resume is required; recoverable resume errors are surfaced instead of silently starting over.
• sendTurn is blocked while recovery is in progress or a recovery token is active; tokens are cleared on explicit starts, interrupts, and stops.
• New tests in CodexAdapter.test.ts verify recovery semantics using observable TestQueue utilities for deterministic event ordering.
• Risk: Any provider activity (non-userMessage items, tool calls, hooks, diffs) disables retry for that turn, aborting recovery silently.

📊 Macroscope summarized 92a148c. 3 files reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5024b059-11d5-4ed6-974c-a7656c1888cb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

🟠 HighLayers/CodexAdapter.ts:1965

A stale usage-limit recovery can stop a newer manual session and leave the thread with no active session. recoverUsageLimitTurn calls startSessionInternal before validating that input.token is still current; if startSession has already deleted the token and installed a replacement, the recovery replaces and then stops that replacement, and finally closes its own runtime. Validate the token before replacing the registered session and atomically guard the replacement.

🤖 Copy this AI Prompt to have your agent fix this:

In file @apps/server/src/provider/Layers/CodexAdapter.ts around line 1965:
A stale usage-limit recovery can stop a newer manual session and leave the thread with no active session. `recoverUsageLimitTurn` calls `startSessionInternal` before validating that `input.token` is still current; if `startSession` has already deleted the token and installed a replacement, the recovery replaces and then stops that replacement, and finally closes its own runtime. Validate the token before replacing the registered session and atomically guard the replacement.

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 92a148c. Configure here.}

[cursor]

Aborted recovery leaves session open

Medium Severity

When post-resume readThread finds provider activity on the failed turn, recoverUsageLimitTurn returns without calling stopSessionInternal. By then startSessionInternal has already stopped the original session and registered the recovery process. Sibling abort paths for wrong thread or model do close that session. An account-selecting executable can therefore leave the thread on a replaced process after refusing to retry, instead of closing the recovery session and surfacing the original usage-limit failure.

Additional Locations (1)

• apps/server/src/provider/Layers/CodexAdapter.ts#L1995-L2007

^{Reviewed by Cursor Bugbot for commit 92a148c. Configure here.}

[macroscopeapp]

Approvability

Verdict: Needs human review

1 blocking correctness issue found. This PR introduces a new automatic recovery feature for usage-limit failures with significant state management complexity (recovery tokens, pending turn tracking, deferred completions). A High-severity finding identifies a potential race condition where stale recovery can interfere with newer sessions.

^{You can customize Macroscope's approvability policy. Learn more.}