fix(cloud): back off relay connector restarts

[nateEc]

Fixes#8760.

An immediately failing relay child was restarted in a tight loop, creating processes and trace records until the server exhausted its heap.

Wait five seconds before retrying a connector exit. The wait intentionally releases the configuration lock so users can disable or change T3 Connect immediately.

Verification: vp fmt --check, focused managed endpoint runtime tests (8 passing), and server typecheck (existing suggestions only).

Model and harness: GPT-5 Codex via Codex CLI.

---

Note

Medium Risk
Changes connector supervision and config locking timing for cloud relay processes; mis-timed restarts or lock handling could affect T3 Connect availability or responsiveness to config changes.

Overview
Fixes a tight restart loop when the Cloudflare relay child exits immediately, which could spawn processes and trace records until the server ran out of heap.

superviseConnector now waits 5 seconds (RELAY_CONNECTOR_RESTART_DELAY) before calling reconcile again. The delay runs outside the reconcile semaphore so applyConfig can still disable or change T3 Connect without blocking behind a failing child. After the wait, it re-reads desiredConfig and only restarts if the same tunnel config is still wanted.

The supervisor test was renamed and updated to use TestClock: it asserts no second spawn for the first 4 seconds after exit, then restart after the full 5 second delay.

^{Reviewed by Cursor Bugbot for commit 14468ba. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Delay relay connector restarts by 5 seconds in ManagedEndpointRuntime

Introduces RELAY_CONNECTOR_RESTART_DELAY (5 seconds) in ManagedEndpointRuntime.ts. After a managed Cloudflare connector exits, the runtime now sleeps before attempting a restart and releases the reconcile semaphore during the wait, so other operations are not blocked. After the delay, it re-validates that the desired config still matches the exited connector before calling reconcgeConfig; if the config changed, the restart is skipped. Updates the supervision test to use TestClock and assert the 5-second wait with no immediate respawn.

• Behavioral Change: restarts of exited relay connectors are now delayed by 5 seconds and gated on a second config validation; callers expecting immediate respawn will observe the delay. The reconcile semaphore is released during the delay, opening a window where config changes can cancel the restart.

^{Macroscope summarized 14468ba.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 44501e4f-4a56-49f1-9e3c-2cb4af9c8bf3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.

---

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — Managed Cloudflare connector restarts now use a fixed five-second delay on an existing production path, with configuration changes able to cancel or supersede the retry. The change is focused and test-covered, but the new default recovery policy warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

[derektrimm]

Heads-up on overlap: #8788 (opened Aug 30) addresses the same issue (#8760) in the same supervisor. It uses exponential backoff (1s doubling to a 60s cap) with an immediate first restart, resets after 30 seconds of stable uptime or an explicit config change, and keeps the delay outside the reconcile semaphore so a user disabling or changing T3 Connect is never blocked behind it. It ships TestClock tests for the doubling, the stable-uptime reset, and config-change preemption, and Macroscope approved it at head. Flagging so the maintainers can compare the two directly rather than review them independently.

[t3dotgg]

Note

🤖 GPT-5.6 Sol responding on behalf of Theo

We're closing this PR as we clean up the T3 Code backlog. Thank you for taking the time to put this together.

We are keeping the relay restart fix in the open PR #8788 instead of maintaining two implementations. It covers the same failing-connector loop as this five-second delay, with capped exponential delays and a reset after stable uptime or a configuration change. Both let configuration changes proceed during the wait. The fix is not on main yet.

If you believe we closed this in error, please reopen the PR and leave a comment explaining what we missed.

[nateEc]

Fixes#8760.

An immediately failing relay child was restarted in a tight loop, creating processes and trace records until the server exhausted its heap.

Wait five seconds before retrying a connector exit. The wait intentionally releases the configuration lock so users can disable or change T3 Connect immediately.

Verification: vp fmt --check, focused managed endpoint runtime tests (8 passing), and server typecheck (existing suggestions only).

Model and harness: GPT-5 Codex via Codex CLI.

---

Note

Medium Risk
Changes connector supervision and config locking timing for cloud relay processes; mis-timed restarts or lock handling could affect T3 Connect availability or responsiveness to config changes.

Overview
Fixes a tight restart loop when the Cloudflare relay child exits immediately, which could spawn processes and trace records until the server ran out of heap.

superviseConnector now waits 5 seconds (RELAY_CONNECTOR_RESTART_DELAY) before calling reconcile again. The delay runs outside the reconcile semaphore so applyConfig can still disable or change T3 Connect without blocking behind a failing child. After the wait, it re-reads desiredConfig and only restarts if the same tunnel config is still wanted.

The supervisor test was renamed and updated to use TestClock: it asserts no second spawn for the first 4 seconds after exit, then restart after the full 5 second delay.

^{Reviewed by Cursor Bugbot for commit 14468ba. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Delay relay connector restarts by 5 seconds in ManagedEndpointRuntime

Introduces RELAY_CONNECTOR_RESTART_DELAY (5 seconds) in ManagedEndpointRuntime.ts. After a managed Cloudflare connector exits, the runtime now sleeps before attempting a restart and releases the reconcile semaphore during the wait, so other operations are not blocked. After the delay, it re-validates that the desired config still matches the exited connector before calling reconcgeConfig; if the config changed, the restart is skipped. Updates the supervision test to use TestClock and assert the 5-second wait with no immediate respawn.

• Behavioral Change: restarts of exited relay connectors are now delayed by 5 seconds and gated on a second config validation; callers expecting immediate respawn will observe the delay. The reconcile semaphore is released during the delay, opening a window where config changes can cancel the restart.

^{Macroscope summarized 14468ba.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 44501e4f-4a56-49f1-9e3c-2cb4af9c8bf3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.

---

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — Managed Cloudflare connector restarts now use a fixed five-second delay on an existing production path, with configuration changes able to cancel or supersede the retry. The change is focused and test-covered, but the new default recovery policy warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

[derektrimm]

Heads-up on overlap: #8788 (opened Aug 30) addresses the same issue (#8760) in the same supervisor. It uses exponential backoff (1s doubling to a 60s cap) with an immediate first restart, resets after 30 seconds of stable uptime or an explicit config change, and keeps the delay outside the reconcile semaphore so a user disabling or changing T3 Connect is never blocked behind it. It ships TestClock tests for the doubling, the stable-uptime reset, and config-change preemption, and Macroscope approved it at head. Flagging so the maintainers can compare the two directly rather than review them independently.

[t3dotgg]

Note

🤖 GPT-5.6 Sol responding on behalf of Theo

We're closing this PR as we clean up the T3 Code backlog. Thank you for taking the time to put this together.

We are keeping the relay restart fix in the open PR #8788 instead of maintaining two implementations. It covers the same failing-connector loop as this five-second delay, with capped exponential delays and a reset after stable uptime or a configuration change. Both let configuration changes proceed during the wait. The fix is not on main yet.

If you believe we closed this in error, please reopen the PR and leave a comment explaining what we missed.

[nateEc]

Fixes#8760.

An immediately failing relay child was restarted in a tight loop, creating processes and trace records until the server exhausted its heap.

Wait five seconds before retrying a connector exit. The wait intentionally releases the configuration lock so users can disable or change T3 Connect immediately.

Verification: vp fmt --check, focused managed endpoint runtime tests (8 passing), and server typecheck (existing suggestions only).

Model and harness: GPT-5 Codex via Codex CLI.

---

Note

Medium Risk
Changes connector supervision and config locking timing for cloud relay processes; mis-timed restarts or lock handling could affect T3 Connect availability or responsiveness to config changes.

Overview
Fixes a tight restart loop when the Cloudflare relay child exits immediately, which could spawn processes and trace records until the server ran out of heap.

superviseConnector now waits 5 seconds (RELAY_CONNECTOR_RESTART_DELAY) before calling reconcile again. The delay runs outside the reconcile semaphore so applyConfig can still disable or change T3 Connect without blocking behind a failing child. After the wait, it re-reads desiredConfig and only restarts if the same tunnel config is still wanted.

The supervisor test was renamed and updated to use TestClock: it asserts no second spawn for the first 4 seconds after exit, then restart after the full 5 second delay.

^{Reviewed by Cursor Bugbot for commit 14468ba. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Delay relay connector restarts by 5 seconds in ManagedEndpointRuntime

Introduces RELAY_CONNECTOR_RESTART_DELAY (5 seconds) in ManagedEndpointRuntime.ts. After a managed Cloudflare connector exits, the runtime now sleeps before attempting a restart and releases the reconcile semaphore during the wait, so other operations are not blocked. After the delay, it re-validates that the desired config still matches the exited connector before calling reconcgeConfig; if the config changed, the restart is skipped. Updates the supervision test to use TestClock and assert the 5-second wait with no immediate respawn.

• Behavioral Change: restarts of exited relay connectors are now delayed by 5 seconds and gated on a second config validation; callers expecting immediate respawn will observe the delay. The reconcile semaphore is released during the delay, opening a window where config changes can cancel the restart.

^{Macroscope summarized 14468ba.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 44501e4f-4a56-49f1-9e3c-2cb4af9c8bf3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.

---

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — Managed Cloudflare connector restarts now use a fixed five-second delay on an existing production path, with configuration changes able to cancel or supersede the retry. The change is focused and test-covered, but the new default recovery policy warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

[derektrimm]

Heads-up on overlap: #8788 (opened Aug 30) addresses the same issue (#8760) in the same supervisor. It uses exponential backoff (1s doubling to a 60s cap) with an immediate first restart, resets after 30 seconds of stable uptime or an explicit config change, and keeps the delay outside the reconcile semaphore so a user disabling or changing T3 Connect is never blocked behind it. It ships TestClock tests for the doubling, the stable-uptime reset, and config-change preemption, and Macroscope approved it at head. Flagging so the maintainers can compare the two directly rather than review them independently.

[t3dotgg]

Note

🤖 GPT-5.6 Sol responding on behalf of Theo

We're closing this PR as we clean up the T3 Code backlog. Thank you for taking the time to put this together.

We are keeping the relay restart fix in the open PR #8788 instead of maintaining two implementations. It covers the same failing-connector loop as this five-second delay, with capped exponential delays and a reset after stable uptime or a configuration change. Both let configuration changes proceed during the wait. The fix is not on main yet.

If you believe we closed this in error, please reopen the PR and leave a comment explaining what we missed.

[nateEc]

Fixes#8760.

An immediately failing relay child was restarted in a tight loop, creating processes and trace records until the server exhausted its heap.

Wait five seconds before retrying a connector exit. The wait intentionally releases the configuration lock so users can disable or change T3 Connect immediately.

Verification: vp fmt --check, focused managed endpoint runtime tests (8 passing), and server typecheck (existing suggestions only).

Model and harness: GPT-5 Codex via Codex CLI.

---

Note

Medium Risk
Changes connector supervision and config locking timing for cloud relay processes; mis-timed restarts or lock handling could affect T3 Connect availability or responsiveness to config changes.

Overview
Fixes a tight restart loop when the Cloudflare relay child exits immediately, which could spawn processes and trace records until the server ran out of heap.

superviseConnector now waits 5 seconds (RELAY_CONNECTOR_RESTART_DELAY) before calling reconcile again. The delay runs outside the reconcile semaphore so applyConfig can still disable or change T3 Connect without blocking behind a failing child. After the wait, it re-reads desiredConfig and only restarts if the same tunnel config is still wanted.

The supervisor test was renamed and updated to use TestClock: it asserts no second spawn for the first 4 seconds after exit, then restart after the full 5 second delay.

^{Reviewed by Cursor Bugbot for commit 14468ba. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Delay relay connector restarts by 5 seconds in ManagedEndpointRuntime

Introduces RELAY_CONNECTOR_RESTART_DELAY (5 seconds) in ManagedEndpointRuntime.ts. After a managed Cloudflare connector exits, the runtime now sleeps before attempting a restart and releases the reconcile semaphore during the wait, so other operations are not blocked. After the delay, it re-validates that the desired config still matches the exited connector before calling reconcgeConfig; if the config changed, the restart is skipped. Updates the supervision test to use TestClock and assert the 5-second wait with no immediate respawn.

• Behavioral Change: restarts of exited relay connectors are now delayed by 5 seconds and gated on a second config validation; callers expecting immediate respawn will observe the delay. The reconcile semaphore is released during the delay, opening a window where config changes can cancel the restart.

^{Macroscope summarized 14468ba.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 44501e4f-4a56-49f1-9e3c-2cb4af9c8bf3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.

---

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — Managed Cloudflare connector restarts now use a fixed five-second delay on an existing production path, with configuration changes able to cancel or supersede the retry. The change is focused and test-covered, but the new default recovery policy warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

[derektrimm]

Heads-up on overlap: #8788 (opened Aug 30) addresses the same issue (#8760) in the same supervisor. It uses exponential backoff (1s doubling to a 60s cap) with an immediate first restart, resets after 30 seconds of stable uptime or an explicit config change, and keeps the delay outside the reconcile semaphore so a user disabling or changing T3 Connect is never blocked behind it. It ships TestClock tests for the doubling, the stable-uptime reset, and config-change preemption, and Macroscope approved it at head. Flagging so the maintainers can compare the two directly rather than review them independently.

[t3dotgg]

Note

🤖 GPT-5.6 Sol responding on behalf of Theo

We're closing this PR as we clean up the T3 Code backlog. Thank you for taking the time to put this together.

We are keeping the relay restart fix in the open PR #8788 instead of maintaining two implementations. It covers the same failing-connector loop as this five-second delay, with capped exponential delays and a reset after stable uptime or a configuration change. Both let configuration changes proceed during the wait. The fix is not on main yet.

If you believe we closed this in error, please reopen the PR and leave a comment explaining what we missed.

[nateEc]

Fixes#8760.

An immediately failing relay child was restarted in a tight loop, creating processes and trace records until the server exhausted its heap.

Wait five seconds before retrying a connector exit. The wait intentionally releases the configuration lock so users can disable or change T3 Connect immediately.

Verification: vp fmt --check, focused managed endpoint runtime tests (8 passing), and server typecheck (existing suggestions only).

Model and harness: GPT-5 Codex via Codex CLI.

---

Note

Medium Risk
Changes connector supervision and config locking timing for cloud relay processes; mis-timed restarts or lock handling could affect T3 Connect availability or responsiveness to config changes.

Overview
Fixes a tight restart loop when the Cloudflare relay child exits immediately, which could spawn processes and trace records until the server ran out of heap.

superviseConnector now waits 5 seconds (RELAY_CONNECTOR_RESTART_DELAY) before calling reconcile again. The delay runs outside the reconcile semaphore so applyConfig can still disable or change T3 Connect without blocking behind a failing child. After the wait, it re-reads desiredConfig and only restarts if the same tunnel config is still wanted.

The supervisor test was renamed and updated to use TestClock: it asserts no second spawn for the first 4 seconds after exit, then restart after the full 5 second delay.

^{Reviewed by Cursor Bugbot for commit 14468ba. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Delay relay connector restarts by 5 seconds in ManagedEndpointRuntime

Introduces RELAY_CONNECTOR_RESTART_DELAY (5 seconds) in ManagedEndpointRuntime.ts. After a managed Cloudflare connector exits, the runtime now sleeps before attempting a restart and releases the reconcile semaphore during the wait, so other operations are not blocked. After the delay, it re-validates that the desired config still matches the exited connector before calling reconcgeConfig; if the config changed, the restart is skipped. Updates the supervision test to use TestClock and assert the 5-second wait with no immediate respawn.

• Behavioral Change: restarts of exited relay connectors are now delayed by 5 seconds and gated on a second config validation; callers expecting immediate respawn will observe the delay. The reconcile semaphore is released during the delay, opening a window where config changes can cancel the restart.

^{Macroscope summarized 14468ba.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 44501e4f-4a56-49f1-9e3c-2cb4af9c8bf3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.

---

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — Managed Cloudflare connector restarts now use a fixed five-second delay on an existing production path, with configuration changes able to cancel or supersede the retry. The change is focused and test-covered, but the new default recovery policy warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

[derektrimm]

Heads-up on overlap: #8788 (opened Aug 30) addresses the same issue (#8760) in the same supervisor. It uses exponential backoff (1s doubling to a 60s cap) with an immediate first restart, resets after 30 seconds of stable uptime or an explicit config change, and keeps the delay outside the reconcile semaphore so a user disabling or changing T3 Connect is never blocked behind it. It ships TestClock tests for the doubling, the stable-uptime reset, and config-change preemption, and Macroscope approved it at head. Flagging so the maintainers can compare the two directly rather than review them independently.

[t3dotgg]

Note

🤖 GPT-5.6 Sol responding on behalf of Theo

We're closing this PR as we clean up the T3 Code backlog. Thank you for taking the time to put this together.

We are keeping the relay restart fix in the open PR #8788 instead of maintaining two implementations. It covers the same failing-connector loop as this five-second delay, with capped exponential delays and a reset after stable uptime or a configuration change. Both let configuration changes proceed during the wait. The fix is not on main yet.

If you believe we closed this in error, please reopen the PR and leave a comment explaining what we missed.

[nateEc]

Fixes#8760.

An immediately failing relay child was restarted in a tight loop, creating processes and trace records until the server exhausted its heap.

Wait five seconds before retrying a connector exit. The wait intentionally releases the configuration lock so users can disable or change T3 Connect immediately.

Verification: vp fmt --check, focused managed endpoint runtime tests (8 passing), and server typecheck (existing suggestions only).

Model and harness: GPT-5 Codex via Codex CLI.

---

Note

Medium Risk
Changes connector supervision and config locking timing for cloud relay processes; mis-timed restarts or lock handling could affect T3 Connect availability or responsiveness to config changes.

Overview
Fixes a tight restart loop when the Cloudflare relay child exits immediately, which could spawn processes and trace records until the server ran out of heap.

superviseConnector now waits 5 seconds (RELAY_CONNECTOR_RESTART_DELAY) before calling reconcile again. The delay runs outside the reconcile semaphore so applyConfig can still disable or change T3 Connect without blocking behind a failing child. After the wait, it re-reads desiredConfig and only restarts if the same tunnel config is still wanted.

The supervisor test was renamed and updated to use TestClock: it asserts no second spawn for the first 4 seconds after exit, then restart after the full 5 second delay.

^{Reviewed by Cursor Bugbot for commit 14468ba. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Delay relay connector restarts by 5 seconds in ManagedEndpointRuntime

Introduces RELAY_CONNECTOR_RESTART_DELAY (5 seconds) in ManagedEndpointRuntime.ts. After a managed Cloudflare connector exits, the runtime now sleeps before attempting a restart and releases the reconcile semaphore during the wait, so other operations are not blocked. After the delay, it re-validates that the desired config still matches the exited connector before calling reconcgeConfig; if the config changed, the restart is skipped. Updates the supervision test to use TestClock and assert the 5-second wait with no immediate respawn.

• Behavioral Change: restarts of exited relay connectors are now delayed by 5 seconds and gated on a second config validation; callers expecting immediate respawn will observe the delay. The reconcile semaphore is released during the delay, opening a window where config changes can cancel the restart.

^{Macroscope summarized 14468ba.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 44501e4f-4a56-49f1-9e3c-2cb4af9c8bf3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.

---

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — Managed Cloudflare connector restarts now use a fixed five-second delay on an existing production path, with configuration changes able to cancel or supersede the retry. The change is focused and test-covered, but the new default recovery policy warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

[derektrimm]

Heads-up on overlap: #8788 (opened Aug 30) addresses the same issue (#8760) in the same supervisor. It uses exponential backoff (1s doubling to a 60s cap) with an immediate first restart, resets after 30 seconds of stable uptime or an explicit config change, and keeps the delay outside the reconcile semaphore so a user disabling or changing T3 Connect is never blocked behind it. It ships TestClock tests for the doubling, the stable-uptime reset, and config-change preemption, and Macroscope approved it at head. Flagging so the maintainers can compare the two directly rather than review them independently.

[t3dotgg]

Note

🤖 GPT-5.6 Sol responding on behalf of Theo

We're closing this PR as we clean up the T3 Code backlog. Thank you for taking the time to put this together.

We are keeping the relay restart fix in the open PR #8788 instead of maintaining two implementations. It covers the same failing-connector loop as this five-second delay, with capped exponential delays and a reset after stable uptime or a configuration change. Both let configuration changes proceed during the wait. The fix is not on main yet.

If you believe we closed this in error, please reopen the PR and leave a comment explaining what we missed.

[nateEc]

Fixes#8760.

An immediately failing relay child was restarted in a tight loop, creating processes and trace records until the server exhausted its heap.

Wait five seconds before retrying a connector exit. The wait intentionally releases the configuration lock so users can disable or change T3 Connect immediately.

Verification: vp fmt --check, focused managed endpoint runtime tests (8 passing), and server typecheck (existing suggestions only).

Model and harness: GPT-5 Codex via Codex CLI.

---

Note

Medium Risk
Changes connector supervision and config locking timing for cloud relay processes; mis-timed restarts or lock handling could affect T3 Connect availability or responsiveness to config changes.

Overview
Fixes a tight restart loop when the Cloudflare relay child exits immediately, which could spawn processes and trace records until the server ran out of heap.

superviseConnector now waits 5 seconds (RELAY_CONNECTOR_RESTART_DELAY) before calling reconcile again. The delay runs outside the reconcile semaphore so applyConfig can still disable or change T3 Connect without blocking behind a failing child. After the wait, it re-reads desiredConfig and only restarts if the same tunnel config is still wanted.

The supervisor test was renamed and updated to use TestClock: it asserts no second spawn for the first 4 seconds after exit, then restart after the full 5 second delay.

^{Reviewed by Cursor Bugbot for commit 14468ba. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Delay relay connector restarts by 5 seconds in ManagedEndpointRuntime

Introduces RELAY_CONNECTOR_RESTART_DELAY (5 seconds) in ManagedEndpointRuntime.ts. After a managed Cloudflare connector exits, the runtime now sleeps before attempting a restart and releases the reconcile semaphore during the wait, so other operations are not blocked. After the delay, it re-validates that the desired config still matches the exited connector before calling reconcgeConfig; if the config changed, the restart is skipped. Updates the supervision test to use TestClock and assert the 5-second wait with no immediate respawn.

• Behavioral Change: restarts of exited relay connectors are now delayed by 5 seconds and gated on a second config validation; callers expecting immediate respawn will observe the delay. The reconcile semaphore is released during the delay, opening a window where config changes can cancel the restart.

^{Macroscope summarized 14468ba.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 44501e4f-4a56-49f1-9e3c-2cb4af9c8bf3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.

---

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — Managed Cloudflare connector restarts now use a fixed five-second delay on an existing production path, with configuration changes able to cancel or supersede the retry. The change is focused and test-covered, but the new default recovery policy warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

[derektrimm]

Heads-up on overlap: #8788 (opened Aug 30) addresses the same issue (#8760) in the same supervisor. It uses exponential backoff (1s doubling to a 60s cap) with an immediate first restart, resets after 30 seconds of stable uptime or an explicit config change, and keeps the delay outside the reconcile semaphore so a user disabling or changing T3 Connect is never blocked behind it. It ships TestClock tests for the doubling, the stable-uptime reset, and config-change preemption, and Macroscope approved it at head. Flagging so the maintainers can compare the two directly rather than review them independently.

[t3dotgg]

Note

🤖 GPT-5.6 Sol responding on behalf of Theo

We're closing this PR as we clean up the T3 Code backlog. Thank you for taking the time to put this together.

We are keeping the relay restart fix in the open PR #8788 instead of maintaining two implementations. It covers the same failing-connector loop as this five-second delay, with capped exponential delays and a reset after stable uptime or a configuration change. Both let configuration changes proceed during the wait. The fix is not on main yet.

If you believe we closed this in error, please reopen the PR and leave a comment explaining what we missed.

[nateEc]

Fixes#8760.

An immediately failing relay child was restarted in a tight loop, creating processes and trace records until the server exhausted its heap.

Wait five seconds before retrying a connector exit. The wait intentionally releases the configuration lock so users can disable or change T3 Connect immediately.

Verification: vp fmt --check, focused managed endpoint runtime tests (8 passing), and server typecheck (existing suggestions only).

Model and harness: GPT-5 Codex via Codex CLI.

---

Note

Medium Risk
Changes connector supervision and config locking timing for cloud relay processes; mis-timed restarts or lock handling could affect T3 Connect availability or responsiveness to config changes.

Overview
Fixes a tight restart loop when the Cloudflare relay child exits immediately, which could spawn processes and trace records until the server ran out of heap.

superviseConnector now waits 5 seconds (RELAY_CONNECTOR_RESTART_DELAY) before calling reconcile again. The delay runs outside the reconcile semaphore so applyConfig can still disable or change T3 Connect without blocking behind a failing child. After the wait, it re-reads desiredConfig and only restarts if the same tunnel config is still wanted.

The supervisor test was renamed and updated to use TestClock: it asserts no second spawn for the first 4 seconds after exit, then restart after the full 5 second delay.

^{Reviewed by Cursor Bugbot for commit 14468ba. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Delay relay connector restarts by 5 seconds in ManagedEndpointRuntime

Introduces RELAY_CONNECTOR_RESTART_DELAY (5 seconds) in ManagedEndpointRuntime.ts. After a managed Cloudflare connector exits, the runtime now sleeps before attempting a restart and releases the reconcile semaphore during the wait, so other operations are not blocked. After the delay, it re-validates that the desired config still matches the exited connector before calling reconcgeConfig; if the config changed, the restart is skipped. Updates the supervision test to use TestClock and assert the 5-second wait with no immediate respawn.

• Behavioral Change: restarts of exited relay connectors are now delayed by 5 seconds and gated on a second config validation; callers expecting immediate respawn will observe the delay. The reconcile semaphore is released during the delay, opening a window where config changes can cancel the restart.

^{Macroscope summarized 14468ba.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 44501e4f-4a56-49f1-9e3c-2cb4af9c8bf3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.

---

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — Managed Cloudflare connector restarts now use a fixed five-second delay on an existing production path, with configuration changes able to cancel or supersede the retry. The change is focused and test-covered, but the new default recovery policy warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

[derektrimm]

Heads-up on overlap: #8788 (opened Aug 30) addresses the same issue (#8760) in the same supervisor. It uses exponential backoff (1s doubling to a 60s cap) with an immediate first restart, resets after 30 seconds of stable uptime or an explicit config change, and keeps the delay outside the reconcile semaphore so a user disabling or changing T3 Connect is never blocked behind it. It ships TestClock tests for the doubling, the stable-uptime reset, and config-change preemption, and Macroscope approved it at head. Flagging so the maintainers can compare the two directly rather than review them independently.

[t3dotgg]

Note

🤖 GPT-5.6 Sol responding on behalf of Theo

We're closing this PR as we clean up the T3 Code backlog. Thank you for taking the time to put this together.

We are keeping the relay restart fix in the open PR #8788 instead of maintaining two implementations. It covers the same failing-connector loop as this five-second delay, with capped exponential delays and a reset after stable uptime or a configuration change. Both let configuration changes proceed during the wait. The fix is not on main yet.

If you believe we closed this in error, please reopen the PR and leave a comment explaining what we missed.