fix(server): isolate provider command lanes by thread - #7071

Open
clintebbesen wants to merge 2 commits into
pingdotgg:mainfrom
clintebbesen:fix/provider-command-cross-thread-lockout
Open

fix(server): isolate provider command lanes by thread#7071
clintebbesen wants to merge 2 commits into
pingdotgg:mainfrom
clintebbesen:fix/provider-command-cross-thread-lockout

Conversation

@clintebbesen

@clintebbesenclintebbesen commented Aug 15, 2026

Copy link
Copy Markdown

Fixes#6517.

ProviderCommandReactor used one global DrainableWorker for lifecycle, turn, response, and stop commands. A provider start/restart that never resolves therefore prevented unrelated threads from starting and left their statuses blank or stuck on Connecting.

This change extends the existing DrainableWorker owner with keyed FIFO lanes. Commands for different threads now process independently; commands for the same thread retain order. Idle lanes remove themselves after draining so the reactor does not retain a queue and fiber for every historical thread.

Focused regression coverage proves independent progress for different keys and FIFO ordering for the same key. The existing ProviderCommandReactor test file still passes (47 tests); the shared worker tests pass (2 tests). Server/shared typechecks, targeted lint, formatting, and done-check --base upstream/main --no-tests passed. Tests were run in the isolated worktree; no live T3 state was used.

Scope disposition: this PR fixes the cross-thread scheduling lockout only. It does not duplicate the still-open interrupt responsiveness work in #6531, nor fold in the separately scoped lifecycle timeout/stale-session/restart-recovery work tracked by #6560, #4944, and #4584. The Codex framing defect remains separate in #5389 and fork coordination issue clintebbesen#1.

Verification environment: Codex harness, gpt-5.6-sol.


Note

Medium Risk
Changes orchestration concurrency for provider lifecycle/turn commands; same-thread ordering is preserved but cross-thread behavior is no longer globally serialized.

Overview
Fixes cross-thread lockout where a single global provider command queue let one thread’s stuck start/restart block every other thread’s lifecycle and turn handling.

Adds makeKeyedDrainableWorker in shared DrainableWorker: one FIFO drainable worker per key, concurrent across keys, FIFO within a key, with idle lanes torn down after drain so historical threads don’t leak queues/fibers. ProviderCommandReactor now keys lanes on event.payload.threadId instead of makeDrainableWorker.

Regression test covers independent progress for different keys and FIFO for the same key.

Reviewed by Cursor Bugbot for commit 4822cc3. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Isolate provider command processing into per-thread FIFO lanes

  • Adds makeKeyedDrainableWorker to DrainableWorker.ts, which maintains a per-key makeDrainableWorker lane, creating and cleaning up lanes dynamically as work arrives and drains.
  • Updates ProviderCommandReactor.ts to key the worker by event.payload.threadId, so events on different threads are processed concurrently while preserving FIFO order within each thread.
  • Behavioral Change: previously all provider command events shared a single FIFO queue; now concurrency is possible across threads.
📊 Macroscope summarized 4822cc3. 1 file reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

@coderabbitai

coderabbitaiBot commented Aug 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c6db509b-8e09-4829-b3fc-91dfc1be273c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Aug 15, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit bf77599. Configure here.

Comment threadpackages/shared/src/DrainableWorker.ts
@macroscopeapp

macroscopeappBot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

1 blocking correctness issue found. This PR introduces significant concurrency changes by switching from a single worker queue to per-thread isolated lanes. There is also an unresolved High severity finding about a race condition in lane creation that could break the intended per-key FIFO ordering.

You can customize Macroscope's approvability policy. Learn more.

const enqueue = (item: A): Effect.Effect<void> =>
Effect.gen(function* () {
const key = keyOf(item);
const existing = entries.get(key);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Highsrc/DrainableWorker.ts:65

Concurrent first enqueues for the same key create separate workers, so their items are processed concurrently instead of in per-key FIFO order; the later entries.set also hides the first lane from drain and idle cleanup. Serialize the keyed lookup, lane creation, and initial enqueue (or reserve the key before yielding) so only one lane can be created per key.

🤖 Copy this AI Prompt to have your agent fix this:
In file @packages/shared/src/DrainableWorker.ts around line 65:
Concurrent first enqueues for the same key create separate workers, so their items are processed concurrently instead of in per-key FIFO order; the later `entries.set` also hides the first lane from `drain` and idle cleanup. Serialize the keyed lookup, lane creation, and initial enqueue (or reserve the key before yielding) so only one lane can be created per key.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M30-99 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Changing runtime mode mid-turn hangs ProviderCommandReactor; all new sessions stay Connecting or show no status

1 participant

@clintebbesen
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

fix(server): isolate provider command lanes by thread - #7071

Open
clintebbesen wants to merge 2 commits into
pingdotgg:mainfrom
clintebbesen:fix/provider-command-cross-thread-lockout
Open

fix(server): isolate provider command lanes by thread#7071
clintebbesen wants to merge 2 commits into
pingdotgg:mainfrom
clintebbesen:fix/provider-command-cross-thread-lockout

Conversation

@clintebbesen

@clintebbesenclintebbesen commented Aug 15, 2026

Copy link
Copy Markdown

Fixes#6517.

ProviderCommandReactor used one global DrainableWorker for lifecycle, turn, response, and stop commands. A provider start/restart that never resolves therefore prevented unrelated threads from starting and left their statuses blank or stuck on Connecting.

This change extends the existing DrainableWorker owner with keyed FIFO lanes. Commands for different threads now process independently; commands for the same thread retain order. Idle lanes remove themselves after draining so the reactor does not retain a queue and fiber for every historical thread.

Focused regression coverage proves independent progress for different keys and FIFO ordering for the same key. The existing ProviderCommandReactor test file still passes (47 tests); the shared worker tests pass (2 tests). Server/shared typechecks, targeted lint, formatting, and done-check --base upstream/main --no-tests passed. Tests were run in the isolated worktree; no live T3 state was used.

Scope disposition: this PR fixes the cross-thread scheduling lockout only. It does not duplicate the still-open interrupt responsiveness work in #6531, nor fold in the separately scoped lifecycle timeout/stale-session/restart-recovery work tracked by #6560, #4944, and #4584. The Codex framing defect remains separate in #5389 and fork coordination issue clintebbesen#1.

Verification environment: Codex harness, gpt-5.6-sol.


Note

Medium Risk
Changes orchestration concurrency for provider lifecycle/turn commands; same-thread ordering is preserved but cross-thread behavior is no longer globally serialized.

Overview
Fixes cross-thread lockout where a single global provider command queue let one thread’s stuck start/restart block every other thread’s lifecycle and turn handling.

Adds makeKeyedDrainableWorker in shared DrainableWorker: one FIFO drainable worker per key, concurrent across keys, FIFO within a key, with idle lanes torn down after drain so historical threads don’t leak queues/fibers. ProviderCommandReactor now keys lanes on event.payload.threadId instead of makeDrainableWorker.

Regression test covers independent progress for different keys and FIFO for the same key.

Reviewed by Cursor Bugbot for commit 4822cc3. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Isolate provider command processing into per-thread FIFO lanes

  • Adds makeKeyedDrainableWorker to DrainableWorker.ts, which maintains a per-key makeDrainableWorker lane, creating and cleaning up lanes dynamically as work arrives and drains.
  • Updates ProviderCommandReactor.ts to key the worker by event.payload.threadId, so events on different threads are processed concurrently while preserving FIFO order within each thread.
  • Behavioral Change: previously all provider command events shared a single FIFO queue; now concurrency is possible across threads.
📊 Macroscope summarized 4822cc3. 1 file reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

@coderabbitai

coderabbitaiBot commented Aug 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c6db509b-8e09-4829-b3fc-91dfc1be273c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Aug 15, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit bf77599. Configure here.

Comment threadpackages/shared/src/DrainableWorker.ts
@macroscopeapp

macroscopeappBot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

1 blocking correctness issue found. This PR introduces significant concurrency changes by switching from a single worker queue to per-thread isolated lanes. There is also an unresolved High severity finding about a race condition in lane creation that could break the intended per-key FIFO ordering.

You can customize Macroscope's approvability policy. Learn more.

const enqueue = (item: A): Effect.Effect<void> =>
Effect.gen(function* () {
const key = keyOf(item);
const existing = entries.get(key);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Highsrc/DrainableWorker.ts:65

Concurrent first enqueues for the same key create separate workers, so their items are processed concurrently instead of in per-key FIFO order; the later entries.set also hides the first lane from drain and idle cleanup. Serialize the keyed lookup, lane creation, and initial enqueue (or reserve the key before yielding) so only one lane can be created per key.

🤖 Copy this AI Prompt to have your agent fix this:
In file @packages/shared/src/DrainableWorker.ts around line 65:
Concurrent first enqueues for the same key create separate workers, so their items are processed concurrently instead of in per-key FIFO order; the later `entries.set` also hides the first lane from `drain` and idle cleanup. Serialize the keyed lookup, lane creation, and initial enqueue (or reserve the key before yielding) so only one lane can be created per key.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M30-99 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Changing runtime mode mid-turn hangs ProviderCommandReactor; all new sessions stay Connecting or show no status

1 participant

@clintebbesen
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(server): isolate provider command lanes by thread - #7071

Open
clintebbesen wants to merge 2 commits into
pingdotgg:mainfrom
clintebbesen:fix/provider-command-cross-thread-lockout
Open

fix(server): isolate provider command lanes by thread#7071
clintebbesen wants to merge 2 commits into
pingdotgg:mainfrom
clintebbesen:fix/provider-command-cross-thread-lockout

Conversation

@clintebbesen

@clintebbesenclintebbesen commented Aug 15, 2026

Copy link
Copy Markdown

Fixes#6517.

ProviderCommandReactor used one global DrainableWorker for lifecycle, turn, response, and stop commands. A provider start/restart that never resolves therefore prevented unrelated threads from starting and left their statuses blank or stuck on Connecting.

This change extends the existing DrainableWorker owner with keyed FIFO lanes. Commands for different threads now process independently; commands for the same thread retain order. Idle lanes remove themselves after draining so the reactor does not retain a queue and fiber for every historical thread.

Focused regression coverage proves independent progress for different keys and FIFO ordering for the same key. The existing ProviderCommandReactor test file still passes (47 tests); the shared worker tests pass (2 tests). Server/shared typechecks, targeted lint, formatting, and done-check --base upstream/main --no-tests passed. Tests were run in the isolated worktree; no live T3 state was used.

Scope disposition: this PR fixes the cross-thread scheduling lockout only. It does not duplicate the still-open interrupt responsiveness work in #6531, nor fold in the separately scoped lifecycle timeout/stale-session/restart-recovery work tracked by #6560, #4944, and #4584. The Codex framing defect remains separate in #5389 and fork coordination issue clintebbesen#1.

Verification environment: Codex harness, gpt-5.6-sol.


Note

Medium Risk
Changes orchestration concurrency for provider lifecycle/turn commands; same-thread ordering is preserved but cross-thread behavior is no longer globally serialized.

Overview
Fixes cross-thread lockout where a single global provider command queue let one thread’s stuck start/restart block every other thread’s lifecycle and turn handling.

Adds makeKeyedDrainableWorker in shared DrainableWorker: one FIFO drainable worker per key, concurrent across keys, FIFO within a key, with idle lanes torn down after drain so historical threads don’t leak queues/fibers. ProviderCommandReactor now keys lanes on event.payload.threadId instead of makeDrainableWorker.

Regression test covers independent progress for different keys and FIFO for the same key.

Reviewed by Cursor Bugbot for commit 4822cc3. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Isolate provider command processing into per-thread FIFO lanes

  • Adds makeKeyedDrainableWorker to DrainableWorker.ts, which maintains a per-key makeDrainableWorker lane, creating and cleaning up lanes dynamically as work arrives and drains.
  • Updates ProviderCommandReactor.ts to key the worker by event.payload.threadId, so events on different threads are processed concurrently while preserving FIFO order within each thread.
  • Behavioral Change: previously all provider command events shared a single FIFO queue; now concurrency is possible across threads.
📊 Macroscope summarized 4822cc3. 1 file reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

@coderabbitai

coderabbitaiBot commented Aug 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c6db509b-8e09-4829-b3fc-91dfc1be273c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Aug 15, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit bf77599. Configure here.

Comment threadpackages/shared/src/DrainableWorker.ts
@macroscopeapp

macroscopeappBot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

1 blocking correctness issue found. This PR introduces significant concurrency changes by switching from a single worker queue to per-thread isolated lanes. There is also an unresolved High severity finding about a race condition in lane creation that could break the intended per-key FIFO ordering.

You can customize Macroscope's approvability policy. Learn more.

const enqueue = (item: A): Effect.Effect<void> =>
Effect.gen(function* () {
const key = keyOf(item);
const existing = entries.get(key);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Highsrc/DrainableWorker.ts:65

Concurrent first enqueues for the same key create separate workers, so their items are processed concurrently instead of in per-key FIFO order; the later entries.set also hides the first lane from drain and idle cleanup. Serialize the keyed lookup, lane creation, and initial enqueue (or reserve the key before yielding) so only one lane can be created per key.

🤖 Copy this AI Prompt to have your agent fix this:
In file @packages/shared/src/DrainableWorker.ts around line 65:
Concurrent first enqueues for the same key create separate workers, so their items are processed concurrently instead of in per-key FIFO order; the later `entries.set` also hides the first lane from `drain` and idle cleanup. Serialize the keyed lookup, lane creation, and initial enqueue (or reserve the key before yielding) so only one lane can be created per key.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M30-99 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Changing runtime mode mid-turn hangs ProviderCommandReactor; all new sessions stay Connecting or show no status

1 participant

@clintebbesen
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(server): isolate provider command lanes by thread - #7071

Open
clintebbesen wants to merge 2 commits into
pingdotgg:mainfrom
clintebbesen:fix/provider-command-cross-thread-lockout
Open

fix(server): isolate provider command lanes by thread#7071
clintebbesen wants to merge 2 commits into
pingdotgg:mainfrom
clintebbesen:fix/provider-command-cross-thread-lockout

Conversation

@clintebbesen

@clintebbesenclintebbesen commented Aug 15, 2026

Copy link
Copy Markdown

Fixes#6517.

ProviderCommandReactor used one global DrainableWorker for lifecycle, turn, response, and stop commands. A provider start/restart that never resolves therefore prevented unrelated threads from starting and left their statuses blank or stuck on Connecting.

This change extends the existing DrainableWorker owner with keyed FIFO lanes. Commands for different threads now process independently; commands for the same thread retain order. Idle lanes remove themselves after draining so the reactor does not retain a queue and fiber for every historical thread.

Focused regression coverage proves independent progress for different keys and FIFO ordering for the same key. The existing ProviderCommandReactor test file still passes (47 tests); the shared worker tests pass (2 tests). Server/shared typechecks, targeted lint, formatting, and done-check --base upstream/main --no-tests passed. Tests were run in the isolated worktree; no live T3 state was used.

Scope disposition: this PR fixes the cross-thread scheduling lockout only. It does not duplicate the still-open interrupt responsiveness work in #6531, nor fold in the separately scoped lifecycle timeout/stale-session/restart-recovery work tracked by #6560, #4944, and #4584. The Codex framing defect remains separate in #5389 and fork coordination issue clintebbesen#1.

Verification environment: Codex harness, gpt-5.6-sol.


Note

Medium Risk
Changes orchestration concurrency for provider lifecycle/turn commands; same-thread ordering is preserved but cross-thread behavior is no longer globally serialized.

Overview
Fixes cross-thread lockout where a single global provider command queue let one thread’s stuck start/restart block every other thread’s lifecycle and turn handling.

Adds makeKeyedDrainableWorker in shared DrainableWorker: one FIFO drainable worker per key, concurrent across keys, FIFO within a key, with idle lanes torn down after drain so historical threads don’t leak queues/fibers. ProviderCommandReactor now keys lanes on event.payload.threadId instead of makeDrainableWorker.

Regression test covers independent progress for different keys and FIFO for the same key.

Reviewed by Cursor Bugbot for commit 4822cc3. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Isolate provider command processing into per-thread FIFO lanes

  • Adds makeKeyedDrainableWorker to DrainableWorker.ts, which maintains a per-key makeDrainableWorker lane, creating and cleaning up lanes dynamically as work arrives and drains.
  • Updates ProviderCommandReactor.ts to key the worker by event.payload.threadId, so events on different threads are processed concurrently while preserving FIFO order within each thread.
  • Behavioral Change: previously all provider command events shared a single FIFO queue; now concurrency is possible across threads.
📊 Macroscope summarized 4822cc3. 1 file reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

@coderabbitai

coderabbitaiBot commented Aug 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c6db509b-8e09-4829-b3fc-91dfc1be273c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Aug 15, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit bf77599. Configure here.

Comment threadpackages/shared/src/DrainableWorker.ts
@macroscopeapp

macroscopeappBot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

1 blocking correctness issue found. This PR introduces significant concurrency changes by switching from a single worker queue to per-thread isolated lanes. There is also an unresolved High severity finding about a race condition in lane creation that could break the intended per-key FIFO ordering.

You can customize Macroscope's approvability policy. Learn more.

const enqueue = (item: A): Effect.Effect<void> =>
Effect.gen(function* () {
const key = keyOf(item);
const existing = entries.get(key);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Highsrc/DrainableWorker.ts:65

Concurrent first enqueues for the same key create separate workers, so their items are processed concurrently instead of in per-key FIFO order; the later entries.set also hides the first lane from drain and idle cleanup. Serialize the keyed lookup, lane creation, and initial enqueue (or reserve the key before yielding) so only one lane can be created per key.

🤖 Copy this AI Prompt to have your agent fix this:
In file @packages/shared/src/DrainableWorker.ts around line 65:
Concurrent first enqueues for the same key create separate workers, so their items are processed concurrently instead of in per-key FIFO order; the later `entries.set` also hides the first lane from `drain` and idle cleanup. Serialize the keyed lookup, lane creation, and initial enqueue (or reserve the key before yielding) so only one lane can be created per key.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M30-99 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Changing runtime mode mid-turn hangs ProviderCommandReactor; all new sessions stay Connecting or show no status

1 participant

@clintebbesen
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

fix(server): isolate provider command lanes by thread - #7071

Open
clintebbesen wants to merge 2 commits into
pingdotgg:mainfrom
clintebbesen:fix/provider-command-cross-thread-lockout
Open

fix(server): isolate provider command lanes by thread#7071
clintebbesen wants to merge 2 commits into
pingdotgg:mainfrom
clintebbesen:fix/provider-command-cross-thread-lockout

Conversation

@clintebbesen

@clintebbesenclintebbesen commented Aug 15, 2026

Copy link
Copy Markdown

Fixes#6517.

ProviderCommandReactor used one global DrainableWorker for lifecycle, turn, response, and stop commands. A provider start/restart that never resolves therefore prevented unrelated threads from starting and left their statuses blank or stuck on Connecting.

This change extends the existing DrainableWorker owner with keyed FIFO lanes. Commands for different threads now process independently; commands for the same thread retain order. Idle lanes remove themselves after draining so the reactor does not retain a queue and fiber for every historical thread.

Focused regression coverage proves independent progress for different keys and FIFO ordering for the same key. The existing ProviderCommandReactor test file still passes (47 tests); the shared worker tests pass (2 tests). Server/shared typechecks, targeted lint, formatting, and done-check --base upstream/main --no-tests passed. Tests were run in the isolated worktree; no live T3 state was used.

Scope disposition: this PR fixes the cross-thread scheduling lockout only. It does not duplicate the still-open interrupt responsiveness work in #6531, nor fold in the separately scoped lifecycle timeout/stale-session/restart-recovery work tracked by #6560, #4944, and #4584. The Codex framing defect remains separate in #5389 and fork coordination issue clintebbesen#1.

Verification environment: Codex harness, gpt-5.6-sol.


Note

Medium Risk
Changes orchestration concurrency for provider lifecycle/turn commands; same-thread ordering is preserved but cross-thread behavior is no longer globally serialized.

Overview
Fixes cross-thread lockout where a single global provider command queue let one thread’s stuck start/restart block every other thread’s lifecycle and turn handling.

Adds makeKeyedDrainableWorker in shared DrainableWorker: one FIFO drainable worker per key, concurrent across keys, FIFO within a key, with idle lanes torn down after drain so historical threads don’t leak queues/fibers. ProviderCommandReactor now keys lanes on event.payload.threadId instead of makeDrainableWorker.

Regression test covers independent progress for different keys and FIFO for the same key.

Reviewed by Cursor Bugbot for commit 4822cc3. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Isolate provider command processing into per-thread FIFO lanes

  • Adds makeKeyedDrainableWorker to DrainableWorker.ts, which maintains a per-key makeDrainableWorker lane, creating and cleaning up lanes dynamically as work arrives and drains.
  • Updates ProviderCommandReactor.ts to key the worker by event.payload.threadId, so events on different threads are processed concurrently while preserving FIFO order within each thread.
  • Behavioral Change: previously all provider command events shared a single FIFO queue; now concurrency is possible across threads.
📊 Macroscope summarized 4822cc3. 1 file reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

@coderabbitai

coderabbitaiBot commented Aug 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c6db509b-8e09-4829-b3fc-91dfc1be273c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Aug 15, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit bf77599. Configure here.

Comment threadpackages/shared/src/DrainableWorker.ts
@macroscopeapp

macroscopeappBot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

1 blocking correctness issue found. This PR introduces significant concurrency changes by switching from a single worker queue to per-thread isolated lanes. There is also an unresolved High severity finding about a race condition in lane creation that could break the intended per-key FIFO ordering.

You can customize Macroscope's approvability policy. Learn more.

const enqueue = (item: A): Effect.Effect<void> =>
Effect.gen(function* () {
const key = keyOf(item);
const existing = entries.get(key);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Highsrc/DrainableWorker.ts:65

Concurrent first enqueues for the same key create separate workers, so their items are processed concurrently instead of in per-key FIFO order; the later entries.set also hides the first lane from drain and idle cleanup. Serialize the keyed lookup, lane creation, and initial enqueue (or reserve the key before yielding) so only one lane can be created per key.

🤖 Copy this AI Prompt to have your agent fix this:
In file @packages/shared/src/DrainableWorker.ts around line 65:
Concurrent first enqueues for the same key create separate workers, so their items are processed concurrently instead of in per-key FIFO order; the later `entries.set` also hides the first lane from `drain` and idle cleanup. Serialize the keyed lookup, lane creation, and initial enqueue (or reserve the key before yielding) so only one lane can be created per key.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M30-99 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Changing runtime mode mid-turn hangs ProviderCommandReactor; all new sessions stay Connecting or show no status

1 participant

@clintebbesen
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(server): isolate provider command lanes by thread - #7071

Open
clintebbesen wants to merge 2 commits into
pingdotgg:mainfrom
clintebbesen:fix/provider-command-cross-thread-lockout
Open

fix(server): isolate provider command lanes by thread#7071
clintebbesen wants to merge 2 commits into
pingdotgg:mainfrom
clintebbesen:fix/provider-command-cross-thread-lockout

Conversation

@clintebbesen

@clintebbesenclintebbesen commented Aug 15, 2026

Copy link
Copy Markdown

Fixes#6517.

ProviderCommandReactor used one global DrainableWorker for lifecycle, turn, response, and stop commands. A provider start/restart that never resolves therefore prevented unrelated threads from starting and left their statuses blank or stuck on Connecting.

This change extends the existing DrainableWorker owner with keyed FIFO lanes. Commands for different threads now process independently; commands for the same thread retain order. Idle lanes remove themselves after draining so the reactor does not retain a queue and fiber for every historical thread.

Focused regression coverage proves independent progress for different keys and FIFO ordering for the same key. The existing ProviderCommandReactor test file still passes (47 tests); the shared worker tests pass (2 tests). Server/shared typechecks, targeted lint, formatting, and done-check --base upstream/main --no-tests passed. Tests were run in the isolated worktree; no live T3 state was used.

Scope disposition: this PR fixes the cross-thread scheduling lockout only. It does not duplicate the still-open interrupt responsiveness work in #6531, nor fold in the separately scoped lifecycle timeout/stale-session/restart-recovery work tracked by #6560, #4944, and #4584. The Codex framing defect remains separate in #5389 and fork coordination issue clintebbesen#1.

Verification environment: Codex harness, gpt-5.6-sol.


Note

Medium Risk
Changes orchestration concurrency for provider lifecycle/turn commands; same-thread ordering is preserved but cross-thread behavior is no longer globally serialized.

Overview
Fixes cross-thread lockout where a single global provider command queue let one thread’s stuck start/restart block every other thread’s lifecycle and turn handling.

Adds makeKeyedDrainableWorker in shared DrainableWorker: one FIFO drainable worker per key, concurrent across keys, FIFO within a key, with idle lanes torn down after drain so historical threads don’t leak queues/fibers. ProviderCommandReactor now keys lanes on event.payload.threadId instead of makeDrainableWorker.

Regression test covers independent progress for different keys and FIFO for the same key.

Reviewed by Cursor Bugbot for commit 4822cc3. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Isolate provider command processing into per-thread FIFO lanes

  • Adds makeKeyedDrainableWorker to DrainableWorker.ts, which maintains a per-key makeDrainableWorker lane, creating and cleaning up lanes dynamically as work arrives and drains.
  • Updates ProviderCommandReactor.ts to key the worker by event.payload.threadId, so events on different threads are processed concurrently while preserving FIFO order within each thread.
  • Behavioral Change: previously all provider command events shared a single FIFO queue; now concurrency is possible across threads.
📊 Macroscope summarized 4822cc3. 1 file reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

@coderabbitai

coderabbitaiBot commented Aug 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c6db509b-8e09-4829-b3fc-91dfc1be273c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Aug 15, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit bf77599. Configure here.

Comment threadpackages/shared/src/DrainableWorker.ts
@macroscopeapp

macroscopeappBot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

1 blocking correctness issue found. This PR introduces significant concurrency changes by switching from a single worker queue to per-thread isolated lanes. There is also an unresolved High severity finding about a race condition in lane creation that could break the intended per-key FIFO ordering.

You can customize Macroscope's approvability policy. Learn more.

const enqueue = (item: A): Effect.Effect<void> =>
Effect.gen(function* () {
const key = keyOf(item);
const existing = entries.get(key);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Highsrc/DrainableWorker.ts:65

Concurrent first enqueues for the same key create separate workers, so their items are processed concurrently instead of in per-key FIFO order; the later entries.set also hides the first lane from drain and idle cleanup. Serialize the keyed lookup, lane creation, and initial enqueue (or reserve the key before yielding) so only one lane can be created per key.

🤖 Copy this AI Prompt to have your agent fix this:
In file @packages/shared/src/DrainableWorker.ts around line 65:
Concurrent first enqueues for the same key create separate workers, so their items are processed concurrently instead of in per-key FIFO order; the later `entries.set` also hides the first lane from `drain` and idle cleanup. Serialize the keyed lookup, lane creation, and initial enqueue (or reserve the key before yielding) so only one lane can be created per key.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M30-99 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Changing runtime mode mid-turn hangs ProviderCommandReactor; all new sessions stay Connecting or show no status

1 participant

@clintebbesen
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(server): isolate provider command lanes by thread - #7071

Open
clintebbesen wants to merge 2 commits into
pingdotgg:mainfrom
clintebbesen:fix/provider-command-cross-thread-lockout
Open

fix(server): isolate provider command lanes by thread#7071
clintebbesen wants to merge 2 commits into
pingdotgg:mainfrom
clintebbesen:fix/provider-command-cross-thread-lockout

Conversation

@clintebbesen

@clintebbesenclintebbesen commented Aug 15, 2026

Copy link
Copy Markdown

Fixes#6517.

ProviderCommandReactor used one global DrainableWorker for lifecycle, turn, response, and stop commands. A provider start/restart that never resolves therefore prevented unrelated threads from starting and left their statuses blank or stuck on Connecting.

This change extends the existing DrainableWorker owner with keyed FIFO lanes. Commands for different threads now process independently; commands for the same thread retain order. Idle lanes remove themselves after draining so the reactor does not retain a queue and fiber for every historical thread.

Focused regression coverage proves independent progress for different keys and FIFO ordering for the same key. The existing ProviderCommandReactor test file still passes (47 tests); the shared worker tests pass (2 tests). Server/shared typechecks, targeted lint, formatting, and done-check --base upstream/main --no-tests passed. Tests were run in the isolated worktree; no live T3 state was used.

Scope disposition: this PR fixes the cross-thread scheduling lockout only. It does not duplicate the still-open interrupt responsiveness work in #6531, nor fold in the separately scoped lifecycle timeout/stale-session/restart-recovery work tracked by #6560, #4944, and #4584. The Codex framing defect remains separate in #5389 and fork coordination issue clintebbesen#1.

Verification environment: Codex harness, gpt-5.6-sol.


Note

Medium Risk
Changes orchestration concurrency for provider lifecycle/turn commands; same-thread ordering is preserved but cross-thread behavior is no longer globally serialized.

Overview
Fixes cross-thread lockout where a single global provider command queue let one thread’s stuck start/restart block every other thread’s lifecycle and turn handling.

Adds makeKeyedDrainableWorker in shared DrainableWorker: one FIFO drainable worker per key, concurrent across keys, FIFO within a key, with idle lanes torn down after drain so historical threads don’t leak queues/fibers. ProviderCommandReactor now keys lanes on event.payload.threadId instead of makeDrainableWorker.

Regression test covers independent progress for different keys and FIFO for the same key.

Reviewed by Cursor Bugbot for commit 4822cc3. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Isolate provider command processing into per-thread FIFO lanes

  • Adds makeKeyedDrainableWorker to DrainableWorker.ts, which maintains a per-key makeDrainableWorker lane, creating and cleaning up lanes dynamically as work arrives and drains.
  • Updates ProviderCommandReactor.ts to key the worker by event.payload.threadId, so events on different threads are processed concurrently while preserving FIFO order within each thread.
  • Behavioral Change: previously all provider command events shared a single FIFO queue; now concurrency is possible across threads.
📊 Macroscope summarized 4822cc3. 1 file reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

@coderabbitai

coderabbitaiBot commented Aug 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c6db509b-8e09-4829-b3fc-91dfc1be273c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Aug 15, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit bf77599. Configure here.

Comment threadpackages/shared/src/DrainableWorker.ts
@macroscopeapp

macroscopeappBot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

1 blocking correctness issue found. This PR introduces significant concurrency changes by switching from a single worker queue to per-thread isolated lanes. There is also an unresolved High severity finding about a race condition in lane creation that could break the intended per-key FIFO ordering.

You can customize Macroscope's approvability policy. Learn more.

const enqueue = (item: A): Effect.Effect<void> =>
Effect.gen(function* () {
const key = keyOf(item);
const existing = entries.get(key);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Highsrc/DrainableWorker.ts:65

Concurrent first enqueues for the same key create separate workers, so their items are processed concurrently instead of in per-key FIFO order; the later entries.set also hides the first lane from drain and idle cleanup. Serialize the keyed lookup, lane creation, and initial enqueue (or reserve the key before yielding) so only one lane can be created per key.

🤖 Copy this AI Prompt to have your agent fix this:
In file @packages/shared/src/DrainableWorker.ts around line 65:
Concurrent first enqueues for the same key create separate workers, so their items are processed concurrently instead of in per-key FIFO order; the later `entries.set` also hides the first lane from `drain` and idle cleanup. Serialize the keyed lookup, lane creation, and initial enqueue (or reserve the key before yielding) so only one lane can be created per key.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M30-99 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Changing runtime mode mid-turn hangs ProviderCommandReactor; all new sessions stay Connecting or show no status

1 participant

@clintebbesen
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

fix(server): isolate provider command lanes by thread - #7071

Open
clintebbesen wants to merge 2 commits into
pingdotgg:mainfrom
clintebbesen:fix/provider-command-cross-thread-lockout
Open

fix(server): isolate provider command lanes by thread#7071
clintebbesen wants to merge 2 commits into
pingdotgg:mainfrom
clintebbesen:fix/provider-command-cross-thread-lockout

Conversation

@clintebbesen

@clintebbesenclintebbesen commented Aug 15, 2026

Copy link
Copy Markdown

Fixes#6517.

ProviderCommandReactor used one global DrainableWorker for lifecycle, turn, response, and stop commands. A provider start/restart that never resolves therefore prevented unrelated threads from starting and left their statuses blank or stuck on Connecting.

This change extends the existing DrainableWorker owner with keyed FIFO lanes. Commands for different threads now process independently; commands for the same thread retain order. Idle lanes remove themselves after draining so the reactor does not retain a queue and fiber for every historical thread.

Focused regression coverage proves independent progress for different keys and FIFO ordering for the same key. The existing ProviderCommandReactor test file still passes (47 tests); the shared worker tests pass (2 tests). Server/shared typechecks, targeted lint, formatting, and done-check --base upstream/main --no-tests passed. Tests were run in the isolated worktree; no live T3 state was used.

Scope disposition: this PR fixes the cross-thread scheduling lockout only. It does not duplicate the still-open interrupt responsiveness work in #6531, nor fold in the separately scoped lifecycle timeout/stale-session/restart-recovery work tracked by #6560, #4944, and #4584. The Codex framing defect remains separate in #5389 and fork coordination issue clintebbesen#1.

Verification environment: Codex harness, gpt-5.6-sol.


Note

Medium Risk
Changes orchestration concurrency for provider lifecycle/turn commands; same-thread ordering is preserved but cross-thread behavior is no longer globally serialized.

Overview
Fixes cross-thread lockout where a single global provider command queue let one thread’s stuck start/restart block every other thread’s lifecycle and turn handling.

Adds makeKeyedDrainableWorker in shared DrainableWorker: one FIFO drainable worker per key, concurrent across keys, FIFO within a key, with idle lanes torn down after drain so historical threads don’t leak queues/fibers. ProviderCommandReactor now keys lanes on event.payload.threadId instead of makeDrainableWorker.

Regression test covers independent progress for different keys and FIFO for the same key.

Reviewed by Cursor Bugbot for commit 4822cc3. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Isolate provider command processing into per-thread FIFO lanes

  • Adds makeKeyedDrainableWorker to DrainableWorker.ts, which maintains a per-key makeDrainableWorker lane, creating and cleaning up lanes dynamically as work arrives and drains.
  • Updates ProviderCommandReactor.ts to key the worker by event.payload.threadId, so events on different threads are processed concurrently while preserving FIFO order within each thread.
  • Behavioral Change: previously all provider command events shared a single FIFO queue; now concurrency is possible across threads.
📊 Macroscope summarized 4822cc3. 1 file reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

@coderabbitai

coderabbitaiBot commented Aug 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c6db509b-8e09-4829-b3fc-91dfc1be273c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Aug 15, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit bf77599. Configure here.

Comment threadpackages/shared/src/DrainableWorker.ts
@macroscopeapp

macroscopeappBot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

1 blocking correctness issue found. This PR introduces significant concurrency changes by switching from a single worker queue to per-thread isolated lanes. There is also an unresolved High severity finding about a race condition in lane creation that could break the intended per-key FIFO ordering.

You can customize Macroscope's approvability policy. Learn more.

const enqueue = (item: A): Effect.Effect<void> =>
Effect.gen(function* () {
const key = keyOf(item);
const existing = entries.get(key);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Highsrc/DrainableWorker.ts:65

Concurrent first enqueues for the same key create separate workers, so their items are processed concurrently instead of in per-key FIFO order; the later entries.set also hides the first lane from drain and idle cleanup. Serialize the keyed lookup, lane creation, and initial enqueue (or reserve the key before yielding) so only one lane can be created per key.

🤖 Copy this AI Prompt to have your agent fix this:
In file @packages/shared/src/DrainableWorker.ts around line 65:
Concurrent first enqueues for the same key create separate workers, so their items are processed concurrently instead of in per-key FIFO order; the later `entries.set` also hides the first lane from `drain` and idle cleanup. Serialize the keyed lookup, lane creation, and initial enqueue (or reserve the key before yielding) so only one lane can be created per key.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M30-99 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Changing runtime mode mid-turn hangs ProviderCommandReactor; all new sessions stay Connecting or show no status

1 participant

@clintebbesen