feat: allow switching provider/model mid-conversation - #3799

Closed
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation
Closed

feat: allow switching provider/model mid-conversation#3799
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation

Conversation

@Brechard

@BrechardBrechard commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Closes#3797.

Heads up on scope: this is a large change (~1.1k lines) and overlaps the in-flight orchestrator-v2 work. I'm opening it for visibility per the linked issue and CONTRIBUTING's "issues first" — happy to close, split, or defer if it's not wanted as-is.

What changed

Allows switching provider/model mid-conversation, without resuming the old provider's session (which #2365 found doesn't work across providers).

  • On a cross-provider switch, ensureSessionForThread starts a fresh session on the target provider (dropping the old resume cursor) and persists the new modelSelection, instead of rejecting the switch.
  • providerHandoffTranscript renders the prior conversation — user/assistant messages plus a structured tool/command trail (commands, file edits, MCP/skill calls, exit codes, output) — and prepends it to the first turn on the new provider, so it inherits the actual work, not just prose.
  • An inline "Switched from X to Y" activity is rendered as a timeline notice above the triggering message.
  • The picker is unlocked on started threads; the in-thread block is kept only for providers that require a new thread for a model change.

Why this differs from prior attempts

Notes

  • Adds no DB migrations.
  • Same-driver model switches are unchanged; this only adds the cross-driver handoff path.
  • Verified end-to-end: a cross-provider switch carries context, including command output and non-zero exit codes the previous assistant never narrated.
  • Tradeoff: the new provider sees a rendered transcript, not native tool-call state, so per-turn rollback/checkpointing doesn't span the switch boundary.

Before / after

pr3799-provider-AFTER-switched-notice

Note

High Risk
Changes core turn-start/session binding and provider input shaping in orchestration; incorrect handoff detection or transcript handling could mis-route turns or leak stale resume state, though behavior is heavily covered by new tests.

Overview
Mid-conversation provider/model switches are allowed when the target instance uses a different driver or continuation key. The server no longer rejects those turns; it starts a fresh provider session (no old resumeCursor), persists the new modelSelection, and prepends a [Conversation handoff] transcript to the first turn on the new provider via new renderProviderHandoffPrelude (messages plus deduped tool/command/MCP trail, size-capped).

ProviderCommandReactor detects handoffs, skips in-session model-change rules for them, appends thread.model-changed activities after a successful (re)start, and resolves “current” instance from the bound session when needed. ProviderService tolerates transient binding mismatches during handoff and dedupes overlapping live sessions per thread.

The web UI unlocks the model picker on started threads, only blocks same-driver/same-continuation switches for requiresNewThreadForModelChange providers, and shows inline “Switched from X to Y” timeline notices from those activities.

Reviewed by Cursor Bugbot for commit 853c8d5. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Allow switching provider/model mid-conversation with transcript handoff

  • When a user switches to a provider with a different driver or continuation group, the orchestrator starts a fresh session on the new provider instead of rejecting the change, prepending a [Conversation handoff] transcript prelude to the next turn so the new provider has prior context.
  • A thread.model-changed activity is appended to the thread on handoff; the UI renders this as a pill-styled notice row in the conversation timeline via a new NoticeTimelineRow component.
  • listSessions in ProviderService no longer dies on provider/instance binding mismatches; it returns at most one session per thread, preferring the authoritative binding match.
  • The getStartedThreadModelChangeBlockReason logic is relaxed so cross-instance switches are allowed unless both instances share the same driver and continuation group.
  • Resume cursors are cleared when rebinding to a different provider instance to prevent stale cursor reuse across drivers.
  • Risk: the handoff prelude is capped at 80,000 chars and will silently omit older history when truncated, which may cause the new provider to miss earlier context.

Macroscope summarized 853c8d5.

Threads were pinned to their initial provider once started: the server
rejected cross-driver/continuation switches and the UI locked the picker.
This lets a thread hand off to a different provider mid-conversation.
- Server: replace the switch-rejection guards in ensureSessionForThread
with a handoff path that starts a fresh session on the new provider
(dropping the old resume cursor), persists the new modelSelection, and
tolerates the transient binding mismatch during rebind.
- Handoff context: new providerHandoffTranscript renderer replays prior
user/assistant messages plus a structured tool/command trail (commands,
file edits, MCP/skill calls, exit codes, output) into the first turn on
the new provider, so it inherits the actual work done, not just prose.
- Notice: emit a thread.model-changed activity on model/provider change,
rendered as an inline "Switched from X to Y" timeline row above the
triggering user message.
- UI: unlock the model picker on started threads; keep the in-thread
block only for providers that require a new thread for a model change.
Adds no DB migrations. Server + web suites, typecheck, and lint pass;
verified end-to-end (cross-provider switch carries context, incl. hidden
command output and non-zero exit codes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitaiBot commented Jul 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: da904db3-b1f6-4b05-b0f9-69c5e67e6d8d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 8, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b747679. Configure here.

Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/web/src/components/ChatView.logic.ts Outdated
Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
Comment threadapps/server/src/provider/Layers/ProviderService.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
@macroscopeapp

macroscopeappBot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a significant new feature enabling provider/model switching mid-conversation, changing previously blocked behavior to a complex handoff flow. The changes span orchestration, session management, and UI layers (~1300 additions), introducing new runtime behavior that warrants human review.

You can customize Macroscope's approvability policy. Learn more.

From Cursor Bugbot + Macroscope on the PR:
- Handoff replay no longer requires a prior assistant message; the renderer
already returns undefined when there's nothing to carry, so user-only /
tool-only threads keep their context on switch.
- Handoff detection prefers the last-bound session's provider over the
thread's stored modelSelection, so a switch is still detected after the
session is stopped/restarted (or the selection was advanced early).
- The "Switched model" notice is emitted only after the session is
confirmed (re)started, not before — a failed start no longer leaves a
stale success notice.
- Transcript extraction also reads stderr, data.rawOutput.exitCode, and
data.toolCallId (dedup key), so failed-command diagnostics and exit codes
survive the handoff and lifecycle events collapse correctly.
- listSessions dedupes by threadId, preferring the binding-matched session,
so a mid-handoff stale session can't produce two entries for one thread.
- getStartedThreadModelChangeBlockReason only treats a cross-instance switch
as an unblocked handoff when the driver/continuation actually differ;
same-driver+continuation instances stay blocked for providers that
require a new thread.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
collectToolTrailEntries kept the longest line per tool-call group, so a
verbose in-progress tool.updated could shadow the shorter tool.completed/
task.completed line that actually carries the exit code and final output.
The next provider then inherited stale partial tool state. Prefer the
terminal lifecycle event, falling back to longest line only on ties.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 333d44e)
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 replaces the V1 orchestration engine wholesale, and the behavior this PR adds already landed there natively — so this change is a no-op against the new base.

Thanks for the work here, and sorry for the churn on a long-lived branch. Once #2829 merges, please rebase onto main and reopen if the behavior is still missing — happy to take it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL500-999 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Switch provider/model mid-conversation via transcript handoff

3 participants

@Brechard@juliusmarminge@rodrigoB-CB
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

feat: allow switching provider/model mid-conversation - #3799

Closed
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation
Closed

feat: allow switching provider/model mid-conversation#3799
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation

Conversation

@Brechard

@BrechardBrechard commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Closes#3797.

Heads up on scope: this is a large change (~1.1k lines) and overlaps the in-flight orchestrator-v2 work. I'm opening it for visibility per the linked issue and CONTRIBUTING's "issues first" — happy to close, split, or defer if it's not wanted as-is.

What changed

Allows switching provider/model mid-conversation, without resuming the old provider's session (which #2365 found doesn't work across providers).

  • On a cross-provider switch, ensureSessionForThread starts a fresh session on the target provider (dropping the old resume cursor) and persists the new modelSelection, instead of rejecting the switch.
  • providerHandoffTranscript renders the prior conversation — user/assistant messages plus a structured tool/command trail (commands, file edits, MCP/skill calls, exit codes, output) — and prepends it to the first turn on the new provider, so it inherits the actual work, not just prose.
  • An inline "Switched from X to Y" activity is rendered as a timeline notice above the triggering message.
  • The picker is unlocked on started threads; the in-thread block is kept only for providers that require a new thread for a model change.

Why this differs from prior attempts

Notes

  • Adds no DB migrations.
  • Same-driver model switches are unchanged; this only adds the cross-driver handoff path.
  • Verified end-to-end: a cross-provider switch carries context, including command output and non-zero exit codes the previous assistant never narrated.
  • Tradeoff: the new provider sees a rendered transcript, not native tool-call state, so per-turn rollback/checkpointing doesn't span the switch boundary.

Before / after

pr3799-provider-AFTER-switched-notice

Note

High Risk
Changes core turn-start/session binding and provider input shaping in orchestration; incorrect handoff detection or transcript handling could mis-route turns or leak stale resume state, though behavior is heavily covered by new tests.

Overview
Mid-conversation provider/model switches are allowed when the target instance uses a different driver or continuation key. The server no longer rejects those turns; it starts a fresh provider session (no old resumeCursor), persists the new modelSelection, and prepends a [Conversation handoff] transcript to the first turn on the new provider via new renderProviderHandoffPrelude (messages plus deduped tool/command/MCP trail, size-capped).

ProviderCommandReactor detects handoffs, skips in-session model-change rules for them, appends thread.model-changed activities after a successful (re)start, and resolves “current” instance from the bound session when needed. ProviderService tolerates transient binding mismatches during handoff and dedupes overlapping live sessions per thread.

The web UI unlocks the model picker on started threads, only blocks same-driver/same-continuation switches for requiresNewThreadForModelChange providers, and shows inline “Switched from X to Y” timeline notices from those activities.

Reviewed by Cursor Bugbot for commit 853c8d5. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Allow switching provider/model mid-conversation with transcript handoff

  • When a user switches to a provider with a different driver or continuation group, the orchestrator starts a fresh session on the new provider instead of rejecting the change, prepending a [Conversation handoff] transcript prelude to the next turn so the new provider has prior context.
  • A thread.model-changed activity is appended to the thread on handoff; the UI renders this as a pill-styled notice row in the conversation timeline via a new NoticeTimelineRow component.
  • listSessions in ProviderService no longer dies on provider/instance binding mismatches; it returns at most one session per thread, preferring the authoritative binding match.
  • The getStartedThreadModelChangeBlockReason logic is relaxed so cross-instance switches are allowed unless both instances share the same driver and continuation group.
  • Resume cursors are cleared when rebinding to a different provider instance to prevent stale cursor reuse across drivers.
  • Risk: the handoff prelude is capped at 80,000 chars and will silently omit older history when truncated, which may cause the new provider to miss earlier context.

Macroscope summarized 853c8d5.

Threads were pinned to their initial provider once started: the server
rejected cross-driver/continuation switches and the UI locked the picker.
This lets a thread hand off to a different provider mid-conversation.
- Server: replace the switch-rejection guards in ensureSessionForThread
with a handoff path that starts a fresh session on the new provider
(dropping the old resume cursor), persists the new modelSelection, and
tolerates the transient binding mismatch during rebind.
- Handoff context: new providerHandoffTranscript renderer replays prior
user/assistant messages plus a structured tool/command trail (commands,
file edits, MCP/skill calls, exit codes, output) into the first turn on
the new provider, so it inherits the actual work done, not just prose.
- Notice: emit a thread.model-changed activity on model/provider change,
rendered as an inline "Switched from X to Y" timeline row above the
triggering user message.
- UI: unlock the model picker on started threads; keep the in-thread
block only for providers that require a new thread for a model change.
Adds no DB migrations. Server + web suites, typecheck, and lint pass;
verified end-to-end (cross-provider switch carries context, incl. hidden
command output and non-zero exit codes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitaiBot commented Jul 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: da904db3-b1f6-4b05-b0f9-69c5e67e6d8d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 8, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b747679. Configure here.

Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/web/src/components/ChatView.logic.ts Outdated
Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
Comment threadapps/server/src/provider/Layers/ProviderService.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
@macroscopeapp

macroscopeappBot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a significant new feature enabling provider/model switching mid-conversation, changing previously blocked behavior to a complex handoff flow. The changes span orchestration, session management, and UI layers (~1300 additions), introducing new runtime behavior that warrants human review.

You can customize Macroscope's approvability policy. Learn more.

From Cursor Bugbot + Macroscope on the PR:
- Handoff replay no longer requires a prior assistant message; the renderer
already returns undefined when there's nothing to carry, so user-only /
tool-only threads keep their context on switch.
- Handoff detection prefers the last-bound session's provider over the
thread's stored modelSelection, so a switch is still detected after the
session is stopped/restarted (or the selection was advanced early).
- The "Switched model" notice is emitted only after the session is
confirmed (re)started, not before — a failed start no longer leaves a
stale success notice.
- Transcript extraction also reads stderr, data.rawOutput.exitCode, and
data.toolCallId (dedup key), so failed-command diagnostics and exit codes
survive the handoff and lifecycle events collapse correctly.
- listSessions dedupes by threadId, preferring the binding-matched session,
so a mid-handoff stale session can't produce two entries for one thread.
- getStartedThreadModelChangeBlockReason only treats a cross-instance switch
as an unblocked handoff when the driver/continuation actually differ;
same-driver+continuation instances stay blocked for providers that
require a new thread.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
collectToolTrailEntries kept the longest line per tool-call group, so a
verbose in-progress tool.updated could shadow the shorter tool.completed/
task.completed line that actually carries the exit code and final output.
The next provider then inherited stale partial tool state. Prefer the
terminal lifecycle event, falling back to longest line only on ties.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 333d44e)
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 replaces the V1 orchestration engine wholesale, and the behavior this PR adds already landed there natively — so this change is a no-op against the new base.

Thanks for the work here, and sorry for the churn on a long-lived branch. Once #2829 merges, please rebase onto main and reopen if the behavior is still missing — happy to take it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL500-999 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Switch provider/model mid-conversation via transcript handoff

3 participants

@Brechard@juliusmarminge@rodrigoB-CB
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat: allow switching provider/model mid-conversation - #3799

Closed
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation
Closed

feat: allow switching provider/model mid-conversation#3799
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation

Conversation

@Brechard

@BrechardBrechard commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Closes#3797.

Heads up on scope: this is a large change (~1.1k lines) and overlaps the in-flight orchestrator-v2 work. I'm opening it for visibility per the linked issue and CONTRIBUTING's "issues first" — happy to close, split, or defer if it's not wanted as-is.

What changed

Allows switching provider/model mid-conversation, without resuming the old provider's session (which #2365 found doesn't work across providers).

  • On a cross-provider switch, ensureSessionForThread starts a fresh session on the target provider (dropping the old resume cursor) and persists the new modelSelection, instead of rejecting the switch.
  • providerHandoffTranscript renders the prior conversation — user/assistant messages plus a structured tool/command trail (commands, file edits, MCP/skill calls, exit codes, output) — and prepends it to the first turn on the new provider, so it inherits the actual work, not just prose.
  • An inline "Switched from X to Y" activity is rendered as a timeline notice above the triggering message.
  • The picker is unlocked on started threads; the in-thread block is kept only for providers that require a new thread for a model change.

Why this differs from prior attempts

Notes

  • Adds no DB migrations.
  • Same-driver model switches are unchanged; this only adds the cross-driver handoff path.
  • Verified end-to-end: a cross-provider switch carries context, including command output and non-zero exit codes the previous assistant never narrated.
  • Tradeoff: the new provider sees a rendered transcript, not native tool-call state, so per-turn rollback/checkpointing doesn't span the switch boundary.

Before / after

pr3799-provider-AFTER-switched-notice

Note

High Risk
Changes core turn-start/session binding and provider input shaping in orchestration; incorrect handoff detection or transcript handling could mis-route turns or leak stale resume state, though behavior is heavily covered by new tests.

Overview
Mid-conversation provider/model switches are allowed when the target instance uses a different driver or continuation key. The server no longer rejects those turns; it starts a fresh provider session (no old resumeCursor), persists the new modelSelection, and prepends a [Conversation handoff] transcript to the first turn on the new provider via new renderProviderHandoffPrelude (messages plus deduped tool/command/MCP trail, size-capped).

ProviderCommandReactor detects handoffs, skips in-session model-change rules for them, appends thread.model-changed activities after a successful (re)start, and resolves “current” instance from the bound session when needed. ProviderService tolerates transient binding mismatches during handoff and dedupes overlapping live sessions per thread.

The web UI unlocks the model picker on started threads, only blocks same-driver/same-continuation switches for requiresNewThreadForModelChange providers, and shows inline “Switched from X to Y” timeline notices from those activities.

Reviewed by Cursor Bugbot for commit 853c8d5. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Allow switching provider/model mid-conversation with transcript handoff

  • When a user switches to a provider with a different driver or continuation group, the orchestrator starts a fresh session on the new provider instead of rejecting the change, prepending a [Conversation handoff] transcript prelude to the next turn so the new provider has prior context.
  • A thread.model-changed activity is appended to the thread on handoff; the UI renders this as a pill-styled notice row in the conversation timeline via a new NoticeTimelineRow component.
  • listSessions in ProviderService no longer dies on provider/instance binding mismatches; it returns at most one session per thread, preferring the authoritative binding match.
  • The getStartedThreadModelChangeBlockReason logic is relaxed so cross-instance switches are allowed unless both instances share the same driver and continuation group.
  • Resume cursors are cleared when rebinding to a different provider instance to prevent stale cursor reuse across drivers.
  • Risk: the handoff prelude is capped at 80,000 chars and will silently omit older history when truncated, which may cause the new provider to miss earlier context.

Macroscope summarized 853c8d5.

Threads were pinned to their initial provider once started: the server
rejected cross-driver/continuation switches and the UI locked the picker.
This lets a thread hand off to a different provider mid-conversation.
- Server: replace the switch-rejection guards in ensureSessionForThread
with a handoff path that starts a fresh session on the new provider
(dropping the old resume cursor), persists the new modelSelection, and
tolerates the transient binding mismatch during rebind.
- Handoff context: new providerHandoffTranscript renderer replays prior
user/assistant messages plus a structured tool/command trail (commands,
file edits, MCP/skill calls, exit codes, output) into the first turn on
the new provider, so it inherits the actual work done, not just prose.
- Notice: emit a thread.model-changed activity on model/provider change,
rendered as an inline "Switched from X to Y" timeline row above the
triggering user message.
- UI: unlock the model picker on started threads; keep the in-thread
block only for providers that require a new thread for a model change.
Adds no DB migrations. Server + web suites, typecheck, and lint pass;
verified end-to-end (cross-provider switch carries context, incl. hidden
command output and non-zero exit codes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitaiBot commented Jul 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: da904db3-b1f6-4b05-b0f9-69c5e67e6d8d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 8, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b747679. Configure here.

Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/web/src/components/ChatView.logic.ts Outdated
Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
Comment threadapps/server/src/provider/Layers/ProviderService.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
@macroscopeapp

macroscopeappBot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a significant new feature enabling provider/model switching mid-conversation, changing previously blocked behavior to a complex handoff flow. The changes span orchestration, session management, and UI layers (~1300 additions), introducing new runtime behavior that warrants human review.

You can customize Macroscope's approvability policy. Learn more.

From Cursor Bugbot + Macroscope on the PR:
- Handoff replay no longer requires a prior assistant message; the renderer
already returns undefined when there's nothing to carry, so user-only /
tool-only threads keep their context on switch.
- Handoff detection prefers the last-bound session's provider over the
thread's stored modelSelection, so a switch is still detected after the
session is stopped/restarted (or the selection was advanced early).
- The "Switched model" notice is emitted only after the session is
confirmed (re)started, not before — a failed start no longer leaves a
stale success notice.
- Transcript extraction also reads stderr, data.rawOutput.exitCode, and
data.toolCallId (dedup key), so failed-command diagnostics and exit codes
survive the handoff and lifecycle events collapse correctly.
- listSessions dedupes by threadId, preferring the binding-matched session,
so a mid-handoff stale session can't produce two entries for one thread.
- getStartedThreadModelChangeBlockReason only treats a cross-instance switch
as an unblocked handoff when the driver/continuation actually differ;
same-driver+continuation instances stay blocked for providers that
require a new thread.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
collectToolTrailEntries kept the longest line per tool-call group, so a
verbose in-progress tool.updated could shadow the shorter tool.completed/
task.completed line that actually carries the exit code and final output.
The next provider then inherited stale partial tool state. Prefer the
terminal lifecycle event, falling back to longest line only on ties.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 333d44e)
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 replaces the V1 orchestration engine wholesale, and the behavior this PR adds already landed there natively — so this change is a no-op against the new base.

Thanks for the work here, and sorry for the churn on a long-lived branch. Once #2829 merges, please rebase onto main and reopen if the behavior is still missing — happy to take it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL500-999 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Switch provider/model mid-conversation via transcript handoff

3 participants

@Brechard@juliusmarminge@rodrigoB-CB
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat: allow switching provider/model mid-conversation - #3799

Closed
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation
Closed

feat: allow switching provider/model mid-conversation#3799
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation

Conversation

@Brechard

@BrechardBrechard commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Closes#3797.

Heads up on scope: this is a large change (~1.1k lines) and overlaps the in-flight orchestrator-v2 work. I'm opening it for visibility per the linked issue and CONTRIBUTING's "issues first" — happy to close, split, or defer if it's not wanted as-is.

What changed

Allows switching provider/model mid-conversation, without resuming the old provider's session (which #2365 found doesn't work across providers).

  • On a cross-provider switch, ensureSessionForThread starts a fresh session on the target provider (dropping the old resume cursor) and persists the new modelSelection, instead of rejecting the switch.
  • providerHandoffTranscript renders the prior conversation — user/assistant messages plus a structured tool/command trail (commands, file edits, MCP/skill calls, exit codes, output) — and prepends it to the first turn on the new provider, so it inherits the actual work, not just prose.
  • An inline "Switched from X to Y" activity is rendered as a timeline notice above the triggering message.
  • The picker is unlocked on started threads; the in-thread block is kept only for providers that require a new thread for a model change.

Why this differs from prior attempts

Notes

  • Adds no DB migrations.
  • Same-driver model switches are unchanged; this only adds the cross-driver handoff path.
  • Verified end-to-end: a cross-provider switch carries context, including command output and non-zero exit codes the previous assistant never narrated.
  • Tradeoff: the new provider sees a rendered transcript, not native tool-call state, so per-turn rollback/checkpointing doesn't span the switch boundary.

Before / after

pr3799-provider-AFTER-switched-notice

Note

High Risk
Changes core turn-start/session binding and provider input shaping in orchestration; incorrect handoff detection or transcript handling could mis-route turns or leak stale resume state, though behavior is heavily covered by new tests.

Overview
Mid-conversation provider/model switches are allowed when the target instance uses a different driver or continuation key. The server no longer rejects those turns; it starts a fresh provider session (no old resumeCursor), persists the new modelSelection, and prepends a [Conversation handoff] transcript to the first turn on the new provider via new renderProviderHandoffPrelude (messages plus deduped tool/command/MCP trail, size-capped).

ProviderCommandReactor detects handoffs, skips in-session model-change rules for them, appends thread.model-changed activities after a successful (re)start, and resolves “current” instance from the bound session when needed. ProviderService tolerates transient binding mismatches during handoff and dedupes overlapping live sessions per thread.

The web UI unlocks the model picker on started threads, only blocks same-driver/same-continuation switches for requiresNewThreadForModelChange providers, and shows inline “Switched from X to Y” timeline notices from those activities.

Reviewed by Cursor Bugbot for commit 853c8d5. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Allow switching provider/model mid-conversation with transcript handoff

  • When a user switches to a provider with a different driver or continuation group, the orchestrator starts a fresh session on the new provider instead of rejecting the change, prepending a [Conversation handoff] transcript prelude to the next turn so the new provider has prior context.
  • A thread.model-changed activity is appended to the thread on handoff; the UI renders this as a pill-styled notice row in the conversation timeline via a new NoticeTimelineRow component.
  • listSessions in ProviderService no longer dies on provider/instance binding mismatches; it returns at most one session per thread, preferring the authoritative binding match.
  • The getStartedThreadModelChangeBlockReason logic is relaxed so cross-instance switches are allowed unless both instances share the same driver and continuation group.
  • Resume cursors are cleared when rebinding to a different provider instance to prevent stale cursor reuse across drivers.
  • Risk: the handoff prelude is capped at 80,000 chars and will silently omit older history when truncated, which may cause the new provider to miss earlier context.

Macroscope summarized 853c8d5.

Threads were pinned to their initial provider once started: the server
rejected cross-driver/continuation switches and the UI locked the picker.
This lets a thread hand off to a different provider mid-conversation.
- Server: replace the switch-rejection guards in ensureSessionForThread
with a handoff path that starts a fresh session on the new provider
(dropping the old resume cursor), persists the new modelSelection, and
tolerates the transient binding mismatch during rebind.
- Handoff context: new providerHandoffTranscript renderer replays prior
user/assistant messages plus a structured tool/command trail (commands,
file edits, MCP/skill calls, exit codes, output) into the first turn on
the new provider, so it inherits the actual work done, not just prose.
- Notice: emit a thread.model-changed activity on model/provider change,
rendered as an inline "Switched from X to Y" timeline row above the
triggering user message.
- UI: unlock the model picker on started threads; keep the in-thread
block only for providers that require a new thread for a model change.
Adds no DB migrations. Server + web suites, typecheck, and lint pass;
verified end-to-end (cross-provider switch carries context, incl. hidden
command output and non-zero exit codes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitaiBot commented Jul 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: da904db3-b1f6-4b05-b0f9-69c5e67e6d8d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 8, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b747679. Configure here.

Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/web/src/components/ChatView.logic.ts Outdated
Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
Comment threadapps/server/src/provider/Layers/ProviderService.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
@macroscopeapp

macroscopeappBot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a significant new feature enabling provider/model switching mid-conversation, changing previously blocked behavior to a complex handoff flow. The changes span orchestration, session management, and UI layers (~1300 additions), introducing new runtime behavior that warrants human review.

You can customize Macroscope's approvability policy. Learn more.

From Cursor Bugbot + Macroscope on the PR:
- Handoff replay no longer requires a prior assistant message; the renderer
already returns undefined when there's nothing to carry, so user-only /
tool-only threads keep their context on switch.
- Handoff detection prefers the last-bound session's provider over the
thread's stored modelSelection, so a switch is still detected after the
session is stopped/restarted (or the selection was advanced early).
- The "Switched model" notice is emitted only after the session is
confirmed (re)started, not before — a failed start no longer leaves a
stale success notice.
- Transcript extraction also reads stderr, data.rawOutput.exitCode, and
data.toolCallId (dedup key), so failed-command diagnostics and exit codes
survive the handoff and lifecycle events collapse correctly.
- listSessions dedupes by threadId, preferring the binding-matched session,
so a mid-handoff stale session can't produce two entries for one thread.
- getStartedThreadModelChangeBlockReason only treats a cross-instance switch
as an unblocked handoff when the driver/continuation actually differ;
same-driver+continuation instances stay blocked for providers that
require a new thread.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
collectToolTrailEntries kept the longest line per tool-call group, so a
verbose in-progress tool.updated could shadow the shorter tool.completed/
task.completed line that actually carries the exit code and final output.
The next provider then inherited stale partial tool state. Prefer the
terminal lifecycle event, falling back to longest line only on ties.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 333d44e)
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 replaces the V1 orchestration engine wholesale, and the behavior this PR adds already landed there natively — so this change is a no-op against the new base.

Thanks for the work here, and sorry for the churn on a long-lived branch. Once #2829 merges, please rebase onto main and reopen if the behavior is still missing — happy to take it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL500-999 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Switch provider/model mid-conversation via transcript handoff

3 participants

@Brechard@juliusmarminge@rodrigoB-CB
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

feat: allow switching provider/model mid-conversation - #3799

Closed
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation
Closed

feat: allow switching provider/model mid-conversation#3799
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation

Conversation

@Brechard

@BrechardBrechard commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Closes#3797.

Heads up on scope: this is a large change (~1.1k lines) and overlaps the in-flight orchestrator-v2 work. I'm opening it for visibility per the linked issue and CONTRIBUTING's "issues first" — happy to close, split, or defer if it's not wanted as-is.

What changed

Allows switching provider/model mid-conversation, without resuming the old provider's session (which #2365 found doesn't work across providers).

  • On a cross-provider switch, ensureSessionForThread starts a fresh session on the target provider (dropping the old resume cursor) and persists the new modelSelection, instead of rejecting the switch.
  • providerHandoffTranscript renders the prior conversation — user/assistant messages plus a structured tool/command trail (commands, file edits, MCP/skill calls, exit codes, output) — and prepends it to the first turn on the new provider, so it inherits the actual work, not just prose.
  • An inline "Switched from X to Y" activity is rendered as a timeline notice above the triggering message.
  • The picker is unlocked on started threads; the in-thread block is kept only for providers that require a new thread for a model change.

Why this differs from prior attempts

Notes

  • Adds no DB migrations.
  • Same-driver model switches are unchanged; this only adds the cross-driver handoff path.
  • Verified end-to-end: a cross-provider switch carries context, including command output and non-zero exit codes the previous assistant never narrated.
  • Tradeoff: the new provider sees a rendered transcript, not native tool-call state, so per-turn rollback/checkpointing doesn't span the switch boundary.

Before / after

pr3799-provider-AFTER-switched-notice

Note

High Risk
Changes core turn-start/session binding and provider input shaping in orchestration; incorrect handoff detection or transcript handling could mis-route turns or leak stale resume state, though behavior is heavily covered by new tests.

Overview
Mid-conversation provider/model switches are allowed when the target instance uses a different driver or continuation key. The server no longer rejects those turns; it starts a fresh provider session (no old resumeCursor), persists the new modelSelection, and prepends a [Conversation handoff] transcript to the first turn on the new provider via new renderProviderHandoffPrelude (messages plus deduped tool/command/MCP trail, size-capped).

ProviderCommandReactor detects handoffs, skips in-session model-change rules for them, appends thread.model-changed activities after a successful (re)start, and resolves “current” instance from the bound session when needed. ProviderService tolerates transient binding mismatches during handoff and dedupes overlapping live sessions per thread.

The web UI unlocks the model picker on started threads, only blocks same-driver/same-continuation switches for requiresNewThreadForModelChange providers, and shows inline “Switched from X to Y” timeline notices from those activities.

Reviewed by Cursor Bugbot for commit 853c8d5. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Allow switching provider/model mid-conversation with transcript handoff

  • When a user switches to a provider with a different driver or continuation group, the orchestrator starts a fresh session on the new provider instead of rejecting the change, prepending a [Conversation handoff] transcript prelude to the next turn so the new provider has prior context.
  • A thread.model-changed activity is appended to the thread on handoff; the UI renders this as a pill-styled notice row in the conversation timeline via a new NoticeTimelineRow component.
  • listSessions in ProviderService no longer dies on provider/instance binding mismatches; it returns at most one session per thread, preferring the authoritative binding match.
  • The getStartedThreadModelChangeBlockReason logic is relaxed so cross-instance switches are allowed unless both instances share the same driver and continuation group.
  • Resume cursors are cleared when rebinding to a different provider instance to prevent stale cursor reuse across drivers.
  • Risk: the handoff prelude is capped at 80,000 chars and will silently omit older history when truncated, which may cause the new provider to miss earlier context.

Macroscope summarized 853c8d5.

Threads were pinned to their initial provider once started: the server
rejected cross-driver/continuation switches and the UI locked the picker.
This lets a thread hand off to a different provider mid-conversation.
- Server: replace the switch-rejection guards in ensureSessionForThread
with a handoff path that starts a fresh session on the new provider
(dropping the old resume cursor), persists the new modelSelection, and
tolerates the transient binding mismatch during rebind.
- Handoff context: new providerHandoffTranscript renderer replays prior
user/assistant messages plus a structured tool/command trail (commands,
file edits, MCP/skill calls, exit codes, output) into the first turn on
the new provider, so it inherits the actual work done, not just prose.
- Notice: emit a thread.model-changed activity on model/provider change,
rendered as an inline "Switched from X to Y" timeline row above the
triggering user message.
- UI: unlock the model picker on started threads; keep the in-thread
block only for providers that require a new thread for a model change.
Adds no DB migrations. Server + web suites, typecheck, and lint pass;
verified end-to-end (cross-provider switch carries context, incl. hidden
command output and non-zero exit codes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitaiBot commented Jul 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: da904db3-b1f6-4b05-b0f9-69c5e67e6d8d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 8, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b747679. Configure here.

Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/web/src/components/ChatView.logic.ts Outdated
Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
Comment threadapps/server/src/provider/Layers/ProviderService.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
@macroscopeapp

macroscopeappBot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a significant new feature enabling provider/model switching mid-conversation, changing previously blocked behavior to a complex handoff flow. The changes span orchestration, session management, and UI layers (~1300 additions), introducing new runtime behavior that warrants human review.

You can customize Macroscope's approvability policy. Learn more.

From Cursor Bugbot + Macroscope on the PR:
- Handoff replay no longer requires a prior assistant message; the renderer
already returns undefined when there's nothing to carry, so user-only /
tool-only threads keep their context on switch.
- Handoff detection prefers the last-bound session's provider over the
thread's stored modelSelection, so a switch is still detected after the
session is stopped/restarted (or the selection was advanced early).
- The "Switched model" notice is emitted only after the session is
confirmed (re)started, not before — a failed start no longer leaves a
stale success notice.
- Transcript extraction also reads stderr, data.rawOutput.exitCode, and
data.toolCallId (dedup key), so failed-command diagnostics and exit codes
survive the handoff and lifecycle events collapse correctly.
- listSessions dedupes by threadId, preferring the binding-matched session,
so a mid-handoff stale session can't produce two entries for one thread.
- getStartedThreadModelChangeBlockReason only treats a cross-instance switch
as an unblocked handoff when the driver/continuation actually differ;
same-driver+continuation instances stay blocked for providers that
require a new thread.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
collectToolTrailEntries kept the longest line per tool-call group, so a
verbose in-progress tool.updated could shadow the shorter tool.completed/
task.completed line that actually carries the exit code and final output.
The next provider then inherited stale partial tool state. Prefer the
terminal lifecycle event, falling back to longest line only on ties.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 333d44e)
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 replaces the V1 orchestration engine wholesale, and the behavior this PR adds already landed there natively — so this change is a no-op against the new base.

Thanks for the work here, and sorry for the churn on a long-lived branch. Once #2829 merges, please rebase onto main and reopen if the behavior is still missing — happy to take it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL500-999 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Switch provider/model mid-conversation via transcript handoff

3 participants

@Brechard@juliusmarminge@rodrigoB-CB
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat: allow switching provider/model mid-conversation - #3799

Closed
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation
Closed

feat: allow switching provider/model mid-conversation#3799
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation

Conversation

@Brechard

@BrechardBrechard commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Closes#3797.

Heads up on scope: this is a large change (~1.1k lines) and overlaps the in-flight orchestrator-v2 work. I'm opening it for visibility per the linked issue and CONTRIBUTING's "issues first" — happy to close, split, or defer if it's not wanted as-is.

What changed

Allows switching provider/model mid-conversation, without resuming the old provider's session (which #2365 found doesn't work across providers).

  • On a cross-provider switch, ensureSessionForThread starts a fresh session on the target provider (dropping the old resume cursor) and persists the new modelSelection, instead of rejecting the switch.
  • providerHandoffTranscript renders the prior conversation — user/assistant messages plus a structured tool/command trail (commands, file edits, MCP/skill calls, exit codes, output) — and prepends it to the first turn on the new provider, so it inherits the actual work, not just prose.
  • An inline "Switched from X to Y" activity is rendered as a timeline notice above the triggering message.
  • The picker is unlocked on started threads; the in-thread block is kept only for providers that require a new thread for a model change.

Why this differs from prior attempts

Notes

  • Adds no DB migrations.
  • Same-driver model switches are unchanged; this only adds the cross-driver handoff path.
  • Verified end-to-end: a cross-provider switch carries context, including command output and non-zero exit codes the previous assistant never narrated.
  • Tradeoff: the new provider sees a rendered transcript, not native tool-call state, so per-turn rollback/checkpointing doesn't span the switch boundary.

Before / after

pr3799-provider-AFTER-switched-notice

Note

High Risk
Changes core turn-start/session binding and provider input shaping in orchestration; incorrect handoff detection or transcript handling could mis-route turns or leak stale resume state, though behavior is heavily covered by new tests.

Overview
Mid-conversation provider/model switches are allowed when the target instance uses a different driver or continuation key. The server no longer rejects those turns; it starts a fresh provider session (no old resumeCursor), persists the new modelSelection, and prepends a [Conversation handoff] transcript to the first turn on the new provider via new renderProviderHandoffPrelude (messages plus deduped tool/command/MCP trail, size-capped).

ProviderCommandReactor detects handoffs, skips in-session model-change rules for them, appends thread.model-changed activities after a successful (re)start, and resolves “current” instance from the bound session when needed. ProviderService tolerates transient binding mismatches during handoff and dedupes overlapping live sessions per thread.

The web UI unlocks the model picker on started threads, only blocks same-driver/same-continuation switches for requiresNewThreadForModelChange providers, and shows inline “Switched from X to Y” timeline notices from those activities.

Reviewed by Cursor Bugbot for commit 853c8d5. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Allow switching provider/model mid-conversation with transcript handoff

  • When a user switches to a provider with a different driver or continuation group, the orchestrator starts a fresh session on the new provider instead of rejecting the change, prepending a [Conversation handoff] transcript prelude to the next turn so the new provider has prior context.
  • A thread.model-changed activity is appended to the thread on handoff; the UI renders this as a pill-styled notice row in the conversation timeline via a new NoticeTimelineRow component.
  • listSessions in ProviderService no longer dies on provider/instance binding mismatches; it returns at most one session per thread, preferring the authoritative binding match.
  • The getStartedThreadModelChangeBlockReason logic is relaxed so cross-instance switches are allowed unless both instances share the same driver and continuation group.
  • Resume cursors are cleared when rebinding to a different provider instance to prevent stale cursor reuse across drivers.
  • Risk: the handoff prelude is capped at 80,000 chars and will silently omit older history when truncated, which may cause the new provider to miss earlier context.

Macroscope summarized 853c8d5.

Threads were pinned to their initial provider once started: the server
rejected cross-driver/continuation switches and the UI locked the picker.
This lets a thread hand off to a different provider mid-conversation.
- Server: replace the switch-rejection guards in ensureSessionForThread
with a handoff path that starts a fresh session on the new provider
(dropping the old resume cursor), persists the new modelSelection, and
tolerates the transient binding mismatch during rebind.
- Handoff context: new providerHandoffTranscript renderer replays prior
user/assistant messages plus a structured tool/command trail (commands,
file edits, MCP/skill calls, exit codes, output) into the first turn on
the new provider, so it inherits the actual work done, not just prose.
- Notice: emit a thread.model-changed activity on model/provider change,
rendered as an inline "Switched from X to Y" timeline row above the
triggering user message.
- UI: unlock the model picker on started threads; keep the in-thread
block only for providers that require a new thread for a model change.
Adds no DB migrations. Server + web suites, typecheck, and lint pass;
verified end-to-end (cross-provider switch carries context, incl. hidden
command output and non-zero exit codes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitaiBot commented Jul 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: da904db3-b1f6-4b05-b0f9-69c5e67e6d8d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 8, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b747679. Configure here.

Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/web/src/components/ChatView.logic.ts Outdated
Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
Comment threadapps/server/src/provider/Layers/ProviderService.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
@macroscopeapp

macroscopeappBot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a significant new feature enabling provider/model switching mid-conversation, changing previously blocked behavior to a complex handoff flow. The changes span orchestration, session management, and UI layers (~1300 additions), introducing new runtime behavior that warrants human review.

You can customize Macroscope's approvability policy. Learn more.

From Cursor Bugbot + Macroscope on the PR:
- Handoff replay no longer requires a prior assistant message; the renderer
already returns undefined when there's nothing to carry, so user-only /
tool-only threads keep their context on switch.
- Handoff detection prefers the last-bound session's provider over the
thread's stored modelSelection, so a switch is still detected after the
session is stopped/restarted (or the selection was advanced early).
- The "Switched model" notice is emitted only after the session is
confirmed (re)started, not before — a failed start no longer leaves a
stale success notice.
- Transcript extraction also reads stderr, data.rawOutput.exitCode, and
data.toolCallId (dedup key), so failed-command diagnostics and exit codes
survive the handoff and lifecycle events collapse correctly.
- listSessions dedupes by threadId, preferring the binding-matched session,
so a mid-handoff stale session can't produce two entries for one thread.
- getStartedThreadModelChangeBlockReason only treats a cross-instance switch
as an unblocked handoff when the driver/continuation actually differ;
same-driver+continuation instances stay blocked for providers that
require a new thread.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
collectToolTrailEntries kept the longest line per tool-call group, so a
verbose in-progress tool.updated could shadow the shorter tool.completed/
task.completed line that actually carries the exit code and final output.
The next provider then inherited stale partial tool state. Prefer the
terminal lifecycle event, falling back to longest line only on ties.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 333d44e)
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 replaces the V1 orchestration engine wholesale, and the behavior this PR adds already landed there natively — so this change is a no-op against the new base.

Thanks for the work here, and sorry for the churn on a long-lived branch. Once #2829 merges, please rebase onto main and reopen if the behavior is still missing — happy to take it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL500-999 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Switch provider/model mid-conversation via transcript handoff

3 participants

@Brechard@juliusmarminge@rodrigoB-CB
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat: allow switching provider/model mid-conversation - #3799

Closed
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation
Closed

feat: allow switching provider/model mid-conversation#3799
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation

Conversation

@Brechard

@BrechardBrechard commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Closes#3797.

Heads up on scope: this is a large change (~1.1k lines) and overlaps the in-flight orchestrator-v2 work. I'm opening it for visibility per the linked issue and CONTRIBUTING's "issues first" — happy to close, split, or defer if it's not wanted as-is.

What changed

Allows switching provider/model mid-conversation, without resuming the old provider's session (which #2365 found doesn't work across providers).

  • On a cross-provider switch, ensureSessionForThread starts a fresh session on the target provider (dropping the old resume cursor) and persists the new modelSelection, instead of rejecting the switch.
  • providerHandoffTranscript renders the prior conversation — user/assistant messages plus a structured tool/command trail (commands, file edits, MCP/skill calls, exit codes, output) — and prepends it to the first turn on the new provider, so it inherits the actual work, not just prose.
  • An inline "Switched from X to Y" activity is rendered as a timeline notice above the triggering message.
  • The picker is unlocked on started threads; the in-thread block is kept only for providers that require a new thread for a model change.

Why this differs from prior attempts

Notes

  • Adds no DB migrations.
  • Same-driver model switches are unchanged; this only adds the cross-driver handoff path.
  • Verified end-to-end: a cross-provider switch carries context, including command output and non-zero exit codes the previous assistant never narrated.
  • Tradeoff: the new provider sees a rendered transcript, not native tool-call state, so per-turn rollback/checkpointing doesn't span the switch boundary.

Before / after

pr3799-provider-AFTER-switched-notice

Note

High Risk
Changes core turn-start/session binding and provider input shaping in orchestration; incorrect handoff detection or transcript handling could mis-route turns or leak stale resume state, though behavior is heavily covered by new tests.

Overview
Mid-conversation provider/model switches are allowed when the target instance uses a different driver or continuation key. The server no longer rejects those turns; it starts a fresh provider session (no old resumeCursor), persists the new modelSelection, and prepends a [Conversation handoff] transcript to the first turn on the new provider via new renderProviderHandoffPrelude (messages plus deduped tool/command/MCP trail, size-capped).

ProviderCommandReactor detects handoffs, skips in-session model-change rules for them, appends thread.model-changed activities after a successful (re)start, and resolves “current” instance from the bound session when needed. ProviderService tolerates transient binding mismatches during handoff and dedupes overlapping live sessions per thread.

The web UI unlocks the model picker on started threads, only blocks same-driver/same-continuation switches for requiresNewThreadForModelChange providers, and shows inline “Switched from X to Y” timeline notices from those activities.

Reviewed by Cursor Bugbot for commit 853c8d5. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Allow switching provider/model mid-conversation with transcript handoff

  • When a user switches to a provider with a different driver or continuation group, the orchestrator starts a fresh session on the new provider instead of rejecting the change, prepending a [Conversation handoff] transcript prelude to the next turn so the new provider has prior context.
  • A thread.model-changed activity is appended to the thread on handoff; the UI renders this as a pill-styled notice row in the conversation timeline via a new NoticeTimelineRow component.
  • listSessions in ProviderService no longer dies on provider/instance binding mismatches; it returns at most one session per thread, preferring the authoritative binding match.
  • The getStartedThreadModelChangeBlockReason logic is relaxed so cross-instance switches are allowed unless both instances share the same driver and continuation group.
  • Resume cursors are cleared when rebinding to a different provider instance to prevent stale cursor reuse across drivers.
  • Risk: the handoff prelude is capped at 80,000 chars and will silently omit older history when truncated, which may cause the new provider to miss earlier context.

Macroscope summarized 853c8d5.

Threads were pinned to their initial provider once started: the server
rejected cross-driver/continuation switches and the UI locked the picker.
This lets a thread hand off to a different provider mid-conversation.
- Server: replace the switch-rejection guards in ensureSessionForThread
with a handoff path that starts a fresh session on the new provider
(dropping the old resume cursor), persists the new modelSelection, and
tolerates the transient binding mismatch during rebind.
- Handoff context: new providerHandoffTranscript renderer replays prior
user/assistant messages plus a structured tool/command trail (commands,
file edits, MCP/skill calls, exit codes, output) into the first turn on
the new provider, so it inherits the actual work done, not just prose.
- Notice: emit a thread.model-changed activity on model/provider change,
rendered as an inline "Switched from X to Y" timeline row above the
triggering user message.
- UI: unlock the model picker on started threads; keep the in-thread
block only for providers that require a new thread for a model change.
Adds no DB migrations. Server + web suites, typecheck, and lint pass;
verified end-to-end (cross-provider switch carries context, incl. hidden
command output and non-zero exit codes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitaiBot commented Jul 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: da904db3-b1f6-4b05-b0f9-69c5e67e6d8d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 8, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b747679. Configure here.

Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/web/src/components/ChatView.logic.ts Outdated
Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
Comment threadapps/server/src/provider/Layers/ProviderService.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
@macroscopeapp

macroscopeappBot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a significant new feature enabling provider/model switching mid-conversation, changing previously blocked behavior to a complex handoff flow. The changes span orchestration, session management, and UI layers (~1300 additions), introducing new runtime behavior that warrants human review.

You can customize Macroscope's approvability policy. Learn more.

From Cursor Bugbot + Macroscope on the PR:
- Handoff replay no longer requires a prior assistant message; the renderer
already returns undefined when there's nothing to carry, so user-only /
tool-only threads keep their context on switch.
- Handoff detection prefers the last-bound session's provider over the
thread's stored modelSelection, so a switch is still detected after the
session is stopped/restarted (or the selection was advanced early).
- The "Switched model" notice is emitted only after the session is
confirmed (re)started, not before — a failed start no longer leaves a
stale success notice.
- Transcript extraction also reads stderr, data.rawOutput.exitCode, and
data.toolCallId (dedup key), so failed-command diagnostics and exit codes
survive the handoff and lifecycle events collapse correctly.
- listSessions dedupes by threadId, preferring the binding-matched session,
so a mid-handoff stale session can't produce two entries for one thread.
- getStartedThreadModelChangeBlockReason only treats a cross-instance switch
as an unblocked handoff when the driver/continuation actually differ;
same-driver+continuation instances stay blocked for providers that
require a new thread.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
collectToolTrailEntries kept the longest line per tool-call group, so a
verbose in-progress tool.updated could shadow the shorter tool.completed/
task.completed line that actually carries the exit code and final output.
The next provider then inherited stale partial tool state. Prefer the
terminal lifecycle event, falling back to longest line only on ties.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 333d44e)
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 replaces the V1 orchestration engine wholesale, and the behavior this PR adds already landed there natively — so this change is a no-op against the new base.

Thanks for the work here, and sorry for the churn on a long-lived branch. Once #2829 merges, please rebase onto main and reopen if the behavior is still missing — happy to take it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL500-999 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Switch provider/model mid-conversation via transcript handoff

3 participants

@Brechard@juliusmarminge@rodrigoB-CB
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

feat: allow switching provider/model mid-conversation - #3799

Closed
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation
Closed

feat: allow switching provider/model mid-conversation#3799
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation

Conversation

@Brechard

@BrechardBrechard commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Closes#3797.

Heads up on scope: this is a large change (~1.1k lines) and overlaps the in-flight orchestrator-v2 work. I'm opening it for visibility per the linked issue and CONTRIBUTING's "issues first" — happy to close, split, or defer if it's not wanted as-is.

What changed

Allows switching provider/model mid-conversation, without resuming the old provider's session (which #2365 found doesn't work across providers).

  • On a cross-provider switch, ensureSessionForThread starts a fresh session on the target provider (dropping the old resume cursor) and persists the new modelSelection, instead of rejecting the switch.
  • providerHandoffTranscript renders the prior conversation — user/assistant messages plus a structured tool/command trail (commands, file edits, MCP/skill calls, exit codes, output) — and prepends it to the first turn on the new provider, so it inherits the actual work, not just prose.
  • An inline "Switched from X to Y" activity is rendered as a timeline notice above the triggering message.
  • The picker is unlocked on started threads; the in-thread block is kept only for providers that require a new thread for a model change.

Why this differs from prior attempts

Notes

  • Adds no DB migrations.
  • Same-driver model switches are unchanged; this only adds the cross-driver handoff path.
  • Verified end-to-end: a cross-provider switch carries context, including command output and non-zero exit codes the previous assistant never narrated.
  • Tradeoff: the new provider sees a rendered transcript, not native tool-call state, so per-turn rollback/checkpointing doesn't span the switch boundary.

Before / after

pr3799-provider-AFTER-switched-notice

Note

High Risk
Changes core turn-start/session binding and provider input shaping in orchestration; incorrect handoff detection or transcript handling could mis-route turns or leak stale resume state, though behavior is heavily covered by new tests.

Overview
Mid-conversation provider/model switches are allowed when the target instance uses a different driver or continuation key. The server no longer rejects those turns; it starts a fresh provider session (no old resumeCursor), persists the new modelSelection, and prepends a [Conversation handoff] transcript to the first turn on the new provider via new renderProviderHandoffPrelude (messages plus deduped tool/command/MCP trail, size-capped).

ProviderCommandReactor detects handoffs, skips in-session model-change rules for them, appends thread.model-changed activities after a successful (re)start, and resolves “current” instance from the bound session when needed. ProviderService tolerates transient binding mismatches during handoff and dedupes overlapping live sessions per thread.

The web UI unlocks the model picker on started threads, only blocks same-driver/same-continuation switches for requiresNewThreadForModelChange providers, and shows inline “Switched from X to Y” timeline notices from those activities.

Reviewed by Cursor Bugbot for commit 853c8d5. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Allow switching provider/model mid-conversation with transcript handoff

  • When a user switches to a provider with a different driver or continuation group, the orchestrator starts a fresh session on the new provider instead of rejecting the change, prepending a [Conversation handoff] transcript prelude to the next turn so the new provider has prior context.
  • A thread.model-changed activity is appended to the thread on handoff; the UI renders this as a pill-styled notice row in the conversation timeline via a new NoticeTimelineRow component.
  • listSessions in ProviderService no longer dies on provider/instance binding mismatches; it returns at most one session per thread, preferring the authoritative binding match.
  • The getStartedThreadModelChangeBlockReason logic is relaxed so cross-instance switches are allowed unless both instances share the same driver and continuation group.
  • Resume cursors are cleared when rebinding to a different provider instance to prevent stale cursor reuse across drivers.
  • Risk: the handoff prelude is capped at 80,000 chars and will silently omit older history when truncated, which may cause the new provider to miss earlier context.

Macroscope summarized 853c8d5.

Threads were pinned to their initial provider once started: the server
rejected cross-driver/continuation switches and the UI locked the picker.
This lets a thread hand off to a different provider mid-conversation.
- Server: replace the switch-rejection guards in ensureSessionForThread
with a handoff path that starts a fresh session on the new provider
(dropping the old resume cursor), persists the new modelSelection, and
tolerates the transient binding mismatch during rebind.
- Handoff context: new providerHandoffTranscript renderer replays prior
user/assistant messages plus a structured tool/command trail (commands,
file edits, MCP/skill calls, exit codes, output) into the first turn on
the new provider, so it inherits the actual work done, not just prose.
- Notice: emit a thread.model-changed activity on model/provider change,
rendered as an inline "Switched from X to Y" timeline row above the
triggering user message.
- UI: unlock the model picker on started threads; keep the in-thread
block only for providers that require a new thread for a model change.
Adds no DB migrations. Server + web suites, typecheck, and lint pass;
verified end-to-end (cross-provider switch carries context, incl. hidden
command output and non-zero exit codes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitaiBot commented Jul 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: da904db3-b1f6-4b05-b0f9-69c5e67e6d8d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actionsgithub-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 8, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b747679. Configure here.

Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/web/src/components/ChatView.logic.ts Outdated
Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
Comment threadapps/server/src/provider/Layers/ProviderService.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
Comment threadapps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
@macroscopeapp

macroscopeappBot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a significant new feature enabling provider/model switching mid-conversation, changing previously blocked behavior to a complex handoff flow. The changes span orchestration, session management, and UI layers (~1300 additions), introducing new runtime behavior that warrants human review.

You can customize Macroscope's approvability policy. Learn more.

From Cursor Bugbot + Macroscope on the PR:
- Handoff replay no longer requires a prior assistant message; the renderer
already returns undefined when there's nothing to carry, so user-only /
tool-only threads keep their context on switch.
- Handoff detection prefers the last-bound session's provider over the
thread's stored modelSelection, so a switch is still detected after the
session is stopped/restarted (or the selection was advanced early).
- The "Switched model" notice is emitted only after the session is
confirmed (re)started, not before — a failed start no longer leaves a
stale success notice.
- Transcript extraction also reads stderr, data.rawOutput.exitCode, and
data.toolCallId (dedup key), so failed-command diagnostics and exit codes
survive the handoff and lifecycle events collapse correctly.
- listSessions dedupes by threadId, preferring the binding-matched session,
so a mid-handoff stale session can't produce two entries for one thread.
- getStartedThreadModelChangeBlockReason only treats a cross-instance switch
as an unblocked handoff when the driver/continuation actually differ;
same-driver+continuation instances stay blocked for providers that
require a new thread.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment threadapps/server/src/orchestration/providerHandoffTranscript.ts
collectToolTrailEntries kept the longest line per tool-call group, so a
verbose in-progress tool.updated could shadow the shorter tool.completed/
task.completed line that actually carries the exit code and final output.
The next provider then inherited stale partial tool state. Prefer the
terminal lifecycle event, falling back to longest line only on ties.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 333d44e)
@juliusmarminge

Copy link
Copy Markdown
Member

Closing in favor of #2829 (orchestration V2).

#2829 replaces the V1 orchestration engine wholesale, and the behavior this PR adds already landed there natively — so this change is a no-op against the new base.

Thanks for the work here, and sorry for the churn on a long-lived branch. Once #2829 merges, please rebase onto main and reopen if the behavior is still missing — happy to take it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL500-999 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Switch provider/model mid-conversation via transcript handoff

3 participants

@Brechard@juliusmarminge@rodrigoB-CB