fix(runtime): seal partial thinking before retrying mid-stream network cuts - #4393

Open
chinawch007 wants to merge 1 commit into
apache:mainfrom
chinawch007:fix-4284-sealed-thinking-network-retry
Open

fix(runtime): seal partial thinking before retrying mid-stream network cuts#4393
chinawch007 wants to merge 1 commit into
apache:mainfrom
chinawch007:fix-4284-sealed-thinking-network-retry

Conversation

@chinawch007

Copy link
Copy Markdown
Contributor

Summary

A thinking-only streaming attempt that died from a retryable network error
ended the turn, while the identical attempt state recovered when the 120s
idle watchdog fired first — in the #4284 incident an ECONNRESET landed
seconds before the watchdog would have, so the same fault got opposite
outcomes depending on which detector noticed it. Recovery safety depends on
what the attempt emitted, not on which side detected the cut.

The plain retryable path now accepts thinking-only attempts under the seal
contract the watchdog path already uses: flushStep() closes the partial
thinking as a message under its own id, the retry streams into a fresh id,
and the sealed fragment stays out of the retried request's provider context.
It carries its own budget (MAX_SEALED_THINKING_RETRIES_PER_STEP = 1) so a
gateway that systematically cuts long thinking streams fails fast instead of
accumulating fragments across the full attempt budget. The new
sealedThinkingRecovery flag is mutually exclusive with the other three
retry paths by construction.

Answer text, tool activity, and provider continuation metadata remain
non-retryable; the watchdog and truncated-stream paths are unchanged.

Fixes#4284

Verification

  • npm run format:check — clean (8 formatting nits auto-fixed and folded in)
  • npm run lint — 2304 files, no issues
  • npm run typecheck
  • npm --workspace @maka/runtime run build — clean
  • npm --workspace @maka/runtime run test:dist — 3107 tests, 3094 pass, 13 skip

Not run: the full root-level battery (npm run lint, format:check,
workspace-wide tests, knip).

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: ZCode — analyzed the issue, authored the implementation
and tests from that analysis under a plan reviewed by the contributor; every
step was human-reviewed and verified locally.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@github-actionsgithub-actionsBot added the effort/M Under 500 readable lines label Aug 31, 2026

@hqhq1025hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a P0-P3 correctness issue in the exact-head change. The new path permits only retryable failures whose physical request emitted thinking but no answer text, tool activity, continuation metadata, or completed step boundary; it seals that fragment under the current message id, rotates the id, and allows one such recovery per provider step. The added tests cover the successful retry, no-output retry, the one-fragment budget, and the continuation-metadata exclusion.

Validation on this exact head passed: clean install, npm run build:test, full repository typecheck, lint, format, git diff --check, and the complete Runtime suite (3081 passed, 13 skipped). The PR is currently CONFLICTING/DIRTY against main and has no hosted test result. I also performed a local semantic conflict resolution that preserves current main’s ordered reasoning-parts accumulator; the merged tree passed build:test, full typecheck, and the complete Runtime suite (3105 passed, 13 skipped). This is therefore a technical code-review GO for the reviewed head, not a merge-ready decision; the author still needs to rebase and obtain current-head hosted checks.

I did not exercise a real paid provider or reproduce the original macOS gateway incident.

Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.

@chinawch007
chinawch007force-pushed the fix-4284-sealed-thinking-network-retry branch from d046ea5 to 304e6cdCompareSeptember 1, 2026 04:05
@chinawch007

Copy link
Copy Markdown
ContributorAuthor

Thanks for your attention, resolved the conflicts.

…k cuts
A thinking-only attempt that failed with a retryable provider/network
error (the 2026-08-28 ECONNRESET incident: reset landed seconds before
the 120s idle watchdog would have) ended the turn, while the identical
attempt state recovered when the watchdog noticed the failure first.
Recovery safety depends on what the attempt emitted, not on which side
detected the cut: thinking is sealable, so the plain retryable path now
accepts thinking-only attempts with the same flushStep() +
fresh-message-id contract as the watchdog path, under its own budget of
one recovery per step so a systematically cutting gateway fails fast
instead of accumulating sealed fragments across the full attempt budget.
The sealed fragment stays out of the retried request's provider
context. Answer text, tool activity, and provider continuation metadata
remain non-retryable; the watchdog and truncated-stream paths are
unchanged.
Fixesapache#4284
Generated-by: ZCode
@chinawch007
chinawch007force-pushed the fix-4284-sealed-thinking-network-retry branch from 304e6cd to 99720efCompareSeptember 1, 2026 04:13
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/MUnder 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(runtime): Mid-stream ECONNRESET after thinking output is terminal, while idle-watchdog timeout on the same state recovers

2 participants

@chinawch007@hqhq1025
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

fix(runtime): seal partial thinking before retrying mid-stream network cuts - #4393

Open
chinawch007 wants to merge 1 commit into
apache:mainfrom
chinawch007:fix-4284-sealed-thinking-network-retry
Open

fix(runtime): seal partial thinking before retrying mid-stream network cuts#4393
chinawch007 wants to merge 1 commit into
apache:mainfrom
chinawch007:fix-4284-sealed-thinking-network-retry

Conversation

@chinawch007

Copy link
Copy Markdown
Contributor

Summary

A thinking-only streaming attempt that died from a retryable network error
ended the turn, while the identical attempt state recovered when the 120s
idle watchdog fired first — in the #4284 incident an ECONNRESET landed
seconds before the watchdog would have, so the same fault got opposite
outcomes depending on which detector noticed it. Recovery safety depends on
what the attempt emitted, not on which side detected the cut.

The plain retryable path now accepts thinking-only attempts under the seal
contract the watchdog path already uses: flushStep() closes the partial
thinking as a message under its own id, the retry streams into a fresh id,
and the sealed fragment stays out of the retried request's provider context.
It carries its own budget (MAX_SEALED_THINKING_RETRIES_PER_STEP = 1) so a
gateway that systematically cuts long thinking streams fails fast instead of
accumulating fragments across the full attempt budget. The new
sealedThinkingRecovery flag is mutually exclusive with the other three
retry paths by construction.

Answer text, tool activity, and provider continuation metadata remain
non-retryable; the watchdog and truncated-stream paths are unchanged.

Fixes#4284

Verification

  • npm run format:check — clean (8 formatting nits auto-fixed and folded in)
  • npm run lint — 2304 files, no issues
  • npm run typecheck
  • npm --workspace @maka/runtime run build — clean
  • npm --workspace @maka/runtime run test:dist — 3107 tests, 3094 pass, 13 skip

Not run: the full root-level battery (npm run lint, format:check,
workspace-wide tests, knip).

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: ZCode — analyzed the issue, authored the implementation
and tests from that analysis under a plan reviewed by the contributor; every
step was human-reviewed and verified locally.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@github-actionsgithub-actionsBot added the effort/M Under 500 readable lines label Aug 31, 2026

@hqhq1025hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a P0-P3 correctness issue in the exact-head change. The new path permits only retryable failures whose physical request emitted thinking but no answer text, tool activity, continuation metadata, or completed step boundary; it seals that fragment under the current message id, rotates the id, and allows one such recovery per provider step. The added tests cover the successful retry, no-output retry, the one-fragment budget, and the continuation-metadata exclusion.

Validation on this exact head passed: clean install, npm run build:test, full repository typecheck, lint, format, git diff --check, and the complete Runtime suite (3081 passed, 13 skipped). The PR is currently CONFLICTING/DIRTY against main and has no hosted test result. I also performed a local semantic conflict resolution that preserves current main’s ordered reasoning-parts accumulator; the merged tree passed build:test, full typecheck, and the complete Runtime suite (3105 passed, 13 skipped). This is therefore a technical code-review GO for the reviewed head, not a merge-ready decision; the author still needs to rebase and obtain current-head hosted checks.

I did not exercise a real paid provider or reproduce the original macOS gateway incident.

Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.

@chinawch007
chinawch007force-pushed the fix-4284-sealed-thinking-network-retry branch from d046ea5 to 304e6cdCompareSeptember 1, 2026 04:05
@chinawch007

Copy link
Copy Markdown
ContributorAuthor

Thanks for your attention, resolved the conflicts.

…k cuts
A thinking-only attempt that failed with a retryable provider/network
error (the 2026-08-28 ECONNRESET incident: reset landed seconds before
the 120s idle watchdog would have) ended the turn, while the identical
attempt state recovered when the watchdog noticed the failure first.
Recovery safety depends on what the attempt emitted, not on which side
detected the cut: thinking is sealable, so the plain retryable path now
accepts thinking-only attempts with the same flushStep() +
fresh-message-id contract as the watchdog path, under its own budget of
one recovery per step so a systematically cutting gateway fails fast
instead of accumulating sealed fragments across the full attempt budget.
The sealed fragment stays out of the retried request's provider
context. Answer text, tool activity, and provider continuation metadata
remain non-retryable; the watchdog and truncated-stream paths are
unchanged.
Fixesapache#4284
Generated-by: ZCode
@chinawch007
chinawch007force-pushed the fix-4284-sealed-thinking-network-retry branch from 304e6cd to 99720efCompareSeptember 1, 2026 04:13
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/MUnder 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(runtime): Mid-stream ECONNRESET after thinking output is terminal, while idle-watchdog timeout on the same state recovers

2 participants

@chinawch007@hqhq1025
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(runtime): seal partial thinking before retrying mid-stream network cuts - #4393

Open
chinawch007 wants to merge 1 commit into
apache:mainfrom
chinawch007:fix-4284-sealed-thinking-network-retry
Open

fix(runtime): seal partial thinking before retrying mid-stream network cuts#4393
chinawch007 wants to merge 1 commit into
apache:mainfrom
chinawch007:fix-4284-sealed-thinking-network-retry

Conversation

@chinawch007

Copy link
Copy Markdown
Contributor

Summary

A thinking-only streaming attempt that died from a retryable network error
ended the turn, while the identical attempt state recovered when the 120s
idle watchdog fired first — in the #4284 incident an ECONNRESET landed
seconds before the watchdog would have, so the same fault got opposite
outcomes depending on which detector noticed it. Recovery safety depends on
what the attempt emitted, not on which side detected the cut.

The plain retryable path now accepts thinking-only attempts under the seal
contract the watchdog path already uses: flushStep() closes the partial
thinking as a message under its own id, the retry streams into a fresh id,
and the sealed fragment stays out of the retried request's provider context.
It carries its own budget (MAX_SEALED_THINKING_RETRIES_PER_STEP = 1) so a
gateway that systematically cuts long thinking streams fails fast instead of
accumulating fragments across the full attempt budget. The new
sealedThinkingRecovery flag is mutually exclusive with the other three
retry paths by construction.

Answer text, tool activity, and provider continuation metadata remain
non-retryable; the watchdog and truncated-stream paths are unchanged.

Fixes#4284

Verification

  • npm run format:check — clean (8 formatting nits auto-fixed and folded in)
  • npm run lint — 2304 files, no issues
  • npm run typecheck
  • npm --workspace @maka/runtime run build — clean
  • npm --workspace @maka/runtime run test:dist — 3107 tests, 3094 pass, 13 skip

Not run: the full root-level battery (npm run lint, format:check,
workspace-wide tests, knip).

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: ZCode — analyzed the issue, authored the implementation
and tests from that analysis under a plan reviewed by the contributor; every
step was human-reviewed and verified locally.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@github-actionsgithub-actionsBot added the effort/M Under 500 readable lines label Aug 31, 2026

@hqhq1025hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a P0-P3 correctness issue in the exact-head change. The new path permits only retryable failures whose physical request emitted thinking but no answer text, tool activity, continuation metadata, or completed step boundary; it seals that fragment under the current message id, rotates the id, and allows one such recovery per provider step. The added tests cover the successful retry, no-output retry, the one-fragment budget, and the continuation-metadata exclusion.

Validation on this exact head passed: clean install, npm run build:test, full repository typecheck, lint, format, git diff --check, and the complete Runtime suite (3081 passed, 13 skipped). The PR is currently CONFLICTING/DIRTY against main and has no hosted test result. I also performed a local semantic conflict resolution that preserves current main’s ordered reasoning-parts accumulator; the merged tree passed build:test, full typecheck, and the complete Runtime suite (3105 passed, 13 skipped). This is therefore a technical code-review GO for the reviewed head, not a merge-ready decision; the author still needs to rebase and obtain current-head hosted checks.

I did not exercise a real paid provider or reproduce the original macOS gateway incident.

Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.

@chinawch007
chinawch007force-pushed the fix-4284-sealed-thinking-network-retry branch from d046ea5 to 304e6cdCompareSeptember 1, 2026 04:05
@chinawch007

Copy link
Copy Markdown
ContributorAuthor

Thanks for your attention, resolved the conflicts.

…k cuts
A thinking-only attempt that failed with a retryable provider/network
error (the 2026-08-28 ECONNRESET incident: reset landed seconds before
the 120s idle watchdog would have) ended the turn, while the identical
attempt state recovered when the watchdog noticed the failure first.
Recovery safety depends on what the attempt emitted, not on which side
detected the cut: thinking is sealable, so the plain retryable path now
accepts thinking-only attempts with the same flushStep() +
fresh-message-id contract as the watchdog path, under its own budget of
one recovery per step so a systematically cutting gateway fails fast
instead of accumulating sealed fragments across the full attempt budget.
The sealed fragment stays out of the retried request's provider
context. Answer text, tool activity, and provider continuation metadata
remain non-retryable; the watchdog and truncated-stream paths are
unchanged.
Fixesapache#4284
Generated-by: ZCode
@chinawch007
chinawch007force-pushed the fix-4284-sealed-thinking-network-retry branch from 304e6cd to 99720efCompareSeptember 1, 2026 04:13
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/MUnder 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(runtime): Mid-stream ECONNRESET after thinking output is terminal, while idle-watchdog timeout on the same state recovers

2 participants

@chinawch007@hqhq1025
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(runtime): seal partial thinking before retrying mid-stream network cuts - #4393

Open
chinawch007 wants to merge 1 commit into
apache:mainfrom
chinawch007:fix-4284-sealed-thinking-network-retry
Open

fix(runtime): seal partial thinking before retrying mid-stream network cuts#4393
chinawch007 wants to merge 1 commit into
apache:mainfrom
chinawch007:fix-4284-sealed-thinking-network-retry

Conversation

@chinawch007

Copy link
Copy Markdown
Contributor

Summary

A thinking-only streaming attempt that died from a retryable network error
ended the turn, while the identical attempt state recovered when the 120s
idle watchdog fired first — in the #4284 incident an ECONNRESET landed
seconds before the watchdog would have, so the same fault got opposite
outcomes depending on which detector noticed it. Recovery safety depends on
what the attempt emitted, not on which side detected the cut.

The plain retryable path now accepts thinking-only attempts under the seal
contract the watchdog path already uses: flushStep() closes the partial
thinking as a message under its own id, the retry streams into a fresh id,
and the sealed fragment stays out of the retried request's provider context.
It carries its own budget (MAX_SEALED_THINKING_RETRIES_PER_STEP = 1) so a
gateway that systematically cuts long thinking streams fails fast instead of
accumulating fragments across the full attempt budget. The new
sealedThinkingRecovery flag is mutually exclusive with the other three
retry paths by construction.

Answer text, tool activity, and provider continuation metadata remain
non-retryable; the watchdog and truncated-stream paths are unchanged.

Fixes#4284

Verification

  • npm run format:check — clean (8 formatting nits auto-fixed and folded in)
  • npm run lint — 2304 files, no issues
  • npm run typecheck
  • npm --workspace @maka/runtime run build — clean
  • npm --workspace @maka/runtime run test:dist — 3107 tests, 3094 pass, 13 skip

Not run: the full root-level battery (npm run lint, format:check,
workspace-wide tests, knip).

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: ZCode — analyzed the issue, authored the implementation
and tests from that analysis under a plan reviewed by the contributor; every
step was human-reviewed and verified locally.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@github-actionsgithub-actionsBot added the effort/M Under 500 readable lines label Aug 31, 2026

@hqhq1025hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a P0-P3 correctness issue in the exact-head change. The new path permits only retryable failures whose physical request emitted thinking but no answer text, tool activity, continuation metadata, or completed step boundary; it seals that fragment under the current message id, rotates the id, and allows one such recovery per provider step. The added tests cover the successful retry, no-output retry, the one-fragment budget, and the continuation-metadata exclusion.

Validation on this exact head passed: clean install, npm run build:test, full repository typecheck, lint, format, git diff --check, and the complete Runtime suite (3081 passed, 13 skipped). The PR is currently CONFLICTING/DIRTY against main and has no hosted test result. I also performed a local semantic conflict resolution that preserves current main’s ordered reasoning-parts accumulator; the merged tree passed build:test, full typecheck, and the complete Runtime suite (3105 passed, 13 skipped). This is therefore a technical code-review GO for the reviewed head, not a merge-ready decision; the author still needs to rebase and obtain current-head hosted checks.

I did not exercise a real paid provider or reproduce the original macOS gateway incident.

Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.

@chinawch007
chinawch007force-pushed the fix-4284-sealed-thinking-network-retry branch from d046ea5 to 304e6cdCompareSeptember 1, 2026 04:05
@chinawch007

Copy link
Copy Markdown
ContributorAuthor

Thanks for your attention, resolved the conflicts.

…k cuts
A thinking-only attempt that failed with a retryable provider/network
error (the 2026-08-28 ECONNRESET incident: reset landed seconds before
the 120s idle watchdog would have) ended the turn, while the identical
attempt state recovered when the watchdog noticed the failure first.
Recovery safety depends on what the attempt emitted, not on which side
detected the cut: thinking is sealable, so the plain retryable path now
accepts thinking-only attempts with the same flushStep() +
fresh-message-id contract as the watchdog path, under its own budget of
one recovery per step so a systematically cutting gateway fails fast
instead of accumulating sealed fragments across the full attempt budget.
The sealed fragment stays out of the retried request's provider
context. Answer text, tool activity, and provider continuation metadata
remain non-retryable; the watchdog and truncated-stream paths are
unchanged.
Fixesapache#4284
Generated-by: ZCode
@chinawch007
chinawch007force-pushed the fix-4284-sealed-thinking-network-retry branch from 304e6cd to 99720efCompareSeptember 1, 2026 04:13
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/MUnder 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(runtime): Mid-stream ECONNRESET after thinking output is terminal, while idle-watchdog timeout on the same state recovers

2 participants

@chinawch007@hqhq1025
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

fix(runtime): seal partial thinking before retrying mid-stream network cuts - #4393

Open
chinawch007 wants to merge 1 commit into
apache:mainfrom
chinawch007:fix-4284-sealed-thinking-network-retry
Open

fix(runtime): seal partial thinking before retrying mid-stream network cuts#4393
chinawch007 wants to merge 1 commit into
apache:mainfrom
chinawch007:fix-4284-sealed-thinking-network-retry

Conversation

@chinawch007

Copy link
Copy Markdown
Contributor

Summary

A thinking-only streaming attempt that died from a retryable network error
ended the turn, while the identical attempt state recovered when the 120s
idle watchdog fired first — in the #4284 incident an ECONNRESET landed
seconds before the watchdog would have, so the same fault got opposite
outcomes depending on which detector noticed it. Recovery safety depends on
what the attempt emitted, not on which side detected the cut.

The plain retryable path now accepts thinking-only attempts under the seal
contract the watchdog path already uses: flushStep() closes the partial
thinking as a message under its own id, the retry streams into a fresh id,
and the sealed fragment stays out of the retried request's provider context.
It carries its own budget (MAX_SEALED_THINKING_RETRIES_PER_STEP = 1) so a
gateway that systematically cuts long thinking streams fails fast instead of
accumulating fragments across the full attempt budget. The new
sealedThinkingRecovery flag is mutually exclusive with the other three
retry paths by construction.

Answer text, tool activity, and provider continuation metadata remain
non-retryable; the watchdog and truncated-stream paths are unchanged.

Fixes#4284

Verification

  • npm run format:check — clean (8 formatting nits auto-fixed and folded in)
  • npm run lint — 2304 files, no issues
  • npm run typecheck
  • npm --workspace @maka/runtime run build — clean
  • npm --workspace @maka/runtime run test:dist — 3107 tests, 3094 pass, 13 skip

Not run: the full root-level battery (npm run lint, format:check,
workspace-wide tests, knip).

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: ZCode — analyzed the issue, authored the implementation
and tests from that analysis under a plan reviewed by the contributor; every
step was human-reviewed and verified locally.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@github-actionsgithub-actionsBot added the effort/M Under 500 readable lines label Aug 31, 2026

@hqhq1025hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a P0-P3 correctness issue in the exact-head change. The new path permits only retryable failures whose physical request emitted thinking but no answer text, tool activity, continuation metadata, or completed step boundary; it seals that fragment under the current message id, rotates the id, and allows one such recovery per provider step. The added tests cover the successful retry, no-output retry, the one-fragment budget, and the continuation-metadata exclusion.

Validation on this exact head passed: clean install, npm run build:test, full repository typecheck, lint, format, git diff --check, and the complete Runtime suite (3081 passed, 13 skipped). The PR is currently CONFLICTING/DIRTY against main and has no hosted test result. I also performed a local semantic conflict resolution that preserves current main’s ordered reasoning-parts accumulator; the merged tree passed build:test, full typecheck, and the complete Runtime suite (3105 passed, 13 skipped). This is therefore a technical code-review GO for the reviewed head, not a merge-ready decision; the author still needs to rebase and obtain current-head hosted checks.

I did not exercise a real paid provider or reproduce the original macOS gateway incident.

Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.

@chinawch007
chinawch007force-pushed the fix-4284-sealed-thinking-network-retry branch from d046ea5 to 304e6cdCompareSeptember 1, 2026 04:05
@chinawch007

Copy link
Copy Markdown
ContributorAuthor

Thanks for your attention, resolved the conflicts.

…k cuts
A thinking-only attempt that failed with a retryable provider/network
error (the 2026-08-28 ECONNRESET incident: reset landed seconds before
the 120s idle watchdog would have) ended the turn, while the identical
attempt state recovered when the watchdog noticed the failure first.
Recovery safety depends on what the attempt emitted, not on which side
detected the cut: thinking is sealable, so the plain retryable path now
accepts thinking-only attempts with the same flushStep() +
fresh-message-id contract as the watchdog path, under its own budget of
one recovery per step so a systematically cutting gateway fails fast
instead of accumulating sealed fragments across the full attempt budget.
The sealed fragment stays out of the retried request's provider
context. Answer text, tool activity, and provider continuation metadata
remain non-retryable; the watchdog and truncated-stream paths are
unchanged.
Fixesapache#4284
Generated-by: ZCode
@chinawch007
chinawch007force-pushed the fix-4284-sealed-thinking-network-retry branch from 304e6cd to 99720efCompareSeptember 1, 2026 04:13
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/MUnder 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(runtime): Mid-stream ECONNRESET after thinking output is terminal, while idle-watchdog timeout on the same state recovers

2 participants

@chinawch007@hqhq1025
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(runtime): seal partial thinking before retrying mid-stream network cuts - #4393

Open
chinawch007 wants to merge 1 commit into
apache:mainfrom
chinawch007:fix-4284-sealed-thinking-network-retry
Open

fix(runtime): seal partial thinking before retrying mid-stream network cuts#4393
chinawch007 wants to merge 1 commit into
apache:mainfrom
chinawch007:fix-4284-sealed-thinking-network-retry

Conversation

@chinawch007

Copy link
Copy Markdown
Contributor

Summary

A thinking-only streaming attempt that died from a retryable network error
ended the turn, while the identical attempt state recovered when the 120s
idle watchdog fired first — in the #4284 incident an ECONNRESET landed
seconds before the watchdog would have, so the same fault got opposite
outcomes depending on which detector noticed it. Recovery safety depends on
what the attempt emitted, not on which side detected the cut.

The plain retryable path now accepts thinking-only attempts under the seal
contract the watchdog path already uses: flushStep() closes the partial
thinking as a message under its own id, the retry streams into a fresh id,
and the sealed fragment stays out of the retried request's provider context.
It carries its own budget (MAX_SEALED_THINKING_RETRIES_PER_STEP = 1) so a
gateway that systematically cuts long thinking streams fails fast instead of
accumulating fragments across the full attempt budget. The new
sealedThinkingRecovery flag is mutually exclusive with the other three
retry paths by construction.

Answer text, tool activity, and provider continuation metadata remain
non-retryable; the watchdog and truncated-stream paths are unchanged.

Fixes#4284

Verification

  • npm run format:check — clean (8 formatting nits auto-fixed and folded in)
  • npm run lint — 2304 files, no issues
  • npm run typecheck
  • npm --workspace @maka/runtime run build — clean
  • npm --workspace @maka/runtime run test:dist — 3107 tests, 3094 pass, 13 skip

Not run: the full root-level battery (npm run lint, format:check,
workspace-wide tests, knip).

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: ZCode — analyzed the issue, authored the implementation
and tests from that analysis under a plan reviewed by the contributor; every
step was human-reviewed and verified locally.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@github-actionsgithub-actionsBot added the effort/M Under 500 readable lines label Aug 31, 2026

@hqhq1025hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a P0-P3 correctness issue in the exact-head change. The new path permits only retryable failures whose physical request emitted thinking but no answer text, tool activity, continuation metadata, or completed step boundary; it seals that fragment under the current message id, rotates the id, and allows one such recovery per provider step. The added tests cover the successful retry, no-output retry, the one-fragment budget, and the continuation-metadata exclusion.

Validation on this exact head passed: clean install, npm run build:test, full repository typecheck, lint, format, git diff --check, and the complete Runtime suite (3081 passed, 13 skipped). The PR is currently CONFLICTING/DIRTY against main and has no hosted test result. I also performed a local semantic conflict resolution that preserves current main’s ordered reasoning-parts accumulator; the merged tree passed build:test, full typecheck, and the complete Runtime suite (3105 passed, 13 skipped). This is therefore a technical code-review GO for the reviewed head, not a merge-ready decision; the author still needs to rebase and obtain current-head hosted checks.

I did not exercise a real paid provider or reproduce the original macOS gateway incident.

Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.

@chinawch007
chinawch007force-pushed the fix-4284-sealed-thinking-network-retry branch from d046ea5 to 304e6cdCompareSeptember 1, 2026 04:05
@chinawch007

Copy link
Copy Markdown
ContributorAuthor

Thanks for your attention, resolved the conflicts.

…k cuts
A thinking-only attempt that failed with a retryable provider/network
error (the 2026-08-28 ECONNRESET incident: reset landed seconds before
the 120s idle watchdog would have) ended the turn, while the identical
attempt state recovered when the watchdog noticed the failure first.
Recovery safety depends on what the attempt emitted, not on which side
detected the cut: thinking is sealable, so the plain retryable path now
accepts thinking-only attempts with the same flushStep() +
fresh-message-id contract as the watchdog path, under its own budget of
one recovery per step so a systematically cutting gateway fails fast
instead of accumulating sealed fragments across the full attempt budget.
The sealed fragment stays out of the retried request's provider
context. Answer text, tool activity, and provider continuation metadata
remain non-retryable; the watchdog and truncated-stream paths are
unchanged.
Fixesapache#4284
Generated-by: ZCode
@chinawch007
chinawch007force-pushed the fix-4284-sealed-thinking-network-retry branch from 304e6cd to 99720efCompareSeptember 1, 2026 04:13
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/MUnder 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(runtime): Mid-stream ECONNRESET after thinking output is terminal, while idle-watchdog timeout on the same state recovers

2 participants

@chinawch007@hqhq1025
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(runtime): seal partial thinking before retrying mid-stream network cuts - #4393

Open
chinawch007 wants to merge 1 commit into
apache:mainfrom
chinawch007:fix-4284-sealed-thinking-network-retry
Open

fix(runtime): seal partial thinking before retrying mid-stream network cuts#4393
chinawch007 wants to merge 1 commit into
apache:mainfrom
chinawch007:fix-4284-sealed-thinking-network-retry

Conversation

@chinawch007

Copy link
Copy Markdown
Contributor

Summary

A thinking-only streaming attempt that died from a retryable network error
ended the turn, while the identical attempt state recovered when the 120s
idle watchdog fired first — in the #4284 incident an ECONNRESET landed
seconds before the watchdog would have, so the same fault got opposite
outcomes depending on which detector noticed it. Recovery safety depends on
what the attempt emitted, not on which side detected the cut.

The plain retryable path now accepts thinking-only attempts under the seal
contract the watchdog path already uses: flushStep() closes the partial
thinking as a message under its own id, the retry streams into a fresh id,
and the sealed fragment stays out of the retried request's provider context.
It carries its own budget (MAX_SEALED_THINKING_RETRIES_PER_STEP = 1) so a
gateway that systematically cuts long thinking streams fails fast instead of
accumulating fragments across the full attempt budget. The new
sealedThinkingRecovery flag is mutually exclusive with the other three
retry paths by construction.

Answer text, tool activity, and provider continuation metadata remain
non-retryable; the watchdog and truncated-stream paths are unchanged.

Fixes#4284

Verification

  • npm run format:check — clean (8 formatting nits auto-fixed and folded in)
  • npm run lint — 2304 files, no issues
  • npm run typecheck
  • npm --workspace @maka/runtime run build — clean
  • npm --workspace @maka/runtime run test:dist — 3107 tests, 3094 pass, 13 skip

Not run: the full root-level battery (npm run lint, format:check,
workspace-wide tests, knip).

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: ZCode — analyzed the issue, authored the implementation
and tests from that analysis under a plan reviewed by the contributor; every
step was human-reviewed and verified locally.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@github-actionsgithub-actionsBot added the effort/M Under 500 readable lines label Aug 31, 2026

@hqhq1025hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a P0-P3 correctness issue in the exact-head change. The new path permits only retryable failures whose physical request emitted thinking but no answer text, tool activity, continuation metadata, or completed step boundary; it seals that fragment under the current message id, rotates the id, and allows one such recovery per provider step. The added tests cover the successful retry, no-output retry, the one-fragment budget, and the continuation-metadata exclusion.

Validation on this exact head passed: clean install, npm run build:test, full repository typecheck, lint, format, git diff --check, and the complete Runtime suite (3081 passed, 13 skipped). The PR is currently CONFLICTING/DIRTY against main and has no hosted test result. I also performed a local semantic conflict resolution that preserves current main’s ordered reasoning-parts accumulator; the merged tree passed build:test, full typecheck, and the complete Runtime suite (3105 passed, 13 skipped). This is therefore a technical code-review GO for the reviewed head, not a merge-ready decision; the author still needs to rebase and obtain current-head hosted checks.

I did not exercise a real paid provider or reproduce the original macOS gateway incident.

Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.

@chinawch007
chinawch007force-pushed the fix-4284-sealed-thinking-network-retry branch from d046ea5 to 304e6cdCompareSeptember 1, 2026 04:05
@chinawch007

Copy link
Copy Markdown
ContributorAuthor

Thanks for your attention, resolved the conflicts.

…k cuts
A thinking-only attempt that failed with a retryable provider/network
error (the 2026-08-28 ECONNRESET incident: reset landed seconds before
the 120s idle watchdog would have) ended the turn, while the identical
attempt state recovered when the watchdog noticed the failure first.
Recovery safety depends on what the attempt emitted, not on which side
detected the cut: thinking is sealable, so the plain retryable path now
accepts thinking-only attempts with the same flushStep() +
fresh-message-id contract as the watchdog path, under its own budget of
one recovery per step so a systematically cutting gateway fails fast
instead of accumulating sealed fragments across the full attempt budget.
The sealed fragment stays out of the retried request's provider
context. Answer text, tool activity, and provider continuation metadata
remain non-retryable; the watchdog and truncated-stream paths are
unchanged.
Fixesapache#4284
Generated-by: ZCode
@chinawch007
chinawch007force-pushed the fix-4284-sealed-thinking-network-retry branch from 304e6cd to 99720efCompareSeptember 1, 2026 04:13
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/MUnder 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(runtime): Mid-stream ECONNRESET after thinking output is terminal, while idle-watchdog timeout on the same state recovers

2 participants

@chinawch007@hqhq1025
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

fix(runtime): seal partial thinking before retrying mid-stream network cuts - #4393

Open
chinawch007 wants to merge 1 commit into
apache:mainfrom
chinawch007:fix-4284-sealed-thinking-network-retry
Open

fix(runtime): seal partial thinking before retrying mid-stream network cuts#4393
chinawch007 wants to merge 1 commit into
apache:mainfrom
chinawch007:fix-4284-sealed-thinking-network-retry

Conversation

@chinawch007

Copy link
Copy Markdown
Contributor

Summary

A thinking-only streaming attempt that died from a retryable network error
ended the turn, while the identical attempt state recovered when the 120s
idle watchdog fired first — in the #4284 incident an ECONNRESET landed
seconds before the watchdog would have, so the same fault got opposite
outcomes depending on which detector noticed it. Recovery safety depends on
what the attempt emitted, not on which side detected the cut.

The plain retryable path now accepts thinking-only attempts under the seal
contract the watchdog path already uses: flushStep() closes the partial
thinking as a message under its own id, the retry streams into a fresh id,
and the sealed fragment stays out of the retried request's provider context.
It carries its own budget (MAX_SEALED_THINKING_RETRIES_PER_STEP = 1) so a
gateway that systematically cuts long thinking streams fails fast instead of
accumulating fragments across the full attempt budget. The new
sealedThinkingRecovery flag is mutually exclusive with the other three
retry paths by construction.

Answer text, tool activity, and provider continuation metadata remain
non-retryable; the watchdog and truncated-stream paths are unchanged.

Fixes#4284

Verification

  • npm run format:check — clean (8 formatting nits auto-fixed and folded in)
  • npm run lint — 2304 files, no issues
  • npm run typecheck
  • npm --workspace @maka/runtime run build — clean
  • npm --workspace @maka/runtime run test:dist — 3107 tests, 3094 pass, 13 skip

Not run: the full root-level battery (npm run lint, format:check,
workspace-wide tests, knip).

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: ZCode — analyzed the issue, authored the implementation
and tests from that analysis under a plan reviewed by the contributor; every
step was human-reviewed and verified locally.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@github-actionsgithub-actionsBot added the effort/M Under 500 readable lines label Aug 31, 2026

@hqhq1025hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find a P0-P3 correctness issue in the exact-head change. The new path permits only retryable failures whose physical request emitted thinking but no answer text, tool activity, continuation metadata, or completed step boundary; it seals that fragment under the current message id, rotates the id, and allows one such recovery per provider step. The added tests cover the successful retry, no-output retry, the one-fragment budget, and the continuation-metadata exclusion.

Validation on this exact head passed: clean install, npm run build:test, full repository typecheck, lint, format, git diff --check, and the complete Runtime suite (3081 passed, 13 skipped). The PR is currently CONFLICTING/DIRTY against main and has no hosted test result. I also performed a local semantic conflict resolution that preserves current main’s ordered reasoning-parts accumulator; the merged tree passed build:test, full typecheck, and the complete Runtime suite (3105 passed, 13 skipped). This is therefore a technical code-review GO for the reviewed head, not a merge-ready decision; the author still needs to rebase and obtain current-head hosted checks.

I did not exercise a real paid provider or reproduce the original macOS gateway incident.

Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.

@chinawch007
chinawch007force-pushed the fix-4284-sealed-thinking-network-retry branch from d046ea5 to 304e6cdCompareSeptember 1, 2026 04:05
@chinawch007

Copy link
Copy Markdown
ContributorAuthor

Thanks for your attention, resolved the conflicts.

…k cuts
A thinking-only attempt that failed with a retryable provider/network
error (the 2026-08-28 ECONNRESET incident: reset landed seconds before
the 120s idle watchdog would have) ended the turn, while the identical
attempt state recovered when the watchdog noticed the failure first.
Recovery safety depends on what the attempt emitted, not on which side
detected the cut: thinking is sealable, so the plain retryable path now
accepts thinking-only attempts with the same flushStep() +
fresh-message-id contract as the watchdog path, under its own budget of
one recovery per step so a systematically cutting gateway fails fast
instead of accumulating sealed fragments across the full attempt budget.
The sealed fragment stays out of the retried request's provider
context. Answer text, tool activity, and provider continuation metadata
remain non-retryable; the watchdog and truncated-stream paths are
unchanged.
Fixesapache#4284
Generated-by: ZCode
@chinawch007
chinawch007force-pushed the fix-4284-sealed-thinking-network-retry branch from 304e6cd to 99720efCompareSeptember 1, 2026 04:13
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/MUnder 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(runtime): Mid-stream ECONNRESET after thinking output is terminal, while idle-watchdog timeout on the same state recovers

2 participants

@chinawch007@hqhq1025