Fix OleTxTests.Recovery CI timeout by bounding remote process wall-clock time - #125813

Merged
danmoseley merged 4 commits into
mainfrom
copilot/fix-recovery-timeout-error
Mar 20, 2026
Merged

Fix OleTxTests.Recovery CI timeout by bounding remote process wall-clock time#125813
danmoseley merged 4 commits into
mainfrom
copilot/fix-recovery-timeout-error

Conversation

CopilotAI commented Mar 20, 2026

Copy link
Copy Markdown
Contributor

Description

OleTxTests.Recovery repeatedly times out the CI work item (the entire System.Transactions.Local.Tests work item is killed). The CI error gives no detail beyond the timeout — no stack traces or indication of where the hang occurs.

The most likely scenario is that MSDTC is unresponsive on certain CI machines, causing the child process spawned by RemoteExecutor.Invoke to hang (either stuck in MSDTC COM interop, or sitting in its intentional Thread.Sleep waiting for a commit callback that never arrives). Since RemoteInvokeOptions.TimeOut defaults to Timeout.Infinite, RemoteInvokeHandle.Dispose() blocks indefinitely waiting for the child to exit, and the test never completes.

It's also possible the main test thread hangs in tx.Commit() or one of the WaitOne(Timeout) calls. This change would not help in that case — only the child process is bounded. If the timeout continues to repro after this change, the main thread is the more likely culprit and would need its own timeout or the test should be skipped on affected CI legs.

Change:

  • Add TimeOut = 120_000 to RemoteInvokeOptions so RemoteInvokeHandle.Dispose() will force-kill the child process after 2 minutes (with a diagnostic dump at the halfway point) instead of blocking indefinitely. This bounds the test's wall-clock time and should prevent it from consuming the entire work item timeout budget.

Changes

  • src/libraries/System.Transactions.Local/tests/OleTxTests.cs

Testing

This change targets a flaky/hanging test; no new test logic is added. The fix will be validated by the reduction in CI work item timeouts on Windows runs. The test requires MSDTC and cannot be run locally in most environments (the ConditionalFact skips it when MSDTC is unavailable).

…all-clock time
Co-authored-by: danmoseley <6385855+danmoseley@users.noreply.github.com>
CopilotAI changed the title [WIP] Fix recovery timeout error in OleTx testsFix OleTxTests.Recovery CI timeout by bounding remote process and test wall-clock timeMar 20, 2026
CopilotAI requested a review from danmoseleyMarch 20, 2026 00:37
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @SamMonoRT, @dotnet/efteam
See info in area-owners.md if you want to be subscribed.

@danmoseley
danmoseley requested a review from CopilotMarch 20, 2026 00:50

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@danmoseley
danmoseley requested a review from CopilotMarch 20, 2026 01:16

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

Comment threadsrc/libraries/System.Transactions.Local/tests/OleTxTests.cs Outdated
Comment threadsrc/libraries/System.Transactions.Local/tests/OleTxTests.cs
…out on async methods)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseleydanmoseley changed the title Fix OleTxTests.Recovery CI timeout by bounding remote process and test wall-clock timeFix OleTxTests.Recovery CI timeout by bounding remote process wall-clock timeMar 20, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseley
danmoseley marked this pull request as ready for review March 20, 2026 02:19
CopilotAI review requested due to automatic review settings March 20, 2026 02:19

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@AndriySvyryd
AndriySvyryd requested a review from rojiMarch 20, 2026 20:47
@danmoseley
danmoseley merged commit 4521881 into mainMar 20, 2026
83 of 92 checks passed
@danmoseley
danmoseley deleted the copilot/fix-recovery-timeout-error branch March 20, 2026 21:52
danmoseley pushed a commit that referenced this pull request Apr 8, 2026
The Recovery test consistently times out (20+ min) under stress modes
(fullpgo, jitstress2_jitstressregs) — confirmed from Helix logs across
multiple hits. PR #125813 added a 120s child-process timeout but the
main thread still hangs waiting for MSDTC under slow runtimes.
This is a libraries-level test exercising MSDTC/OLE transaction recovery;
running it under JIT stress provides no additional signal. Skip it on
all non-regular CoreCLR test modes.
Fixes#126304
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Apr 20, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

System.Transactions.Tests.OleTxTests.Recovery timeout

5 participants

@danmoseley@AndriySvyryd@jkotas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Fix OleTxTests.Recovery CI timeout by bounding remote process wall-clock time - #125813

Merged
danmoseley merged 4 commits into
mainfrom
copilot/fix-recovery-timeout-error
Mar 20, 2026
Merged

Fix OleTxTests.Recovery CI timeout by bounding remote process wall-clock time#125813
danmoseley merged 4 commits into
mainfrom
copilot/fix-recovery-timeout-error

Conversation

CopilotAI commented Mar 20, 2026

Copy link
Copy Markdown
Contributor

Description

OleTxTests.Recovery repeatedly times out the CI work item (the entire System.Transactions.Local.Tests work item is killed). The CI error gives no detail beyond the timeout — no stack traces or indication of where the hang occurs.

The most likely scenario is that MSDTC is unresponsive on certain CI machines, causing the child process spawned by RemoteExecutor.Invoke to hang (either stuck in MSDTC COM interop, or sitting in its intentional Thread.Sleep waiting for a commit callback that never arrives). Since RemoteInvokeOptions.TimeOut defaults to Timeout.Infinite, RemoteInvokeHandle.Dispose() blocks indefinitely waiting for the child to exit, and the test never completes.

It's also possible the main test thread hangs in tx.Commit() or one of the WaitOne(Timeout) calls. This change would not help in that case — only the child process is bounded. If the timeout continues to repro after this change, the main thread is the more likely culprit and would need its own timeout or the test should be skipped on affected CI legs.

Change:

  • Add TimeOut = 120_000 to RemoteInvokeOptions so RemoteInvokeHandle.Dispose() will force-kill the child process after 2 minutes (with a diagnostic dump at the halfway point) instead of blocking indefinitely. This bounds the test's wall-clock time and should prevent it from consuming the entire work item timeout budget.

Changes

  • src/libraries/System.Transactions.Local/tests/OleTxTests.cs

Testing

This change targets a flaky/hanging test; no new test logic is added. The fix will be validated by the reduction in CI work item timeouts on Windows runs. The test requires MSDTC and cannot be run locally in most environments (the ConditionalFact skips it when MSDTC is unavailable).

…all-clock time
Co-authored-by: danmoseley <6385855+danmoseley@users.noreply.github.com>
CopilotAI changed the title [WIP] Fix recovery timeout error in OleTx testsFix OleTxTests.Recovery CI timeout by bounding remote process and test wall-clock timeMar 20, 2026
CopilotAI requested a review from danmoseleyMarch 20, 2026 00:37
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @SamMonoRT, @dotnet/efteam
See info in area-owners.md if you want to be subscribed.

@danmoseley
danmoseley requested a review from CopilotMarch 20, 2026 00:50

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@danmoseley
danmoseley requested a review from CopilotMarch 20, 2026 01:16

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

Comment threadsrc/libraries/System.Transactions.Local/tests/OleTxTests.cs Outdated
Comment threadsrc/libraries/System.Transactions.Local/tests/OleTxTests.cs
…out on async methods)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseleydanmoseley changed the title Fix OleTxTests.Recovery CI timeout by bounding remote process and test wall-clock timeFix OleTxTests.Recovery CI timeout by bounding remote process wall-clock timeMar 20, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseley
danmoseley marked this pull request as ready for review March 20, 2026 02:19
CopilotAI review requested due to automatic review settings March 20, 2026 02:19

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@AndriySvyryd
AndriySvyryd requested a review from rojiMarch 20, 2026 20:47
@danmoseley
danmoseley merged commit 4521881 into mainMar 20, 2026
83 of 92 checks passed
@danmoseley
danmoseley deleted the copilot/fix-recovery-timeout-error branch March 20, 2026 21:52
danmoseley pushed a commit that referenced this pull request Apr 8, 2026
The Recovery test consistently times out (20+ min) under stress modes
(fullpgo, jitstress2_jitstressregs) — confirmed from Helix logs across
multiple hits. PR #125813 added a 120s child-process timeout but the
main thread still hangs waiting for MSDTC under slow runtimes.
This is a libraries-level test exercising MSDTC/OLE transaction recovery;
running it under JIT stress provides no additional signal. Skip it on
all non-regular CoreCLR test modes.
Fixes#126304
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Apr 20, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

System.Transactions.Tests.OleTxTests.Recovery timeout

5 participants

@danmoseley@AndriySvyryd@jkotas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix OleTxTests.Recovery CI timeout by bounding remote process wall-clock time - #125813

Merged
danmoseley merged 4 commits into
mainfrom
copilot/fix-recovery-timeout-error
Mar 20, 2026
Merged

Fix OleTxTests.Recovery CI timeout by bounding remote process wall-clock time#125813
danmoseley merged 4 commits into
mainfrom
copilot/fix-recovery-timeout-error

Conversation

CopilotAI commented Mar 20, 2026

Copy link
Copy Markdown
Contributor

Description

OleTxTests.Recovery repeatedly times out the CI work item (the entire System.Transactions.Local.Tests work item is killed). The CI error gives no detail beyond the timeout — no stack traces or indication of where the hang occurs.

The most likely scenario is that MSDTC is unresponsive on certain CI machines, causing the child process spawned by RemoteExecutor.Invoke to hang (either stuck in MSDTC COM interop, or sitting in its intentional Thread.Sleep waiting for a commit callback that never arrives). Since RemoteInvokeOptions.TimeOut defaults to Timeout.Infinite, RemoteInvokeHandle.Dispose() blocks indefinitely waiting for the child to exit, and the test never completes.

It's also possible the main test thread hangs in tx.Commit() or one of the WaitOne(Timeout) calls. This change would not help in that case — only the child process is bounded. If the timeout continues to repro after this change, the main thread is the more likely culprit and would need its own timeout or the test should be skipped on affected CI legs.

Change:

  • Add TimeOut = 120_000 to RemoteInvokeOptions so RemoteInvokeHandle.Dispose() will force-kill the child process after 2 minutes (with a diagnostic dump at the halfway point) instead of blocking indefinitely. This bounds the test's wall-clock time and should prevent it from consuming the entire work item timeout budget.

Changes

  • src/libraries/System.Transactions.Local/tests/OleTxTests.cs

Testing

This change targets a flaky/hanging test; no new test logic is added. The fix will be validated by the reduction in CI work item timeouts on Windows runs. The test requires MSDTC and cannot be run locally in most environments (the ConditionalFact skips it when MSDTC is unavailable).

…all-clock time
Co-authored-by: danmoseley <6385855+danmoseley@users.noreply.github.com>
CopilotAI changed the title [WIP] Fix recovery timeout error in OleTx testsFix OleTxTests.Recovery CI timeout by bounding remote process and test wall-clock timeMar 20, 2026
CopilotAI requested a review from danmoseleyMarch 20, 2026 00:37
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @SamMonoRT, @dotnet/efteam
See info in area-owners.md if you want to be subscribed.

@danmoseley
danmoseley requested a review from CopilotMarch 20, 2026 00:50

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@danmoseley
danmoseley requested a review from CopilotMarch 20, 2026 01:16

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

Comment threadsrc/libraries/System.Transactions.Local/tests/OleTxTests.cs Outdated
Comment threadsrc/libraries/System.Transactions.Local/tests/OleTxTests.cs
…out on async methods)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseleydanmoseley changed the title Fix OleTxTests.Recovery CI timeout by bounding remote process and test wall-clock timeFix OleTxTests.Recovery CI timeout by bounding remote process wall-clock timeMar 20, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseley
danmoseley marked this pull request as ready for review March 20, 2026 02:19
CopilotAI review requested due to automatic review settings March 20, 2026 02:19

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@AndriySvyryd
AndriySvyryd requested a review from rojiMarch 20, 2026 20:47
@danmoseley
danmoseley merged commit 4521881 into mainMar 20, 2026
83 of 92 checks passed
@danmoseley
danmoseley deleted the copilot/fix-recovery-timeout-error branch March 20, 2026 21:52
danmoseley pushed a commit that referenced this pull request Apr 8, 2026
The Recovery test consistently times out (20+ min) under stress modes
(fullpgo, jitstress2_jitstressregs) — confirmed from Helix logs across
multiple hits. PR #125813 added a 120s child-process timeout but the
main thread still hangs waiting for MSDTC under slow runtimes.
This is a libraries-level test exercising MSDTC/OLE transaction recovery;
running it under JIT stress provides no additional signal. Skip it on
all non-regular CoreCLR test modes.
Fixes#126304
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Apr 20, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

System.Transactions.Tests.OleTxTests.Recovery timeout

5 participants

@danmoseley@AndriySvyryd@jkotas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix OleTxTests.Recovery CI timeout by bounding remote process wall-clock time - #125813

Merged
danmoseley merged 4 commits into
mainfrom
copilot/fix-recovery-timeout-error
Mar 20, 2026
Merged

Fix OleTxTests.Recovery CI timeout by bounding remote process wall-clock time#125813
danmoseley merged 4 commits into
mainfrom
copilot/fix-recovery-timeout-error

Conversation

CopilotAI commented Mar 20, 2026

Copy link
Copy Markdown
Contributor

Description

OleTxTests.Recovery repeatedly times out the CI work item (the entire System.Transactions.Local.Tests work item is killed). The CI error gives no detail beyond the timeout — no stack traces or indication of where the hang occurs.

The most likely scenario is that MSDTC is unresponsive on certain CI machines, causing the child process spawned by RemoteExecutor.Invoke to hang (either stuck in MSDTC COM interop, or sitting in its intentional Thread.Sleep waiting for a commit callback that never arrives). Since RemoteInvokeOptions.TimeOut defaults to Timeout.Infinite, RemoteInvokeHandle.Dispose() blocks indefinitely waiting for the child to exit, and the test never completes.

It's also possible the main test thread hangs in tx.Commit() or one of the WaitOne(Timeout) calls. This change would not help in that case — only the child process is bounded. If the timeout continues to repro after this change, the main thread is the more likely culprit and would need its own timeout or the test should be skipped on affected CI legs.

Change:

  • Add TimeOut = 120_000 to RemoteInvokeOptions so RemoteInvokeHandle.Dispose() will force-kill the child process after 2 minutes (with a diagnostic dump at the halfway point) instead of blocking indefinitely. This bounds the test's wall-clock time and should prevent it from consuming the entire work item timeout budget.

Changes

  • src/libraries/System.Transactions.Local/tests/OleTxTests.cs

Testing

This change targets a flaky/hanging test; no new test logic is added. The fix will be validated by the reduction in CI work item timeouts on Windows runs. The test requires MSDTC and cannot be run locally in most environments (the ConditionalFact skips it when MSDTC is unavailable).

…all-clock time
Co-authored-by: danmoseley <6385855+danmoseley@users.noreply.github.com>
CopilotAI changed the title [WIP] Fix recovery timeout error in OleTx testsFix OleTxTests.Recovery CI timeout by bounding remote process and test wall-clock timeMar 20, 2026
CopilotAI requested a review from danmoseleyMarch 20, 2026 00:37
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @SamMonoRT, @dotnet/efteam
See info in area-owners.md if you want to be subscribed.

@danmoseley
danmoseley requested a review from CopilotMarch 20, 2026 00:50

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@danmoseley
danmoseley requested a review from CopilotMarch 20, 2026 01:16

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

Comment threadsrc/libraries/System.Transactions.Local/tests/OleTxTests.cs Outdated
Comment threadsrc/libraries/System.Transactions.Local/tests/OleTxTests.cs
…out on async methods)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseleydanmoseley changed the title Fix OleTxTests.Recovery CI timeout by bounding remote process and test wall-clock timeFix OleTxTests.Recovery CI timeout by bounding remote process wall-clock timeMar 20, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseley
danmoseley marked this pull request as ready for review March 20, 2026 02:19
CopilotAI review requested due to automatic review settings March 20, 2026 02:19

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@AndriySvyryd
AndriySvyryd requested a review from rojiMarch 20, 2026 20:47
@danmoseley
danmoseley merged commit 4521881 into mainMar 20, 2026
83 of 92 checks passed
@danmoseley
danmoseley deleted the copilot/fix-recovery-timeout-error branch March 20, 2026 21:52
danmoseley pushed a commit that referenced this pull request Apr 8, 2026
The Recovery test consistently times out (20+ min) under stress modes
(fullpgo, jitstress2_jitstressregs) — confirmed from Helix logs across
multiple hits. PR #125813 added a 120s child-process timeout but the
main thread still hangs waiting for MSDTC under slow runtimes.
This is a libraries-level test exercising MSDTC/OLE transaction recovery;
running it under JIT stress provides no additional signal. Skip it on
all non-regular CoreCLR test modes.
Fixes#126304
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Apr 20, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

System.Transactions.Tests.OleTxTests.Recovery timeout

5 participants

@danmoseley@AndriySvyryd@jkotas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Fix OleTxTests.Recovery CI timeout by bounding remote process wall-clock time - #125813

Merged
danmoseley merged 4 commits into
mainfrom
copilot/fix-recovery-timeout-error
Mar 20, 2026
Merged

Fix OleTxTests.Recovery CI timeout by bounding remote process wall-clock time#125813
danmoseley merged 4 commits into
mainfrom
copilot/fix-recovery-timeout-error

Conversation

CopilotAI commented Mar 20, 2026

Copy link
Copy Markdown
Contributor

Description

OleTxTests.Recovery repeatedly times out the CI work item (the entire System.Transactions.Local.Tests work item is killed). The CI error gives no detail beyond the timeout — no stack traces or indication of where the hang occurs.

The most likely scenario is that MSDTC is unresponsive on certain CI machines, causing the child process spawned by RemoteExecutor.Invoke to hang (either stuck in MSDTC COM interop, or sitting in its intentional Thread.Sleep waiting for a commit callback that never arrives). Since RemoteInvokeOptions.TimeOut defaults to Timeout.Infinite, RemoteInvokeHandle.Dispose() blocks indefinitely waiting for the child to exit, and the test never completes.

It's also possible the main test thread hangs in tx.Commit() or one of the WaitOne(Timeout) calls. This change would not help in that case — only the child process is bounded. If the timeout continues to repro after this change, the main thread is the more likely culprit and would need its own timeout or the test should be skipped on affected CI legs.

Change:

  • Add TimeOut = 120_000 to RemoteInvokeOptions so RemoteInvokeHandle.Dispose() will force-kill the child process after 2 minutes (with a diagnostic dump at the halfway point) instead of blocking indefinitely. This bounds the test's wall-clock time and should prevent it from consuming the entire work item timeout budget.

Changes

  • src/libraries/System.Transactions.Local/tests/OleTxTests.cs

Testing

This change targets a flaky/hanging test; no new test logic is added. The fix will be validated by the reduction in CI work item timeouts on Windows runs. The test requires MSDTC and cannot be run locally in most environments (the ConditionalFact skips it when MSDTC is unavailable).

…all-clock time
Co-authored-by: danmoseley <6385855+danmoseley@users.noreply.github.com>
CopilotAI changed the title [WIP] Fix recovery timeout error in OleTx testsFix OleTxTests.Recovery CI timeout by bounding remote process and test wall-clock timeMar 20, 2026
CopilotAI requested a review from danmoseleyMarch 20, 2026 00:37
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @SamMonoRT, @dotnet/efteam
See info in area-owners.md if you want to be subscribed.

@danmoseley
danmoseley requested a review from CopilotMarch 20, 2026 00:50

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@danmoseley
danmoseley requested a review from CopilotMarch 20, 2026 01:16

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

Comment threadsrc/libraries/System.Transactions.Local/tests/OleTxTests.cs Outdated
Comment threadsrc/libraries/System.Transactions.Local/tests/OleTxTests.cs
…out on async methods)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseleydanmoseley changed the title Fix OleTxTests.Recovery CI timeout by bounding remote process and test wall-clock timeFix OleTxTests.Recovery CI timeout by bounding remote process wall-clock timeMar 20, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseley
danmoseley marked this pull request as ready for review March 20, 2026 02:19
CopilotAI review requested due to automatic review settings March 20, 2026 02:19

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@AndriySvyryd
AndriySvyryd requested a review from rojiMarch 20, 2026 20:47
@danmoseley
danmoseley merged commit 4521881 into mainMar 20, 2026
83 of 92 checks passed
@danmoseley
danmoseley deleted the copilot/fix-recovery-timeout-error branch March 20, 2026 21:52
danmoseley pushed a commit that referenced this pull request Apr 8, 2026
The Recovery test consistently times out (20+ min) under stress modes
(fullpgo, jitstress2_jitstressregs) — confirmed from Helix logs across
multiple hits. PR #125813 added a 120s child-process timeout but the
main thread still hangs waiting for MSDTC under slow runtimes.
This is a libraries-level test exercising MSDTC/OLE transaction recovery;
running it under JIT stress provides no additional signal. Skip it on
all non-regular CoreCLR test modes.
Fixes#126304
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Apr 20, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

System.Transactions.Tests.OleTxTests.Recovery timeout

5 participants

@danmoseley@AndriySvyryd@jkotas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix OleTxTests.Recovery CI timeout by bounding remote process wall-clock time - #125813

Merged
danmoseley merged 4 commits into
mainfrom
copilot/fix-recovery-timeout-error
Mar 20, 2026
Merged

Fix OleTxTests.Recovery CI timeout by bounding remote process wall-clock time#125813
danmoseley merged 4 commits into
mainfrom
copilot/fix-recovery-timeout-error

Conversation

CopilotAI commented Mar 20, 2026

Copy link
Copy Markdown
Contributor

Description

OleTxTests.Recovery repeatedly times out the CI work item (the entire System.Transactions.Local.Tests work item is killed). The CI error gives no detail beyond the timeout — no stack traces or indication of where the hang occurs.

The most likely scenario is that MSDTC is unresponsive on certain CI machines, causing the child process spawned by RemoteExecutor.Invoke to hang (either stuck in MSDTC COM interop, or sitting in its intentional Thread.Sleep waiting for a commit callback that never arrives). Since RemoteInvokeOptions.TimeOut defaults to Timeout.Infinite, RemoteInvokeHandle.Dispose() blocks indefinitely waiting for the child to exit, and the test never completes.

It's also possible the main test thread hangs in tx.Commit() or one of the WaitOne(Timeout) calls. This change would not help in that case — only the child process is bounded. If the timeout continues to repro after this change, the main thread is the more likely culprit and would need its own timeout or the test should be skipped on affected CI legs.

Change:

  • Add TimeOut = 120_000 to RemoteInvokeOptions so RemoteInvokeHandle.Dispose() will force-kill the child process after 2 minutes (with a diagnostic dump at the halfway point) instead of blocking indefinitely. This bounds the test's wall-clock time and should prevent it from consuming the entire work item timeout budget.

Changes

  • src/libraries/System.Transactions.Local/tests/OleTxTests.cs

Testing

This change targets a flaky/hanging test; no new test logic is added. The fix will be validated by the reduction in CI work item timeouts on Windows runs. The test requires MSDTC and cannot be run locally in most environments (the ConditionalFact skips it when MSDTC is unavailable).

…all-clock time
Co-authored-by: danmoseley <6385855+danmoseley@users.noreply.github.com>
CopilotAI changed the title [WIP] Fix recovery timeout error in OleTx testsFix OleTxTests.Recovery CI timeout by bounding remote process and test wall-clock timeMar 20, 2026
CopilotAI requested a review from danmoseleyMarch 20, 2026 00:37
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @SamMonoRT, @dotnet/efteam
See info in area-owners.md if you want to be subscribed.

@danmoseley
danmoseley requested a review from CopilotMarch 20, 2026 00:50

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@danmoseley
danmoseley requested a review from CopilotMarch 20, 2026 01:16

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

Comment threadsrc/libraries/System.Transactions.Local/tests/OleTxTests.cs Outdated
Comment threadsrc/libraries/System.Transactions.Local/tests/OleTxTests.cs
…out on async methods)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseleydanmoseley changed the title Fix OleTxTests.Recovery CI timeout by bounding remote process and test wall-clock timeFix OleTxTests.Recovery CI timeout by bounding remote process wall-clock timeMar 20, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseley
danmoseley marked this pull request as ready for review March 20, 2026 02:19
CopilotAI review requested due to automatic review settings March 20, 2026 02:19

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@AndriySvyryd
AndriySvyryd requested a review from rojiMarch 20, 2026 20:47
@danmoseley
danmoseley merged commit 4521881 into mainMar 20, 2026
83 of 92 checks passed
@danmoseley
danmoseley deleted the copilot/fix-recovery-timeout-error branch March 20, 2026 21:52
danmoseley pushed a commit that referenced this pull request Apr 8, 2026
The Recovery test consistently times out (20+ min) under stress modes
(fullpgo, jitstress2_jitstressregs) — confirmed from Helix logs across
multiple hits. PR #125813 added a 120s child-process timeout but the
main thread still hangs waiting for MSDTC under slow runtimes.
This is a libraries-level test exercising MSDTC/OLE transaction recovery;
running it under JIT stress provides no additional signal. Skip it on
all non-regular CoreCLR test modes.
Fixes#126304
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Apr 20, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

System.Transactions.Tests.OleTxTests.Recovery timeout

5 participants

@danmoseley@AndriySvyryd@jkotas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix OleTxTests.Recovery CI timeout by bounding remote process wall-clock time - #125813

Merged
danmoseley merged 4 commits into
mainfrom
copilot/fix-recovery-timeout-error
Mar 20, 2026
Merged

Fix OleTxTests.Recovery CI timeout by bounding remote process wall-clock time#125813
danmoseley merged 4 commits into
mainfrom
copilot/fix-recovery-timeout-error

Conversation

CopilotAI commented Mar 20, 2026

Copy link
Copy Markdown
Contributor

Description

OleTxTests.Recovery repeatedly times out the CI work item (the entire System.Transactions.Local.Tests work item is killed). The CI error gives no detail beyond the timeout — no stack traces or indication of where the hang occurs.

The most likely scenario is that MSDTC is unresponsive on certain CI machines, causing the child process spawned by RemoteExecutor.Invoke to hang (either stuck in MSDTC COM interop, or sitting in its intentional Thread.Sleep waiting for a commit callback that never arrives). Since RemoteInvokeOptions.TimeOut defaults to Timeout.Infinite, RemoteInvokeHandle.Dispose() blocks indefinitely waiting for the child to exit, and the test never completes.

It's also possible the main test thread hangs in tx.Commit() or one of the WaitOne(Timeout) calls. This change would not help in that case — only the child process is bounded. If the timeout continues to repro after this change, the main thread is the more likely culprit and would need its own timeout or the test should be skipped on affected CI legs.

Change:

  • Add TimeOut = 120_000 to RemoteInvokeOptions so RemoteInvokeHandle.Dispose() will force-kill the child process after 2 minutes (with a diagnostic dump at the halfway point) instead of blocking indefinitely. This bounds the test's wall-clock time and should prevent it from consuming the entire work item timeout budget.

Changes

  • src/libraries/System.Transactions.Local/tests/OleTxTests.cs

Testing

This change targets a flaky/hanging test; no new test logic is added. The fix will be validated by the reduction in CI work item timeouts on Windows runs. The test requires MSDTC and cannot be run locally in most environments (the ConditionalFact skips it when MSDTC is unavailable).

…all-clock time
Co-authored-by: danmoseley <6385855+danmoseley@users.noreply.github.com>
CopilotAI changed the title [WIP] Fix recovery timeout error in OleTx testsFix OleTxTests.Recovery CI timeout by bounding remote process and test wall-clock timeMar 20, 2026
CopilotAI requested a review from danmoseleyMarch 20, 2026 00:37
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @SamMonoRT, @dotnet/efteam
See info in area-owners.md if you want to be subscribed.

@danmoseley
danmoseley requested a review from CopilotMarch 20, 2026 00:50

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@danmoseley
danmoseley requested a review from CopilotMarch 20, 2026 01:16

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

Comment threadsrc/libraries/System.Transactions.Local/tests/OleTxTests.cs Outdated
Comment threadsrc/libraries/System.Transactions.Local/tests/OleTxTests.cs
…out on async methods)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseleydanmoseley changed the title Fix OleTxTests.Recovery CI timeout by bounding remote process and test wall-clock timeFix OleTxTests.Recovery CI timeout by bounding remote process wall-clock timeMar 20, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseley
danmoseley marked this pull request as ready for review March 20, 2026 02:19
CopilotAI review requested due to automatic review settings March 20, 2026 02:19

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@AndriySvyryd
AndriySvyryd requested a review from rojiMarch 20, 2026 20:47
@danmoseley
danmoseley merged commit 4521881 into mainMar 20, 2026
83 of 92 checks passed
@danmoseley
danmoseley deleted the copilot/fix-recovery-timeout-error branch March 20, 2026 21:52
danmoseley pushed a commit that referenced this pull request Apr 8, 2026
The Recovery test consistently times out (20+ min) under stress modes
(fullpgo, jitstress2_jitstressregs) — confirmed from Helix logs across
multiple hits. PR #125813 added a 120s child-process timeout but the
main thread still hangs waiting for MSDTC under slow runtimes.
This is a libraries-level test exercising MSDTC/OLE transaction recovery;
running it under JIT stress provides no additional signal. Skip it on
all non-regular CoreCLR test modes.
Fixes#126304
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Apr 20, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

System.Transactions.Tests.OleTxTests.Recovery timeout

5 participants

@danmoseley@AndriySvyryd@jkotas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Fix OleTxTests.Recovery CI timeout by bounding remote process wall-clock time - #125813

Merged
danmoseley merged 4 commits into
mainfrom
copilot/fix-recovery-timeout-error
Mar 20, 2026
Merged

Fix OleTxTests.Recovery CI timeout by bounding remote process wall-clock time#125813
danmoseley merged 4 commits into
mainfrom
copilot/fix-recovery-timeout-error

Conversation

CopilotAI commented Mar 20, 2026

Copy link
Copy Markdown
Contributor

Description

OleTxTests.Recovery repeatedly times out the CI work item (the entire System.Transactions.Local.Tests work item is killed). The CI error gives no detail beyond the timeout — no stack traces or indication of where the hang occurs.

The most likely scenario is that MSDTC is unresponsive on certain CI machines, causing the child process spawned by RemoteExecutor.Invoke to hang (either stuck in MSDTC COM interop, or sitting in its intentional Thread.Sleep waiting for a commit callback that never arrives). Since RemoteInvokeOptions.TimeOut defaults to Timeout.Infinite, RemoteInvokeHandle.Dispose() blocks indefinitely waiting for the child to exit, and the test never completes.

It's also possible the main test thread hangs in tx.Commit() or one of the WaitOne(Timeout) calls. This change would not help in that case — only the child process is bounded. If the timeout continues to repro after this change, the main thread is the more likely culprit and would need its own timeout or the test should be skipped on affected CI legs.

Change:

  • Add TimeOut = 120_000 to RemoteInvokeOptions so RemoteInvokeHandle.Dispose() will force-kill the child process after 2 minutes (with a diagnostic dump at the halfway point) instead of blocking indefinitely. This bounds the test's wall-clock time and should prevent it from consuming the entire work item timeout budget.

Changes

  • src/libraries/System.Transactions.Local/tests/OleTxTests.cs

Testing

This change targets a flaky/hanging test; no new test logic is added. The fix will be validated by the reduction in CI work item timeouts on Windows runs. The test requires MSDTC and cannot be run locally in most environments (the ConditionalFact skips it when MSDTC is unavailable).

…all-clock time
Co-authored-by: danmoseley <6385855+danmoseley@users.noreply.github.com>
CopilotAI changed the title [WIP] Fix recovery timeout error in OleTx testsFix OleTxTests.Recovery CI timeout by bounding remote process and test wall-clock timeMar 20, 2026
CopilotAI requested a review from danmoseleyMarch 20, 2026 00:37
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @SamMonoRT, @dotnet/efteam
See info in area-owners.md if you want to be subscribed.

@danmoseley
danmoseley requested a review from CopilotMarch 20, 2026 00:50

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@danmoseley
danmoseley requested a review from CopilotMarch 20, 2026 01:16

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

Comment threadsrc/libraries/System.Transactions.Local/tests/OleTxTests.cs Outdated
Comment threadsrc/libraries/System.Transactions.Local/tests/OleTxTests.cs
…out on async methods)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseleydanmoseley changed the title Fix OleTxTests.Recovery CI timeout by bounding remote process and test wall-clock timeFix OleTxTests.Recovery CI timeout by bounding remote process wall-clock timeMar 20, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danmoseley
danmoseley marked this pull request as ready for review March 20, 2026 02:19
CopilotAI review requested due to automatic review settings March 20, 2026 02:19

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@AndriySvyryd
AndriySvyryd requested a review from rojiMarch 20, 2026 20:47
@danmoseley
danmoseley merged commit 4521881 into mainMar 20, 2026
83 of 92 checks passed
@danmoseley
danmoseley deleted the copilot/fix-recovery-timeout-error branch March 20, 2026 21:52
danmoseley pushed a commit that referenced this pull request Apr 8, 2026
The Recovery test consistently times out (20+ min) under stress modes
(fullpgo, jitstress2_jitstressregs) — confirmed from Helix logs across
multiple hits. PR #125813 added a 120s child-process timeout but the
main thread still hangs waiting for MSDTC under slow runtimes.
This is a libraries-level test exercising MSDTC/OLE transaction recovery;
running it under JIT stress provides no additional signal. Skip it on
all non-regular CoreCLR test modes.
Fixes#126304
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Apr 20, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

System.Transactions.Tests.OleTxTests.Recovery timeout

5 participants

@danmoseley@AndriySvyryd@jkotas