http2: fix write deadlock exposed by larger window sizes - #65440

Merged
nodejs-github-bot merged 4 commits into
nodejs:mainfrom
pimterry:fix-h2-write-deadlock
Aug 26, 2026
Merged

http2: fix write deadlock exposed by larger window sizes#65440
nodejs-github-bot merged 4 commits into
nodejs:mainfrom
pimterry:fix-h2-write-deadlock

Conversation

@pimterry

Copy link
Copy Markdown
Member

This completes the fix from #65079. That PR resolved one test flake (test-worker-terminate-http2-respond-with-file), and reduced the second (test-stream-pipeline-http2) but left an underlying issue there which is still causing flakes.

This took more work, as it's a bit complicated. The issue is a real bug which can result in deadlocks between two Node HTTP/2 peers, not a test issue. This was preexisting though very hard to hit, but is exposed in some cases recently by the new window size update (#64623).

This PR fixes this by dropping a security guard (no reading while writing) completely. I've put it in two commits: the first does the tiny fix to drop the guard & tests it, the second removes various code which is now unreachable without the guard.

Dropping this guard needs careful review, but I think that dropping this guard is safe due to the various other mechanisms in place. As long as we're happy that this is safe, it has a lot of upsides: it fixes the flake, resolves a real deadlock, simply deletes some code, and provides some performance boosts.

An example deadlock flow looks roughly like this:

  • Both peers are reading & writing.
  • Peers write enough to fill the remote peer's TCP receive buffers (only now easily reachable, due to the larger window sizes).
  • While TCP receive buffers are full, both peers try to write at the same time.
  • The security guard here blocks reading while writing, so both peers stop reading.
  • The buffers are all full, so TCP backpressure stops the writes completing - they wait for space.
  • Both peers wait for their writes to complete, which requires a read to empty the buffers, but neither is reading => πŸ”’

This PR fixes that deadlock, by removing the "no reading while writing" guard completely, in its two forms.

These were added as part of a larger set of many HTTP/2 DoS mitigations in 2019 by @addaleax in #29122, so this needs careful review please!

I do think it's safe though: the key thing it's protecting against (one peer forcing the other to buffer large amounts of outgoing data through manipulation of flow control & window updates) is covered by other existing mechanisms here (most notably maxSessionMemory). I can't find any attacks that depend on this guard alone. I've added an additional test which covers CVE-2019-9517 directly, and confirms that maxSessionMemory blocks the key attack scenario regardless.

Removing this guard has notable performance benefits. @mcollina previously looked at this same issue briefly with #63009 testing small focused tweaks on the same guard. AFAICT tweaks alone had minimal benefit, but the guard is still a real perf problem, because it blocks all reads on the whole H2 session while writing any frames anywhere. That means during e.g. a large read (server large upload/client large download), every sent window update to read more data from any stream pauses reading on all streams until the write completes.

Many different dimensions where removing this entirely should improve things, but I've included one example in the benchmark here: this saturates TLS bidirectional streaming, and with this change you get roughly 25% throughput boost. This is using default settings except it shrinks the stream window size back to the standard H2 default - without that you can't benchmark against previous versions, because they deadlock.

This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/http2
  • @nodejs/net
  • @nodejs/performance

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. http2 Issues and PRs related to the http2 subsystem. needs-ci PRs that need a full CI run. labels Aug 20, 2026
@pimterry
pimterryforce-pushed the fix-h2-write-deadlock branch from e2733d2 to a46d64fCompareAugust 20, 2026 18:18
@codecov

codecovBot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 90.15%. Comparing base (fd5b135) to head (a68ec45).
⚠️ Report is 131 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #65440 +/- ##
==========================================
+ Coverage 90.12% 90.15% +0.03% 
==========================================
Files 752 751 -1 Lines 252325 253473 +1148 Branches 47456 47740 +284 ==========================================
+ Hits 227407 228520 +1113 - Misses 16217 16221 +4 - Partials 8701 8732 +31 
Files with missing linesCoverage Ξ”
src/node_http2.cc81.75% <100.00%> (+0.03%)⬆️
src/node_http2.h92.22% <ΓΈ> (-0.05%)⬇️

... and 107 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@nodejs-github-bot

This comment has been minimized.

This rewrites it to focus on the key case: single stream backpressure is
still applied correctly when writes block even if the remote peer H2
window allows more data. The previous case covered multiple streams
which is protected by maxSessionMemory, but kernel buffering makes the
behaviour variable on other platforms (not breaking the security
guarantees, but failing the test) and this isn't the clearest
representation of the key issue we need to guard against
(CVE-2019-9517).
@pimterry

Copy link
Copy Markdown
MemberAuthor

The previous extra security test I added wasn't stable on MacOS, seemingly due to differences in buffer & batching delivery. It didn't fail the security assertions, it just never completed.

This new test is aiming to cover the relevant security behaviour and show it passes with and without this change. I've now rewritten the test completely, to focus directly on the important CVE (my understanding: HTTP/2 writes must correctly apply transport backpressure: they should not silently accept & buffer content when writes stall but the HTTP/2 window is wide open). This now uses a raw socket client and readable with specific config to tightly target this in a way that should behave identically across platforms (I hope 🀞).

Open to adding other security tests here if anybody thinks there's more specific constraints we should validate. I have done even more digging, and I still can't find any cases this guard protects against that other mitigations don't already cover. A review from @nodejs/security would be nice though, especially if anybody was involved in the original 2019 HTTP/2 tightening.

@nodejs-github-bot

This comment has been minimized.

@pimterrypimterry added the commit-queue-squash PRs the Commit Queue should land as one squashed commit. label Aug 21, 2026
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@trivikr

Copy link
Copy Markdown
Member

Ci is failing on Windows

not ok 666 parallel/test-http2-stream-backpressure-unread-peer --- duration_ms: 186.00800 severity: fail exitcode: 1 stack: |- node:internal/assert/utils:146 throw error; ^ AssertionError [ERR_ASSERTION]: Expected values to be strictly equal: 32768 !== 81920 at Immediate.<anonymous> (c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-http2-stream-backpressure-unread-peer.js:115:14) at Immediate._onImmediate (c:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:511:15) at process.processImmediate (node:internal/timers:574:21) { generatedMessage: true, code: 'ERR_ASSERTION', actual: 32768, expected: 81920, operator: 'strictEqual', diff: 'simple' } Node.js v27.0.0-pre ...

https://ci.nodejs.org/job/node-test-binary-windows-js-suites/42576/RUN_SUBSET=3,nodes=win10-COMPILED_BY-vs2022_clang/console

Signed-off-by: Tim Perry <pimterry@gmail.com>
@nodejs-github-bot

This comment was marked as outdated.

@pimterry

pimterry commented Aug 24, 2026

Copy link
Copy Markdown
MemberAuthor

Thanks @trivikr! The previous test assertions made assumptions that didn't hold on Windows. Now fixed by widening the accepted range to properly match the real goal instead of being too specific, hopefully CI will confirm.

I've also now got a passing stress test result (from the previous commit, but relevant tests and code haven't changed), showing this fixes the test-stream-pipeline-http2 flake on MacOS16-x64: https://ci.nodejs.org/job/node-stress-single-test/nodes=macos15-x64/840/console

Main job for the same test still pending but will hopefully flake a bunch to clearly back that up.

https://ci.nodejs.org/job/node-stress-single-test/nodes=macos15-x64/842/ has now failed - 21 times out of 1000 runs, so this is a nice clear flake fix.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@panvapanva added the flaky-test Issues and PRs involving tests that fail intermittently in CI. label Aug 26, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@pimterrypimterry added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 26, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 37887a3 into nodejs:mainAug 26, 2026
68 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 37887a3

@nodejs-github-botnodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 26, 2026
aduh95 pushed a commit that referenced this pull request Aug 29, 2026
This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: #65440
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
aduh95 pushed a commit that referenced this pull request Sep 3, 2026
This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: #65440
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.commit-queue-squashPRs the Commit Queue should land as one squashed commit.flaky-testIssues and PRs involving tests that fail intermittently in CI.http2Issues and PRs related to the http2 subsystem.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@pimterry@nodejs-github-bot@trivikr@mcollina@panva@jasnell
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

http2: fix write deadlock exposed by larger window sizes - #65440

Merged
nodejs-github-bot merged 4 commits into
nodejs:mainfrom
pimterry:fix-h2-write-deadlock
Aug 26, 2026
Merged

http2: fix write deadlock exposed by larger window sizes#65440
nodejs-github-bot merged 4 commits into
nodejs:mainfrom
pimterry:fix-h2-write-deadlock

Conversation

@pimterry

Copy link
Copy Markdown
Member

This completes the fix from #65079. That PR resolved one test flake (test-worker-terminate-http2-respond-with-file), and reduced the second (test-stream-pipeline-http2) but left an underlying issue there which is still causing flakes.

This took more work, as it's a bit complicated. The issue is a real bug which can result in deadlocks between two Node HTTP/2 peers, not a test issue. This was preexisting though very hard to hit, but is exposed in some cases recently by the new window size update (#64623).

This PR fixes this by dropping a security guard (no reading while writing) completely. I've put it in two commits: the first does the tiny fix to drop the guard & tests it, the second removes various code which is now unreachable without the guard.

Dropping this guard needs careful review, but I think that dropping this guard is safe due to the various other mechanisms in place. As long as we're happy that this is safe, it has a lot of upsides: it fixes the flake, resolves a real deadlock, simply deletes some code, and provides some performance boosts.

An example deadlock flow looks roughly like this:

  • Both peers are reading & writing.
  • Peers write enough to fill the remote peer's TCP receive buffers (only now easily reachable, due to the larger window sizes).
  • While TCP receive buffers are full, both peers try to write at the same time.
  • The security guard here blocks reading while writing, so both peers stop reading.
  • The buffers are all full, so TCP backpressure stops the writes completing - they wait for space.
  • Both peers wait for their writes to complete, which requires a read to empty the buffers, but neither is reading => πŸ”’

This PR fixes that deadlock, by removing the "no reading while writing" guard completely, in its two forms.

These were added as part of a larger set of many HTTP/2 DoS mitigations in 2019 by @addaleax in #29122, so this needs careful review please!

I do think it's safe though: the key thing it's protecting against (one peer forcing the other to buffer large amounts of outgoing data through manipulation of flow control & window updates) is covered by other existing mechanisms here (most notably maxSessionMemory). I can't find any attacks that depend on this guard alone. I've added an additional test which covers CVE-2019-9517 directly, and confirms that maxSessionMemory blocks the key attack scenario regardless.

Removing this guard has notable performance benefits. @mcollina previously looked at this same issue briefly with #63009 testing small focused tweaks on the same guard. AFAICT tweaks alone had minimal benefit, but the guard is still a real perf problem, because it blocks all reads on the whole H2 session while writing any frames anywhere. That means during e.g. a large read (server large upload/client large download), every sent window update to read more data from any stream pauses reading on all streams until the write completes.

Many different dimensions where removing this entirely should improve things, but I've included one example in the benchmark here: this saturates TLS bidirectional streaming, and with this change you get roughly 25% throughput boost. This is using default settings except it shrinks the stream window size back to the standard H2 default - without that you can't benchmark against previous versions, because they deadlock.

This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/http2
  • @nodejs/net
  • @nodejs/performance

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. http2 Issues and PRs related to the http2 subsystem. needs-ci PRs that need a full CI run. labels Aug 20, 2026
@pimterry
pimterryforce-pushed the fix-h2-write-deadlock branch from e2733d2 to a46d64fCompareAugust 20, 2026 18:18
@codecov

codecovBot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 90.15%. Comparing base (fd5b135) to head (a68ec45).
⚠️ Report is 131 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #65440 +/- ##
==========================================
+ Coverage 90.12% 90.15% +0.03% 
==========================================
Files 752 751 -1 Lines 252325 253473 +1148 Branches 47456 47740 +284 ==========================================
+ Hits 227407 228520 +1113 - Misses 16217 16221 +4 - Partials 8701 8732 +31 
Files with missing linesCoverage Ξ”
src/node_http2.cc81.75% <100.00%> (+0.03%)⬆️
src/node_http2.h92.22% <ΓΈ> (-0.05%)⬇️

... and 107 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@nodejs-github-bot

This comment has been minimized.

This rewrites it to focus on the key case: single stream backpressure is
still applied correctly when writes block even if the remote peer H2
window allows more data. The previous case covered multiple streams
which is protected by maxSessionMemory, but kernel buffering makes the
behaviour variable on other platforms (not breaking the security
guarantees, but failing the test) and this isn't the clearest
representation of the key issue we need to guard against
(CVE-2019-9517).
@pimterry

Copy link
Copy Markdown
MemberAuthor

The previous extra security test I added wasn't stable on MacOS, seemingly due to differences in buffer & batching delivery. It didn't fail the security assertions, it just never completed.

This new test is aiming to cover the relevant security behaviour and show it passes with and without this change. I've now rewritten the test completely, to focus directly on the important CVE (my understanding: HTTP/2 writes must correctly apply transport backpressure: they should not silently accept & buffer content when writes stall but the HTTP/2 window is wide open). This now uses a raw socket client and readable with specific config to tightly target this in a way that should behave identically across platforms (I hope 🀞).

Open to adding other security tests here if anybody thinks there's more specific constraints we should validate. I have done even more digging, and I still can't find any cases this guard protects against that other mitigations don't already cover. A review from @nodejs/security would be nice though, especially if anybody was involved in the original 2019 HTTP/2 tightening.

@nodejs-github-bot

This comment has been minimized.

@pimterrypimterry added the commit-queue-squash PRs the Commit Queue should land as one squashed commit. label Aug 21, 2026
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@trivikr

Copy link
Copy Markdown
Member

Ci is failing on Windows

not ok 666 parallel/test-http2-stream-backpressure-unread-peer --- duration_ms: 186.00800 severity: fail exitcode: 1 stack: |- node:internal/assert/utils:146 throw error; ^ AssertionError [ERR_ASSERTION]: Expected values to be strictly equal: 32768 !== 81920 at Immediate.<anonymous> (c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-http2-stream-backpressure-unread-peer.js:115:14) at Immediate._onImmediate (c:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:511:15) at process.processImmediate (node:internal/timers:574:21) { generatedMessage: true, code: 'ERR_ASSERTION', actual: 32768, expected: 81920, operator: 'strictEqual', diff: 'simple' } Node.js v27.0.0-pre ...

https://ci.nodejs.org/job/node-test-binary-windows-js-suites/42576/RUN_SUBSET=3,nodes=win10-COMPILED_BY-vs2022_clang/console

Signed-off-by: Tim Perry <pimterry@gmail.com>
@nodejs-github-bot

This comment was marked as outdated.

@pimterry

pimterry commented Aug 24, 2026

Copy link
Copy Markdown
MemberAuthor

Thanks @trivikr! The previous test assertions made assumptions that didn't hold on Windows. Now fixed by widening the accepted range to properly match the real goal instead of being too specific, hopefully CI will confirm.

I've also now got a passing stress test result (from the previous commit, but relevant tests and code haven't changed), showing this fixes the test-stream-pipeline-http2 flake on MacOS16-x64: https://ci.nodejs.org/job/node-stress-single-test/nodes=macos15-x64/840/console

Main job for the same test still pending but will hopefully flake a bunch to clearly back that up.

https://ci.nodejs.org/job/node-stress-single-test/nodes=macos15-x64/842/ has now failed - 21 times out of 1000 runs, so this is a nice clear flake fix.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@panvapanva added the flaky-test Issues and PRs involving tests that fail intermittently in CI. label Aug 26, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@pimterrypimterry added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 26, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 37887a3 into nodejs:mainAug 26, 2026
68 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 37887a3

@nodejs-github-botnodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 26, 2026
aduh95 pushed a commit that referenced this pull request Aug 29, 2026
This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: #65440
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
aduh95 pushed a commit that referenced this pull request Sep 3, 2026
This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: #65440
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.commit-queue-squashPRs the Commit Queue should land as one squashed commit.flaky-testIssues and PRs involving tests that fail intermittently in CI.http2Issues and PRs related to the http2 subsystem.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@pimterry@nodejs-github-bot@trivikr@mcollina@panva@jasnell
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

http2: fix write deadlock exposed by larger window sizes - #65440

Merged
nodejs-github-bot merged 4 commits into
nodejs:mainfrom
pimterry:fix-h2-write-deadlock
Aug 26, 2026
Merged

http2: fix write deadlock exposed by larger window sizes#65440
nodejs-github-bot merged 4 commits into
nodejs:mainfrom
pimterry:fix-h2-write-deadlock

Conversation

@pimterry

Copy link
Copy Markdown
Member

This completes the fix from #65079. That PR resolved one test flake (test-worker-terminate-http2-respond-with-file), and reduced the second (test-stream-pipeline-http2) but left an underlying issue there which is still causing flakes.

This took more work, as it's a bit complicated. The issue is a real bug which can result in deadlocks between two Node HTTP/2 peers, not a test issue. This was preexisting though very hard to hit, but is exposed in some cases recently by the new window size update (#64623).

This PR fixes this by dropping a security guard (no reading while writing) completely. I've put it in two commits: the first does the tiny fix to drop the guard & tests it, the second removes various code which is now unreachable without the guard.

Dropping this guard needs careful review, but I think that dropping this guard is safe due to the various other mechanisms in place. As long as we're happy that this is safe, it has a lot of upsides: it fixes the flake, resolves a real deadlock, simply deletes some code, and provides some performance boosts.

An example deadlock flow looks roughly like this:

  • Both peers are reading & writing.
  • Peers write enough to fill the remote peer's TCP receive buffers (only now easily reachable, due to the larger window sizes).
  • While TCP receive buffers are full, both peers try to write at the same time.
  • The security guard here blocks reading while writing, so both peers stop reading.
  • The buffers are all full, so TCP backpressure stops the writes completing - they wait for space.
  • Both peers wait for their writes to complete, which requires a read to empty the buffers, but neither is reading => πŸ”’

This PR fixes that deadlock, by removing the "no reading while writing" guard completely, in its two forms.

These were added as part of a larger set of many HTTP/2 DoS mitigations in 2019 by @addaleax in #29122, so this needs careful review please!

I do think it's safe though: the key thing it's protecting against (one peer forcing the other to buffer large amounts of outgoing data through manipulation of flow control & window updates) is covered by other existing mechanisms here (most notably maxSessionMemory). I can't find any attacks that depend on this guard alone. I've added an additional test which covers CVE-2019-9517 directly, and confirms that maxSessionMemory blocks the key attack scenario regardless.

Removing this guard has notable performance benefits. @mcollina previously looked at this same issue briefly with #63009 testing small focused tweaks on the same guard. AFAICT tweaks alone had minimal benefit, but the guard is still a real perf problem, because it blocks all reads on the whole H2 session while writing any frames anywhere. That means during e.g. a large read (server large upload/client large download), every sent window update to read more data from any stream pauses reading on all streams until the write completes.

Many different dimensions where removing this entirely should improve things, but I've included one example in the benchmark here: this saturates TLS bidirectional streaming, and with this change you get roughly 25% throughput boost. This is using default settings except it shrinks the stream window size back to the standard H2 default - without that you can't benchmark against previous versions, because they deadlock.

This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/http2
  • @nodejs/net
  • @nodejs/performance

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. http2 Issues and PRs related to the http2 subsystem. needs-ci PRs that need a full CI run. labels Aug 20, 2026
@pimterry
pimterryforce-pushed the fix-h2-write-deadlock branch from e2733d2 to a46d64fCompareAugust 20, 2026 18:18
@codecov

codecovBot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 90.15%. Comparing base (fd5b135) to head (a68ec45).
⚠️ Report is 131 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #65440 +/- ##
==========================================
+ Coverage 90.12% 90.15% +0.03% 
==========================================
Files 752 751 -1 Lines 252325 253473 +1148 Branches 47456 47740 +284 ==========================================
+ Hits 227407 228520 +1113 - Misses 16217 16221 +4 - Partials 8701 8732 +31 
Files with missing linesCoverage Ξ”
src/node_http2.cc81.75% <100.00%> (+0.03%)⬆️
src/node_http2.h92.22% <ΓΈ> (-0.05%)⬇️

... and 107 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@nodejs-github-bot

This comment has been minimized.

This rewrites it to focus on the key case: single stream backpressure is
still applied correctly when writes block even if the remote peer H2
window allows more data. The previous case covered multiple streams
which is protected by maxSessionMemory, but kernel buffering makes the
behaviour variable on other platforms (not breaking the security
guarantees, but failing the test) and this isn't the clearest
representation of the key issue we need to guard against
(CVE-2019-9517).
@pimterry

Copy link
Copy Markdown
MemberAuthor

The previous extra security test I added wasn't stable on MacOS, seemingly due to differences in buffer & batching delivery. It didn't fail the security assertions, it just never completed.

This new test is aiming to cover the relevant security behaviour and show it passes with and without this change. I've now rewritten the test completely, to focus directly on the important CVE (my understanding: HTTP/2 writes must correctly apply transport backpressure: they should not silently accept & buffer content when writes stall but the HTTP/2 window is wide open). This now uses a raw socket client and readable with specific config to tightly target this in a way that should behave identically across platforms (I hope 🀞).

Open to adding other security tests here if anybody thinks there's more specific constraints we should validate. I have done even more digging, and I still can't find any cases this guard protects against that other mitigations don't already cover. A review from @nodejs/security would be nice though, especially if anybody was involved in the original 2019 HTTP/2 tightening.

@nodejs-github-bot

This comment has been minimized.

@pimterrypimterry added the commit-queue-squash PRs the Commit Queue should land as one squashed commit. label Aug 21, 2026
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@trivikr

Copy link
Copy Markdown
Member

Ci is failing on Windows

not ok 666 parallel/test-http2-stream-backpressure-unread-peer --- duration_ms: 186.00800 severity: fail exitcode: 1 stack: |- node:internal/assert/utils:146 throw error; ^ AssertionError [ERR_ASSERTION]: Expected values to be strictly equal: 32768 !== 81920 at Immediate.<anonymous> (c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-http2-stream-backpressure-unread-peer.js:115:14) at Immediate._onImmediate (c:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:511:15) at process.processImmediate (node:internal/timers:574:21) { generatedMessage: true, code: 'ERR_ASSERTION', actual: 32768, expected: 81920, operator: 'strictEqual', diff: 'simple' } Node.js v27.0.0-pre ...

https://ci.nodejs.org/job/node-test-binary-windows-js-suites/42576/RUN_SUBSET=3,nodes=win10-COMPILED_BY-vs2022_clang/console

Signed-off-by: Tim Perry <pimterry@gmail.com>
@nodejs-github-bot

This comment was marked as outdated.

@pimterry

pimterry commented Aug 24, 2026

Copy link
Copy Markdown
MemberAuthor

Thanks @trivikr! The previous test assertions made assumptions that didn't hold on Windows. Now fixed by widening the accepted range to properly match the real goal instead of being too specific, hopefully CI will confirm.

I've also now got a passing stress test result (from the previous commit, but relevant tests and code haven't changed), showing this fixes the test-stream-pipeline-http2 flake on MacOS16-x64: https://ci.nodejs.org/job/node-stress-single-test/nodes=macos15-x64/840/console

Main job for the same test still pending but will hopefully flake a bunch to clearly back that up.

https://ci.nodejs.org/job/node-stress-single-test/nodes=macos15-x64/842/ has now failed - 21 times out of 1000 runs, so this is a nice clear flake fix.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@panvapanva added the flaky-test Issues and PRs involving tests that fail intermittently in CI. label Aug 26, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@pimterrypimterry added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 26, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 37887a3 into nodejs:mainAug 26, 2026
68 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 37887a3

@nodejs-github-botnodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 26, 2026
aduh95 pushed a commit that referenced this pull request Aug 29, 2026
This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: #65440
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
aduh95 pushed a commit that referenced this pull request Sep 3, 2026
This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: #65440
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.commit-queue-squashPRs the Commit Queue should land as one squashed commit.flaky-testIssues and PRs involving tests that fail intermittently in CI.http2Issues and PRs related to the http2 subsystem.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@pimterry@nodejs-github-bot@trivikr@mcollina@panva@jasnell
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

http2: fix write deadlock exposed by larger window sizes - #65440

Merged
nodejs-github-bot merged 4 commits into
nodejs:mainfrom
pimterry:fix-h2-write-deadlock
Aug 26, 2026
Merged

http2: fix write deadlock exposed by larger window sizes#65440
nodejs-github-bot merged 4 commits into
nodejs:mainfrom
pimterry:fix-h2-write-deadlock

Conversation

@pimterry

Copy link
Copy Markdown
Member

This completes the fix from #65079. That PR resolved one test flake (test-worker-terminate-http2-respond-with-file), and reduced the second (test-stream-pipeline-http2) but left an underlying issue there which is still causing flakes.

This took more work, as it's a bit complicated. The issue is a real bug which can result in deadlocks between two Node HTTP/2 peers, not a test issue. This was preexisting though very hard to hit, but is exposed in some cases recently by the new window size update (#64623).

This PR fixes this by dropping a security guard (no reading while writing) completely. I've put it in two commits: the first does the tiny fix to drop the guard & tests it, the second removes various code which is now unreachable without the guard.

Dropping this guard needs careful review, but I think that dropping this guard is safe due to the various other mechanisms in place. As long as we're happy that this is safe, it has a lot of upsides: it fixes the flake, resolves a real deadlock, simply deletes some code, and provides some performance boosts.

An example deadlock flow looks roughly like this:

  • Both peers are reading & writing.
  • Peers write enough to fill the remote peer's TCP receive buffers (only now easily reachable, due to the larger window sizes).
  • While TCP receive buffers are full, both peers try to write at the same time.
  • The security guard here blocks reading while writing, so both peers stop reading.
  • The buffers are all full, so TCP backpressure stops the writes completing - they wait for space.
  • Both peers wait for their writes to complete, which requires a read to empty the buffers, but neither is reading => πŸ”’

This PR fixes that deadlock, by removing the "no reading while writing" guard completely, in its two forms.

These were added as part of a larger set of many HTTP/2 DoS mitigations in 2019 by @addaleax in #29122, so this needs careful review please!

I do think it's safe though: the key thing it's protecting against (one peer forcing the other to buffer large amounts of outgoing data through manipulation of flow control & window updates) is covered by other existing mechanisms here (most notably maxSessionMemory). I can't find any attacks that depend on this guard alone. I've added an additional test which covers CVE-2019-9517 directly, and confirms that maxSessionMemory blocks the key attack scenario regardless.

Removing this guard has notable performance benefits. @mcollina previously looked at this same issue briefly with #63009 testing small focused tweaks on the same guard. AFAICT tweaks alone had minimal benefit, but the guard is still a real perf problem, because it blocks all reads on the whole H2 session while writing any frames anywhere. That means during e.g. a large read (server large upload/client large download), every sent window update to read more data from any stream pauses reading on all streams until the write completes.

Many different dimensions where removing this entirely should improve things, but I've included one example in the benchmark here: this saturates TLS bidirectional streaming, and with this change you get roughly 25% throughput boost. This is using default settings except it shrinks the stream window size back to the standard H2 default - without that you can't benchmark against previous versions, because they deadlock.

This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/http2
  • @nodejs/net
  • @nodejs/performance

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. http2 Issues and PRs related to the http2 subsystem. needs-ci PRs that need a full CI run. labels Aug 20, 2026
@pimterry
pimterryforce-pushed the fix-h2-write-deadlock branch from e2733d2 to a46d64fCompareAugust 20, 2026 18:18
@codecov

codecovBot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 90.15%. Comparing base (fd5b135) to head (a68ec45).
⚠️ Report is 131 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #65440 +/- ##
==========================================
+ Coverage 90.12% 90.15% +0.03% 
==========================================
Files 752 751 -1 Lines 252325 253473 +1148 Branches 47456 47740 +284 ==========================================
+ Hits 227407 228520 +1113 - Misses 16217 16221 +4 - Partials 8701 8732 +31 
Files with missing linesCoverage Ξ”
src/node_http2.cc81.75% <100.00%> (+0.03%)⬆️
src/node_http2.h92.22% <ΓΈ> (-0.05%)⬇️

... and 107 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@nodejs-github-bot

This comment has been minimized.

This rewrites it to focus on the key case: single stream backpressure is
still applied correctly when writes block even if the remote peer H2
window allows more data. The previous case covered multiple streams
which is protected by maxSessionMemory, but kernel buffering makes the
behaviour variable on other platforms (not breaking the security
guarantees, but failing the test) and this isn't the clearest
representation of the key issue we need to guard against
(CVE-2019-9517).
@pimterry

Copy link
Copy Markdown
MemberAuthor

The previous extra security test I added wasn't stable on MacOS, seemingly due to differences in buffer & batching delivery. It didn't fail the security assertions, it just never completed.

This new test is aiming to cover the relevant security behaviour and show it passes with and without this change. I've now rewritten the test completely, to focus directly on the important CVE (my understanding: HTTP/2 writes must correctly apply transport backpressure: they should not silently accept & buffer content when writes stall but the HTTP/2 window is wide open). This now uses a raw socket client and readable with specific config to tightly target this in a way that should behave identically across platforms (I hope 🀞).

Open to adding other security tests here if anybody thinks there's more specific constraints we should validate. I have done even more digging, and I still can't find any cases this guard protects against that other mitigations don't already cover. A review from @nodejs/security would be nice though, especially if anybody was involved in the original 2019 HTTP/2 tightening.

@nodejs-github-bot

This comment has been minimized.

@pimterrypimterry added the commit-queue-squash PRs the Commit Queue should land as one squashed commit. label Aug 21, 2026
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@trivikr

Copy link
Copy Markdown
Member

Ci is failing on Windows

not ok 666 parallel/test-http2-stream-backpressure-unread-peer --- duration_ms: 186.00800 severity: fail exitcode: 1 stack: |- node:internal/assert/utils:146 throw error; ^ AssertionError [ERR_ASSERTION]: Expected values to be strictly equal: 32768 !== 81920 at Immediate.<anonymous> (c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-http2-stream-backpressure-unread-peer.js:115:14) at Immediate._onImmediate (c:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:511:15) at process.processImmediate (node:internal/timers:574:21) { generatedMessage: true, code: 'ERR_ASSERTION', actual: 32768, expected: 81920, operator: 'strictEqual', diff: 'simple' } Node.js v27.0.0-pre ...

https://ci.nodejs.org/job/node-test-binary-windows-js-suites/42576/RUN_SUBSET=3,nodes=win10-COMPILED_BY-vs2022_clang/console

Signed-off-by: Tim Perry <pimterry@gmail.com>
@nodejs-github-bot

This comment was marked as outdated.

@pimterry

pimterry commented Aug 24, 2026

Copy link
Copy Markdown
MemberAuthor

Thanks @trivikr! The previous test assertions made assumptions that didn't hold on Windows. Now fixed by widening the accepted range to properly match the real goal instead of being too specific, hopefully CI will confirm.

I've also now got a passing stress test result (from the previous commit, but relevant tests and code haven't changed), showing this fixes the test-stream-pipeline-http2 flake on MacOS16-x64: https://ci.nodejs.org/job/node-stress-single-test/nodes=macos15-x64/840/console

Main job for the same test still pending but will hopefully flake a bunch to clearly back that up.

https://ci.nodejs.org/job/node-stress-single-test/nodes=macos15-x64/842/ has now failed - 21 times out of 1000 runs, so this is a nice clear flake fix.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@panvapanva added the flaky-test Issues and PRs involving tests that fail intermittently in CI. label Aug 26, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@pimterrypimterry added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 26, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 37887a3 into nodejs:mainAug 26, 2026
68 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 37887a3

@nodejs-github-botnodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 26, 2026
aduh95 pushed a commit that referenced this pull request Aug 29, 2026
This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: #65440
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
aduh95 pushed a commit that referenced this pull request Sep 3, 2026
This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: #65440
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.commit-queue-squashPRs the Commit Queue should land as one squashed commit.flaky-testIssues and PRs involving tests that fail intermittently in CI.http2Issues and PRs related to the http2 subsystem.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@pimterry@nodejs-github-bot@trivikr@mcollina@panva@jasnell
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

http2: fix write deadlock exposed by larger window sizes - #65440

Merged
nodejs-github-bot merged 4 commits into
nodejs:mainfrom
pimterry:fix-h2-write-deadlock
Aug 26, 2026
Merged

http2: fix write deadlock exposed by larger window sizes#65440
nodejs-github-bot merged 4 commits into
nodejs:mainfrom
pimterry:fix-h2-write-deadlock

Conversation

@pimterry

Copy link
Copy Markdown
Member

This completes the fix from #65079. That PR resolved one test flake (test-worker-terminate-http2-respond-with-file), and reduced the second (test-stream-pipeline-http2) but left an underlying issue there which is still causing flakes.

This took more work, as it's a bit complicated. The issue is a real bug which can result in deadlocks between two Node HTTP/2 peers, not a test issue. This was preexisting though very hard to hit, but is exposed in some cases recently by the new window size update (#64623).

This PR fixes this by dropping a security guard (no reading while writing) completely. I've put it in two commits: the first does the tiny fix to drop the guard & tests it, the second removes various code which is now unreachable without the guard.

Dropping this guard needs careful review, but I think that dropping this guard is safe due to the various other mechanisms in place. As long as we're happy that this is safe, it has a lot of upsides: it fixes the flake, resolves a real deadlock, simply deletes some code, and provides some performance boosts.

An example deadlock flow looks roughly like this:

  • Both peers are reading & writing.
  • Peers write enough to fill the remote peer's TCP receive buffers (only now easily reachable, due to the larger window sizes).
  • While TCP receive buffers are full, both peers try to write at the same time.
  • The security guard here blocks reading while writing, so both peers stop reading.
  • The buffers are all full, so TCP backpressure stops the writes completing - they wait for space.
  • Both peers wait for their writes to complete, which requires a read to empty the buffers, but neither is reading => πŸ”’

This PR fixes that deadlock, by removing the "no reading while writing" guard completely, in its two forms.

These were added as part of a larger set of many HTTP/2 DoS mitigations in 2019 by @addaleax in #29122, so this needs careful review please!

I do think it's safe though: the key thing it's protecting against (one peer forcing the other to buffer large amounts of outgoing data through manipulation of flow control & window updates) is covered by other existing mechanisms here (most notably maxSessionMemory). I can't find any attacks that depend on this guard alone. I've added an additional test which covers CVE-2019-9517 directly, and confirms that maxSessionMemory blocks the key attack scenario regardless.

Removing this guard has notable performance benefits. @mcollina previously looked at this same issue briefly with #63009 testing small focused tweaks on the same guard. AFAICT tweaks alone had minimal benefit, but the guard is still a real perf problem, because it blocks all reads on the whole H2 session while writing any frames anywhere. That means during e.g. a large read (server large upload/client large download), every sent window update to read more data from any stream pauses reading on all streams until the write completes.

Many different dimensions where removing this entirely should improve things, but I've included one example in the benchmark here: this saturates TLS bidirectional streaming, and with this change you get roughly 25% throughput boost. This is using default settings except it shrinks the stream window size back to the standard H2 default - without that you can't benchmark against previous versions, because they deadlock.

This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/http2
  • @nodejs/net
  • @nodejs/performance

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. http2 Issues and PRs related to the http2 subsystem. needs-ci PRs that need a full CI run. labels Aug 20, 2026
@pimterry
pimterryforce-pushed the fix-h2-write-deadlock branch from e2733d2 to a46d64fCompareAugust 20, 2026 18:18
@codecov

codecovBot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 90.15%. Comparing base (fd5b135) to head (a68ec45).
⚠️ Report is 131 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #65440 +/- ##
==========================================
+ Coverage 90.12% 90.15% +0.03% 
==========================================
Files 752 751 -1 Lines 252325 253473 +1148 Branches 47456 47740 +284 ==========================================
+ Hits 227407 228520 +1113 - Misses 16217 16221 +4 - Partials 8701 8732 +31 
Files with missing linesCoverage Ξ”
src/node_http2.cc81.75% <100.00%> (+0.03%)⬆️
src/node_http2.h92.22% <ΓΈ> (-0.05%)⬇️

... and 107 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@nodejs-github-bot

This comment has been minimized.

This rewrites it to focus on the key case: single stream backpressure is
still applied correctly when writes block even if the remote peer H2
window allows more data. The previous case covered multiple streams
which is protected by maxSessionMemory, but kernel buffering makes the
behaviour variable on other platforms (not breaking the security
guarantees, but failing the test) and this isn't the clearest
representation of the key issue we need to guard against
(CVE-2019-9517).
@pimterry

Copy link
Copy Markdown
MemberAuthor

The previous extra security test I added wasn't stable on MacOS, seemingly due to differences in buffer & batching delivery. It didn't fail the security assertions, it just never completed.

This new test is aiming to cover the relevant security behaviour and show it passes with and without this change. I've now rewritten the test completely, to focus directly on the important CVE (my understanding: HTTP/2 writes must correctly apply transport backpressure: they should not silently accept & buffer content when writes stall but the HTTP/2 window is wide open). This now uses a raw socket client and readable with specific config to tightly target this in a way that should behave identically across platforms (I hope 🀞).

Open to adding other security tests here if anybody thinks there's more specific constraints we should validate. I have done even more digging, and I still can't find any cases this guard protects against that other mitigations don't already cover. A review from @nodejs/security would be nice though, especially if anybody was involved in the original 2019 HTTP/2 tightening.

@nodejs-github-bot

This comment has been minimized.

@pimterrypimterry added the commit-queue-squash PRs the Commit Queue should land as one squashed commit. label Aug 21, 2026
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@trivikr

Copy link
Copy Markdown
Member

Ci is failing on Windows

not ok 666 parallel/test-http2-stream-backpressure-unread-peer --- duration_ms: 186.00800 severity: fail exitcode: 1 stack: |- node:internal/assert/utils:146 throw error; ^ AssertionError [ERR_ASSERTION]: Expected values to be strictly equal: 32768 !== 81920 at Immediate.<anonymous> (c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-http2-stream-backpressure-unread-peer.js:115:14) at Immediate._onImmediate (c:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:511:15) at process.processImmediate (node:internal/timers:574:21) { generatedMessage: true, code: 'ERR_ASSERTION', actual: 32768, expected: 81920, operator: 'strictEqual', diff: 'simple' } Node.js v27.0.0-pre ...

https://ci.nodejs.org/job/node-test-binary-windows-js-suites/42576/RUN_SUBSET=3,nodes=win10-COMPILED_BY-vs2022_clang/console

Signed-off-by: Tim Perry <pimterry@gmail.com>
@nodejs-github-bot

This comment was marked as outdated.

@pimterry

pimterry commented Aug 24, 2026

Copy link
Copy Markdown
MemberAuthor

Thanks @trivikr! The previous test assertions made assumptions that didn't hold on Windows. Now fixed by widening the accepted range to properly match the real goal instead of being too specific, hopefully CI will confirm.

I've also now got a passing stress test result (from the previous commit, but relevant tests and code haven't changed), showing this fixes the test-stream-pipeline-http2 flake on MacOS16-x64: https://ci.nodejs.org/job/node-stress-single-test/nodes=macos15-x64/840/console

Main job for the same test still pending but will hopefully flake a bunch to clearly back that up.

https://ci.nodejs.org/job/node-stress-single-test/nodes=macos15-x64/842/ has now failed - 21 times out of 1000 runs, so this is a nice clear flake fix.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@panvapanva added the flaky-test Issues and PRs involving tests that fail intermittently in CI. label Aug 26, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@pimterrypimterry added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 26, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 37887a3 into nodejs:mainAug 26, 2026
68 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 37887a3

@nodejs-github-botnodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 26, 2026
aduh95 pushed a commit that referenced this pull request Aug 29, 2026
This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: #65440
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
aduh95 pushed a commit that referenced this pull request Sep 3, 2026
This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: #65440
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.commit-queue-squashPRs the Commit Queue should land as one squashed commit.flaky-testIssues and PRs involving tests that fail intermittently in CI.http2Issues and PRs related to the http2 subsystem.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@pimterry@nodejs-github-bot@trivikr@mcollina@panva@jasnell
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

http2: fix write deadlock exposed by larger window sizes - #65440

Merged
nodejs-github-bot merged 4 commits into
nodejs:mainfrom
pimterry:fix-h2-write-deadlock
Aug 26, 2026
Merged

http2: fix write deadlock exposed by larger window sizes#65440
nodejs-github-bot merged 4 commits into
nodejs:mainfrom
pimterry:fix-h2-write-deadlock

Conversation

@pimterry

Copy link
Copy Markdown
Member

This completes the fix from #65079. That PR resolved one test flake (test-worker-terminate-http2-respond-with-file), and reduced the second (test-stream-pipeline-http2) but left an underlying issue there which is still causing flakes.

This took more work, as it's a bit complicated. The issue is a real bug which can result in deadlocks between two Node HTTP/2 peers, not a test issue. This was preexisting though very hard to hit, but is exposed in some cases recently by the new window size update (#64623).

This PR fixes this by dropping a security guard (no reading while writing) completely. I've put it in two commits: the first does the tiny fix to drop the guard & tests it, the second removes various code which is now unreachable without the guard.

Dropping this guard needs careful review, but I think that dropping this guard is safe due to the various other mechanisms in place. As long as we're happy that this is safe, it has a lot of upsides: it fixes the flake, resolves a real deadlock, simply deletes some code, and provides some performance boosts.

An example deadlock flow looks roughly like this:

  • Both peers are reading & writing.
  • Peers write enough to fill the remote peer's TCP receive buffers (only now easily reachable, due to the larger window sizes).
  • While TCP receive buffers are full, both peers try to write at the same time.
  • The security guard here blocks reading while writing, so both peers stop reading.
  • The buffers are all full, so TCP backpressure stops the writes completing - they wait for space.
  • Both peers wait for their writes to complete, which requires a read to empty the buffers, but neither is reading => πŸ”’

This PR fixes that deadlock, by removing the "no reading while writing" guard completely, in its two forms.

These were added as part of a larger set of many HTTP/2 DoS mitigations in 2019 by @addaleax in #29122, so this needs careful review please!

I do think it's safe though: the key thing it's protecting against (one peer forcing the other to buffer large amounts of outgoing data through manipulation of flow control & window updates) is covered by other existing mechanisms here (most notably maxSessionMemory). I can't find any attacks that depend on this guard alone. I've added an additional test which covers CVE-2019-9517 directly, and confirms that maxSessionMemory blocks the key attack scenario regardless.

Removing this guard has notable performance benefits. @mcollina previously looked at this same issue briefly with #63009 testing small focused tweaks on the same guard. AFAICT tweaks alone had minimal benefit, but the guard is still a real perf problem, because it blocks all reads on the whole H2 session while writing any frames anywhere. That means during e.g. a large read (server large upload/client large download), every sent window update to read more data from any stream pauses reading on all streams until the write completes.

Many different dimensions where removing this entirely should improve things, but I've included one example in the benchmark here: this saturates TLS bidirectional streaming, and with this change you get roughly 25% throughput boost. This is using default settings except it shrinks the stream window size back to the standard H2 default - without that you can't benchmark against previous versions, because they deadlock.

This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/http2
  • @nodejs/net
  • @nodejs/performance

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. http2 Issues and PRs related to the http2 subsystem. needs-ci PRs that need a full CI run. labels Aug 20, 2026
@pimterry
pimterryforce-pushed the fix-h2-write-deadlock branch from e2733d2 to a46d64fCompareAugust 20, 2026 18:18
@codecov

codecovBot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 90.15%. Comparing base (fd5b135) to head (a68ec45).
⚠️ Report is 131 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #65440 +/- ##
==========================================
+ Coverage 90.12% 90.15% +0.03% 
==========================================
Files 752 751 -1 Lines 252325 253473 +1148 Branches 47456 47740 +284 ==========================================
+ Hits 227407 228520 +1113 - Misses 16217 16221 +4 - Partials 8701 8732 +31 
Files with missing linesCoverage Ξ”
src/node_http2.cc81.75% <100.00%> (+0.03%)⬆️
src/node_http2.h92.22% <ΓΈ> (-0.05%)⬇️

... and 107 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@nodejs-github-bot

This comment has been minimized.

This rewrites it to focus on the key case: single stream backpressure is
still applied correctly when writes block even if the remote peer H2
window allows more data. The previous case covered multiple streams
which is protected by maxSessionMemory, but kernel buffering makes the
behaviour variable on other platforms (not breaking the security
guarantees, but failing the test) and this isn't the clearest
representation of the key issue we need to guard against
(CVE-2019-9517).
@pimterry

Copy link
Copy Markdown
MemberAuthor

The previous extra security test I added wasn't stable on MacOS, seemingly due to differences in buffer & batching delivery. It didn't fail the security assertions, it just never completed.

This new test is aiming to cover the relevant security behaviour and show it passes with and without this change. I've now rewritten the test completely, to focus directly on the important CVE (my understanding: HTTP/2 writes must correctly apply transport backpressure: they should not silently accept & buffer content when writes stall but the HTTP/2 window is wide open). This now uses a raw socket client and readable with specific config to tightly target this in a way that should behave identically across platforms (I hope 🀞).

Open to adding other security tests here if anybody thinks there's more specific constraints we should validate. I have done even more digging, and I still can't find any cases this guard protects against that other mitigations don't already cover. A review from @nodejs/security would be nice though, especially if anybody was involved in the original 2019 HTTP/2 tightening.

@nodejs-github-bot

This comment has been minimized.

@pimterrypimterry added the commit-queue-squash PRs the Commit Queue should land as one squashed commit. label Aug 21, 2026
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@trivikr

Copy link
Copy Markdown
Member

Ci is failing on Windows

not ok 666 parallel/test-http2-stream-backpressure-unread-peer --- duration_ms: 186.00800 severity: fail exitcode: 1 stack: |- node:internal/assert/utils:146 throw error; ^ AssertionError [ERR_ASSERTION]: Expected values to be strictly equal: 32768 !== 81920 at Immediate.<anonymous> (c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-http2-stream-backpressure-unread-peer.js:115:14) at Immediate._onImmediate (c:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:511:15) at process.processImmediate (node:internal/timers:574:21) { generatedMessage: true, code: 'ERR_ASSERTION', actual: 32768, expected: 81920, operator: 'strictEqual', diff: 'simple' } Node.js v27.0.0-pre ...

https://ci.nodejs.org/job/node-test-binary-windows-js-suites/42576/RUN_SUBSET=3,nodes=win10-COMPILED_BY-vs2022_clang/console

Signed-off-by: Tim Perry <pimterry@gmail.com>
@nodejs-github-bot

This comment was marked as outdated.

@pimterry

pimterry commented Aug 24, 2026

Copy link
Copy Markdown
MemberAuthor

Thanks @trivikr! The previous test assertions made assumptions that didn't hold on Windows. Now fixed by widening the accepted range to properly match the real goal instead of being too specific, hopefully CI will confirm.

I've also now got a passing stress test result (from the previous commit, but relevant tests and code haven't changed), showing this fixes the test-stream-pipeline-http2 flake on MacOS16-x64: https://ci.nodejs.org/job/node-stress-single-test/nodes=macos15-x64/840/console

Main job for the same test still pending but will hopefully flake a bunch to clearly back that up.

https://ci.nodejs.org/job/node-stress-single-test/nodes=macos15-x64/842/ has now failed - 21 times out of 1000 runs, so this is a nice clear flake fix.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@panvapanva added the flaky-test Issues and PRs involving tests that fail intermittently in CI. label Aug 26, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@pimterrypimterry added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 26, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 37887a3 into nodejs:mainAug 26, 2026
68 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 37887a3

@nodejs-github-botnodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 26, 2026
aduh95 pushed a commit that referenced this pull request Aug 29, 2026
This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: #65440
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
aduh95 pushed a commit that referenced this pull request Sep 3, 2026
This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: #65440
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.commit-queue-squashPRs the Commit Queue should land as one squashed commit.flaky-testIssues and PRs involving tests that fail intermittently in CI.http2Issues and PRs related to the http2 subsystem.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@pimterry@nodejs-github-bot@trivikr@mcollina@panva@jasnell
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

http2: fix write deadlock exposed by larger window sizes - #65440

Merged
nodejs-github-bot merged 4 commits into
nodejs:mainfrom
pimterry:fix-h2-write-deadlock
Aug 26, 2026
Merged

http2: fix write deadlock exposed by larger window sizes#65440
nodejs-github-bot merged 4 commits into
nodejs:mainfrom
pimterry:fix-h2-write-deadlock

Conversation

@pimterry

Copy link
Copy Markdown
Member

This completes the fix from #65079. That PR resolved one test flake (test-worker-terminate-http2-respond-with-file), and reduced the second (test-stream-pipeline-http2) but left an underlying issue there which is still causing flakes.

This took more work, as it's a bit complicated. The issue is a real bug which can result in deadlocks between two Node HTTP/2 peers, not a test issue. This was preexisting though very hard to hit, but is exposed in some cases recently by the new window size update (#64623).

This PR fixes this by dropping a security guard (no reading while writing) completely. I've put it in two commits: the first does the tiny fix to drop the guard & tests it, the second removes various code which is now unreachable without the guard.

Dropping this guard needs careful review, but I think that dropping this guard is safe due to the various other mechanisms in place. As long as we're happy that this is safe, it has a lot of upsides: it fixes the flake, resolves a real deadlock, simply deletes some code, and provides some performance boosts.

An example deadlock flow looks roughly like this:

  • Both peers are reading & writing.
  • Peers write enough to fill the remote peer's TCP receive buffers (only now easily reachable, due to the larger window sizes).
  • While TCP receive buffers are full, both peers try to write at the same time.
  • The security guard here blocks reading while writing, so both peers stop reading.
  • The buffers are all full, so TCP backpressure stops the writes completing - they wait for space.
  • Both peers wait for their writes to complete, which requires a read to empty the buffers, but neither is reading => πŸ”’

This PR fixes that deadlock, by removing the "no reading while writing" guard completely, in its two forms.

These were added as part of a larger set of many HTTP/2 DoS mitigations in 2019 by @addaleax in #29122, so this needs careful review please!

I do think it's safe though: the key thing it's protecting against (one peer forcing the other to buffer large amounts of outgoing data through manipulation of flow control & window updates) is covered by other existing mechanisms here (most notably maxSessionMemory). I can't find any attacks that depend on this guard alone. I've added an additional test which covers CVE-2019-9517 directly, and confirms that maxSessionMemory blocks the key attack scenario regardless.

Removing this guard has notable performance benefits. @mcollina previously looked at this same issue briefly with #63009 testing small focused tweaks on the same guard. AFAICT tweaks alone had minimal benefit, but the guard is still a real perf problem, because it blocks all reads on the whole H2 session while writing any frames anywhere. That means during e.g. a large read (server large upload/client large download), every sent window update to read more data from any stream pauses reading on all streams until the write completes.

Many different dimensions where removing this entirely should improve things, but I've included one example in the benchmark here: this saturates TLS bidirectional streaming, and with this change you get roughly 25% throughput boost. This is using default settings except it shrinks the stream window size back to the standard H2 default - without that you can't benchmark against previous versions, because they deadlock.

This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/http2
  • @nodejs/net
  • @nodejs/performance

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. http2 Issues and PRs related to the http2 subsystem. needs-ci PRs that need a full CI run. labels Aug 20, 2026
@pimterry
pimterryforce-pushed the fix-h2-write-deadlock branch from e2733d2 to a46d64fCompareAugust 20, 2026 18:18
@codecov

codecovBot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 90.15%. Comparing base (fd5b135) to head (a68ec45).
⚠️ Report is 131 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #65440 +/- ##
==========================================
+ Coverage 90.12% 90.15% +0.03% 
==========================================
Files 752 751 -1 Lines 252325 253473 +1148 Branches 47456 47740 +284 ==========================================
+ Hits 227407 228520 +1113 - Misses 16217 16221 +4 - Partials 8701 8732 +31 
Files with missing linesCoverage Ξ”
src/node_http2.cc81.75% <100.00%> (+0.03%)⬆️
src/node_http2.h92.22% <ΓΈ> (-0.05%)⬇️

... and 107 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@nodejs-github-bot

This comment has been minimized.

This rewrites it to focus on the key case: single stream backpressure is
still applied correctly when writes block even if the remote peer H2
window allows more data. The previous case covered multiple streams
which is protected by maxSessionMemory, but kernel buffering makes the
behaviour variable on other platforms (not breaking the security
guarantees, but failing the test) and this isn't the clearest
representation of the key issue we need to guard against
(CVE-2019-9517).
@pimterry

Copy link
Copy Markdown
MemberAuthor

The previous extra security test I added wasn't stable on MacOS, seemingly due to differences in buffer & batching delivery. It didn't fail the security assertions, it just never completed.

This new test is aiming to cover the relevant security behaviour and show it passes with and without this change. I've now rewritten the test completely, to focus directly on the important CVE (my understanding: HTTP/2 writes must correctly apply transport backpressure: they should not silently accept & buffer content when writes stall but the HTTP/2 window is wide open). This now uses a raw socket client and readable with specific config to tightly target this in a way that should behave identically across platforms (I hope 🀞).

Open to adding other security tests here if anybody thinks there's more specific constraints we should validate. I have done even more digging, and I still can't find any cases this guard protects against that other mitigations don't already cover. A review from @nodejs/security would be nice though, especially if anybody was involved in the original 2019 HTTP/2 tightening.

@nodejs-github-bot

This comment has been minimized.

@pimterrypimterry added the commit-queue-squash PRs the Commit Queue should land as one squashed commit. label Aug 21, 2026
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@trivikr

Copy link
Copy Markdown
Member

Ci is failing on Windows

not ok 666 parallel/test-http2-stream-backpressure-unread-peer --- duration_ms: 186.00800 severity: fail exitcode: 1 stack: |- node:internal/assert/utils:146 throw error; ^ AssertionError [ERR_ASSERTION]: Expected values to be strictly equal: 32768 !== 81920 at Immediate.<anonymous> (c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-http2-stream-backpressure-unread-peer.js:115:14) at Immediate._onImmediate (c:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:511:15) at process.processImmediate (node:internal/timers:574:21) { generatedMessage: true, code: 'ERR_ASSERTION', actual: 32768, expected: 81920, operator: 'strictEqual', diff: 'simple' } Node.js v27.0.0-pre ...

https://ci.nodejs.org/job/node-test-binary-windows-js-suites/42576/RUN_SUBSET=3,nodes=win10-COMPILED_BY-vs2022_clang/console

Signed-off-by: Tim Perry <pimterry@gmail.com>
@nodejs-github-bot

This comment was marked as outdated.

@pimterry

pimterry commented Aug 24, 2026

Copy link
Copy Markdown
MemberAuthor

Thanks @trivikr! The previous test assertions made assumptions that didn't hold on Windows. Now fixed by widening the accepted range to properly match the real goal instead of being too specific, hopefully CI will confirm.

I've also now got a passing stress test result (from the previous commit, but relevant tests and code haven't changed), showing this fixes the test-stream-pipeline-http2 flake on MacOS16-x64: https://ci.nodejs.org/job/node-stress-single-test/nodes=macos15-x64/840/console

Main job for the same test still pending but will hopefully flake a bunch to clearly back that up.

https://ci.nodejs.org/job/node-stress-single-test/nodes=macos15-x64/842/ has now failed - 21 times out of 1000 runs, so this is a nice clear flake fix.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@panvapanva added the flaky-test Issues and PRs involving tests that fail intermittently in CI. label Aug 26, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@pimterrypimterry added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 26, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 37887a3 into nodejs:mainAug 26, 2026
68 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 37887a3

@nodejs-github-botnodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 26, 2026
aduh95 pushed a commit that referenced this pull request Aug 29, 2026
This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: #65440
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
aduh95 pushed a commit that referenced this pull request Sep 3, 2026
This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: #65440
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.commit-queue-squashPRs the Commit Queue should land as one squashed commit.flaky-testIssues and PRs involving tests that fail intermittently in CI.http2Issues and PRs related to the http2 subsystem.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@pimterry@nodejs-github-bot@trivikr@mcollina@panva@jasnell
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

http2: fix write deadlock exposed by larger window sizes - #65440

Merged
nodejs-github-bot merged 4 commits into
nodejs:mainfrom
pimterry:fix-h2-write-deadlock
Aug 26, 2026
Merged

http2: fix write deadlock exposed by larger window sizes#65440
nodejs-github-bot merged 4 commits into
nodejs:mainfrom
pimterry:fix-h2-write-deadlock

Conversation

@pimterry

Copy link
Copy Markdown
Member

This completes the fix from #65079. That PR resolved one test flake (test-worker-terminate-http2-respond-with-file), and reduced the second (test-stream-pipeline-http2) but left an underlying issue there which is still causing flakes.

This took more work, as it's a bit complicated. The issue is a real bug which can result in deadlocks between two Node HTTP/2 peers, not a test issue. This was preexisting though very hard to hit, but is exposed in some cases recently by the new window size update (#64623).

This PR fixes this by dropping a security guard (no reading while writing) completely. I've put it in two commits: the first does the tiny fix to drop the guard & tests it, the second removes various code which is now unreachable without the guard.

Dropping this guard needs careful review, but I think that dropping this guard is safe due to the various other mechanisms in place. As long as we're happy that this is safe, it has a lot of upsides: it fixes the flake, resolves a real deadlock, simply deletes some code, and provides some performance boosts.

An example deadlock flow looks roughly like this:

  • Both peers are reading & writing.
  • Peers write enough to fill the remote peer's TCP receive buffers (only now easily reachable, due to the larger window sizes).
  • While TCP receive buffers are full, both peers try to write at the same time.
  • The security guard here blocks reading while writing, so both peers stop reading.
  • The buffers are all full, so TCP backpressure stops the writes completing - they wait for space.
  • Both peers wait for their writes to complete, which requires a read to empty the buffers, but neither is reading => πŸ”’

This PR fixes that deadlock, by removing the "no reading while writing" guard completely, in its two forms.

These were added as part of a larger set of many HTTP/2 DoS mitigations in 2019 by @addaleax in #29122, so this needs careful review please!

I do think it's safe though: the key thing it's protecting against (one peer forcing the other to buffer large amounts of outgoing data through manipulation of flow control & window updates) is covered by other existing mechanisms here (most notably maxSessionMemory). I can't find any attacks that depend on this guard alone. I've added an additional test which covers CVE-2019-9517 directly, and confirms that maxSessionMemory blocks the key attack scenario regardless.

Removing this guard has notable performance benefits. @mcollina previously looked at this same issue briefly with #63009 testing small focused tweaks on the same guard. AFAICT tweaks alone had minimal benefit, but the guard is still a real perf problem, because it blocks all reads on the whole H2 session while writing any frames anywhere. That means during e.g. a large read (server large upload/client large download), every sent window update to read more data from any stream pauses reading on all streams until the write completes.

Many different dimensions where removing this entirely should improve things, but I've included one example in the benchmark here: this saturates TLS bidirectional streaming, and with this change you get roughly 25% throughput boost. This is using default settings except it shrinks the stream window size back to the standard H2 default - without that you can't benchmark against previous versions, because they deadlock.

This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/http2
  • @nodejs/net
  • @nodejs/performance

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. http2 Issues and PRs related to the http2 subsystem. needs-ci PRs that need a full CI run. labels Aug 20, 2026
@pimterry
pimterryforce-pushed the fix-h2-write-deadlock branch from e2733d2 to a46d64fCompareAugust 20, 2026 18:18
@codecov

codecovBot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 90.15%. Comparing base (fd5b135) to head (a68ec45).
⚠️ Report is 131 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #65440 +/- ##
==========================================
+ Coverage 90.12% 90.15% +0.03% 
==========================================
Files 752 751 -1 Lines 252325 253473 +1148 Branches 47456 47740 +284 ==========================================
+ Hits 227407 228520 +1113 - Misses 16217 16221 +4 - Partials 8701 8732 +31 
Files with missing linesCoverage Ξ”
src/node_http2.cc81.75% <100.00%> (+0.03%)⬆️
src/node_http2.h92.22% <ΓΈ> (-0.05%)⬇️

... and 107 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 21, 2026
@nodejs-github-bot

This comment has been minimized.

This rewrites it to focus on the key case: single stream backpressure is
still applied correctly when writes block even if the remote peer H2
window allows more data. The previous case covered multiple streams
which is protected by maxSessionMemory, but kernel buffering makes the
behaviour variable on other platforms (not breaking the security
guarantees, but failing the test) and this isn't the clearest
representation of the key issue we need to guard against
(CVE-2019-9517).
@pimterry

Copy link
Copy Markdown
MemberAuthor

The previous extra security test I added wasn't stable on MacOS, seemingly due to differences in buffer & batching delivery. It didn't fail the security assertions, it just never completed.

This new test is aiming to cover the relevant security behaviour and show it passes with and without this change. I've now rewritten the test completely, to focus directly on the important CVE (my understanding: HTTP/2 writes must correctly apply transport backpressure: they should not silently accept & buffer content when writes stall but the HTTP/2 window is wide open). This now uses a raw socket client and readable with specific config to tightly target this in a way that should behave identically across platforms (I hope 🀞).

Open to adding other security tests here if anybody thinks there's more specific constraints we should validate. I have done even more digging, and I still can't find any cases this guard protects against that other mitigations don't already cover. A review from @nodejs/security would be nice though, especially if anybody was involved in the original 2019 HTTP/2 tightening.

@nodejs-github-bot

This comment has been minimized.

@pimterrypimterry added the commit-queue-squash PRs the Commit Queue should land as one squashed commit. label Aug 21, 2026
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@trivikr

Copy link
Copy Markdown
Member

Ci is failing on Windows

not ok 666 parallel/test-http2-stream-backpressure-unread-peer --- duration_ms: 186.00800 severity: fail exitcode: 1 stack: |- node:internal/assert/utils:146 throw error; ^ AssertionError [ERR_ASSERTION]: Expected values to be strictly equal: 32768 !== 81920 at Immediate.<anonymous> (c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-http2-stream-backpressure-unread-peer.js:115:14) at Immediate._onImmediate (c:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:511:15) at process.processImmediate (node:internal/timers:574:21) { generatedMessage: true, code: 'ERR_ASSERTION', actual: 32768, expected: 81920, operator: 'strictEqual', diff: 'simple' } Node.js v27.0.0-pre ...

https://ci.nodejs.org/job/node-test-binary-windows-js-suites/42576/RUN_SUBSET=3,nodes=win10-COMPILED_BY-vs2022_clang/console

Signed-off-by: Tim Perry <pimterry@gmail.com>
@nodejs-github-bot

This comment was marked as outdated.

@pimterry

pimterry commented Aug 24, 2026

Copy link
Copy Markdown
MemberAuthor

Thanks @trivikr! The previous test assertions made assumptions that didn't hold on Windows. Now fixed by widening the accepted range to properly match the real goal instead of being too specific, hopefully CI will confirm.

I've also now got a passing stress test result (from the previous commit, but relevant tests and code haven't changed), showing this fixes the test-stream-pipeline-http2 flake on MacOS16-x64: https://ci.nodejs.org/job/node-stress-single-test/nodes=macos15-x64/840/console

Main job for the same test still pending but will hopefully flake a bunch to clearly back that up.

https://ci.nodejs.org/job/node-stress-single-test/nodes=macos15-x64/842/ has now failed - 21 times out of 1000 runs, so this is a nice clear flake fix.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@nodejs-github-bot

This comment has been minimized.

@panvapanva added the flaky-test Issues and PRs involving tests that fail intermittently in CI. label Aug 26, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@pimterrypimterry added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 26, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 37887a3 into nodejs:mainAug 26, 2026
68 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 37887a3

@nodejs-github-botnodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 26, 2026
aduh95 pushed a commit that referenced this pull request Aug 29, 2026
This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: #65440
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
aduh95 pushed a commit that referenced this pull request Sep 3, 2026
This removes a guard (no reads while write pending) that creates
this deadlock, which was added as a security mechanism. This guard is
redundant given then other existing mechanisms, and a test is added to
demonstrate that.
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: #65440
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.commit-queue-squashPRs the Commit Queue should land as one squashed commit.flaky-testIssues and PRs involving tests that fail intermittently in CI.http2Issues and PRs related to the http2 subsystem.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@pimterry@nodejs-github-bot@trivikr@mcollina@panva@jasnell