fs: write files in one thread pool round trip - #65489

Closed
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip
Closed

fs: write files in one thread pool round trip#65489
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip

Conversation

@codebytere

@codebyterecodebytere commented Aug 22, 2026

Copy link
Copy Markdown
Member

Makes fs.writeFile(path, data) and fsPromises.writeFile(path, data) do open + write + close as one thread pool request (WriteFileJob, next to the ReadFileJob from #65327) instead of one request per step.

benchmark/fs/writefile-promises.js (buf, duration=2), 20 runs, this branch vs main; fs.writeFile() measured the same way with an equivalent callback loop:

concurrent=1concurrent=10
fsPromises.writeFile, 2 B+108 %+51 %
fsPromises.writeFile, 1 KiB+108 %+51 %
fsPromises.writeFile, 64 KiB+76 %+30 %
fs.writeFile, 1 KiB+26 %+38 %

(1 MiB stays on the chunked FileHandle path and is unchanged.)

fs.writeFile() took three libuv round trips (open, write, close), each its own request with its own queue wait, completion callback and JS/C++ crossing, and fsPromises.writeFile() did the same through a FileHandle. For the small files applications write most (settings, state, caches) the round trips are the cost, and each one occupies a pool slot that concurrent fs, dns.lookup() and crypto work is also queueing for.

WriteFileJob (an AsyncWrap + ThreadPoolWork) opens, writes the whole buffer (looping on short writes) and closes as one pool task, keeping the buffer alive until it is done. fs.writeFile() uses it for path arguments without flush; fsPromises.writeFile() additionally keeps data above one write chunk (512 KiB) and iterables on the FileHandle path, so large writes stay abortable between chunks as before. File descriptors, FileHandles, flush: true and an active VFS keep their existing paths. Open failures report syscall open with the path and write failures write, permission errors arrive through the callback/promise, an abort signalled while the write is in flight is still reported as an AbortError, and the job is an FSREQCALLBACK async resource that emits the write fs trace event.

Refs: #65327

The job holds the buffer's backing store, so the memory stays valid if the buffer is detached or collected before the write finishes; a resizable ArrayBuffer could still have its pages decommitted by a shrink, so its contents are copied when the job is created.

Tests: new test-fs-writefile-one-roundtrip.js (flags a/wx, mode, failing syscall and path, every ArrayBufferView kind, a multi-megabyte buffer, both APIs); tests that used fs.writeFile() as a proxy for open/close trace events or injected FileHandle faults into path-based writes are adjusted to keep testing what they test; test-fs-*write*, *append*, test-fs-promises*, permission and async-hooks fs tests pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Aug 22, 2026
@codecov

codecovBot commented Aug 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.02994% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.94%. Comparing base (927dfad) to head (f86c085).
⚠️ Report is 1 commits behind head on main.

Files with missing linesPatch %Lines
src/node_file.cc81.57%7 Missing and 14 partials ⚠️
lib/internal/fs/promises.js88.57%4 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #65489 +/- ##
==========================================
- Coverage 89.96% 89.94% -0.02% 
==========================================
Files 757 757 Lines 258049 258215 +166 Branches 48925 48955 +30 ==========================================
+ Hits 232144 232243 +99 - Misses 16965 17028 +63 - Partials 8940 8944 +4 
Files with missing linesCoverage Δ
lib/fs.js97.30% <100.00%> (-0.04%)⬇️
lib/internal/fs/promises.js91.24% <88.57%> (-0.09%)⬇️
src/node_file.cc74.90% <81.57%> (+0.41%)⬆️

... and 31 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 9fb687f to 4c3fcd6CompareAugust 22, 2026 18:17
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 23, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 23, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadsrc/node_file.cc Outdated
Comment threadtest/parallel/test-fs-writefile-one-roundtrip.js
@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 4c3fcd6 to a1d02deCompareAugust 25, 2026 12:26
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from a1d02de to c8c92fbCompareAugust 27, 2026 08:09
Comment threadsrc/node_file.cc
@addaleaxaddaleax added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Aug 27, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from c8c92fb to 85ce7e1CompareAugust 28, 2026 15:54
@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 28, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 29, 2026
@nodejs-github-bot

This comment was marked as outdated.

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 30, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 30, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 31, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 31, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 85ce7e1 to db16496CompareSeptember 1, 2026 00:35
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added commit-queue PRs queued for automated landing through the Commit Queue. and removed author ready PRs with CI started, the required approvals, and no outstanding review comments. commit-queue PRs queued for automated landing through the Commit Queue. labels Sep 2, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from db16496 to f86c085CompareSeptember 2, 2026 20:46
@codebyterecodebytere added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Sep 2, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

codebytere added a commit that referenced this pull request Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65489
Refs: #65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@codebytere

Copy link
Copy Markdown
MemberAuthor

Landed in e324419

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author readyPRs with CI started, the required approvals, and no outstanding review comments.c++Issues and PRs that require attention from people who are familiar with C++.fsIssues and PRs related to file-system APIs and the fs module.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@codebytere@nodejs-github-bot@mcollina@jasnell@addaleax@anonrig
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

fs: write files in one thread pool round trip - #65489

Closed
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip
Closed

fs: write files in one thread pool round trip#65489
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip

Conversation

@codebytere

@codebyterecodebytere commented Aug 22, 2026

Copy link
Copy Markdown
Member

Makes fs.writeFile(path, data) and fsPromises.writeFile(path, data) do open + write + close as one thread pool request (WriteFileJob, next to the ReadFileJob from #65327) instead of one request per step.

benchmark/fs/writefile-promises.js (buf, duration=2), 20 runs, this branch vs main; fs.writeFile() measured the same way with an equivalent callback loop:

concurrent=1concurrent=10
fsPromises.writeFile, 2 B+108 %+51 %
fsPromises.writeFile, 1 KiB+108 %+51 %
fsPromises.writeFile, 64 KiB+76 %+30 %
fs.writeFile, 1 KiB+26 %+38 %

(1 MiB stays on the chunked FileHandle path and is unchanged.)

fs.writeFile() took three libuv round trips (open, write, close), each its own request with its own queue wait, completion callback and JS/C++ crossing, and fsPromises.writeFile() did the same through a FileHandle. For the small files applications write most (settings, state, caches) the round trips are the cost, and each one occupies a pool slot that concurrent fs, dns.lookup() and crypto work is also queueing for.

WriteFileJob (an AsyncWrap + ThreadPoolWork) opens, writes the whole buffer (looping on short writes) and closes as one pool task, keeping the buffer alive until it is done. fs.writeFile() uses it for path arguments without flush; fsPromises.writeFile() additionally keeps data above one write chunk (512 KiB) and iterables on the FileHandle path, so large writes stay abortable between chunks as before. File descriptors, FileHandles, flush: true and an active VFS keep their existing paths. Open failures report syscall open with the path and write failures write, permission errors arrive through the callback/promise, an abort signalled while the write is in flight is still reported as an AbortError, and the job is an FSREQCALLBACK async resource that emits the write fs trace event.

Refs: #65327

The job holds the buffer's backing store, so the memory stays valid if the buffer is detached or collected before the write finishes; a resizable ArrayBuffer could still have its pages decommitted by a shrink, so its contents are copied when the job is created.

Tests: new test-fs-writefile-one-roundtrip.js (flags a/wx, mode, failing syscall and path, every ArrayBufferView kind, a multi-megabyte buffer, both APIs); tests that used fs.writeFile() as a proxy for open/close trace events or injected FileHandle faults into path-based writes are adjusted to keep testing what they test; test-fs-*write*, *append*, test-fs-promises*, permission and async-hooks fs tests pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Aug 22, 2026
@codecov

codecovBot commented Aug 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.02994% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.94%. Comparing base (927dfad) to head (f86c085).
⚠️ Report is 1 commits behind head on main.

Files with missing linesPatch %Lines
src/node_file.cc81.57%7 Missing and 14 partials ⚠️
lib/internal/fs/promises.js88.57%4 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #65489 +/- ##
==========================================
- Coverage 89.96% 89.94% -0.02% 
==========================================
Files 757 757 Lines 258049 258215 +166 Branches 48925 48955 +30 ==========================================
+ Hits 232144 232243 +99 - Misses 16965 17028 +63 - Partials 8940 8944 +4 
Files with missing linesCoverage Δ
lib/fs.js97.30% <100.00%> (-0.04%)⬇️
lib/internal/fs/promises.js91.24% <88.57%> (-0.09%)⬇️
src/node_file.cc74.90% <81.57%> (+0.41%)⬆️

... and 31 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 9fb687f to 4c3fcd6CompareAugust 22, 2026 18:17
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 23, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 23, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadsrc/node_file.cc Outdated
Comment threadtest/parallel/test-fs-writefile-one-roundtrip.js
@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 4c3fcd6 to a1d02deCompareAugust 25, 2026 12:26
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from a1d02de to c8c92fbCompareAugust 27, 2026 08:09
Comment threadsrc/node_file.cc
@addaleaxaddaleax added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Aug 27, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from c8c92fb to 85ce7e1CompareAugust 28, 2026 15:54
@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 28, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 29, 2026
@nodejs-github-bot

This comment was marked as outdated.

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 30, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 30, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 31, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 31, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 85ce7e1 to db16496CompareSeptember 1, 2026 00:35
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added commit-queue PRs queued for automated landing through the Commit Queue. and removed author ready PRs with CI started, the required approvals, and no outstanding review comments. commit-queue PRs queued for automated landing through the Commit Queue. labels Sep 2, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from db16496 to f86c085CompareSeptember 2, 2026 20:46
@codebyterecodebytere added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Sep 2, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

codebytere added a commit that referenced this pull request Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65489
Refs: #65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@codebytere

Copy link
Copy Markdown
MemberAuthor

Landed in e324419

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author readyPRs with CI started, the required approvals, and no outstanding review comments.c++Issues and PRs that require attention from people who are familiar with C++.fsIssues and PRs related to file-system APIs and the fs module.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@codebytere@nodejs-github-bot@mcollina@jasnell@addaleax@anonrig
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fs: write files in one thread pool round trip - #65489

Closed
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip
Closed

fs: write files in one thread pool round trip#65489
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip

Conversation

@codebytere

@codebyterecodebytere commented Aug 22, 2026

Copy link
Copy Markdown
Member

Makes fs.writeFile(path, data) and fsPromises.writeFile(path, data) do open + write + close as one thread pool request (WriteFileJob, next to the ReadFileJob from #65327) instead of one request per step.

benchmark/fs/writefile-promises.js (buf, duration=2), 20 runs, this branch vs main; fs.writeFile() measured the same way with an equivalent callback loop:

concurrent=1concurrent=10
fsPromises.writeFile, 2 B+108 %+51 %
fsPromises.writeFile, 1 KiB+108 %+51 %
fsPromises.writeFile, 64 KiB+76 %+30 %
fs.writeFile, 1 KiB+26 %+38 %

(1 MiB stays on the chunked FileHandle path and is unchanged.)

fs.writeFile() took three libuv round trips (open, write, close), each its own request with its own queue wait, completion callback and JS/C++ crossing, and fsPromises.writeFile() did the same through a FileHandle. For the small files applications write most (settings, state, caches) the round trips are the cost, and each one occupies a pool slot that concurrent fs, dns.lookup() and crypto work is also queueing for.

WriteFileJob (an AsyncWrap + ThreadPoolWork) opens, writes the whole buffer (looping on short writes) and closes as one pool task, keeping the buffer alive until it is done. fs.writeFile() uses it for path arguments without flush; fsPromises.writeFile() additionally keeps data above one write chunk (512 KiB) and iterables on the FileHandle path, so large writes stay abortable between chunks as before. File descriptors, FileHandles, flush: true and an active VFS keep their existing paths. Open failures report syscall open with the path and write failures write, permission errors arrive through the callback/promise, an abort signalled while the write is in flight is still reported as an AbortError, and the job is an FSREQCALLBACK async resource that emits the write fs trace event.

Refs: #65327

The job holds the buffer's backing store, so the memory stays valid if the buffer is detached or collected before the write finishes; a resizable ArrayBuffer could still have its pages decommitted by a shrink, so its contents are copied when the job is created.

Tests: new test-fs-writefile-one-roundtrip.js (flags a/wx, mode, failing syscall and path, every ArrayBufferView kind, a multi-megabyte buffer, both APIs); tests that used fs.writeFile() as a proxy for open/close trace events or injected FileHandle faults into path-based writes are adjusted to keep testing what they test; test-fs-*write*, *append*, test-fs-promises*, permission and async-hooks fs tests pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Aug 22, 2026
@codecov

codecovBot commented Aug 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.02994% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.94%. Comparing base (927dfad) to head (f86c085).
⚠️ Report is 1 commits behind head on main.

Files with missing linesPatch %Lines
src/node_file.cc81.57%7 Missing and 14 partials ⚠️
lib/internal/fs/promises.js88.57%4 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #65489 +/- ##
==========================================
- Coverage 89.96% 89.94% -0.02% 
==========================================
Files 757 757 Lines 258049 258215 +166 Branches 48925 48955 +30 ==========================================
+ Hits 232144 232243 +99 - Misses 16965 17028 +63 - Partials 8940 8944 +4 
Files with missing linesCoverage Δ
lib/fs.js97.30% <100.00%> (-0.04%)⬇️
lib/internal/fs/promises.js91.24% <88.57%> (-0.09%)⬇️
src/node_file.cc74.90% <81.57%> (+0.41%)⬆️

... and 31 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 9fb687f to 4c3fcd6CompareAugust 22, 2026 18:17
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 23, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 23, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadsrc/node_file.cc Outdated
Comment threadtest/parallel/test-fs-writefile-one-roundtrip.js
@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 4c3fcd6 to a1d02deCompareAugust 25, 2026 12:26
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from a1d02de to c8c92fbCompareAugust 27, 2026 08:09
Comment threadsrc/node_file.cc
@addaleaxaddaleax added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Aug 27, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from c8c92fb to 85ce7e1CompareAugust 28, 2026 15:54
@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 28, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 29, 2026
@nodejs-github-bot

This comment was marked as outdated.

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 30, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 30, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 31, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 31, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 85ce7e1 to db16496CompareSeptember 1, 2026 00:35
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added commit-queue PRs queued for automated landing through the Commit Queue. and removed author ready PRs with CI started, the required approvals, and no outstanding review comments. commit-queue PRs queued for automated landing through the Commit Queue. labels Sep 2, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from db16496 to f86c085CompareSeptember 2, 2026 20:46
@codebyterecodebytere added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Sep 2, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

codebytere added a commit that referenced this pull request Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65489
Refs: #65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@codebytere

Copy link
Copy Markdown
MemberAuthor

Landed in e324419

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author readyPRs with CI started, the required approvals, and no outstanding review comments.c++Issues and PRs that require attention from people who are familiar with C++.fsIssues and PRs related to file-system APIs and the fs module.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@codebytere@nodejs-github-bot@mcollina@jasnell@addaleax@anonrig
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fs: write files in one thread pool round trip - #65489

Closed
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip
Closed

fs: write files in one thread pool round trip#65489
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip

Conversation

@codebytere

@codebyterecodebytere commented Aug 22, 2026

Copy link
Copy Markdown
Member

Makes fs.writeFile(path, data) and fsPromises.writeFile(path, data) do open + write + close as one thread pool request (WriteFileJob, next to the ReadFileJob from #65327) instead of one request per step.

benchmark/fs/writefile-promises.js (buf, duration=2), 20 runs, this branch vs main; fs.writeFile() measured the same way with an equivalent callback loop:

concurrent=1concurrent=10
fsPromises.writeFile, 2 B+108 %+51 %
fsPromises.writeFile, 1 KiB+108 %+51 %
fsPromises.writeFile, 64 KiB+76 %+30 %
fs.writeFile, 1 KiB+26 %+38 %

(1 MiB stays on the chunked FileHandle path and is unchanged.)

fs.writeFile() took three libuv round trips (open, write, close), each its own request with its own queue wait, completion callback and JS/C++ crossing, and fsPromises.writeFile() did the same through a FileHandle. For the small files applications write most (settings, state, caches) the round trips are the cost, and each one occupies a pool slot that concurrent fs, dns.lookup() and crypto work is also queueing for.

WriteFileJob (an AsyncWrap + ThreadPoolWork) opens, writes the whole buffer (looping on short writes) and closes as one pool task, keeping the buffer alive until it is done. fs.writeFile() uses it for path arguments without flush; fsPromises.writeFile() additionally keeps data above one write chunk (512 KiB) and iterables on the FileHandle path, so large writes stay abortable between chunks as before. File descriptors, FileHandles, flush: true and an active VFS keep their existing paths. Open failures report syscall open with the path and write failures write, permission errors arrive through the callback/promise, an abort signalled while the write is in flight is still reported as an AbortError, and the job is an FSREQCALLBACK async resource that emits the write fs trace event.

Refs: #65327

The job holds the buffer's backing store, so the memory stays valid if the buffer is detached or collected before the write finishes; a resizable ArrayBuffer could still have its pages decommitted by a shrink, so its contents are copied when the job is created.

Tests: new test-fs-writefile-one-roundtrip.js (flags a/wx, mode, failing syscall and path, every ArrayBufferView kind, a multi-megabyte buffer, both APIs); tests that used fs.writeFile() as a proxy for open/close trace events or injected FileHandle faults into path-based writes are adjusted to keep testing what they test; test-fs-*write*, *append*, test-fs-promises*, permission and async-hooks fs tests pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Aug 22, 2026
@codecov

codecovBot commented Aug 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.02994% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.94%. Comparing base (927dfad) to head (f86c085).
⚠️ Report is 1 commits behind head on main.

Files with missing linesPatch %Lines
src/node_file.cc81.57%7 Missing and 14 partials ⚠️
lib/internal/fs/promises.js88.57%4 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #65489 +/- ##
==========================================
- Coverage 89.96% 89.94% -0.02% 
==========================================
Files 757 757 Lines 258049 258215 +166 Branches 48925 48955 +30 ==========================================
+ Hits 232144 232243 +99 - Misses 16965 17028 +63 - Partials 8940 8944 +4 
Files with missing linesCoverage Δ
lib/fs.js97.30% <100.00%> (-0.04%)⬇️
lib/internal/fs/promises.js91.24% <88.57%> (-0.09%)⬇️
src/node_file.cc74.90% <81.57%> (+0.41%)⬆️

... and 31 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 9fb687f to 4c3fcd6CompareAugust 22, 2026 18:17
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 23, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 23, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadsrc/node_file.cc Outdated
Comment threadtest/parallel/test-fs-writefile-one-roundtrip.js
@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 4c3fcd6 to a1d02deCompareAugust 25, 2026 12:26
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from a1d02de to c8c92fbCompareAugust 27, 2026 08:09
Comment threadsrc/node_file.cc
@addaleaxaddaleax added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Aug 27, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from c8c92fb to 85ce7e1CompareAugust 28, 2026 15:54
@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 28, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 29, 2026
@nodejs-github-bot

This comment was marked as outdated.

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 30, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 30, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 31, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 31, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 85ce7e1 to db16496CompareSeptember 1, 2026 00:35
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added commit-queue PRs queued for automated landing through the Commit Queue. and removed author ready PRs with CI started, the required approvals, and no outstanding review comments. commit-queue PRs queued for automated landing through the Commit Queue. labels Sep 2, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from db16496 to f86c085CompareSeptember 2, 2026 20:46
@codebyterecodebytere added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Sep 2, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

codebytere added a commit that referenced this pull request Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65489
Refs: #65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@codebytere

Copy link
Copy Markdown
MemberAuthor

Landed in e324419

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author readyPRs with CI started, the required approvals, and no outstanding review comments.c++Issues and PRs that require attention from people who are familiar with C++.fsIssues and PRs related to file-system APIs and the fs module.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@codebytere@nodejs-github-bot@mcollina@jasnell@addaleax@anonrig
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

fs: write files in one thread pool round trip - #65489

Closed
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip
Closed

fs: write files in one thread pool round trip#65489
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip

Conversation

@codebytere

@codebyterecodebytere commented Aug 22, 2026

Copy link
Copy Markdown
Member

Makes fs.writeFile(path, data) and fsPromises.writeFile(path, data) do open + write + close as one thread pool request (WriteFileJob, next to the ReadFileJob from #65327) instead of one request per step.

benchmark/fs/writefile-promises.js (buf, duration=2), 20 runs, this branch vs main; fs.writeFile() measured the same way with an equivalent callback loop:

concurrent=1concurrent=10
fsPromises.writeFile, 2 B+108 %+51 %
fsPromises.writeFile, 1 KiB+108 %+51 %
fsPromises.writeFile, 64 KiB+76 %+30 %
fs.writeFile, 1 KiB+26 %+38 %

(1 MiB stays on the chunked FileHandle path and is unchanged.)

fs.writeFile() took three libuv round trips (open, write, close), each its own request with its own queue wait, completion callback and JS/C++ crossing, and fsPromises.writeFile() did the same through a FileHandle. For the small files applications write most (settings, state, caches) the round trips are the cost, and each one occupies a pool slot that concurrent fs, dns.lookup() and crypto work is also queueing for.

WriteFileJob (an AsyncWrap + ThreadPoolWork) opens, writes the whole buffer (looping on short writes) and closes as one pool task, keeping the buffer alive until it is done. fs.writeFile() uses it for path arguments without flush; fsPromises.writeFile() additionally keeps data above one write chunk (512 KiB) and iterables on the FileHandle path, so large writes stay abortable between chunks as before. File descriptors, FileHandles, flush: true and an active VFS keep their existing paths. Open failures report syscall open with the path and write failures write, permission errors arrive through the callback/promise, an abort signalled while the write is in flight is still reported as an AbortError, and the job is an FSREQCALLBACK async resource that emits the write fs trace event.

Refs: #65327

The job holds the buffer's backing store, so the memory stays valid if the buffer is detached or collected before the write finishes; a resizable ArrayBuffer could still have its pages decommitted by a shrink, so its contents are copied when the job is created.

Tests: new test-fs-writefile-one-roundtrip.js (flags a/wx, mode, failing syscall and path, every ArrayBufferView kind, a multi-megabyte buffer, both APIs); tests that used fs.writeFile() as a proxy for open/close trace events or injected FileHandle faults into path-based writes are adjusted to keep testing what they test; test-fs-*write*, *append*, test-fs-promises*, permission and async-hooks fs tests pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Aug 22, 2026
@codecov

codecovBot commented Aug 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.02994% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.94%. Comparing base (927dfad) to head (f86c085).
⚠️ Report is 1 commits behind head on main.

Files with missing linesPatch %Lines
src/node_file.cc81.57%7 Missing and 14 partials ⚠️
lib/internal/fs/promises.js88.57%4 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #65489 +/- ##
==========================================
- Coverage 89.96% 89.94% -0.02% 
==========================================
Files 757 757 Lines 258049 258215 +166 Branches 48925 48955 +30 ==========================================
+ Hits 232144 232243 +99 - Misses 16965 17028 +63 - Partials 8940 8944 +4 
Files with missing linesCoverage Δ
lib/fs.js97.30% <100.00%> (-0.04%)⬇️
lib/internal/fs/promises.js91.24% <88.57%> (-0.09%)⬇️
src/node_file.cc74.90% <81.57%> (+0.41%)⬆️

... and 31 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 9fb687f to 4c3fcd6CompareAugust 22, 2026 18:17
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 23, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 23, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadsrc/node_file.cc Outdated
Comment threadtest/parallel/test-fs-writefile-one-roundtrip.js
@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 4c3fcd6 to a1d02deCompareAugust 25, 2026 12:26
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from a1d02de to c8c92fbCompareAugust 27, 2026 08:09
Comment threadsrc/node_file.cc
@addaleaxaddaleax added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Aug 27, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from c8c92fb to 85ce7e1CompareAugust 28, 2026 15:54
@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 28, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 29, 2026
@nodejs-github-bot

This comment was marked as outdated.

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 30, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 30, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 31, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 31, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 85ce7e1 to db16496CompareSeptember 1, 2026 00:35
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added commit-queue PRs queued for automated landing through the Commit Queue. and removed author ready PRs with CI started, the required approvals, and no outstanding review comments. commit-queue PRs queued for automated landing through the Commit Queue. labels Sep 2, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from db16496 to f86c085CompareSeptember 2, 2026 20:46
@codebyterecodebytere added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Sep 2, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

codebytere added a commit that referenced this pull request Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65489
Refs: #65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@codebytere

Copy link
Copy Markdown
MemberAuthor

Landed in e324419

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author readyPRs with CI started, the required approvals, and no outstanding review comments.c++Issues and PRs that require attention from people who are familiar with C++.fsIssues and PRs related to file-system APIs and the fs module.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@codebytere@nodejs-github-bot@mcollina@jasnell@addaleax@anonrig
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fs: write files in one thread pool round trip - #65489

Closed
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip
Closed

fs: write files in one thread pool round trip#65489
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip

Conversation

@codebytere

@codebyterecodebytere commented Aug 22, 2026

Copy link
Copy Markdown
Member

Makes fs.writeFile(path, data) and fsPromises.writeFile(path, data) do open + write + close as one thread pool request (WriteFileJob, next to the ReadFileJob from #65327) instead of one request per step.

benchmark/fs/writefile-promises.js (buf, duration=2), 20 runs, this branch vs main; fs.writeFile() measured the same way with an equivalent callback loop:

concurrent=1concurrent=10
fsPromises.writeFile, 2 B+108 %+51 %
fsPromises.writeFile, 1 KiB+108 %+51 %
fsPromises.writeFile, 64 KiB+76 %+30 %
fs.writeFile, 1 KiB+26 %+38 %

(1 MiB stays on the chunked FileHandle path and is unchanged.)

fs.writeFile() took three libuv round trips (open, write, close), each its own request with its own queue wait, completion callback and JS/C++ crossing, and fsPromises.writeFile() did the same through a FileHandle. For the small files applications write most (settings, state, caches) the round trips are the cost, and each one occupies a pool slot that concurrent fs, dns.lookup() and crypto work is also queueing for.

WriteFileJob (an AsyncWrap + ThreadPoolWork) opens, writes the whole buffer (looping on short writes) and closes as one pool task, keeping the buffer alive until it is done. fs.writeFile() uses it for path arguments without flush; fsPromises.writeFile() additionally keeps data above one write chunk (512 KiB) and iterables on the FileHandle path, so large writes stay abortable between chunks as before. File descriptors, FileHandles, flush: true and an active VFS keep their existing paths. Open failures report syscall open with the path and write failures write, permission errors arrive through the callback/promise, an abort signalled while the write is in flight is still reported as an AbortError, and the job is an FSREQCALLBACK async resource that emits the write fs trace event.

Refs: #65327

The job holds the buffer's backing store, so the memory stays valid if the buffer is detached or collected before the write finishes; a resizable ArrayBuffer could still have its pages decommitted by a shrink, so its contents are copied when the job is created.

Tests: new test-fs-writefile-one-roundtrip.js (flags a/wx, mode, failing syscall and path, every ArrayBufferView kind, a multi-megabyte buffer, both APIs); tests that used fs.writeFile() as a proxy for open/close trace events or injected FileHandle faults into path-based writes are adjusted to keep testing what they test; test-fs-*write*, *append*, test-fs-promises*, permission and async-hooks fs tests pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Aug 22, 2026
@codecov

codecovBot commented Aug 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.02994% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.94%. Comparing base (927dfad) to head (f86c085).
⚠️ Report is 1 commits behind head on main.

Files with missing linesPatch %Lines
src/node_file.cc81.57%7 Missing and 14 partials ⚠️
lib/internal/fs/promises.js88.57%4 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #65489 +/- ##
==========================================
- Coverage 89.96% 89.94% -0.02% 
==========================================
Files 757 757 Lines 258049 258215 +166 Branches 48925 48955 +30 ==========================================
+ Hits 232144 232243 +99 - Misses 16965 17028 +63 - Partials 8940 8944 +4 
Files with missing linesCoverage Δ
lib/fs.js97.30% <100.00%> (-0.04%)⬇️
lib/internal/fs/promises.js91.24% <88.57%> (-0.09%)⬇️
src/node_file.cc74.90% <81.57%> (+0.41%)⬆️

... and 31 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 9fb687f to 4c3fcd6CompareAugust 22, 2026 18:17
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 23, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 23, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadsrc/node_file.cc Outdated
Comment threadtest/parallel/test-fs-writefile-one-roundtrip.js
@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 4c3fcd6 to a1d02deCompareAugust 25, 2026 12:26
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from a1d02de to c8c92fbCompareAugust 27, 2026 08:09
Comment threadsrc/node_file.cc
@addaleaxaddaleax added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Aug 27, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from c8c92fb to 85ce7e1CompareAugust 28, 2026 15:54
@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 28, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 29, 2026
@nodejs-github-bot

This comment was marked as outdated.

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 30, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 30, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 31, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 31, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 85ce7e1 to db16496CompareSeptember 1, 2026 00:35
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added commit-queue PRs queued for automated landing through the Commit Queue. and removed author ready PRs with CI started, the required approvals, and no outstanding review comments. commit-queue PRs queued for automated landing through the Commit Queue. labels Sep 2, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from db16496 to f86c085CompareSeptember 2, 2026 20:46
@codebyterecodebytere added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Sep 2, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

codebytere added a commit that referenced this pull request Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65489
Refs: #65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@codebytere

Copy link
Copy Markdown
MemberAuthor

Landed in e324419

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author readyPRs with CI started, the required approvals, and no outstanding review comments.c++Issues and PRs that require attention from people who are familiar with C++.fsIssues and PRs related to file-system APIs and the fs module.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@codebytere@nodejs-github-bot@mcollina@jasnell@addaleax@anonrig
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fs: write files in one thread pool round trip - #65489

Closed
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip
Closed

fs: write files in one thread pool round trip#65489
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip

Conversation

@codebytere

@codebyterecodebytere commented Aug 22, 2026

Copy link
Copy Markdown
Member

Makes fs.writeFile(path, data) and fsPromises.writeFile(path, data) do open + write + close as one thread pool request (WriteFileJob, next to the ReadFileJob from #65327) instead of one request per step.

benchmark/fs/writefile-promises.js (buf, duration=2), 20 runs, this branch vs main; fs.writeFile() measured the same way with an equivalent callback loop:

concurrent=1concurrent=10
fsPromises.writeFile, 2 B+108 %+51 %
fsPromises.writeFile, 1 KiB+108 %+51 %
fsPromises.writeFile, 64 KiB+76 %+30 %
fs.writeFile, 1 KiB+26 %+38 %

(1 MiB stays on the chunked FileHandle path and is unchanged.)

fs.writeFile() took three libuv round trips (open, write, close), each its own request with its own queue wait, completion callback and JS/C++ crossing, and fsPromises.writeFile() did the same through a FileHandle. For the small files applications write most (settings, state, caches) the round trips are the cost, and each one occupies a pool slot that concurrent fs, dns.lookup() and crypto work is also queueing for.

WriteFileJob (an AsyncWrap + ThreadPoolWork) opens, writes the whole buffer (looping on short writes) and closes as one pool task, keeping the buffer alive until it is done. fs.writeFile() uses it for path arguments without flush; fsPromises.writeFile() additionally keeps data above one write chunk (512 KiB) and iterables on the FileHandle path, so large writes stay abortable between chunks as before. File descriptors, FileHandles, flush: true and an active VFS keep their existing paths. Open failures report syscall open with the path and write failures write, permission errors arrive through the callback/promise, an abort signalled while the write is in flight is still reported as an AbortError, and the job is an FSREQCALLBACK async resource that emits the write fs trace event.

Refs: #65327

The job holds the buffer's backing store, so the memory stays valid if the buffer is detached or collected before the write finishes; a resizable ArrayBuffer could still have its pages decommitted by a shrink, so its contents are copied when the job is created.

Tests: new test-fs-writefile-one-roundtrip.js (flags a/wx, mode, failing syscall and path, every ArrayBufferView kind, a multi-megabyte buffer, both APIs); tests that used fs.writeFile() as a proxy for open/close trace events or injected FileHandle faults into path-based writes are adjusted to keep testing what they test; test-fs-*write*, *append*, test-fs-promises*, permission and async-hooks fs tests pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Aug 22, 2026
@codecov

codecovBot commented Aug 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.02994% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.94%. Comparing base (927dfad) to head (f86c085).
⚠️ Report is 1 commits behind head on main.

Files with missing linesPatch %Lines
src/node_file.cc81.57%7 Missing and 14 partials ⚠️
lib/internal/fs/promises.js88.57%4 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #65489 +/- ##
==========================================
- Coverage 89.96% 89.94% -0.02% 
==========================================
Files 757 757 Lines 258049 258215 +166 Branches 48925 48955 +30 ==========================================
+ Hits 232144 232243 +99 - Misses 16965 17028 +63 - Partials 8940 8944 +4 
Files with missing linesCoverage Δ
lib/fs.js97.30% <100.00%> (-0.04%)⬇️
lib/internal/fs/promises.js91.24% <88.57%> (-0.09%)⬇️
src/node_file.cc74.90% <81.57%> (+0.41%)⬆️

... and 31 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 9fb687f to 4c3fcd6CompareAugust 22, 2026 18:17
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 23, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 23, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadsrc/node_file.cc Outdated
Comment threadtest/parallel/test-fs-writefile-one-roundtrip.js
@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 4c3fcd6 to a1d02deCompareAugust 25, 2026 12:26
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from a1d02de to c8c92fbCompareAugust 27, 2026 08:09
Comment threadsrc/node_file.cc
@addaleaxaddaleax added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Aug 27, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from c8c92fb to 85ce7e1CompareAugust 28, 2026 15:54
@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 28, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 29, 2026
@nodejs-github-bot

This comment was marked as outdated.

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 30, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 30, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 31, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 31, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 85ce7e1 to db16496CompareSeptember 1, 2026 00:35
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added commit-queue PRs queued for automated landing through the Commit Queue. and removed author ready PRs with CI started, the required approvals, and no outstanding review comments. commit-queue PRs queued for automated landing through the Commit Queue. labels Sep 2, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from db16496 to f86c085CompareSeptember 2, 2026 20:46
@codebyterecodebytere added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Sep 2, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

codebytere added a commit that referenced this pull request Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65489
Refs: #65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@codebytere

Copy link
Copy Markdown
MemberAuthor

Landed in e324419

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author readyPRs with CI started, the required approvals, and no outstanding review comments.c++Issues and PRs that require attention from people who are familiar with C++.fsIssues and PRs related to file-system APIs and the fs module.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@codebytere@nodejs-github-bot@mcollina@jasnell@addaleax@anonrig
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

fs: write files in one thread pool round trip - #65489

Closed
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip
Closed

fs: write files in one thread pool round trip#65489
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip

Conversation

@codebytere

@codebyterecodebytere commented Aug 22, 2026

Copy link
Copy Markdown
Member

Makes fs.writeFile(path, data) and fsPromises.writeFile(path, data) do open + write + close as one thread pool request (WriteFileJob, next to the ReadFileJob from #65327) instead of one request per step.

benchmark/fs/writefile-promises.js (buf, duration=2), 20 runs, this branch vs main; fs.writeFile() measured the same way with an equivalent callback loop:

concurrent=1concurrent=10
fsPromises.writeFile, 2 B+108 %+51 %
fsPromises.writeFile, 1 KiB+108 %+51 %
fsPromises.writeFile, 64 KiB+76 %+30 %
fs.writeFile, 1 KiB+26 %+38 %

(1 MiB stays on the chunked FileHandle path and is unchanged.)

fs.writeFile() took three libuv round trips (open, write, close), each its own request with its own queue wait, completion callback and JS/C++ crossing, and fsPromises.writeFile() did the same through a FileHandle. For the small files applications write most (settings, state, caches) the round trips are the cost, and each one occupies a pool slot that concurrent fs, dns.lookup() and crypto work is also queueing for.

WriteFileJob (an AsyncWrap + ThreadPoolWork) opens, writes the whole buffer (looping on short writes) and closes as one pool task, keeping the buffer alive until it is done. fs.writeFile() uses it for path arguments without flush; fsPromises.writeFile() additionally keeps data above one write chunk (512 KiB) and iterables on the FileHandle path, so large writes stay abortable between chunks as before. File descriptors, FileHandles, flush: true and an active VFS keep their existing paths. Open failures report syscall open with the path and write failures write, permission errors arrive through the callback/promise, an abort signalled while the write is in flight is still reported as an AbortError, and the job is an FSREQCALLBACK async resource that emits the write fs trace event.

Refs: #65327

The job holds the buffer's backing store, so the memory stays valid if the buffer is detached or collected before the write finishes; a resizable ArrayBuffer could still have its pages decommitted by a shrink, so its contents are copied when the job is created.

Tests: new test-fs-writefile-one-roundtrip.js (flags a/wx, mode, failing syscall and path, every ArrayBufferView kind, a multi-megabyte buffer, both APIs); tests that used fs.writeFile() as a proxy for open/close trace events or injected FileHandle faults into path-based writes are adjusted to keep testing what they test; test-fs-*write*, *append*, test-fs-promises*, permission and async-hooks fs tests pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Aug 22, 2026
@codecov

codecovBot commented Aug 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.02994% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.94%. Comparing base (927dfad) to head (f86c085).
⚠️ Report is 1 commits behind head on main.

Files with missing linesPatch %Lines
src/node_file.cc81.57%7 Missing and 14 partials ⚠️
lib/internal/fs/promises.js88.57%4 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #65489 +/- ##
==========================================
- Coverage 89.96% 89.94% -0.02% 
==========================================
Files 757 757 Lines 258049 258215 +166 Branches 48925 48955 +30 ==========================================
+ Hits 232144 232243 +99 - Misses 16965 17028 +63 - Partials 8940 8944 +4 
Files with missing linesCoverage Δ
lib/fs.js97.30% <100.00%> (-0.04%)⬇️
lib/internal/fs/promises.js91.24% <88.57%> (-0.09%)⬇️
src/node_file.cc74.90% <81.57%> (+0.41%)⬆️

... and 31 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 9fb687f to 4c3fcd6CompareAugust 22, 2026 18:17
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 23, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 23, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadsrc/node_file.cc Outdated
Comment threadtest/parallel/test-fs-writefile-one-roundtrip.js
@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 4c3fcd6 to a1d02deCompareAugust 25, 2026 12:26
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from a1d02de to c8c92fbCompareAugust 27, 2026 08:09
Comment threadsrc/node_file.cc
@addaleaxaddaleax added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Aug 27, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from c8c92fb to 85ce7e1CompareAugust 28, 2026 15:54
@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 28, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 29, 2026
@nodejs-github-bot

This comment was marked as outdated.

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 30, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 30, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 31, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 31, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from 85ce7e1 to db16496CompareSeptember 1, 2026 00:35
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added commit-queue PRs queued for automated landing through the Commit Queue. and removed author ready PRs with CI started, the required approvals, and no outstanding review comments. commit-queue PRs queued for automated landing through the Commit Queue. labels Sep 2, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytereforce-pushed the perf/fs-writefile-one-roundtrip branch from db16496 to f86c085CompareSeptember 2, 2026 20:46
@codebyterecodebytere added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Sep 2, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

codebytere added a commit that referenced this pull request Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65489
Refs: #65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@codebytere

Copy link
Copy Markdown
MemberAuthor

Landed in e324419

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author readyPRs with CI started, the required approvals, and no outstanding review comments.c++Issues and PRs that require attention from people who are familiar with C++.fsIssues and PRs related to file-system APIs and the fs module.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@codebytere@nodejs-github-bot@mcollina@jasnell@addaleax@anonrig