Uh oh!
There was an error while loading. Please reload this page.
fs: write files in one thread pool round trip - #65489
Closed
codebytere wants to merge 1 commit into
Closed
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@## main #65489 +/- ##
==========================================
- Coverage 89.96% 89.94% -0.02%
==========================================
Files 757 757 Lines 258049 258215 +166 Branches 48925 48955 +30 ==========================================
+ Hits 232144 232243 +99 - Misses 16965 17028 +63 - Partials 8940 8944 +4
🚀 New features to boost your workflow:
|
codebytereforce-pushed
the
perf/fs-writefile-one-roundtrip
branch
from
August 22, 2026 18:17
9fb687f to
4c3fcd6Comparenodejs-github-bot
commented
Aug 23, 2026
Collaborator
nodejs-github-bot
commented
Aug 23, 2026
Collaborator
anonrig
approved these changes
Aug 24, 2026
nodejs-github-bot
commented
Aug 24, 2026
Collaborator
nodejs-github-bot
commented
Aug 24, 2026
Collaborator
nodejs-github-bot
commented
Aug 24, 2026
Collaborator
nodejs-github-bot
commented
Aug 25, 2026
Collaborator
nodejs-github-bot
commented
Aug 25, 2026
Collaborator
jasnell
reviewed
Aug 25, 2026
Uh oh!
There was an error while loading. Please reload this page.
jasnell
reviewed
Aug 25, 2026
Uh oh!
There was an error while loading. Please reload this page.
codebytereforce-pushed
the
perf/fs-writefile-one-roundtrip
branch
from
August 25, 2026 12:26
4c3fcd6 to
a1d02deComparejasnell
approved these changes
Aug 25, 2026
nodejs-github-bot
commented
Aug 25, 2026
Collaborator
codebytereforce-pushed
the
perf/fs-writefile-one-roundtrip
branch
from
August 27, 2026 08:09
a1d02de to
c8c92fbCompareaddaleax
approved these changes
Aug 27, 2026
Uh oh!
There was an error while loading. Please reload this page.
nodejs-github-bot
commented
Aug 28, 2026
Collaborator
codebytereforce-pushed
the
perf/fs-writefile-one-roundtrip
branch
from
August 28, 2026 15:54
c8c92fb to
85ce7e1Comparejasnell
approved these changes
Aug 29, 2026
This comment was marked as outdated.
This comment was marked as outdated.
nodejs-github-bot
commented
Aug 30, 2026
Collaborator
nodejs-github-bot
commented
Aug 31, 2026
Collaborator
nodejs-github-bot
commented
Aug 31, 2026
Collaborator
codebytereforce-pushed
the
perf/fs-writefile-one-roundtrip
branch
from
September 1, 2026 00:35
85ce7e1 to
db16496Compareanonrig
approved these changes
Sep 1, 2026
nodejs-github-bot
commented
Sep 1, 2026
Collaborator
nodejs-github-bot
commented
Sep 1, 2026
Collaborator
nodejs-github-bot
commented
Sep 2, 2026
Collaborator
3 tasks
fs.writeFile(path, data) took three libuv thread pool round trips (open, write, close), each its own request with its own queue wait, completion callback and JS/C++ crossing, and fs.promises.writeFile() did the same through a FileHandle. For the small files applications write most, the round trips are the cost, and each occupies a pool slot that concurrent fs, dns.lookup() and crypto work is also queueing for. Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork that opens, writes the whole buffer (looping on short writes) and closes as one pool task, keeping the buffer alive until it is done. fs.writeFile() uses it for path arguments without flush; fs.promises.writeFile() additionally keeps data above one write chunk (and iterables) on the FileHandle path, so large writes stay abortable between chunks as before. File descriptors, FileHandles, flush: true and an active VFS keep their existing paths. Behavior is otherwise kept: open failures report syscall 'open' with the path, write failures 'write'; permission errors are delivered through the callback/promise; an abort signalled while the write is in flight is still reported as an AbortError; the job is an FSREQCALLBACK resource for async_hooks and emits the 'write' fs trace event. Tests that used fs.writeFile() as a proxy for open/close trace events, or injected FileHandle faults for path-based writes, are adjusted to keep testing what they test. The job holds the buffer's backing store, so the memory stays valid if the buffer is detached or collected before the write finishes; a resizable ArrayBuffer could still have its pages decommitted by a shrink, so its contents are copied when the job is created. Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
codebytereforce-pushed
the
perf/fs-writefile-one-roundtrip
branch
from
September 2, 2026 20:46
db16496 to
f86c085Comparenodejs-github-bot
commented
Sep 2, 2026
Collaborator
3 tasks
codebytere added a commit
that referenced
this pull request
Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips (open, write, close), each its own request with its own queue wait, completion callback and JS/C++ crossing, and fs.promises.writeFile() did the same through a FileHandle. For the small files applications write most, the round trips are the cost, and each occupies a pool slot that concurrent fs, dns.lookup() and crypto work is also queueing for. Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork that opens, writes the whole buffer (looping on short writes) and closes as one pool task, keeping the buffer alive until it is done. fs.writeFile() uses it for path arguments without flush; fs.promises.writeFile() additionally keeps data above one write chunk (and iterables) on the FileHandle path, so large writes stay abortable between chunks as before. File descriptors, FileHandles, flush: true and an active VFS keep their existing paths. Behavior is otherwise kept: open failures report syscall 'open' with the path, write failures 'write'; permission errors are delivered through the callback/promise; an abort signalled while the write is in flight is still reported as an AbortError; the job is an FSREQCALLBACK resource for async_hooks and emits the 'write' fs trace event. Tests that used fs.writeFile() as a proxy for open/close trace events, or injected FileHandle faults for path-based writes, are adjusted to keep testing what they test. The job holds the buffer's backing store, so the memory stays valid if the buffer is detached or collected before the write finishes; a resizable ArrayBuffer could still have its pages decommitted by a shrink, so its contents are copied when the job is created. Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com> PR-URL: #65489 Refs: #65327 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>
codebytere
commented
Sep 3, 2026
MemberAuthor
Landed in e324419 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes
fs.writeFile(path, data)andfsPromises.writeFile(path, data)do open + write + close as one thread pool request (WriteFileJob, next to theReadFileJobfrom #65327) instead of one request per step.benchmark/fs/writefile-promises.js(buf,duration=2), 20 runs, this branch vsmain;fs.writeFile()measured the same way with an equivalent callback loop:fsPromises.writeFile, 2 BfsPromises.writeFile, 1 KiBfsPromises.writeFile, 64 KiBfs.writeFile, 1 KiB(1 MiB stays on the chunked
FileHandlepath and is unchanged.)fs.writeFile()took three libuv round trips (open, write, close), each its own request with its own queue wait, completion callback and JS/C++ crossing, andfsPromises.writeFile()did the same through aFileHandle. For the small files applications write most (settings, state, caches) the round trips are the cost, and each one occupies a pool slot that concurrent fs,dns.lookup()and crypto work is also queueing for.WriteFileJob(anAsyncWrap+ThreadPoolWork) opens, writes the whole buffer (looping on short writes) and closes as one pool task, keeping the buffer alive until it is done.fs.writeFile()uses it for path arguments withoutflush;fsPromises.writeFile()additionally keeps data above one write chunk (512 KiB) and iterables on theFileHandlepath, so large writes stay abortable between chunks as before. File descriptors,FileHandles,flush: trueand an active VFS keep their existing paths. Open failures report syscallopenwith the path and write failureswrite, permission errors arrive through the callback/promise, an abort signalled while the write is in flight is still reported as anAbortError, and the job is anFSREQCALLBACKasync resource that emits thewritefs trace event.Refs: #65327
The job holds the buffer's backing store, so the memory stays valid if the buffer is detached or collected before the write finishes; a resizable
ArrayBuffercould still have its pages decommitted by a shrink, so its contents are copied when the job is created.Tests: new
test-fs-writefile-one-roundtrip.js(flagsa/wx, mode, failing syscall and path, everyArrayBufferViewkind, a multi-megabyte buffer, both APIs); tests that usedfs.writeFile()as a proxy foropen/closetrace events or injectedFileHandlefaults into path-based writes are adjusted to keep testing what they test;test-fs-*write*,*append*,test-fs-promises*, permission and async-hooks fs tests pass.Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.