fs: read small files in one thread pool round trip - #65327

Merged
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
codebytere:perf/fs-readfile-one-roundtrip
Aug 19, 2026
Merged

fs: read small files in one thread pool round trip#65327
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
codebytere:perf/fs-readfile-one-roundtrip

Conversation

@codebytere

@codebyterecodebytere commented Aug 16, 2026

Copy link
Copy Markdown
Member

fs.readFile() / fs.promises.readFile() of small files get 3–6Γ— faster, and use one libuv thread-pool task instead of four, by doing open + fstat + read + close in a single round trip.

fs/readfile.js len=1024 concurrent=1 *** 204.95 % Β±2.29%
fs/readfile.js len=1024 concurrent=10 *** 299.99 % Β±4.58%
fs/readfile-promises.js len=1024 concurrent=1 *** 238.56 % Β±2.68%
fs/readfile-promises.js len=1024 concurrent=10 *** 500.14 % Β±5.17%
fs/readfile-promises.js len=524288 concurrent=10 encoding='utf-8' *** 132.83 % Β±8.34%
fs/readfile-partitioned.js len=1024 concurrent=10 (vs. zlib work) *** 240.87 %
fs/readfile*.js len β‰₯ 4 MiB ~0 % n.s. (one exception below)
fs.readFile() of 4 KiB files at concurrency 64: ~51 k β†’ ~306 k files/s; mixed with fs.stat + dns.lookup: ~66 k β†’ ~312 k ops/s

(Linux x64, --set duration=2, 30 runs.)

Today a path-based readFile issues open, fstat, read and close as four separate uv_fs_* requests, each with its own queue wait, completion callback and JS↔C++ crossing; the promise API does the same through a FileHandle. For small files those round trips are the whole cost, and each one takes a pool slot away from concurrent dns/zlib/crypto/fs work.

ReadFileJob (an AsyncWrap + ThreadPoolWork, provider FSREQCALLBACK) runs open + fstat + read-to-EOF + close as one task and returns the content. If the file is larger than one chunk (kReadFileBufferLength, 512 KiB) it stops after fstat and hands back the fd and size, and the existing chunked reader continues exactly as today (interleaved, abortable between chunks). Both readFiles use it for path arguments without a user buffer; fds and FileHandles are unchanged.

Preserved on purpose: identical results for every size/encoding; open errors report syscall: 'open' + path, read errors 'read'; permission errors arrive through the callback/promise; an abort that lands while the round trip is in flight still wins; the handed-back fd is tracked and closed like any other; size-0 files (procfs) are read to EOF.

One open point: 16–32 MiB reads via fs.promises.readFile(…, 'utf-8') at concurrency 10 measure βˆ’2…3 % (***), reproducibly; the same sizes as Buffers, via the callback API, or at concurrency 1 are flat. They take the hand-back path with identical syscalls, and direct timing shows ≀2 %, so I haven't pinned it down. If preferred, the promise API can keep its current path and only the callback API changes.

Tests:test-fs-readfile-one-roundtrip.js (new; also passes on current main): sizes across the 512 KiB threshold, encodings, flags, error shapes, abort before/during, fd/FileHandle inputs, no fd leak on hand-back, procfs/sysfs, async_hooks lifecycle. Adjusted to keep testing what they test: test/async-hooks/test-fsreqcallback-readFile.js (accepts one request instead of exactly four), test-graph.fsreq-readFile.js (reads a 512 KiB+1 file so the four-request chain keeps its shape), test-async-exec-resource-match.js (resource + β‰₯1 fs request), test-trace-events-fs-async.js (uses fs.fstat() for the fstat trace instead of readFile as a proxy), test-fs-promises-readfile.js (zero-size-liar case goes through a FileHandle), test-fs-promises-file-handle-{op,aggregate,close}-errors.js (use a >512 KiB file so the patched FileHandle path is taken). fs, async-hooks, permission, worker, process and child_process suites pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Aug 16, 2026
@codebytere
codebytereforce-pushed the perf/fs-readfile-one-roundtrip branch from 0eaf058 to 21db0abCompareAugust 16, 2026 16:54
@codecov

codecovBot commented Aug 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.55975% with 65 lines in your changes missing coverage. Please review.
βœ… Project coverage is 90.10%. Comparing base (30bff4a) to head (b626e93).
⚠️ Report is 43 commits behind head on main.

Files with missing linesPatch %Lines
src/node_file.cc69.34%35 Missing and 26 partials ⚠️
lib/internal/fs/promises.js92.85%4 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #65327 +/- ##
==========================================
- Coverage 90.13% 90.10% -0.04% 
==========================================
Files 752 752 Lines 251568 251915 +347 Branches 47270 47353 +83 ==========================================
+ Hits 226759 226976 +217 - Misses 16168 16266 +98 - Partials 8641 8673 +32 
Files with missing linesCoverage Ξ”
lib/fs.js98.39% <100.00%> (+0.02%)⬆️
lib/internal/fs/promises.js92.52% <92.85%> (-0.48%)⬇️
src/node_file.cc73.92% <69.34%> (-0.27%)⬇️

... and 37 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codebyterecodebytere added request-ci Add this label to start a Jenkins CI on a PR. and removed needs-ci PRs that need a full CI run. labels Aug 16, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 16, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadsrc/node_file.cc Outdated

@jasnelljasnell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test failure on AIX needs to be looked at. Also, are these reads actually abortable in any way?

Comment threadtest/parallel/test-fs-readfile-one-roundtrip.js Outdated
Comment threadsrc/node_file.cc
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytereforce-pushed the perf/fs-readfile-one-roundtrip branch from 21db0ab to b626e93CompareAugust 17, 2026 05:17
@codebytere

Copy link
Copy Markdown
MemberAuthor

@jasnell re aborts: the single round trip itself isn't interruptible once it's on the pool, same as an individual read req now; an already-aborted signal never schedules it, an abort that lands while it's in flight wins when it completes, and anything over one chunk hands the fd back to the existing chunked reader, so those stay abortable between chunks exactly as before.

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 17, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 17, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 19, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 542e2b2 into nodejs:mainAug 19, 2026
70 of 71 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 542e2b2

@nodejs-github-botnodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 19, 2026
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
SimenB added a commit to SimenB/jest that referenced this pull request Aug 27, 2026
mock-fs throws at require time on Node 26.8+ (nodejs/node#65327 made
fs.readFile bypass binding.open, breaking its ReadFileContext sniffing;
tschaub/mock-fs#447). The test only needed empty files on disk for glob
threshold matching, so resolve the relative and glob thresholds against
a checked-in fixture tree instead and drop the dependency.
davidgamero added a commit to davidgamero/javascript that referenced this pull request Aug 30, 2026
Node 26.8.0 rewrote fs.readFile (nodejs/node#65327) to do open, fstat,
read and close in a single thread pool round trip, so binding.open is no
longer called from JS. mock-fs recovers the ReadFileContext prototype by
intercepting binding.open during a dummy readFile, so it now gets
undefined and throws at require time:
TypeError: Cannot read properties of undefined (reading 'read')
at exports.patchReadFileContext (mock-fs/lib/readfilecontext.js:40:30)
Because it throws on require rather than in a test, it takes out
config_test.ts and file_auth_test.ts in full.
The matrix entries are floating majors, so setup-node resolves '26' to
whatever the newest 26.x is at run time. That is why main went red on
the merge of kubernetes-client#3022 without any change to the code under test: the branch
last ran CI on 26.7.0, and by the time it merged five days later the
runner had picked up 26.8.1.
Pinning to 26.7 restores a green build. It is a stopgap: mock-fs has had
no functional release since February 2025 and the upstream report
(tschaub/mock-fs#447) is unanswered, so the durable fix is to stop
depending on it.
codebytere added a commit that referenced this pull request Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65489
Refs: #65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.fsIssues and PRs related to file-system APIs and the fs module.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@codebytere@nodejs-github-bot@jasnell@anonrig
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

fs: read small files in one thread pool round trip - #65327

Merged
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
codebytere:perf/fs-readfile-one-roundtrip
Aug 19, 2026
Merged

fs: read small files in one thread pool round trip#65327
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
codebytere:perf/fs-readfile-one-roundtrip

Conversation

@codebytere

@codebyterecodebytere commented Aug 16, 2026

Copy link
Copy Markdown
Member

fs.readFile() / fs.promises.readFile() of small files get 3–6Γ— faster, and use one libuv thread-pool task instead of four, by doing open + fstat + read + close in a single round trip.

fs/readfile.js len=1024 concurrent=1 *** 204.95 % Β±2.29%
fs/readfile.js len=1024 concurrent=10 *** 299.99 % Β±4.58%
fs/readfile-promises.js len=1024 concurrent=1 *** 238.56 % Β±2.68%
fs/readfile-promises.js len=1024 concurrent=10 *** 500.14 % Β±5.17%
fs/readfile-promises.js len=524288 concurrent=10 encoding='utf-8' *** 132.83 % Β±8.34%
fs/readfile-partitioned.js len=1024 concurrent=10 (vs. zlib work) *** 240.87 %
fs/readfile*.js len β‰₯ 4 MiB ~0 % n.s. (one exception below)
fs.readFile() of 4 KiB files at concurrency 64: ~51 k β†’ ~306 k files/s; mixed with fs.stat + dns.lookup: ~66 k β†’ ~312 k ops/s

(Linux x64, --set duration=2, 30 runs.)

Today a path-based readFile issues open, fstat, read and close as four separate uv_fs_* requests, each with its own queue wait, completion callback and JS↔C++ crossing; the promise API does the same through a FileHandle. For small files those round trips are the whole cost, and each one takes a pool slot away from concurrent dns/zlib/crypto/fs work.

ReadFileJob (an AsyncWrap + ThreadPoolWork, provider FSREQCALLBACK) runs open + fstat + read-to-EOF + close as one task and returns the content. If the file is larger than one chunk (kReadFileBufferLength, 512 KiB) it stops after fstat and hands back the fd and size, and the existing chunked reader continues exactly as today (interleaved, abortable between chunks). Both readFiles use it for path arguments without a user buffer; fds and FileHandles are unchanged.

Preserved on purpose: identical results for every size/encoding; open errors report syscall: 'open' + path, read errors 'read'; permission errors arrive through the callback/promise; an abort that lands while the round trip is in flight still wins; the handed-back fd is tracked and closed like any other; size-0 files (procfs) are read to EOF.

One open point: 16–32 MiB reads via fs.promises.readFile(…, 'utf-8') at concurrency 10 measure βˆ’2…3 % (***), reproducibly; the same sizes as Buffers, via the callback API, or at concurrency 1 are flat. They take the hand-back path with identical syscalls, and direct timing shows ≀2 %, so I haven't pinned it down. If preferred, the promise API can keep its current path and only the callback API changes.

Tests:test-fs-readfile-one-roundtrip.js (new; also passes on current main): sizes across the 512 KiB threshold, encodings, flags, error shapes, abort before/during, fd/FileHandle inputs, no fd leak on hand-back, procfs/sysfs, async_hooks lifecycle. Adjusted to keep testing what they test: test/async-hooks/test-fsreqcallback-readFile.js (accepts one request instead of exactly four), test-graph.fsreq-readFile.js (reads a 512 KiB+1 file so the four-request chain keeps its shape), test-async-exec-resource-match.js (resource + β‰₯1 fs request), test-trace-events-fs-async.js (uses fs.fstat() for the fstat trace instead of readFile as a proxy), test-fs-promises-readfile.js (zero-size-liar case goes through a FileHandle), test-fs-promises-file-handle-{op,aggregate,close}-errors.js (use a >512 KiB file so the patched FileHandle path is taken). fs, async-hooks, permission, worker, process and child_process suites pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Aug 16, 2026
@codebytere
codebytereforce-pushed the perf/fs-readfile-one-roundtrip branch from 0eaf058 to 21db0abCompareAugust 16, 2026 16:54
@codecov

codecovBot commented Aug 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.55975% with 65 lines in your changes missing coverage. Please review.
βœ… Project coverage is 90.10%. Comparing base (30bff4a) to head (b626e93).
⚠️ Report is 43 commits behind head on main.

Files with missing linesPatch %Lines
src/node_file.cc69.34%35 Missing and 26 partials ⚠️
lib/internal/fs/promises.js92.85%4 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #65327 +/- ##
==========================================
- Coverage 90.13% 90.10% -0.04% 
==========================================
Files 752 752 Lines 251568 251915 +347 Branches 47270 47353 +83 ==========================================
+ Hits 226759 226976 +217 - Misses 16168 16266 +98 - Partials 8641 8673 +32 
Files with missing linesCoverage Ξ”
lib/fs.js98.39% <100.00%> (+0.02%)⬆️
lib/internal/fs/promises.js92.52% <92.85%> (-0.48%)⬇️
src/node_file.cc73.92% <69.34%> (-0.27%)⬇️

... and 37 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codebyterecodebytere added request-ci Add this label to start a Jenkins CI on a PR. and removed needs-ci PRs that need a full CI run. labels Aug 16, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 16, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadsrc/node_file.cc Outdated

@jasnelljasnell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test failure on AIX needs to be looked at. Also, are these reads actually abortable in any way?

Comment threadtest/parallel/test-fs-readfile-one-roundtrip.js Outdated
Comment threadsrc/node_file.cc
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytereforce-pushed the perf/fs-readfile-one-roundtrip branch from 21db0ab to b626e93CompareAugust 17, 2026 05:17
@codebytere

Copy link
Copy Markdown
MemberAuthor

@jasnell re aborts: the single round trip itself isn't interruptible once it's on the pool, same as an individual read req now; an already-aborted signal never schedules it, an abort that lands while it's in flight wins when it completes, and anything over one chunk hands the fd back to the existing chunked reader, so those stay abortable between chunks exactly as before.

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 17, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 17, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 19, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 542e2b2 into nodejs:mainAug 19, 2026
70 of 71 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 542e2b2

@nodejs-github-botnodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 19, 2026
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
SimenB added a commit to SimenB/jest that referenced this pull request Aug 27, 2026
mock-fs throws at require time on Node 26.8+ (nodejs/node#65327 made
fs.readFile bypass binding.open, breaking its ReadFileContext sniffing;
tschaub/mock-fs#447). The test only needed empty files on disk for glob
threshold matching, so resolve the relative and glob thresholds against
a checked-in fixture tree instead and drop the dependency.
davidgamero added a commit to davidgamero/javascript that referenced this pull request Aug 30, 2026
Node 26.8.0 rewrote fs.readFile (nodejs/node#65327) to do open, fstat,
read and close in a single thread pool round trip, so binding.open is no
longer called from JS. mock-fs recovers the ReadFileContext prototype by
intercepting binding.open during a dummy readFile, so it now gets
undefined and throws at require time:
TypeError: Cannot read properties of undefined (reading 'read')
at exports.patchReadFileContext (mock-fs/lib/readfilecontext.js:40:30)
Because it throws on require rather than in a test, it takes out
config_test.ts and file_auth_test.ts in full.
The matrix entries are floating majors, so setup-node resolves '26' to
whatever the newest 26.x is at run time. That is why main went red on
the merge of kubernetes-client#3022 without any change to the code under test: the branch
last ran CI on 26.7.0, and by the time it merged five days later the
runner had picked up 26.8.1.
Pinning to 26.7 restores a green build. It is a stopgap: mock-fs has had
no functional release since February 2025 and the upstream report
(tschaub/mock-fs#447) is unanswered, so the durable fix is to stop
depending on it.
codebytere added a commit that referenced this pull request Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65489
Refs: #65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.fsIssues and PRs related to file-system APIs and the fs module.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@codebytere@nodejs-github-bot@jasnell@anonrig
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fs: read small files in one thread pool round trip - #65327

Merged
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
codebytere:perf/fs-readfile-one-roundtrip
Aug 19, 2026
Merged

fs: read small files in one thread pool round trip#65327
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
codebytere:perf/fs-readfile-one-roundtrip

Conversation

@codebytere

@codebyterecodebytere commented Aug 16, 2026

Copy link
Copy Markdown
Member

fs.readFile() / fs.promises.readFile() of small files get 3–6Γ— faster, and use one libuv thread-pool task instead of four, by doing open + fstat + read + close in a single round trip.

fs/readfile.js len=1024 concurrent=1 *** 204.95 % Β±2.29%
fs/readfile.js len=1024 concurrent=10 *** 299.99 % Β±4.58%
fs/readfile-promises.js len=1024 concurrent=1 *** 238.56 % Β±2.68%
fs/readfile-promises.js len=1024 concurrent=10 *** 500.14 % Β±5.17%
fs/readfile-promises.js len=524288 concurrent=10 encoding='utf-8' *** 132.83 % Β±8.34%
fs/readfile-partitioned.js len=1024 concurrent=10 (vs. zlib work) *** 240.87 %
fs/readfile*.js len β‰₯ 4 MiB ~0 % n.s. (one exception below)
fs.readFile() of 4 KiB files at concurrency 64: ~51 k β†’ ~306 k files/s; mixed with fs.stat + dns.lookup: ~66 k β†’ ~312 k ops/s

(Linux x64, --set duration=2, 30 runs.)

Today a path-based readFile issues open, fstat, read and close as four separate uv_fs_* requests, each with its own queue wait, completion callback and JS↔C++ crossing; the promise API does the same through a FileHandle. For small files those round trips are the whole cost, and each one takes a pool slot away from concurrent dns/zlib/crypto/fs work.

ReadFileJob (an AsyncWrap + ThreadPoolWork, provider FSREQCALLBACK) runs open + fstat + read-to-EOF + close as one task and returns the content. If the file is larger than one chunk (kReadFileBufferLength, 512 KiB) it stops after fstat and hands back the fd and size, and the existing chunked reader continues exactly as today (interleaved, abortable between chunks). Both readFiles use it for path arguments without a user buffer; fds and FileHandles are unchanged.

Preserved on purpose: identical results for every size/encoding; open errors report syscall: 'open' + path, read errors 'read'; permission errors arrive through the callback/promise; an abort that lands while the round trip is in flight still wins; the handed-back fd is tracked and closed like any other; size-0 files (procfs) are read to EOF.

One open point: 16–32 MiB reads via fs.promises.readFile(…, 'utf-8') at concurrency 10 measure βˆ’2…3 % (***), reproducibly; the same sizes as Buffers, via the callback API, or at concurrency 1 are flat. They take the hand-back path with identical syscalls, and direct timing shows ≀2 %, so I haven't pinned it down. If preferred, the promise API can keep its current path and only the callback API changes.

Tests:test-fs-readfile-one-roundtrip.js (new; also passes on current main): sizes across the 512 KiB threshold, encodings, flags, error shapes, abort before/during, fd/FileHandle inputs, no fd leak on hand-back, procfs/sysfs, async_hooks lifecycle. Adjusted to keep testing what they test: test/async-hooks/test-fsreqcallback-readFile.js (accepts one request instead of exactly four), test-graph.fsreq-readFile.js (reads a 512 KiB+1 file so the four-request chain keeps its shape), test-async-exec-resource-match.js (resource + β‰₯1 fs request), test-trace-events-fs-async.js (uses fs.fstat() for the fstat trace instead of readFile as a proxy), test-fs-promises-readfile.js (zero-size-liar case goes through a FileHandle), test-fs-promises-file-handle-{op,aggregate,close}-errors.js (use a >512 KiB file so the patched FileHandle path is taken). fs, async-hooks, permission, worker, process and child_process suites pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Aug 16, 2026
@codebytere
codebytereforce-pushed the perf/fs-readfile-one-roundtrip branch from 0eaf058 to 21db0abCompareAugust 16, 2026 16:54
@codecov

codecovBot commented Aug 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.55975% with 65 lines in your changes missing coverage. Please review.
βœ… Project coverage is 90.10%. Comparing base (30bff4a) to head (b626e93).
⚠️ Report is 43 commits behind head on main.

Files with missing linesPatch %Lines
src/node_file.cc69.34%35 Missing and 26 partials ⚠️
lib/internal/fs/promises.js92.85%4 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #65327 +/- ##
==========================================
- Coverage 90.13% 90.10% -0.04% 
==========================================
Files 752 752 Lines 251568 251915 +347 Branches 47270 47353 +83 ==========================================
+ Hits 226759 226976 +217 - Misses 16168 16266 +98 - Partials 8641 8673 +32 
Files with missing linesCoverage Ξ”
lib/fs.js98.39% <100.00%> (+0.02%)⬆️
lib/internal/fs/promises.js92.52% <92.85%> (-0.48%)⬇️
src/node_file.cc73.92% <69.34%> (-0.27%)⬇️

... and 37 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codebyterecodebytere added request-ci Add this label to start a Jenkins CI on a PR. and removed needs-ci PRs that need a full CI run. labels Aug 16, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 16, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadsrc/node_file.cc Outdated

@jasnelljasnell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test failure on AIX needs to be looked at. Also, are these reads actually abortable in any way?

Comment threadtest/parallel/test-fs-readfile-one-roundtrip.js Outdated
Comment threadsrc/node_file.cc
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytereforce-pushed the perf/fs-readfile-one-roundtrip branch from 21db0ab to b626e93CompareAugust 17, 2026 05:17
@codebytere

Copy link
Copy Markdown
MemberAuthor

@jasnell re aborts: the single round trip itself isn't interruptible once it's on the pool, same as an individual read req now; an already-aborted signal never schedules it, an abort that lands while it's in flight wins when it completes, and anything over one chunk hands the fd back to the existing chunked reader, so those stay abortable between chunks exactly as before.

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 17, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 17, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 19, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 542e2b2 into nodejs:mainAug 19, 2026
70 of 71 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 542e2b2

@nodejs-github-botnodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 19, 2026
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
SimenB added a commit to SimenB/jest that referenced this pull request Aug 27, 2026
mock-fs throws at require time on Node 26.8+ (nodejs/node#65327 made
fs.readFile bypass binding.open, breaking its ReadFileContext sniffing;
tschaub/mock-fs#447). The test only needed empty files on disk for glob
threshold matching, so resolve the relative and glob thresholds against
a checked-in fixture tree instead and drop the dependency.
davidgamero added a commit to davidgamero/javascript that referenced this pull request Aug 30, 2026
Node 26.8.0 rewrote fs.readFile (nodejs/node#65327) to do open, fstat,
read and close in a single thread pool round trip, so binding.open is no
longer called from JS. mock-fs recovers the ReadFileContext prototype by
intercepting binding.open during a dummy readFile, so it now gets
undefined and throws at require time:
TypeError: Cannot read properties of undefined (reading 'read')
at exports.patchReadFileContext (mock-fs/lib/readfilecontext.js:40:30)
Because it throws on require rather than in a test, it takes out
config_test.ts and file_auth_test.ts in full.
The matrix entries are floating majors, so setup-node resolves '26' to
whatever the newest 26.x is at run time. That is why main went red on
the merge of kubernetes-client#3022 without any change to the code under test: the branch
last ran CI on 26.7.0, and by the time it merged five days later the
runner had picked up 26.8.1.
Pinning to 26.7 restores a green build. It is a stopgap: mock-fs has had
no functional release since February 2025 and the upstream report
(tschaub/mock-fs#447) is unanswered, so the durable fix is to stop
depending on it.
codebytere added a commit that referenced this pull request Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65489
Refs: #65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.fsIssues and PRs related to file-system APIs and the fs module.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@codebytere@nodejs-github-bot@jasnell@anonrig
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fs: read small files in one thread pool round trip - #65327

Merged
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
codebytere:perf/fs-readfile-one-roundtrip
Aug 19, 2026
Merged

fs: read small files in one thread pool round trip#65327
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
codebytere:perf/fs-readfile-one-roundtrip

Conversation

@codebytere

@codebyterecodebytere commented Aug 16, 2026

Copy link
Copy Markdown
Member

fs.readFile() / fs.promises.readFile() of small files get 3–6Γ— faster, and use one libuv thread-pool task instead of four, by doing open + fstat + read + close in a single round trip.

fs/readfile.js len=1024 concurrent=1 *** 204.95 % Β±2.29%
fs/readfile.js len=1024 concurrent=10 *** 299.99 % Β±4.58%
fs/readfile-promises.js len=1024 concurrent=1 *** 238.56 % Β±2.68%
fs/readfile-promises.js len=1024 concurrent=10 *** 500.14 % Β±5.17%
fs/readfile-promises.js len=524288 concurrent=10 encoding='utf-8' *** 132.83 % Β±8.34%
fs/readfile-partitioned.js len=1024 concurrent=10 (vs. zlib work) *** 240.87 %
fs/readfile*.js len β‰₯ 4 MiB ~0 % n.s. (one exception below)
fs.readFile() of 4 KiB files at concurrency 64: ~51 k β†’ ~306 k files/s; mixed with fs.stat + dns.lookup: ~66 k β†’ ~312 k ops/s

(Linux x64, --set duration=2, 30 runs.)

Today a path-based readFile issues open, fstat, read and close as four separate uv_fs_* requests, each with its own queue wait, completion callback and JS↔C++ crossing; the promise API does the same through a FileHandle. For small files those round trips are the whole cost, and each one takes a pool slot away from concurrent dns/zlib/crypto/fs work.

ReadFileJob (an AsyncWrap + ThreadPoolWork, provider FSREQCALLBACK) runs open + fstat + read-to-EOF + close as one task and returns the content. If the file is larger than one chunk (kReadFileBufferLength, 512 KiB) it stops after fstat and hands back the fd and size, and the existing chunked reader continues exactly as today (interleaved, abortable between chunks). Both readFiles use it for path arguments without a user buffer; fds and FileHandles are unchanged.

Preserved on purpose: identical results for every size/encoding; open errors report syscall: 'open' + path, read errors 'read'; permission errors arrive through the callback/promise; an abort that lands while the round trip is in flight still wins; the handed-back fd is tracked and closed like any other; size-0 files (procfs) are read to EOF.

One open point: 16–32 MiB reads via fs.promises.readFile(…, 'utf-8') at concurrency 10 measure βˆ’2…3 % (***), reproducibly; the same sizes as Buffers, via the callback API, or at concurrency 1 are flat. They take the hand-back path with identical syscalls, and direct timing shows ≀2 %, so I haven't pinned it down. If preferred, the promise API can keep its current path and only the callback API changes.

Tests:test-fs-readfile-one-roundtrip.js (new; also passes on current main): sizes across the 512 KiB threshold, encodings, flags, error shapes, abort before/during, fd/FileHandle inputs, no fd leak on hand-back, procfs/sysfs, async_hooks lifecycle. Adjusted to keep testing what they test: test/async-hooks/test-fsreqcallback-readFile.js (accepts one request instead of exactly four), test-graph.fsreq-readFile.js (reads a 512 KiB+1 file so the four-request chain keeps its shape), test-async-exec-resource-match.js (resource + β‰₯1 fs request), test-trace-events-fs-async.js (uses fs.fstat() for the fstat trace instead of readFile as a proxy), test-fs-promises-readfile.js (zero-size-liar case goes through a FileHandle), test-fs-promises-file-handle-{op,aggregate,close}-errors.js (use a >512 KiB file so the patched FileHandle path is taken). fs, async-hooks, permission, worker, process and child_process suites pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Aug 16, 2026
@codebytere
codebytereforce-pushed the perf/fs-readfile-one-roundtrip branch from 0eaf058 to 21db0abCompareAugust 16, 2026 16:54
@codecov

codecovBot commented Aug 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.55975% with 65 lines in your changes missing coverage. Please review.
βœ… Project coverage is 90.10%. Comparing base (30bff4a) to head (b626e93).
⚠️ Report is 43 commits behind head on main.

Files with missing linesPatch %Lines
src/node_file.cc69.34%35 Missing and 26 partials ⚠️
lib/internal/fs/promises.js92.85%4 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #65327 +/- ##
==========================================
- Coverage 90.13% 90.10% -0.04% 
==========================================
Files 752 752 Lines 251568 251915 +347 Branches 47270 47353 +83 ==========================================
+ Hits 226759 226976 +217 - Misses 16168 16266 +98 - Partials 8641 8673 +32 
Files with missing linesCoverage Ξ”
lib/fs.js98.39% <100.00%> (+0.02%)⬆️
lib/internal/fs/promises.js92.52% <92.85%> (-0.48%)⬇️
src/node_file.cc73.92% <69.34%> (-0.27%)⬇️

... and 37 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codebyterecodebytere added request-ci Add this label to start a Jenkins CI on a PR. and removed needs-ci PRs that need a full CI run. labels Aug 16, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 16, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadsrc/node_file.cc Outdated

@jasnelljasnell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test failure on AIX needs to be looked at. Also, are these reads actually abortable in any way?

Comment threadtest/parallel/test-fs-readfile-one-roundtrip.js Outdated
Comment threadsrc/node_file.cc
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytereforce-pushed the perf/fs-readfile-one-roundtrip branch from 21db0ab to b626e93CompareAugust 17, 2026 05:17
@codebytere

Copy link
Copy Markdown
MemberAuthor

@jasnell re aborts: the single round trip itself isn't interruptible once it's on the pool, same as an individual read req now; an already-aborted signal never schedules it, an abort that lands while it's in flight wins when it completes, and anything over one chunk hands the fd back to the existing chunked reader, so those stay abortable between chunks exactly as before.

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 17, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 17, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 19, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 542e2b2 into nodejs:mainAug 19, 2026
70 of 71 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 542e2b2

@nodejs-github-botnodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 19, 2026
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
SimenB added a commit to SimenB/jest that referenced this pull request Aug 27, 2026
mock-fs throws at require time on Node 26.8+ (nodejs/node#65327 made
fs.readFile bypass binding.open, breaking its ReadFileContext sniffing;
tschaub/mock-fs#447). The test only needed empty files on disk for glob
threshold matching, so resolve the relative and glob thresholds against
a checked-in fixture tree instead and drop the dependency.
davidgamero added a commit to davidgamero/javascript that referenced this pull request Aug 30, 2026
Node 26.8.0 rewrote fs.readFile (nodejs/node#65327) to do open, fstat,
read and close in a single thread pool round trip, so binding.open is no
longer called from JS. mock-fs recovers the ReadFileContext prototype by
intercepting binding.open during a dummy readFile, so it now gets
undefined and throws at require time:
TypeError: Cannot read properties of undefined (reading 'read')
at exports.patchReadFileContext (mock-fs/lib/readfilecontext.js:40:30)
Because it throws on require rather than in a test, it takes out
config_test.ts and file_auth_test.ts in full.
The matrix entries are floating majors, so setup-node resolves '26' to
whatever the newest 26.x is at run time. That is why main went red on
the merge of kubernetes-client#3022 without any change to the code under test: the branch
last ran CI on 26.7.0, and by the time it merged five days later the
runner had picked up 26.8.1.
Pinning to 26.7 restores a green build. It is a stopgap: mock-fs has had
no functional release since February 2025 and the upstream report
(tschaub/mock-fs#447) is unanswered, so the durable fix is to stop
depending on it.
codebytere added a commit that referenced this pull request Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65489
Refs: #65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.fsIssues and PRs related to file-system APIs and the fs module.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@codebytere@nodejs-github-bot@jasnell@anonrig
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

fs: read small files in one thread pool round trip - #65327

Merged
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
codebytere:perf/fs-readfile-one-roundtrip
Aug 19, 2026
Merged

fs: read small files in one thread pool round trip#65327
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
codebytere:perf/fs-readfile-one-roundtrip

Conversation

@codebytere

@codebyterecodebytere commented Aug 16, 2026

Copy link
Copy Markdown
Member

fs.readFile() / fs.promises.readFile() of small files get 3–6Γ— faster, and use one libuv thread-pool task instead of four, by doing open + fstat + read + close in a single round trip.

fs/readfile.js len=1024 concurrent=1 *** 204.95 % Β±2.29%
fs/readfile.js len=1024 concurrent=10 *** 299.99 % Β±4.58%
fs/readfile-promises.js len=1024 concurrent=1 *** 238.56 % Β±2.68%
fs/readfile-promises.js len=1024 concurrent=10 *** 500.14 % Β±5.17%
fs/readfile-promises.js len=524288 concurrent=10 encoding='utf-8' *** 132.83 % Β±8.34%
fs/readfile-partitioned.js len=1024 concurrent=10 (vs. zlib work) *** 240.87 %
fs/readfile*.js len β‰₯ 4 MiB ~0 % n.s. (one exception below)
fs.readFile() of 4 KiB files at concurrency 64: ~51 k β†’ ~306 k files/s; mixed with fs.stat + dns.lookup: ~66 k β†’ ~312 k ops/s

(Linux x64, --set duration=2, 30 runs.)

Today a path-based readFile issues open, fstat, read and close as four separate uv_fs_* requests, each with its own queue wait, completion callback and JS↔C++ crossing; the promise API does the same through a FileHandle. For small files those round trips are the whole cost, and each one takes a pool slot away from concurrent dns/zlib/crypto/fs work.

ReadFileJob (an AsyncWrap + ThreadPoolWork, provider FSREQCALLBACK) runs open + fstat + read-to-EOF + close as one task and returns the content. If the file is larger than one chunk (kReadFileBufferLength, 512 KiB) it stops after fstat and hands back the fd and size, and the existing chunked reader continues exactly as today (interleaved, abortable between chunks). Both readFiles use it for path arguments without a user buffer; fds and FileHandles are unchanged.

Preserved on purpose: identical results for every size/encoding; open errors report syscall: 'open' + path, read errors 'read'; permission errors arrive through the callback/promise; an abort that lands while the round trip is in flight still wins; the handed-back fd is tracked and closed like any other; size-0 files (procfs) are read to EOF.

One open point: 16–32 MiB reads via fs.promises.readFile(…, 'utf-8') at concurrency 10 measure βˆ’2…3 % (***), reproducibly; the same sizes as Buffers, via the callback API, or at concurrency 1 are flat. They take the hand-back path with identical syscalls, and direct timing shows ≀2 %, so I haven't pinned it down. If preferred, the promise API can keep its current path and only the callback API changes.

Tests:test-fs-readfile-one-roundtrip.js (new; also passes on current main): sizes across the 512 KiB threshold, encodings, flags, error shapes, abort before/during, fd/FileHandle inputs, no fd leak on hand-back, procfs/sysfs, async_hooks lifecycle. Adjusted to keep testing what they test: test/async-hooks/test-fsreqcallback-readFile.js (accepts one request instead of exactly four), test-graph.fsreq-readFile.js (reads a 512 KiB+1 file so the four-request chain keeps its shape), test-async-exec-resource-match.js (resource + β‰₯1 fs request), test-trace-events-fs-async.js (uses fs.fstat() for the fstat trace instead of readFile as a proxy), test-fs-promises-readfile.js (zero-size-liar case goes through a FileHandle), test-fs-promises-file-handle-{op,aggregate,close}-errors.js (use a >512 KiB file so the patched FileHandle path is taken). fs, async-hooks, permission, worker, process and child_process suites pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Aug 16, 2026
@codebytere
codebytereforce-pushed the perf/fs-readfile-one-roundtrip branch from 0eaf058 to 21db0abCompareAugust 16, 2026 16:54
@codecov

codecovBot commented Aug 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.55975% with 65 lines in your changes missing coverage. Please review.
βœ… Project coverage is 90.10%. Comparing base (30bff4a) to head (b626e93).
⚠️ Report is 43 commits behind head on main.

Files with missing linesPatch %Lines
src/node_file.cc69.34%35 Missing and 26 partials ⚠️
lib/internal/fs/promises.js92.85%4 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #65327 +/- ##
==========================================
- Coverage 90.13% 90.10% -0.04% 
==========================================
Files 752 752 Lines 251568 251915 +347 Branches 47270 47353 +83 ==========================================
+ Hits 226759 226976 +217 - Misses 16168 16266 +98 - Partials 8641 8673 +32 
Files with missing linesCoverage Ξ”
lib/fs.js98.39% <100.00%> (+0.02%)⬆️
lib/internal/fs/promises.js92.52% <92.85%> (-0.48%)⬇️
src/node_file.cc73.92% <69.34%> (-0.27%)⬇️

... and 37 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codebyterecodebytere added request-ci Add this label to start a Jenkins CI on a PR. and removed needs-ci PRs that need a full CI run. labels Aug 16, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 16, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadsrc/node_file.cc Outdated

@jasnelljasnell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test failure on AIX needs to be looked at. Also, are these reads actually abortable in any way?

Comment threadtest/parallel/test-fs-readfile-one-roundtrip.js Outdated
Comment threadsrc/node_file.cc
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytereforce-pushed the perf/fs-readfile-one-roundtrip branch from 21db0ab to b626e93CompareAugust 17, 2026 05:17
@codebytere

Copy link
Copy Markdown
MemberAuthor

@jasnell re aborts: the single round trip itself isn't interruptible once it's on the pool, same as an individual read req now; an already-aborted signal never schedules it, an abort that lands while it's in flight wins when it completes, and anything over one chunk hands the fd back to the existing chunked reader, so those stay abortable between chunks exactly as before.

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 17, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 17, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 19, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 542e2b2 into nodejs:mainAug 19, 2026
70 of 71 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 542e2b2

@nodejs-github-botnodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 19, 2026
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
SimenB added a commit to SimenB/jest that referenced this pull request Aug 27, 2026
mock-fs throws at require time on Node 26.8+ (nodejs/node#65327 made
fs.readFile bypass binding.open, breaking its ReadFileContext sniffing;
tschaub/mock-fs#447). The test only needed empty files on disk for glob
threshold matching, so resolve the relative and glob thresholds against
a checked-in fixture tree instead and drop the dependency.
davidgamero added a commit to davidgamero/javascript that referenced this pull request Aug 30, 2026
Node 26.8.0 rewrote fs.readFile (nodejs/node#65327) to do open, fstat,
read and close in a single thread pool round trip, so binding.open is no
longer called from JS. mock-fs recovers the ReadFileContext prototype by
intercepting binding.open during a dummy readFile, so it now gets
undefined and throws at require time:
TypeError: Cannot read properties of undefined (reading 'read')
at exports.patchReadFileContext (mock-fs/lib/readfilecontext.js:40:30)
Because it throws on require rather than in a test, it takes out
config_test.ts and file_auth_test.ts in full.
The matrix entries are floating majors, so setup-node resolves '26' to
whatever the newest 26.x is at run time. That is why main went red on
the merge of kubernetes-client#3022 without any change to the code under test: the branch
last ran CI on 26.7.0, and by the time it merged five days later the
runner had picked up 26.8.1.
Pinning to 26.7 restores a green build. It is a stopgap: mock-fs has had
no functional release since February 2025 and the upstream report
(tschaub/mock-fs#447) is unanswered, so the durable fix is to stop
depending on it.
codebytere added a commit that referenced this pull request Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65489
Refs: #65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.fsIssues and PRs related to file-system APIs and the fs module.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@codebytere@nodejs-github-bot@jasnell@anonrig
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fs: read small files in one thread pool round trip - #65327

Merged
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
codebytere:perf/fs-readfile-one-roundtrip
Aug 19, 2026
Merged

fs: read small files in one thread pool round trip#65327
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
codebytere:perf/fs-readfile-one-roundtrip

Conversation

@codebytere

@codebyterecodebytere commented Aug 16, 2026

Copy link
Copy Markdown
Member

fs.readFile() / fs.promises.readFile() of small files get 3–6Γ— faster, and use one libuv thread-pool task instead of four, by doing open + fstat + read + close in a single round trip.

fs/readfile.js len=1024 concurrent=1 *** 204.95 % Β±2.29%
fs/readfile.js len=1024 concurrent=10 *** 299.99 % Β±4.58%
fs/readfile-promises.js len=1024 concurrent=1 *** 238.56 % Β±2.68%
fs/readfile-promises.js len=1024 concurrent=10 *** 500.14 % Β±5.17%
fs/readfile-promises.js len=524288 concurrent=10 encoding='utf-8' *** 132.83 % Β±8.34%
fs/readfile-partitioned.js len=1024 concurrent=10 (vs. zlib work) *** 240.87 %
fs/readfile*.js len β‰₯ 4 MiB ~0 % n.s. (one exception below)
fs.readFile() of 4 KiB files at concurrency 64: ~51 k β†’ ~306 k files/s; mixed with fs.stat + dns.lookup: ~66 k β†’ ~312 k ops/s

(Linux x64, --set duration=2, 30 runs.)

Today a path-based readFile issues open, fstat, read and close as four separate uv_fs_* requests, each with its own queue wait, completion callback and JS↔C++ crossing; the promise API does the same through a FileHandle. For small files those round trips are the whole cost, and each one takes a pool slot away from concurrent dns/zlib/crypto/fs work.

ReadFileJob (an AsyncWrap + ThreadPoolWork, provider FSREQCALLBACK) runs open + fstat + read-to-EOF + close as one task and returns the content. If the file is larger than one chunk (kReadFileBufferLength, 512 KiB) it stops after fstat and hands back the fd and size, and the existing chunked reader continues exactly as today (interleaved, abortable between chunks). Both readFiles use it for path arguments without a user buffer; fds and FileHandles are unchanged.

Preserved on purpose: identical results for every size/encoding; open errors report syscall: 'open' + path, read errors 'read'; permission errors arrive through the callback/promise; an abort that lands while the round trip is in flight still wins; the handed-back fd is tracked and closed like any other; size-0 files (procfs) are read to EOF.

One open point: 16–32 MiB reads via fs.promises.readFile(…, 'utf-8') at concurrency 10 measure βˆ’2…3 % (***), reproducibly; the same sizes as Buffers, via the callback API, or at concurrency 1 are flat. They take the hand-back path with identical syscalls, and direct timing shows ≀2 %, so I haven't pinned it down. If preferred, the promise API can keep its current path and only the callback API changes.

Tests:test-fs-readfile-one-roundtrip.js (new; also passes on current main): sizes across the 512 KiB threshold, encodings, flags, error shapes, abort before/during, fd/FileHandle inputs, no fd leak on hand-back, procfs/sysfs, async_hooks lifecycle. Adjusted to keep testing what they test: test/async-hooks/test-fsreqcallback-readFile.js (accepts one request instead of exactly four), test-graph.fsreq-readFile.js (reads a 512 KiB+1 file so the four-request chain keeps its shape), test-async-exec-resource-match.js (resource + β‰₯1 fs request), test-trace-events-fs-async.js (uses fs.fstat() for the fstat trace instead of readFile as a proxy), test-fs-promises-readfile.js (zero-size-liar case goes through a FileHandle), test-fs-promises-file-handle-{op,aggregate,close}-errors.js (use a >512 KiB file so the patched FileHandle path is taken). fs, async-hooks, permission, worker, process and child_process suites pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Aug 16, 2026
@codebytere
codebytereforce-pushed the perf/fs-readfile-one-roundtrip branch from 0eaf058 to 21db0abCompareAugust 16, 2026 16:54
@codecov

codecovBot commented Aug 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.55975% with 65 lines in your changes missing coverage. Please review.
βœ… Project coverage is 90.10%. Comparing base (30bff4a) to head (b626e93).
⚠️ Report is 43 commits behind head on main.

Files with missing linesPatch %Lines
src/node_file.cc69.34%35 Missing and 26 partials ⚠️
lib/internal/fs/promises.js92.85%4 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #65327 +/- ##
==========================================
- Coverage 90.13% 90.10% -0.04% 
==========================================
Files 752 752 Lines 251568 251915 +347 Branches 47270 47353 +83 ==========================================
+ Hits 226759 226976 +217 - Misses 16168 16266 +98 - Partials 8641 8673 +32 
Files with missing linesCoverage Ξ”
lib/fs.js98.39% <100.00%> (+0.02%)⬆️
lib/internal/fs/promises.js92.52% <92.85%> (-0.48%)⬇️
src/node_file.cc73.92% <69.34%> (-0.27%)⬇️

... and 37 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codebyterecodebytere added request-ci Add this label to start a Jenkins CI on a PR. and removed needs-ci PRs that need a full CI run. labels Aug 16, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 16, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadsrc/node_file.cc Outdated

@jasnelljasnell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test failure on AIX needs to be looked at. Also, are these reads actually abortable in any way?

Comment threadtest/parallel/test-fs-readfile-one-roundtrip.js Outdated
Comment threadsrc/node_file.cc
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytereforce-pushed the perf/fs-readfile-one-roundtrip branch from 21db0ab to b626e93CompareAugust 17, 2026 05:17
@codebytere

Copy link
Copy Markdown
MemberAuthor

@jasnell re aborts: the single round trip itself isn't interruptible once it's on the pool, same as an individual read req now; an already-aborted signal never schedules it, an abort that lands while it's in flight wins when it completes, and anything over one chunk hands the fd back to the existing chunked reader, so those stay abortable between chunks exactly as before.

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 17, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 17, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 19, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 542e2b2 into nodejs:mainAug 19, 2026
70 of 71 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 542e2b2

@nodejs-github-botnodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 19, 2026
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
SimenB added a commit to SimenB/jest that referenced this pull request Aug 27, 2026
mock-fs throws at require time on Node 26.8+ (nodejs/node#65327 made
fs.readFile bypass binding.open, breaking its ReadFileContext sniffing;
tschaub/mock-fs#447). The test only needed empty files on disk for glob
threshold matching, so resolve the relative and glob thresholds against
a checked-in fixture tree instead and drop the dependency.
davidgamero added a commit to davidgamero/javascript that referenced this pull request Aug 30, 2026
Node 26.8.0 rewrote fs.readFile (nodejs/node#65327) to do open, fstat,
read and close in a single thread pool round trip, so binding.open is no
longer called from JS. mock-fs recovers the ReadFileContext prototype by
intercepting binding.open during a dummy readFile, so it now gets
undefined and throws at require time:
TypeError: Cannot read properties of undefined (reading 'read')
at exports.patchReadFileContext (mock-fs/lib/readfilecontext.js:40:30)
Because it throws on require rather than in a test, it takes out
config_test.ts and file_auth_test.ts in full.
The matrix entries are floating majors, so setup-node resolves '26' to
whatever the newest 26.x is at run time. That is why main went red on
the merge of kubernetes-client#3022 without any change to the code under test: the branch
last ran CI on 26.7.0, and by the time it merged five days later the
runner had picked up 26.8.1.
Pinning to 26.7 restores a green build. It is a stopgap: mock-fs has had
no functional release since February 2025 and the upstream report
(tschaub/mock-fs#447) is unanswered, so the durable fix is to stop
depending on it.
codebytere added a commit that referenced this pull request Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65489
Refs: #65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.fsIssues and PRs related to file-system APIs and the fs module.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@codebytere@nodejs-github-bot@jasnell@anonrig
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fs: read small files in one thread pool round trip - #65327

Merged
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
codebytere:perf/fs-readfile-one-roundtrip
Aug 19, 2026
Merged

fs: read small files in one thread pool round trip#65327
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
codebytere:perf/fs-readfile-one-roundtrip

Conversation

@codebytere

@codebyterecodebytere commented Aug 16, 2026

Copy link
Copy Markdown
Member

fs.readFile() / fs.promises.readFile() of small files get 3–6Γ— faster, and use one libuv thread-pool task instead of four, by doing open + fstat + read + close in a single round trip.

fs/readfile.js len=1024 concurrent=1 *** 204.95 % Β±2.29%
fs/readfile.js len=1024 concurrent=10 *** 299.99 % Β±4.58%
fs/readfile-promises.js len=1024 concurrent=1 *** 238.56 % Β±2.68%
fs/readfile-promises.js len=1024 concurrent=10 *** 500.14 % Β±5.17%
fs/readfile-promises.js len=524288 concurrent=10 encoding='utf-8' *** 132.83 % Β±8.34%
fs/readfile-partitioned.js len=1024 concurrent=10 (vs. zlib work) *** 240.87 %
fs/readfile*.js len β‰₯ 4 MiB ~0 % n.s. (one exception below)
fs.readFile() of 4 KiB files at concurrency 64: ~51 k β†’ ~306 k files/s; mixed with fs.stat + dns.lookup: ~66 k β†’ ~312 k ops/s

(Linux x64, --set duration=2, 30 runs.)

Today a path-based readFile issues open, fstat, read and close as four separate uv_fs_* requests, each with its own queue wait, completion callback and JS↔C++ crossing; the promise API does the same through a FileHandle. For small files those round trips are the whole cost, and each one takes a pool slot away from concurrent dns/zlib/crypto/fs work.

ReadFileJob (an AsyncWrap + ThreadPoolWork, provider FSREQCALLBACK) runs open + fstat + read-to-EOF + close as one task and returns the content. If the file is larger than one chunk (kReadFileBufferLength, 512 KiB) it stops after fstat and hands back the fd and size, and the existing chunked reader continues exactly as today (interleaved, abortable between chunks). Both readFiles use it for path arguments without a user buffer; fds and FileHandles are unchanged.

Preserved on purpose: identical results for every size/encoding; open errors report syscall: 'open' + path, read errors 'read'; permission errors arrive through the callback/promise; an abort that lands while the round trip is in flight still wins; the handed-back fd is tracked and closed like any other; size-0 files (procfs) are read to EOF.

One open point: 16–32 MiB reads via fs.promises.readFile(…, 'utf-8') at concurrency 10 measure βˆ’2…3 % (***), reproducibly; the same sizes as Buffers, via the callback API, or at concurrency 1 are flat. They take the hand-back path with identical syscalls, and direct timing shows ≀2 %, so I haven't pinned it down. If preferred, the promise API can keep its current path and only the callback API changes.

Tests:test-fs-readfile-one-roundtrip.js (new; also passes on current main): sizes across the 512 KiB threshold, encodings, flags, error shapes, abort before/during, fd/FileHandle inputs, no fd leak on hand-back, procfs/sysfs, async_hooks lifecycle. Adjusted to keep testing what they test: test/async-hooks/test-fsreqcallback-readFile.js (accepts one request instead of exactly four), test-graph.fsreq-readFile.js (reads a 512 KiB+1 file so the four-request chain keeps its shape), test-async-exec-resource-match.js (resource + β‰₯1 fs request), test-trace-events-fs-async.js (uses fs.fstat() for the fstat trace instead of readFile as a proxy), test-fs-promises-readfile.js (zero-size-liar case goes through a FileHandle), test-fs-promises-file-handle-{op,aggregate,close}-errors.js (use a >512 KiB file so the patched FileHandle path is taken). fs, async-hooks, permission, worker, process and child_process suites pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Aug 16, 2026
@codebytere
codebytereforce-pushed the perf/fs-readfile-one-roundtrip branch from 0eaf058 to 21db0abCompareAugust 16, 2026 16:54
@codecov

codecovBot commented Aug 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.55975% with 65 lines in your changes missing coverage. Please review.
βœ… Project coverage is 90.10%. Comparing base (30bff4a) to head (b626e93).
⚠️ Report is 43 commits behind head on main.

Files with missing linesPatch %Lines
src/node_file.cc69.34%35 Missing and 26 partials ⚠️
lib/internal/fs/promises.js92.85%4 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #65327 +/- ##
==========================================
- Coverage 90.13% 90.10% -0.04% 
==========================================
Files 752 752 Lines 251568 251915 +347 Branches 47270 47353 +83 ==========================================
+ Hits 226759 226976 +217 - Misses 16168 16266 +98 - Partials 8641 8673 +32 
Files with missing linesCoverage Ξ”
lib/fs.js98.39% <100.00%> (+0.02%)⬆️
lib/internal/fs/promises.js92.52% <92.85%> (-0.48%)⬇️
src/node_file.cc73.92% <69.34%> (-0.27%)⬇️

... and 37 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codebyterecodebytere added request-ci Add this label to start a Jenkins CI on a PR. and removed needs-ci PRs that need a full CI run. labels Aug 16, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 16, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadsrc/node_file.cc Outdated

@jasnelljasnell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test failure on AIX needs to be looked at. Also, are these reads actually abortable in any way?

Comment threadtest/parallel/test-fs-readfile-one-roundtrip.js Outdated
Comment threadsrc/node_file.cc
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytereforce-pushed the perf/fs-readfile-one-roundtrip branch from 21db0ab to b626e93CompareAugust 17, 2026 05:17
@codebytere

Copy link
Copy Markdown
MemberAuthor

@jasnell re aborts: the single round trip itself isn't interruptible once it's on the pool, same as an individual read req now; an already-aborted signal never schedules it, an abort that lands while it's in flight wins when it completes, and anything over one chunk hands the fd back to the existing chunked reader, so those stay abortable between chunks exactly as before.

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 17, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 17, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 19, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 542e2b2 into nodejs:mainAug 19, 2026
70 of 71 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 542e2b2

@nodejs-github-botnodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 19, 2026
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
SimenB added a commit to SimenB/jest that referenced this pull request Aug 27, 2026
mock-fs throws at require time on Node 26.8+ (nodejs/node#65327 made
fs.readFile bypass binding.open, breaking its ReadFileContext sniffing;
tschaub/mock-fs#447). The test only needed empty files on disk for glob
threshold matching, so resolve the relative and glob thresholds against
a checked-in fixture tree instead and drop the dependency.
davidgamero added a commit to davidgamero/javascript that referenced this pull request Aug 30, 2026
Node 26.8.0 rewrote fs.readFile (nodejs/node#65327) to do open, fstat,
read and close in a single thread pool round trip, so binding.open is no
longer called from JS. mock-fs recovers the ReadFileContext prototype by
intercepting binding.open during a dummy readFile, so it now gets
undefined and throws at require time:
TypeError: Cannot read properties of undefined (reading 'read')
at exports.patchReadFileContext (mock-fs/lib/readfilecontext.js:40:30)
Because it throws on require rather than in a test, it takes out
config_test.ts and file_auth_test.ts in full.
The matrix entries are floating majors, so setup-node resolves '26' to
whatever the newest 26.x is at run time. That is why main went red on
the merge of kubernetes-client#3022 without any change to the code under test: the branch
last ran CI on 26.7.0, and by the time it merged five days later the
runner had picked up 26.8.1.
Pinning to 26.7 restores a green build. It is a stopgap: mock-fs has had
no functional release since February 2025 and the upstream report
(tschaub/mock-fs#447) is unanswered, so the durable fix is to stop
depending on it.
codebytere added a commit that referenced this pull request Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65489
Refs: #65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.fsIssues and PRs related to file-system APIs and the fs module.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@codebytere@nodejs-github-bot@jasnell@anonrig
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

fs: read small files in one thread pool round trip - #65327

Merged
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
codebytere:perf/fs-readfile-one-roundtrip
Aug 19, 2026
Merged

fs: read small files in one thread pool round trip#65327
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
codebytere:perf/fs-readfile-one-roundtrip

Conversation

@codebytere

@codebyterecodebytere commented Aug 16, 2026

Copy link
Copy Markdown
Member

fs.readFile() / fs.promises.readFile() of small files get 3–6Γ— faster, and use one libuv thread-pool task instead of four, by doing open + fstat + read + close in a single round trip.

fs/readfile.js len=1024 concurrent=1 *** 204.95 % Β±2.29%
fs/readfile.js len=1024 concurrent=10 *** 299.99 % Β±4.58%
fs/readfile-promises.js len=1024 concurrent=1 *** 238.56 % Β±2.68%
fs/readfile-promises.js len=1024 concurrent=10 *** 500.14 % Β±5.17%
fs/readfile-promises.js len=524288 concurrent=10 encoding='utf-8' *** 132.83 % Β±8.34%
fs/readfile-partitioned.js len=1024 concurrent=10 (vs. zlib work) *** 240.87 %
fs/readfile*.js len β‰₯ 4 MiB ~0 % n.s. (one exception below)
fs.readFile() of 4 KiB files at concurrency 64: ~51 k β†’ ~306 k files/s; mixed with fs.stat + dns.lookup: ~66 k β†’ ~312 k ops/s

(Linux x64, --set duration=2, 30 runs.)

Today a path-based readFile issues open, fstat, read and close as four separate uv_fs_* requests, each with its own queue wait, completion callback and JS↔C++ crossing; the promise API does the same through a FileHandle. For small files those round trips are the whole cost, and each one takes a pool slot away from concurrent dns/zlib/crypto/fs work.

ReadFileJob (an AsyncWrap + ThreadPoolWork, provider FSREQCALLBACK) runs open + fstat + read-to-EOF + close as one task and returns the content. If the file is larger than one chunk (kReadFileBufferLength, 512 KiB) it stops after fstat and hands back the fd and size, and the existing chunked reader continues exactly as today (interleaved, abortable between chunks). Both readFiles use it for path arguments without a user buffer; fds and FileHandles are unchanged.

Preserved on purpose: identical results for every size/encoding; open errors report syscall: 'open' + path, read errors 'read'; permission errors arrive through the callback/promise; an abort that lands while the round trip is in flight still wins; the handed-back fd is tracked and closed like any other; size-0 files (procfs) are read to EOF.

One open point: 16–32 MiB reads via fs.promises.readFile(…, 'utf-8') at concurrency 10 measure βˆ’2…3 % (***), reproducibly; the same sizes as Buffers, via the callback API, or at concurrency 1 are flat. They take the hand-back path with identical syscalls, and direct timing shows ≀2 %, so I haven't pinned it down. If preferred, the promise API can keep its current path and only the callback API changes.

Tests:test-fs-readfile-one-roundtrip.js (new; also passes on current main): sizes across the 512 KiB threshold, encodings, flags, error shapes, abort before/during, fd/FileHandle inputs, no fd leak on hand-back, procfs/sysfs, async_hooks lifecycle. Adjusted to keep testing what they test: test/async-hooks/test-fsreqcallback-readFile.js (accepts one request instead of exactly four), test-graph.fsreq-readFile.js (reads a 512 KiB+1 file so the four-request chain keeps its shape), test-async-exec-resource-match.js (resource + β‰₯1 fs request), test-trace-events-fs-async.js (uses fs.fstat() for the fstat trace instead of readFile as a proxy), test-fs-promises-readfile.js (zero-size-liar case goes through a FileHandle), test-fs-promises-file-handle-{op,aggregate,close}-errors.js (use a >512 KiB file so the patched FileHandle path is taken). fs, async-hooks, permission, worker, process and child_process suites pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Aug 16, 2026
@codebytere
codebytereforce-pushed the perf/fs-readfile-one-roundtrip branch from 0eaf058 to 21db0abCompareAugust 16, 2026 16:54
@codecov

codecovBot commented Aug 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.55975% with 65 lines in your changes missing coverage. Please review.
βœ… Project coverage is 90.10%. Comparing base (30bff4a) to head (b626e93).
⚠️ Report is 43 commits behind head on main.

Files with missing linesPatch %Lines
src/node_file.cc69.34%35 Missing and 26 partials ⚠️
lib/internal/fs/promises.js92.85%4 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #65327 +/- ##
==========================================
- Coverage 90.13% 90.10% -0.04% 
==========================================
Files 752 752 Lines 251568 251915 +347 Branches 47270 47353 +83 ==========================================
+ Hits 226759 226976 +217 - Misses 16168 16266 +98 - Partials 8641 8673 +32 
Files with missing linesCoverage Ξ”
lib/fs.js98.39% <100.00%> (+0.02%)⬆️
lib/internal/fs/promises.js92.52% <92.85%> (-0.48%)⬇️
src/node_file.cc73.92% <69.34%> (-0.27%)⬇️

... and 37 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codebyterecodebytere added request-ci Add this label to start a Jenkins CI on a PR. and removed needs-ci PRs that need a full CI run. labels Aug 16, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 16, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadsrc/node_file.cc Outdated

@jasnelljasnell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test failure on AIX needs to be looked at. Also, are these reads actually abortable in any way?

Comment threadtest/parallel/test-fs-readfile-one-roundtrip.js Outdated
Comment threadsrc/node_file.cc
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytereforce-pushed the perf/fs-readfile-one-roundtrip branch from 21db0ab to b626e93CompareAugust 17, 2026 05:17
@codebytere

Copy link
Copy Markdown
MemberAuthor

@jasnell re aborts: the single round trip itself isn't interruptible once it's on the pool, same as an individual read req now; an already-aborted signal never schedules it, an abort that lands while it's in flight wins when it completes, and anything over one chunk hands the fd back to the existing chunked reader, so those stay abortable between chunks exactly as before.

@codebyterecodebytere added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 17, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 17, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@codebyterecodebytere added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 19, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 542e2b2 into nodejs:mainAug 19, 2026
70 of 71 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 542e2b2

@nodejs-github-botnodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 19, 2026
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
fs.readFile(path) took four libuv thread pool round trips for a typical
small file -- open, fstat, read and close, each its own uv_fs request
with its own queue wait, completion callback and JS/C++ crossing -- and
fs.promises.readFile(path) did the same through a FileHandle. For the
small files applications read most, the round trips are the cost, and
each occupies a slot in the pool that concurrent dns.lookup(), fs and
crypto work is also queueing for.
Add ReadFileJob (an AsyncWrap + ThreadPoolWork) that performs
open + fstat + read-to-EOF + close as one pool task and reports the
whole content, or, when the file turns out to be larger than one chunk
(kReadFileBufferLength, 512 KiB), stops after fstat() and hands the fd
and size back so that the existing chunked reader continues unchanged
(large reads stay interleaved and abortable between chunks, and still
save the fstat round trip). fs.readFile() and fs.promises.readFile()
use it for path arguments without a user buffer; file descriptors,
FileHandles, options.buffer and an active VFS keep their paths.
Behavior is otherwise kept: same bytes for every size and encoding;
open failures report syscall 'open' with the path, read failures
'read'; permission errors are delivered through the callback/promise
as before; an abort that arrives while the read is in flight still
wins; the job is an FSREQCALLBACK resource for async_hooks; a handed
back fd is tracked exactly like one from a plain open().
Tests that asserted the internal open/fstat/read/close request chain,
used readFile() as a proxy for an fstat trace event, or injected
faults through FileHandle.prototype for path-based reads are adjusted
to keep testing what they test (a file just over one chunk where the
chain shape matters, fs.fstat() for the fstat trace, a larger file so
the FileHandle path is taken).
fs.readFile() of 4 KiB files at concurrency 64 goes from ~51k to ~306k
files per second, and a mixed stat/readFile/dns.lookup burst from ~66k
to ~312k operations per second.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65327
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
SimenB added a commit to SimenB/jest that referenced this pull request Aug 27, 2026
mock-fs throws at require time on Node 26.8+ (nodejs/node#65327 made
fs.readFile bypass binding.open, breaking its ReadFileContext sniffing;
tschaub/mock-fs#447). The test only needed empty files on disk for glob
threshold matching, so resolve the relative and glob thresholds against
a checked-in fixture tree instead and drop the dependency.
davidgamero added a commit to davidgamero/javascript that referenced this pull request Aug 30, 2026
Node 26.8.0 rewrote fs.readFile (nodejs/node#65327) to do open, fstat,
read and close in a single thread pool round trip, so binding.open is no
longer called from JS. mock-fs recovers the ReadFileContext prototype by
intercepting binding.open during a dummy readFile, so it now gets
undefined and throws at require time:
TypeError: Cannot read properties of undefined (reading 'read')
at exports.patchReadFileContext (mock-fs/lib/readfilecontext.js:40:30)
Because it throws on require rather than in a test, it takes out
config_test.ts and file_auth_test.ts in full.
The matrix entries are floating majors, so setup-node resolves '26' to
whatever the newest 26.x is at run time. That is why main went red on
the merge of kubernetes-client#3022 without any change to the code under test: the branch
last ran CI on 26.7.0, and by the time it merged five days later the
runner had picked up 26.8.1.
Pinning to 26.7 restores a green build. It is a stopgap: mock-fs has had
no functional release since February 2025 and the upstream report
(tschaub/mock-fs#447) is unanswered, so the durable fix is to stop
depending on it.
codebytere added a commit that referenced this pull request Sep 3, 2026
fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.
Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.
Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.
Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.
The job holds the buffer's backing store, so the memory stays valid if
the buffer is detached or collected before the write finishes; a
resizable ArrayBuffer could still have its pages decommitted by a
shrink, so its contents are copied when the job is created.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: #65489
Refs: #65327
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.fsIssues and PRs related to file-system APIs and the fs module.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@codebytere@nodejs-github-bot@jasnell@anonrig