Skip to content

fs: use fast api calls for existsSync#49893

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
littledivy wants to merge 8 commits into from
Closed

fs: use fast api calls for existsSync#49893

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
f413971
fs: improve performance of `existsSync`
littledivy Sep 27, 2023
dd08af9
Format
littledivy Sep 27, 2023
ac4be44
Merge branch 'main' into perf-existsSync
littledivy Sep 27, 2023
69736c2
Add test for fast call
littledivy Sep 27, 2023
176cbdc
Fix lint js
littledivy Sep 27, 2023
8d331ed
Add test for true existsSync
littledivy Sep 27, 2023
7a37e48
Use SetFastMethod
littledivy Sep 27, 2023
e2e7341
Format
littledivy Sep 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions src/node_external_reference.h
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -27,6 +27,11 @@ using CFunctionCallbackWithStrings =
const v8::FastOneByteString& base);
using CFunctionWithUint32 = uint32_t (*)(v8::Local<v8::Value>,
const uint32_t input);
using CFunctionCallbackWithStringOptions =
bool (*)(v8::Local<v8::Object>,
const v8::FastOneByteString& input,
// NOLINTNEXTLINE(runtime/references) This is V8 api.
v8::FastApiCallbackOptions& options);

// This class manages the external references from the V8 heap
// to the C++ addresses in Node.js.
Expand All@@ -41,6 +46,7 @@ class ExternalReferenceRegistry {
V(CFunctionCallbackWithInt64) \
V(CFunctionCallbackWithBool) \
V(CFunctionCallbackWithString) \
V(CFunctionCallbackWithStringOptions) \
V(CFunctionCallbackWithStrings) \
V(CFunctionWithUint32) \
V(const v8::CFunctionInfo*) \
Expand Down
48 changes: 47 additions & 1 deletion src/node_file.cc
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -31,6 +31,7 @@
#include "node_stat_watcher.h"
#include "permission/permission.h"
#include "util-inl.h"
#include "v8-fast-api-calls.h"

#include "tracing/trace_event.h"

Expand All@@ -57,8 +58,11 @@ namespace fs {

using v8::Array;
using v8::BigInt;
using v8::CFunction;
using v8::Context;
using v8::EscapableHandleScope;
using v8::FastApiCallbackOptions;
using v8::FastOneByteString;
using v8::Function;
using v8::FunctionCallbackInfo;
using v8::FunctionTemplate;
Expand DownExpand Up@@ -1095,6 +1099,46 @@ static void ExistsSync(const FunctionCallbackInfo<Value>& args) {
args.GetReturnValue().Set(err == 0);
}

bool FastExistsSync(v8::Local<v8::Object> recv,
const FastOneByteString& string,
// NOLINTNEXTLINE(runtime/references) This is V8 api.
FastApiCallbackOptions& options) {
Environment* env = Environment::GetCurrent(recv->GetCreationContextChecked());

MaybeStackBuffer<char> path;

path.AllocateSufficientStorage(string.length + 1);
memcpy(path.out(), string.data, string.length);
path.SetLengthAndZeroTerminate(string.length);
Comment thread
anonrig marked this conversation as resolved.

if (UNLIKELY(!env->permission()->is_granted(
permission::PermissionScope::kFileSystemRead, path.ToStringView()))) {
options.fallback = true;
return false;
}

uv_fs_t req;
auto make = OnScopeLeave([&req]() { uv_fs_req_cleanup(&req); });
FS_SYNC_TRACE_BEGIN(access);
int err = uv_fs_access(nullptr, &req, path.out(), 0, nullptr);
FS_SYNC_TRACE_END(access);

#ifdef _WIN32
// In case of an invalid symlink, `uv_fs_access` on win32
// will **not** return an error and is therefore not enough.
// Double check with `uv_fs_stat()`.
if (err == 0) {
FS_SYNC_TRACE_BEGIN(stat);
err = uv_fs_stat(nullptr, &req, path.out(), nullptr);
FS_SYNC_TRACE_END(stat);
}
#endif // _WIN32
Comment thread
anonrig marked this conversation as resolved.
Comment on lines +1120 to +1135

@joyeecheungjoyeecheungSep 27, 2023•
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fs operation part can be wrapped in a helper and shared with the slow callback to avoid getting out of sync.


return err == 0;
}

CFunction fast_exists_sync_(CFunction::Make(FastExistsSync));

// Used to speed up module loading. Returns an array [string, boolean]
static void InternalModuleReadJSON(const FunctionCallbackInfo<Value>& args) {
Environment* env = Environment::GetCurrent(args);
Expand DownExpand Up@@ -3385,7 +3429,7 @@ static void CreatePerIsolateProperties(IsolateData* isolate_data,
SetMethod(isolate, target, "accessSync", AccessSync);
SetMethod(isolate, target, "close", Close);
SetMethod(isolate, target, "closeSync", CloseSync);
SetMethod(isolate, target, "existsSync", ExistsSync);
SetFastMethod(isolate, target, "existsSync", ExistsSync, &fast_exists_sync_);
SetMethod(isolate, target, "open", Open);
SetMethod(isolate, target, "openSync", OpenSync);
SetMethod(isolate, target, "openFileHandle", OpenFileHandle);
Expand DownExpand Up@@ -3512,6 +3556,8 @@ void RegisterExternalReferences(ExternalReferenceRegistry* registry) {
registry->Register(Close);
registry->Register(CloseSync);
registry->Register(ExistsSync);
registry->Register(FastExistsSync);
registry->Register(fast_exists_sync_.GetTypeInfo());
registry->Register(Open);
registry->Register(OpenSync);
registry->Register(OpenFileHandle);
Expand Down
9 changes: 9 additions & 0 deletions test/parallel/test-fs-exists.js
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -54,3 +54,12 @@ assert(!fs.existsSync(`${f}-NO`));
assert(!fs.existsSync());
assert(!fs.existsSync({}));
assert(!fs.existsSync(new URL('https://foo')));

{
// This test is to ensure that the v8 fast api works.
const oneBytePath = 'hello.txt';
for (let i = 0; i < 1e5; i++) {
assert(!fs.existsSync(oneBytePath));
Comment thread
anonrig marked this conversation as resolved.

@joyeecheungjoyeecheungSep 27, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be moved to test/pummel instead. But also, in general we need to avoid running tight loops in the tests to avoid introducing timeouts in the CI on the slower machines. Maybe it's already enough that the fast path is exercised in the benchmark..

@targostargosSep 28, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we use V8 natives API like they do unit test fast calls? IIRC you need to %PrepareForOptimization(fn), call the function, %OptimizeOnNextCall(fn), and call it again. That last call should be optimized.

@targostargosSep 29, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, while I'm not good enough with C++ to suggest how to implement it, I think we can put something into place (maybe only in debug builds?). For example, the fast version, when called, increases some counter that we can get from JavaScript for an assertion.

@joyeecheungjoyeecheungOct 2, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can keep an array of booleans for all fast APIs to see if they are called, and expose them to the JS land, toggling a boolean shouldn't be very expensive.

@joyeecheungjoyeecheungOct 2, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this PR though doing %PrepareForOptimization() and %OptimizeOnNextCall() in the tests may be fine. But we also need to check in JS land if the optimizing compiler is enabled at all in the test to avoid failing on builds that turns optimizations off, which would be tricky..

assert(fs.existsSync(__filename));
}
}