Skip to content

fs: add fast api for InternalModuleStat - #51344

Merged
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
anonrig:fast-api-internal-module-stat
Jun 16, 2024
Merged

fs: add fast api for InternalModuleStat#51344
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
anonrig:fast-api-internal-module-stat

Conversation

@anonrig

Copy link
Copy Markdown
Member

Adds V8 Fast API to InternalModuleStat

@anonriganonrig added the performance Issues and PRs related to the performance of Node.js. label Jan 2, 2024
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/loaders

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Jan 2, 2024
H4ad
H4ad approved these changes Jan 2, 2024
Comment threadsrc/node_file.cc Outdated
Comment on lines 1065 to 1058

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just thinking about this, if this is true, it will:

  • ignore the value -1.
  • re-run InternalModuleStat but in the slower version.
  • throw error since the permission is not granted.

I don't know if worthy but maybe one test just to ensure this behavior, maybe a loop and then pass a value that is not allowed.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a test (6 months later)

@H4ad

H4ad commented Jan 3, 2024

Copy link
Copy Markdown
Member
Error: --- stderr ---
npm WARN cli npm v10.2.4 does not support Node.js v22.0.0-pre. This version of npm supports the following node versions: `^18.17.0 || >=20.5.0`. You can find the latest version at [https://nodejs.org/.](https://nodejs.org/)
npm ERR! code MODULE_NOT_FOUND
npm ERR! path /home/runner/work/node/node/deps/npm/node_modules/walk-up-path/package.json
npm ERR! Cannot find module '/home/runner/work/node/node/deps/npm/node_modules/walk-up-path/dist/cjs/index.js'

Failing because of the Node version: https://github.com/nodejs/node/actions/runs/7390448695/job/20105393172?pr=51344#step:6:5719

Who we should ping to solve this kind of issue?

@anonrig

Copy link
Copy Markdown
MemberAuthor
Error: --- stderr ---
npm WARN cli npm v10.2.4 does not support Node.js v22.0.0-pre. This version of npm supports the following node versions: `^18.17.0 || >=20.5.0`. You can find the latest version at [https://nodejs.org/.](https://nodejs.org/)
npm ERR! code MODULE_NOT_FOUND
npm ERR! path /home/runner/work/node/node/deps/npm/node_modules/walk-up-path/package.json
npm ERR! Cannot find module '/home/runner/work/node/node/deps/npm/node_modules/walk-up-path/dist/cjs/index.js'

Failing because of the Node version: https://github.com/nodejs/node/actions/runs/7390448695/job/20105393172?pr=51344#step:6:5719

Who we should ping to solve this kind of issue?

cc @nodejs/build @nodejs/npm Any suggestions on how to fix the error?

@wraithgar

wraithgar commented Jan 3, 2024

Copy link
Copy Markdown
Contributor

There are two separate items in that log. First is a warning that will always show up when you're using a prerelease version of node. This is intentional to remind folks that they're in an unsupported prerelease version of node. npm still will install just fine, the part of npm that prevents itself from being installed globally in an unsupported node version throws a different exception and allows prereleases in its semver check against npm's engines field.

The second error has to do with a missing file that is being required. As to why that file is missing, that's another mystery. It's definitely in the tarball of the published module, and in source control.

@targos

Copy link
Copy Markdown
Member

The error must be related to the changes made in this PR. In other words, there's a bug in the implementation.

@aduh95

Copy link
Copy Markdown
Contributor

This needs a rebase.

@anonrig
anonrigforce-pushed the fast-api-internal-module-stat branch from 11cea33 to 3f32a24CompareJune 13, 2024 21:59
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@anonrig
anonrigforce-pushed the fast-api-internal-module-stat branch from 3f32a24 to 9b6a9f8CompareJune 13, 2024 22:13
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@anonrig
anonrigforce-pushed the fast-api-internal-module-stat branch from 9b6a9f8 to 1a71169CompareJune 14, 2024 13:22
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

is this improving startup time in any way?

@anonrig

Copy link
Copy Markdown
MemberAuthor

is this improving startup time in any way?

I don't know, but better ask our friend "benchmark ci": https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1570/

@anonrig

anonrig commented Jun 16, 2024

Copy link
Copy Markdown
MemberAuthor

@mcollina

misc/startup-cli-version.js count=30 cli='deps/corepack/dist/corepack.js' *** 0.14 % ±0.08% ±0.11% ±0.14%
misc/startup-cli-version.js count=30 cli='deps/npm/bin/npm-cli.js' 0.07 % ±0.19% ±0.25% ±0.33%
misc/startup-cli-version.js count=30 cli='deps/npm/bin/npx-cli.js' *** 0.31 % ±0.17% ±0.23% ±0.30%
misc/startup-cli-version.js count=30 cli='tools/node_modules/eslint/bin/eslint.js' *** -0.12 % ±0.05% ±0.06% ±0.08%
misc/startup-core.js count=30 mode='process' script='benchmark/fixtures/require-builtins' * -0.45 % ±0.36% ±0.48% ±0.62%
misc/startup-core.js count=30 mode='process' script='test/fixtures/semicolon' 2.38 % ±3.17% ±4.22% ±5.50%
misc/startup-core.js count=30 mode='process' script='test/fixtures/snapshot/typescript' *** 0.55 % ±0.05% ±0.06% ±0.08%
misc/startup-core.js count=30 mode='worker' script='benchmark/fixtures/require-builtins' -0.20 % ±0.23% ±0.31% ±0.40%
misc/startup-core.js count=30 mode='worker' script='test/fixtures/semicolon' 0.06 % ±0.37% ±0.49% ±0.65%
misc/startup-core.js count=30 mode='worker' script='test/fixtures/snapshot/typescript' 0.69 % ±0.85% ±1.13% ±1.47%```

@anonriganonrig added the commit-queue Add this label to land a pull request using GitHub Actions. label Jun 16, 2024
@nodejs-github-botnodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Jun 16, 2024
@nodejs-github-bot
nodejs-github-bot merged commit 1aa8a78 into nodejs:mainJun 16, 2024
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 1aa8a78

targos pushed a commit that referenced this pull request Jun 20, 2024
PR-URL: #51344
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
soophoo pushed a commit to soophoo/node that referenced this pull request Jun 20, 2024
PR-URL: nodejs#51344
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
@targostargos mentioned this pull request Jun 25, 2024
codebytere added a commit to electron/electron that referenced this pull request Oct 16, 2024
codebytere added a commit to electron/electron that referenced this pull request Oct 21, 2024
codebytere added a commit to electron/electron that referenced this pull request Oct 21, 2024
codebytere added a commit to electron/electron that referenced this pull request Oct 22, 2024
codebytere added a commit to electron/electron that referenced this pull request Oct 28, 2024
codebytere added a commit to electron/electron that referenced this pull request Oct 29, 2024
codebytere added a commit to electron/electron that referenced this pull request Oct 31, 2024
jkleinsc pushed a commit to electron/electron that referenced this pull request Nov 4, 2024
* chore: bump Node.js to v22.9.0
* build: drop base64 dep in GN build
nodejs/node#52856
* build,tools: make addons tests work with GN
nodejs/node#50737
* fs: add fast api for InternalModuleStat
nodejs/node#51344
* src: move package_json_reader cache to c++
nodejs/node#50322
* crypto: disable PKCS#1 padding for privateDecrypt
nodejs-private/node-private#525
* src: move more crypto code to ncrypto
nodejs/node#54320
* crypto: ensure valid point on elliptic curve in SubtleCrypto.importKey
nodejs/node#50234
* src: shift more crypto impl details to ncrypto
nodejs/node#54028
* src: switch crypto APIs to use Maybe<void>
nodejs/node#54775
* crypto: remove DEFAULT_ENCODING
nodejs/node#47182
* deps: update libuv to 1.47.0
nodejs/node#50650
* build: fix conflict gyp configs
nodejs/node#53605
* lib,src: drop --experimental-network-imports
nodejs/node#53822
* esm: align sync and async load implementations
nodejs/node#49152
* esm: remove unnecessary toNamespacedPath calls
nodejs/node#53656
* module: detect ESM syntax by trying to recompile as SourceTextModule
nodejs/node#52413
* test: adapt debugger tests to V8 11.4
nodejs/node#49639
* lib: update usage of always on Atomics API
nodejs/node#49639
* test: adapt test-fs-write to V8 internal changes
nodejs/node#49639
* test: adapt to new V8 trusted memory spaces
nodejs/node#50115
* deps: update libuv to 1.47.0
nodejs/node#50650
* src: use non-deprecated v8::Uint8Array::kMaxLength
nodejs/node#50115
* src: update default V8 platform to override functions with location
nodejs/node#51362
* src: add missing TryCatch
nodejs/node#51362
* lib,test: handle new Iterator global
nodejs/node#51362
* src: use non-deprecated version of CreateSyntheticModule
nodejs/node#50115
* src: remove calls to recently deprecated V8 APIs
nodejs/node#52996
* src: use new V8 API to define stream accessor
nodejs/node#53084
* src: do not use deprecated V8 API
nodejs/node#53084
* src: do not use soon-to-be-deprecated V8 API
nodejs/node#53174
* src: migrate to new V8 interceptors API
nodejs/node#52745
* src: use supported API to get stalled TLA messages
nodejs/node#51362
* module: print location of unsettled top-level await in entry points
nodejs/node#51999
* test: make snapshot comparison more flexible
nodejs/node#54375
* test: do not set concurrency on parallelized runs
nodejs/node#52177
* src: move FromNamespacedPath to path.cc
nodejs/node#53540
* test: adapt to new V8 trusted memory spaces
nodejs/node#50115
* build: add option to enable clang-cl on Windows
nodejs/node#52870
* chore: fixup patch indices
* chore: add/remove changed files
* esm: drop support for import assertions
nodejs/node#54890
* build: compile with C++20 support
nodejs/node#52838
* deps: update nghttp2 to 1.62.1
nodejs/node#52966
* src: parse inspector profiles with simdjson
nodejs/node#51783
* build: add GN build files
nodejs/node#47637
* deps,lib,src: add experimental web storage
nodejs/node#52435
* build: add missing BoringSSL dep
* src: rewrite task runner in c++
nodejs/node#52609
* fixup! build: add GN build files
* src: stop using deprecated fields of v8::FastApiCallbackOptions
nodejs/node#54077
* fix: shadow variable
* build: add back incorrectly removed SetAccessor patch
* fixup! fixup! build: add GN build files
* crypto: fix integer comparison in crypto for BoringSSL
* src,lib: reducing C++ calls of esm legacy main resolvenodejs/node#48325
* src: move more crypto_dh.cc code to ncrypto
nodejs/node#54459
* chore: fixup GN files for previous commit
* src: move more crypto code to ncrypto
nodejs/node#54320
* Fixup Perfetto ifdef guards
* fix: missing electron_natives dep
* fix: node_use_node_platform = false
* fix: include src/node_snapshot_stub.cc in libnode
* 5507047: [import-attributes] Remove support for import assertions
https://chromium-review.googlesource.com/c/v8/v8/+/5507047
* fix: restore v8-sandbox.h in filenames.json
* fix: re-add original-fs generation logic
* fix: ngtcp2 openssl dep
* test: try removing NAPI_VERSION undef
* chore(deps): bump @types/node
* src: move more crypto_dh.cc code to ncrypto
nodejs/node#54459
* esm: remove unnecessary toNamespacedPath calls
nodejs/node#53656
* buffer: fix out of range for toString
nodejs/node#54553
* lib: rewrite AsyncLocalStorage without async_hooks
nodejs/node#48528
* module: print amount of load time of a cjs module
nodejs/node#52213
* test: skip reproducible snapshot test on 32-bit
nodejs/node#53592
* fixup! src: move more crypto_dh.cc code to ncrypto
* test: adjust emittedUntil return type
* chore: remove redundant wpt streams patch
* fixup! chore(deps): bump @types/node
* fix: gn executable name on Windows
* fix: build on Windows
* fix: rename conflicting win32 symbols in //third_party/sqlite
On Windows otherwise we get:
lld-link: error: duplicate symbol: sqlite3_win32_write_debug
>>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:47987
>>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj
>>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj
lld-link: error: duplicate symbol: sqlite3_win32_sleep
>>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:48042
>>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj
>>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj
lld-link: error: duplicate symbol: sqlite3_win32_is_nt
>>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:48113
>>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj
>>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj
lld-link: error: duplicate symbol: sqlite3_win32_utf8_to_unicode
>>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:48470
>>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj
>>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj
lld-link: error: duplicate symbol: sqlite3_win32_unicode_to_utf8
>>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:48486
>>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj
>>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj
lld-link: error: duplicate symbol: sqlite3_win32_mbcs_to_utf8
>>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:48502
>>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj
>>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj
lld-link: error: duplicate symbol: sqlite3_win32_mbcs_to_utf8_v2
>>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:48518
>>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj
>>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj
lld-link: error: duplicate symbol: sqlite3_win32_utf8_to_mbcs
>>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:48534
>>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj
>>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj
lld-link: error: duplicate symbol: sqlite3_win32_utf8_to_mbcs_v2
>>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:48550
>>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj
>>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj
* docs: remove unnecessary ts-expect-error after types bump
* src: move package resolver to c++
nodejs/node#50322
* build: set ASAN detect_container_overflow=0
nodejs/node#55584
* chore: fixup rebase
* test: disable failing ASAN test
* win: almost fix race detecting ESRCH in uv_kill
libuv/libuv#4341
Comment threadsrc/node_file.cc
return -1;
}

switch (std::filesystem::status(path).type()) {

@joyeecheungjoyeecheungMay 28, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is incorrect: the path should be converted to UTF-16 on windows if it's using std::filesystem::path, otherwise it could crash - this could be the cause of #56670

@joyeecheung

Copy link
Copy Markdown
Member

It seems removing the fast API locally improves performance now (due to the handle scope creation, which would be necessary to get the Environment if this were to be implemented correctly e.g. to name spacing it using env->exec_path)

 confidence improvement accuracy (*) (**) (***)
misc/startup-cli-version.js n=30 cli='deps/corepack/dist/corepack.js' -0.16 % ±1.46% ±1.95% ±2.53%
misc/startup-cli-version.js n=30 cli='deps/npm/bin/npm-cli.js' ** 1.88 % ±1.32% ±1.76% ±2.29%
misc/startup-cli-version.js n=30 cli='deps/npm/bin/npx-cli.js' *** 2.37 % ±1.04% ±1.39% ±1.81%
misc/startup-cli-version.js n=30 cli='tools/eslint/node_modules/eslint/bin/eslint.js' -0.94 % ±1.12% ±1.49% ±1.94%
misc/startup-core.js n=30 mode='process' script='benchmark/fixtures/require-builtins' 0.68 % ±1.69% ±2.25% ±2.92%
misc/startup-core.js n=30 mode='process' script='test/fixtures/semicolon' 0.79 % ±1.08% ±1.44% ±1.88%
misc/startup-core.js n=30 mode='process' script='test/fixtures/snapshot/typescript' -0.21 % ±0.88% ±1.17% ±1.53%
misc/startup-core.js n=30 mode='worker' script='benchmark/fixtures/require-builtins' -0.29 % ±1.04% ±1.39% ±1.80%
misc/startup-core.js n=30 mode='worker' script='test/fixtures/semicolon' 2.43 % ±2.98% ±4.01% ±5.32%
misc/startup-core.js n=30 mode='worker' script='test/fixtures/snapshot/typescript' 0.12 % ±0.70% ±0.93% ±1.21%

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.lib / srcIssues and PRs related to general changes in the lib or src directory.needs-ciPRs that need a full CI run.performanceIssues and PRs related to the performance of Node.js.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

11 participants

@anonrig@nodejs-github-bot@H4ad@wraithgar@targos@aduh95@joyeecheung@mcollina@fhinkel@Qard@GeoffreyBooth