Skip to content

lib,src: implement WebAssembly Web API - #42701

Merged
nodejs-github-bot merged 1 commit into
nodejs:masterfrom
tniessen:wasm-web-api
Apr 23, 2022
Merged

lib,src: implement WebAssembly Web API#42701
nodejs-github-bot merged 1 commit into
nodejs:masterfrom
tniessen:wasm-web-api

Conversation

@tniessen

Copy link
Copy Markdown
Member

Now that we have fetch(), we may as well also have WebAssembly.compileStreaming() and WebAssembly.instantiateStreaming().

Unfortunately, our fetch() implementation cannot be interacted with easily from C++, which is why part of this implementation ended up in JavaScript. The C++ glue code allows controlling the v8::WasmStreaming instance from JavaScript.

This attempts to be a spec-compliant implementation with no node-specific extensions.

Refs: #41749
Fixes: #21130

@tniessentniessen added notable-change PRs with changes that should be highlighted in changelogs. wasm Issues and PRs related to WebAssembly. fetch Issues and PRs related to the Fetch API labels Apr 12, 2022
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/gyp
  • @nodejs/startup

@nodejs-github-botnodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Apr 12, 2022
Comment threadsrc/node_wasm_web_api.cc Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unless i am mistaken, these methods do not allocate. would you be up for adding fast call wrappers? if not i can maybe add them in a followup pr.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will try that tomorrow. So far, every time I tried using fast calls, I can into some V8 limitation, but Push might be a good candidate for once.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, given how web streams are performing so far and how slow everything is, that seems like premature optimization. I might create a follow-up PR to add that or I might push that as a separate commit later, but I'd like to get this landed rather sooner than later and not make it more complicated to review.

Comment threadtest/parallel/test-wasm-web-api.js Outdated
@targostargos added the semver-minor PRs that contain new features and should be released in the next minor version. label Apr 12, 2022
Comment threaddoc/api/errors.md Outdated
Comment threadsrc/node_wasm_web_api.h Outdated
Comment threadsrc/node_wasm_web_api.h Outdated
Comment threadsrc/node_wasm_web_api.h Outdated
Comment threadsrc/node_wasm_web_api.cc Outdated
Comment threadsrc/node_wasm_web_api.cc Outdated
Comment threadsrc/node_wasm_web_api.h Outdated
@tniessen
tniessenforce-pushed the wasm-web-api branch 4 times, most recently from f76bda6 to 8689723CompareApril 13, 2022 00:58
@tniessentniessen added the review wanted PRs that need reviews. label Apr 17, 2022
@tniessentniessen added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 17, 2022
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 17, 2022
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@aduh95aduh95 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add WTP tests? Maybe https://github.com/web-platform-tests/wpt/tree/HEAD/wasm/webapi or a subset of it?

Comment threaddoc/api/errors.md Outdated
@tniessen

Copy link
Copy Markdown
MemberAuthor

Should we add WTP tests?

That's a good idea. It's not straightforward because many WPT tests for this API rely on fetch and we currently mock fetch in WPT.

Of the ones I can run easily, wasm/webapi/empty-body.any.js is failing, presumably due to nodejs/undici#1345.

@aduh95

Copy link
Copy Markdown
Contributor

Should we add WTP tests?

That's a good idea. It's not straightforward because many WPT tests for this API rely on fetch and we currently mock fetch in WPT.

Of the ones I can run easily, wasm/webapi/empty-body.any.js is failing, presumably due to nodejs/undici#1345.

I think you can create a status file that lists all the failing tests, and that can be used as a TODO list for contributors who want to improve Node.js API compliance. But maybe that's a lot of work (I've never done that myself so not sure), and it probably should not block this from landing.

RafaelGSS pushed a commit that referenced this pull request May 10, 2022
Refs: #42701
Refs: nodejs/undici#1346
Refs: #42939
PR-URL: #42960
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
rekmarks added a commit to MetaMask/snaps that referenced this pull request May 14, 2022
Node landed `WebAssembly.compileStreaming` and `instantiateStreaming` in nodejs/node#42701 and made it available in Node 18, so we no longer need to exclude these properties from our WebAssembly endowment. This PR therefore removes the WebAssembly endowment factory, which will now be included in whatever form it exists on the execution environment root realm global.
@juanarbol

Copy link
Copy Markdown
Member

I think it's fine to land it on v16.x, because it's behind the --experimental-fetch flag.

Yeap, it does, It lands fine; except for the bootstrap, tried to fix this by adding the binding wasm_web_api to lib/internal/bootstrap/loaders.js with no sucess, I will label this to request backport:

make[1]: Leaving directory '/home/juanarbol/GitHub/node/test/node-api/test_worker_terminate_finalization/build' [487/5756]
/usr/bin/python3.10 tools/test.py --mode=release \ \  --skip-tests= \  default \  addons js-native-api node-api === release test-bootstrap-modules === Path: parallel/test-bootstrap-modules node:assert:123 throw new AssertionError(obj); ^ AssertionError [ERR_ASSERTION]: These modules were not loaded: Internal Binding wasm_web_api at Object.<anonymous> (/home/juanarbol/GitHub/node/test/parallel/test-bootstrap-modules.js:216:8) at Module._compile (node:internal/modules/cjs/loader:1105:14) at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10) at Module.load (node:internal/modules/cjs/loader:981:32) at Function.Module._load (node:internal/modules/cjs/loader:822:12) at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12) at node:internal/main/run_main_module:17:47 {

juanarbol pushed a commit that referenced this pull request May 31, 2022
Refs: #42660
Refs: #42701
PR-URL: #42836
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Akhil Marsonya <akhil.marsonya27@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
danielleadams pushed a commit that referenced this pull request Jun 27, 2022
Refs: #42660
Refs: #42701
PR-URL: #42836
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Akhil Marsonya <akhil.marsonya27@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
targos pushed a commit that referenced this pull request Jul 12, 2022
Refs: #42660
Refs: #42701
PR-URL: #42836
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Akhil Marsonya <akhil.marsonya27@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
targos pushed a commit that referenced this pull request Jul 31, 2022
Refs: #42660
Refs: #42701
PR-URL: #42836
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Akhil Marsonya <akhil.marsonya27@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
guangwong pushed a commit to noslate-project/node that referenced this pull request Oct 10, 2022
Refs: nodejs/node#42660
Refs: nodejs/node#42701
PR-URL: nodejs/node#42836
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Akhil Marsonya <akhil.marsonya27@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
codebytere added a commit to electron/electron that referenced this pull request Oct 24, 2022
codebytere added a commit to electron/electron that referenced this pull request Oct 24, 2022
codebytere added a commit to electron/electron that referenced this pull request Oct 26, 2022
codebytere added a commit to electron/electron that referenced this pull request Nov 8, 2022
codebytere added a commit to electron/electron that referenced this pull request Nov 8, 2022
codebytere added a commit to electron/electron that referenced this pull request Nov 10, 2022
* chore: update to Node.js v18
* child_process: improve argument validation
nodejs/node#41305
* bootstrap: support configure-time user-land snapshot
nodejs/node#42466
* chore: update GN patch
* src: disambiguate terms used to refer to builtins and addons
nodejs/node#44135
* src: use a typed array internally for process._exiting
nodejs/node#43883
* chore: lib/internal/bootstrap -> lib/internal/process
* src: disambiguate terms used to refer to builtins and addons
nodejs/node#44135
* chore: remove redudant browserGlobals patch
* chore: update BoringSSL patch
* src: allow embedder-provided PageAllocator in NodePlatform
nodejs/node#38362
* chore: fixup Node.js crypto tests
- nodejs/node#44171
- nodejs/node#41600
* lib: add Promise methods to avoid-prototype-pollution lint rule
nodejs/node#43849
* deps: update V8 to 10.1
nodejs/node#42657
* src: add kNoBrowserGlobals flag for Environment
nodejs/node#40532
* chore: consolidate asar initialization patches
* deps: update V8 to 10.1
nodejs/node#42657
* deps: update V8 to 9.8
nodejs/node#41610
* src,crypto: remove AllocatedBuffers from crypto_spkac
nodejs/node#40752
* build: enable V8's shared read-only heap
nodejs/node#42809
* src: fix ssize_t error from nghttp2.h
nodejs/node#44393
* chore: fixup ESM patch
* chore: fixup patch indices
* src: merge NativeModuleEnv into NativeModuleLoader
nodejs/node#43824
* [API] Pass OOMDetails to OOMErrorCallback
https://chromium-review.googlesource.com/c/v8/v8/+/3647827
* src: iwyu in cleanup_queue.cc
* src: return Maybe from a couple of functions
nodejs/node#39603
* src: clean up embedder API
nodejs/node#35897
* src: refactor DH groups to delete crypto_groups.h
nodejs/node#43896
* deps,src: use SIMD for normal base64 encoding
nodejs/node#39775
* chore: remove deleted source file
* chore: update patches
* chore: remove deleted source file
* lib: add fetch
nodejs/node#41749
* chore: remove nonexistent node specs
* test: split report OOM tests
nodejs/node#44389
* src: trace fs async api
nodejs/node#44057
* http: trace http request / response
nodejs/node#44102
* test: split test-crypto-dh.js
nodejs/node#40451
* crypto: introduce X509Certificate API
nodejs/node#36804
* src: split property helpers from node::Environment
nodejs/node#44056
* nodejs/node#38905
bootstrap: implement run-time user-land snapshots via --build-snapshot and --snapshot-blob
* lib,src: implement WebAssembly Web API
nodejs/node#42701
* fixup! deps,src: use SIMD for normal base64 encoding
* fixup! src: refactor DH groups to delete crypto_groups.h
* chore: fixup base64 GN file
* fix: check that node::InitializeContext() returns true
* chore: delete _noBrowserGlobals usage
* chore: disable fetch in renderer procceses
* dns: default to verbatim=true in dns.lookup()
nodejs/node#39987
Co-authored-by: PatchUp <73610968+patchup[bot]@users.noreply.github.com>
khalwa pushed a commit to solarwindscloud/electron that referenced this pull request Feb 22, 2023
* chore: update to Node.js v18
* child_process: improve argument validation
nodejs/node#41305
* bootstrap: support configure-time user-land snapshot
nodejs/node#42466
* chore: update GN patch
* src: disambiguate terms used to refer to builtins and addons
nodejs/node#44135
* src: use a typed array internally for process._exiting
nodejs/node#43883
* chore: lib/internal/bootstrap -> lib/internal/process
* src: disambiguate terms used to refer to builtins and addons
nodejs/node#44135
* chore: remove redudant browserGlobals patch
* chore: update BoringSSL patch
* src: allow embedder-provided PageAllocator in NodePlatform
nodejs/node#38362
* chore: fixup Node.js crypto tests
- nodejs/node#44171
- nodejs/node#41600
* lib: add Promise methods to avoid-prototype-pollution lint rule
nodejs/node#43849
* deps: update V8 to 10.1
nodejs/node#42657
* src: add kNoBrowserGlobals flag for Environment
nodejs/node#40532
* chore: consolidate asar initialization patches
* deps: update V8 to 10.1
nodejs/node#42657
* deps: update V8 to 9.8
nodejs/node#41610
* src,crypto: remove AllocatedBuffers from crypto_spkac
nodejs/node#40752
* build: enable V8's shared read-only heap
nodejs/node#42809
* src: fix ssize_t error from nghttp2.h
nodejs/node#44393
* chore: fixup ESM patch
* chore: fixup patch indices
* src: merge NativeModuleEnv into NativeModuleLoader
nodejs/node#43824
* [API] Pass OOMDetails to OOMErrorCallback
https://chromium-review.googlesource.com/c/v8/v8/+/3647827
* src: iwyu in cleanup_queue.cc
* src: return Maybe from a couple of functions
nodejs/node#39603
* src: clean up embedder API
nodejs/node#35897
* src: refactor DH groups to delete crypto_groups.h
nodejs/node#43896
* deps,src: use SIMD for normal base64 encoding
nodejs/node#39775
* chore: remove deleted source file
* chore: update patches
* chore: remove deleted source file
* lib: add fetch
nodejs/node#41749
* chore: remove nonexistent node specs
* test: split report OOM tests
nodejs/node#44389
* src: trace fs async api
nodejs/node#44057
* http: trace http request / response
nodejs/node#44102
* test: split test-crypto-dh.js
nodejs/node#40451
* crypto: introduce X509Certificate API
nodejs/node#36804
* src: split property helpers from node::Environment
nodejs/node#44056
* nodejs/node#38905
bootstrap: implement run-time user-land snapshots via --build-snapshot and --snapshot-blob
* lib,src: implement WebAssembly Web API
nodejs/node#42701
* fixup! deps,src: use SIMD for normal base64 encoding
* fixup! src: refactor DH groups to delete crypto_groups.h
* chore: fixup base64 GN file
* fix: check that node::InitializeContext() returns true
* chore: delete _noBrowserGlobals usage
* chore: disable fetch in renderer procceses
* dns: default to verbatim=true in dns.lookup()
nodejs/node#39987
Co-authored-by: PatchUp <73610968+patchup[bot]@users.noreply.github.com>
nicolo-ribaudo added a commit to nicolo-ribaudo/browser-compat-data that referenced this pull request Sep 14, 2023
Implemented in nodejs/node#42701. Manually tested that it's present in Node.js 18.1 and not in 18.0.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author readyPRs that have at least one approval, no pending requests for changes, and a CI started.experimentalIssues and PRs related to experimental features.fetchIssues and PRs related to the Fetch APIlib / srcIssues and PRs related to general changes in the lib or src directory.needs-ciPRs that need a full CI run.notable-changePRs with changes that should be highlighted in changelogs.semver-minorPRs that contain new features and should be released in the next minor version.wasmIssues and PRs related to WebAssembly.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support WebAssembly.instantiateStreaming

8 participants

@tniessen@nodejs-github-bot@aduh95@richardlau@juanarbol@devsnek@RaisinTen@targos