bootstrap: implement run-time user-land snapshots via --build-snapshot and --snapshot-blob

[joyeecheung]

This patch introduces --build-snapshot and --snapshot-blob options for creating and using user land snapshots.

For the initial iteration, user land CJS modules and ESM are not yet supported in the snapshot, so only one single file can be snapshotted (users can bundle their applications into a single script with their bundler of choice to build a snapshot though).

A subset of builtins should already work, and support for more builtins are being added. This PR includes tests checking that the TypeScript compiler and the marked markdown renderer (and the builtins they use) can be snapshotted and deserialized.

To generate a snapshot using snapshot.js as entry point and write the snapshot blob to snapshot.blob:

$ echo "globalThis.foo = 'I am from the snapshot'" > snapshot.js
$ node --snapshot-blob snapshot.blob --build-snapshot snapshot.js 

To restore application state from snapshot.blob, with index.js as the entry point script for the deserialized application:

$ echo "console.log(globalThis.foo)" > index.js
$ node --snapshot-blob snapshot.blob index.js
I am from the snapshot

Users can also use the v8.startupSnapshot API to specify an entry point at snapshot building time, thus avoiding the need of an additional entry script at deserialization time:

$ echo "require('v8').startupSnapshot.setDeserializeMainFunction(() => console.log('I am from the snapshot'))" > snapshot.js
$ node --snapshot-blob snapshot.blob --build-snapshot snapshot.js $ node --snapshot-blob snapshot.blob
I am from the snapshot

Note that this patch only adds functionality to the node executable for building run-time user-land snapshots, the generated snapshot is stored into a separate file on disk. Building a single binary with both Node.js and an embedded snapshot has already been possible with the --node-snapshot-main option to the configure script if the user compiles Node.js from source. It would be a different task to enable the node executable to produce a single binary that contains both Node.js and an embedded snapshot without building Node.js from source, which should be layered on top of the SEA (Single Executable Apps) initiative.

Known limitations/bugs that are being fixed in the V8 upstream:

• V8 hits a DCHECK when deserializing certain mutated globals, e.g. Error.stackTraceLimit (it should work fine in the release build, however): https://chromium-review.googlesource.com/c/v8/v8/+/3319481
• Layout of V8's read-only heap can be inconsistent after deserialization, resulting in memory corruption: https://bugs.chromium.org/p/v8/issues/detail?id=12921

More known limitations/bugs in the tracking issue: #44014

Refs: #35711

[bl-ue]

Wow, that's really cool. Can't wait! ❤️

[addaleax]

👍

[targos]

What is the compatibility of the snapshot blob WRT V8/Node version, other CLI flags, etc?

[targos]

And what happens if you try to load a snapshot with the wrong version of the runtime?

[joyeecheung]

What is the compatibility of the snapshot blob WRT V8/Node version, other CLI flags, etc?

@targos The blob requires the version of V8/Node to be the same between build time and load time - V8 checks the versions explicitly, and abort if there is a mismatch, while at the moment Node.js should work as long as the number/kind/order of external references etc. match between the binaries used to build and load the snapshot (what happens when there is a mismatch depends on what the mismatch actually is, I would say it's most likely that the process would abort when some V8 casting type check fails e.g. casting e.g. a String to a Function, because one version expects value at blob index n to be a String and the other expects value at blob index n to be a Function). Maybe we could explicitly check the Node.js versions as well so when there is a mismatch users can understand what's wrong immediately.

The CLI flags can vary between between build time and load time, to some extent, it depends on the actual flags being used.

[targos]

Maybe we could explicitly check the Node.js versions as well so when there is a mismatch users can understand what's wrong immediately.

Seems like a good idea.

[joyeecheung]

Also that reminded me that we need to regenerate process.argv and friends when the process is restored from a user land snapshot, otherwise they will be part of the JS state being captured and stay that way..

[joyeecheung]

Another thing that we need to watch out for: right now some modules cache access to getOptionValue() which means they would ignore the values of these flags at snapshot restoration time (if passed differently). Same goes for process.env. It's hard to say whether always dynamically querying the values of these things internally is a good idea for user land snapshots, depending on what exactly the flags are, but I think for the initial version of this we just need to make sure that if the user uses the same flags to build and restore snapshots then they are fine. We can think about how to allow users to configure the flag behaviors in later iterations. (I think one approach that might work is to have two sets of flags available, one for the flags used at snapshot building time and one for the flags actually used to launch the instance, and allow code to access both to adjust based on their needs).

[addaleax]

It's hard to say whether always dynamically querying the values of these things

Fwiw, I think it’s generally a good idea not to cache these values, so that they become configurable – independently of snapshot support.

[joyeecheung]

Fwiw, I think it’s generally a good idea not to cache these values, so that they become configurable – independently of snapshot support.

At the source code level I agree - not caching it directly via the form of something like const value = getOptionValue('--some-flag') gives us the opportunity to refresh them should we choose to. If a flag shouldn't be dynamically configurable for some reason (maybe security) we can just warn or error from the function that returns these values, but I think those should be the rare case.

[joyeecheung]

Updated the prototype a bit, now it loads the snapshot entry point from JS (the snapshot would only be generated when it runs to completion), this gives us more control in e.g. warning about unsupported modules. I've split some smaller commits to other PRs, there's also a small V8 debuggability thing that I'll upstream later.

Some issues I discovered:

• After src: set PromiseHooks by Environment #38821 the environment keeps a list of weak references to all vm contexts created in it. Right now in the V8 API there is no way to record a weak reference to a context in the snapshot, so building user land snapshots with any vm contexts not yet GC'ed would crash right now (even if promise hooks aren't enabled, because the implementation always track the contexts so that it can add hooks to them on demand later). We could probably work around this for now by always tracking the contexts in the snapshot and setting the references to weak when restoring them, and let V8 GC them later if necessary. In the long term we can add an API in V8 to track weak references in snapshots.
• For some reason spinning the loop at snapshot building time results in inconsistencies in the JS land so e.g. REPL would stop working. There are still a bunch of refactoring to be done to make those runtime stuff queried lazily, but I guess we can just spin the loop for the experimental version, even if it breaks when there are runtime differences between snapshot building time and loading time?

(Also, I realized that many modules just require the CJS module loader at the top level, which would in turn require the ESM loader at the top level, which leaves a promise immediately. Not sure how to work around that promise yet but I guess for now we could just make sure that other modules only load those loaders when they actually need to...)

[jasnell]

At the source code level I agree - not caching it directly via the form of something like const value = getOptionValue('--some-flag') gives us the opportunity to refresh them should we choose to. If a flag shouldn't be dynamically configurable for some reason (maybe security) we can just warn or error from the function that returns these values, but I think those should be the rare case.

It's likely worth opening a separate issue for this particular piece. I agree that getting away from caching as a module-scope const would be a good thing.

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/45797/

[joyeecheung]

Added two more fixups to accommodate recent changes:

• The last commit added serialization and deserialization of the exiting AliasedBuffer for src: use a typed array internally for process._exiting #43883
• The first commit made the creation of the net diagnostics channels lazy for lib: add diagnostics channel and perf hooks detail #43984 so that net.js can be included in the snapshot - it should be fine to create them lazily, if the user needs these channels they would've created these channels first before invoking anything that would publish to these channels. If the users don't need them, then the channels would only be created when builtins that try to publish to them are invoked, and the snapshot currently doesn't support these builtins (that fire network requests) anyway. I added a TODO for supporting the diagnostics channels in the snapshot, and added it to the list in Tracking issue: support of builtins and V8 issues in run-time user-land snapshots #44014.

cc @legendecas@mcollina (feel free to ignore) still LGTY?

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/45800/

[legendecas]

You may need to format the code.

[mcollina]

lgtm

[joyeecheung]

Landed with git clang-format applied in 32ffa5d...48229e5. The commit message of the second commit is slightly too long as it contains a one-line command that demonstrates the feature, I doubt that it's going to break anything so I kept it as-is. (it'd be nice for core-commit-validate to recognize code blocks in the commit message and ignore certain code that's too long, just like how it ignores URLs that are too long)

[joyeecheung]

Note to releasers: this needs to be backported after the two PRs mentioned above #38905 (comment) or there would be conflicts

[danielleadams]

@joyeecheung#43984 needs a backport, so I'm going to mark this as backport-blocked.

[ruyadorno]

It looks like #44203 builds on top of this PR, so I'm marking it with the backport-blocked-v18.x label too and we should look into landing that PR as soon as this one lands 😊

[ruyadorno]

Note to releasers: this needs to be backported after the two PRs mentioned above #38905 (comment) or there would be conflicts

That worked! thanks @joyeecheung!

Removing the backport-blocked-v18.x label here and in other commits that depended on this one.

[gajus]

It would be a different task to enable the node executable to produce a single binary that contains both Node.js and an embedded snapshot without building Node.js from source, which should be layered on top of the SEA (Single Executable Apps) initiative.

Where can I track progress of this?

[joyeecheung]

@gajus The SEA effort is tracked in https://github.com/nodejs/single-executable and currently most activities take place in https://github.com/nodejs/single-executable/discussions