esm: `--experimental-default-type` flag to flip module defaults by GeoffreyBooth · Pull Request #49869 · nodejs/node

GeoffreyBooth · 2023-09-26T00:53:23Z

This PR contains work by @LiviaMedeiros, @JakobJingleheimer, @bmeck adapted from #49531, #49540, #49295, #41552, #31388.

This PR implements most of the proposal in #49432. It creates a new flag, --experimental-default-type, which supports values commonjs (the default) or module. Under --experimental-default-type=module, Node.js interprets the following as ES modules:

String input provided via --eval or STDIN, if --input-type is unspecified.
Files ending in .js or with no extension, if there is no package.json file present in the same folder or any parent folder.
Files ending in .js or with no extension, if the nearest parent package.json field lacks a type field; unless the folder is inside a node_modules folder.

Extensionless files are interpreted as Wasm if --experimental-wasm-modules is passed and the file contains the “magic bytes” Wasm header.

The ESM loader is used for all entry points, and we don’t pass through Module.runMain. Under this flag, we no longer support monkey-patching of that method by files loaded via --require.

This PR does not implement the “parse the entry point as a URL” part of #49432, as there are still some design decisions to be made around that and it’s substantial enough to warrant being implemented as a separate PR.

This is related to #49629 and linked PRs, which include the above behaviors for extensionless files but don’t require a flag. I think we can potentially ship that change on its own, separate from making this flag the new default behavior, because extensionless files currently error in module scopes. I’d prefer to ship it behind the --experimental-default-type flag first to work out any kinks before splitting it out to not require any flag.

If people don’t mind, I’d like if this PR thread can focus on the implementation. If anyone has any thoughts on the design or intentions behind this, there’s already a detailed thread for that at #49432.

cc @nodejs/loaders @nodejs/wasi @nodejs/startup @nodejs/tsc

Notable change

The new flag --experimental-default-type can be used to flip the default module system used by Node.js. Input that is already explicitly defined as ES modules or CommonJS, such as by a package.json"type" field or .mjs/.cjs file extension or the --input-type flag, is unaffected. What is currently implicitly CommonJS would instead be interpreted as ES modules under --experimental-default-type=module:

String input provided via --eval or STDIN, if --input-type is unspecified.
Files ending in .js or with no extension, if there is no package.json file present in the same folder or any parent folder.
Files ending in .js or with no extension, if the nearest parent package.json field lacks a type field; unless the folder is inside a node_modules folder.

In addition, extensionless files are interpreted as Wasm if --experimental-wasm-modules is passed and the file contains the “magic bytes” Wasm header.

nodejs-github-bot · 2023-09-26T00:53:28Z

Review requested:

@nodejs/loaders

guybedford

Thanks @GeoffreyBooth for continuing to push this work, this seems like a step in the right direction to me.

My only suggestion would be considering an alternative flag name like --default-type as an experimental flag or --experimental-default-type so that it is clear from the start that it is not an authoritative flag.

guybedford

Marking for approval conditional on resolving the naming discussion.

PR-URL: #49869 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

`test-wasm-web-api` was moved from `test/parallel` to `test/es-modules`. Update the status files for parallel and es-modules accordingly. Refs: #49869 Refs: #47299 Refs: #47297 PR-URL: #49958 Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Moshe Atlow <moshe@atlow.co.il>

PR-URL: nodejs#49869 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

Notable changes: deps: * (SEMVER-MINOR) update uvwasi to 0.0.19 (Node.js GitHub Bot) #49908 doc: * add H4ad to collaborators (Vinícius Lourenço) #50217 esm: * (SEMVER-MINOR) detect ESM syntax in ambiguous JavaScript (Geoffrey Booth) #50096 * use import attributes instead of import assertions (Antoine du Hamel) #50140 * --experimental-default-type flag to flip module defaults (Geoffrey Booth) #49869 fs: * (SEMVER-MINOR) add flush option to appendFile() functions (Colin Ihrig) #50095 * add flush option to writeFile() functions (Colin Ihrig) #50009 lib: * (SEMVER-MINOR) add WebSocket client (Matthew Aitken) #49830 stream: * (SEMVER-MINOR) allow pass stream class to `stream.compose` (Alex Yang) #50187 * call helper function from push and unshift (Raz Luvaton) #50173 * optimize Writable (Robert Nagy) #50012 test_runner, cli: * (SEMVER-MINOR) add --test-concurrency flag (Colin Ihrig) #49996 vm: * (SEMVER-MINOR) use import attributes instead of import assertions (Antoine du Hamel) #50141 * use default HDO when importModuleDynamically is not set (Joyee Cheung) #49950 wasi: PR-URL: #50682

PR-URL: #49869 Backport-PR-URL: #50669 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

Notable changes: deps: * (SEMVER-MINOR) update uvwasi to 0.0.19 (Node.js GitHub Bot) nodejs#49908 doc: * add H4ad to collaborators (Vinícius Lourenço) nodejs#50217 esm: * (SEMVER-MINOR) detect ESM syntax in ambiguous JavaScript (Geoffrey Booth) nodejs#50096 * use import attributes instead of import assertions (Antoine du Hamel) nodejs#50140 * --experimental-default-type flag to flip module defaults (Geoffrey Booth) nodejs#49869 fs: * (SEMVER-MINOR) add flush option to appendFile() functions (Colin Ihrig) nodejs#50095 * add flush option to writeFile() functions (Colin Ihrig) nodejs#50009 lib: * (SEMVER-MINOR) add WebSocket client (Matthew Aitken) nodejs#49830 stream: * (SEMVER-MINOR) allow pass stream class to `stream.compose` (Alex Yang) nodejs#50187 * call helper function from push and unshift (Raz Luvaton) nodejs#50173 * optimize Writable (Robert Nagy) nodejs#50012 test_runner, cli: * (SEMVER-MINOR) add --test-concurrency flag (Colin Ihrig) nodejs#49996 vm: * (SEMVER-MINOR) use import attributes instead of import assertions (Antoine du Hamel) nodejs#50141 * use default HDO when importModuleDynamically is not set (Joyee Cheung) nodejs#49950 wasi: PR-URL: nodejs#50682

Notable changes: deps: * (SEMVER-MINOR) update uvwasi to 0.0.19 (Node.js GitHub Bot) #49908 * (SEMVER-MINOR) upgrade npm to 10.0.0 (npm team) #49423 doc: * add new TSC members (Michael Dawson) #48841 * move and rename loaders section (Geoffrey Booth) #49261 esm: * use import attributes instead of import assertions (Antoine du Hamel) #50140 * --experimental-default-type flag to flip module defaults (Geoffrey Booth) #49869 * unflag import.meta.resolve (Guy Bedford) #49028 * move hook execution to separate thread (Jacob Smith) #44710 * leverage loaders when resolving subsequent loaders (Maël Nison) #43772 lib: * (SEMVER-MINOR) add api to detect whether source-maps are enabled (翠 / green) #46391 * (SEMVER-MINOR) add tracing channel to diagnostics_channel (Stephen Belanger) #44943 src: * (SEMVER-MINOR) add cjs_module_lexer_version base64_version (Jithil P Ponnan) #45629 stream: * use bitmap in readable state (Benjamin Gruenbaum) #49745 test_runner: * (SEMVER-MINOR) accept `testOnly` in `run` (Moshe Atlow) #49753 * (SEMVER-MINOR) add junit reporter (Moshe Atlow) #49614 * (SEMVER-MINOR) expose location of tests (Colin Ihrig) #48975 * (SEMVER-MINOR) add shards support (Raz Luvaton) #48639 * (SEMVER-MINOR) add initial draft for fakeTimers (Erick Wendel) #47775 test_runner, cli: * (SEMVER-MINOR) add --test-concurrency flag (Colin Ihrig) #49996 tls: * (SEMVER-MINOR) add ALPNCallback server option for dynamic ALPN negotiation (Tim Perry) #45190 vm: * (SEMVER-MINOR) use import attributes instead of import assertions (Antoine du Hamel) #50141 wasi: * (SEMVER-MINOR) updates required for latest uvwasi version (Michael Dawson) #49908 PR-URL: TODO

Notable changes: deps: * (SEMVER-MINOR) update uvwasi to 0.0.19 (Node.js GitHub Bot) #49908 * (SEMVER-MINOR) upgrade npm to 10.2.3 (npm team) #50531 doc: * move and rename loaders section (Geoffrey Booth) #49261 esm: * use import attributes instead of import assertions (Antoine du Hamel) #50140 * --experimental-default-type flag to flip module defaults (Geoffrey Booth) #49869 * unflag import.meta.resolve (Guy Bedford) #49028 * move hook execution to separate thread (Jacob Smith) #44710 * leverage loaders when resolving subsequent loaders (Maël Nison) #43772 lib: * (SEMVER-MINOR) add api to detect whether source-maps are enabled (翠 / green) #46391 * (SEMVER-MINOR) add tracing channel to diagnostics_channel (Stephen Belanger) #44943 src: * (SEMVER-MINOR) add cjs_module_lexer_version base64_version (Jithil P Ponnan) #45629 stream: * use bitmap in readable state (Benjamin Gruenbaum) #49745 test_runner: * (SEMVER-MINOR) accept `testOnly` in `run` (Moshe Atlow) #49753 * (SEMVER-MINOR) add junit reporter (Moshe Atlow) #49614 * (SEMVER-MINOR) expose location of tests (Colin Ihrig) #48975 * (SEMVER-MINOR) add shards support (Raz Luvaton) #48639 * (SEMVER-MINOR) add initial draft for fakeTimers (Erick Wendel) #47775 test_runner, cli: * (SEMVER-MINOR) add --test-concurrency flag (Colin Ihrig) #49996 tls: * (SEMVER-MINOR) add ALPNCallback server option for dynamic ALPN negotiation (Tim Perry) #45190 vm: * (SEMVER-MINOR) use import attributes instead of import assertions (Antoine du Hamel) #50141 PR-URL: #50953

**What's the problem this PR addresses?** Node.js v18.19 has been released and contains nodejs/node#49869 so we need to update tests. **How did you fix it?** Updated the version range. **Checklist** - [x] I have read the [Contributing Guide](https://yarnpkg.com/advanced/contributing). - [x] I have set the packages that need to be released for my changes to be effective. - [x] I will check that all automated PR checks pass before the PR gets reviewed.

**What's the problem this PR addresses?** Node.js v18.19 has been released and contains nodejs/node#49869 so we need to update tests. **How did you fix it?** Updated the version range. **Checklist** - [x] I have read the [Contributing Guide](https://yarnpkg.com/advanced/contributing). - [x] I have set the packages that need to be released for my changes to be effective. - [x] I will check that all automated PR checks pass before the PR gets reviewed. (cherry picked from commit 6ca73db)

PR-URL: nodejs#49869 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

`test-wasm-web-api` was moved from `test/parallel` to `test/es-modules`. Update the status files for parallel and es-modules accordingly. Refs: nodejs#49869 Refs: nodejs#47299 Refs: nodejs#47297 PR-URL: nodejs#49958 Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Moshe Atlow <moshe@atlow.co.il>

PR-URL: nodejs/node#49869 Backport-PR-URL: nodejs/node#50669 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

Notable changes: deps: * (SEMVER-MINOR) update uvwasi to 0.0.19 (Node.js GitHub Bot) nodejs/node#49908 * (SEMVER-MINOR) upgrade npm to 10.2.3 (npm team) nodejs/node#50531 doc: * move and rename loaders section (Geoffrey Booth) nodejs/node#49261 esm: * use import attributes instead of import assertions (Antoine du Hamel) nodejs/node#50140 * --experimental-default-type flag to flip module defaults (Geoffrey Booth) nodejs/node#49869 * unflag import.meta.resolve (Guy Bedford) nodejs/node#49028 * move hook execution to separate thread (Jacob Smith) nodejs/node#44710 * leverage loaders when resolving subsequent loaders (Maël Nison) nodejs/node#43772 lib: * (SEMVER-MINOR) add api to detect whether source-maps are enabled (翠 / green) nodejs/node#46391 * (SEMVER-MINOR) add tracing channel to diagnostics_channel (Stephen Belanger) nodejs/node#44943 src: * (SEMVER-MINOR) add cjs_module_lexer_version base64_version (Jithil P Ponnan) nodejs/node#45629 stream: * use bitmap in readable state (Benjamin Gruenbaum) nodejs/node#49745 test_runner: * (SEMVER-MINOR) accept `testOnly` in `run` (Moshe Atlow) nodejs/node#49753 * (SEMVER-MINOR) add junit reporter (Moshe Atlow) nodejs/node#49614 * (SEMVER-MINOR) expose location of tests (Colin Ihrig) nodejs/node#48975 * (SEMVER-MINOR) add shards support (Raz Luvaton) nodejs/node#48639 * (SEMVER-MINOR) add initial draft for fakeTimers (Erick Wendel) nodejs/node#47775 test_runner, cli: * (SEMVER-MINOR) add --test-concurrency flag (Colin Ihrig) nodejs/node#49996 tls: * (SEMVER-MINOR) add ALPNCallback server option for dynamic ALPN negotiation (Tim Perry) nodejs/node#45190 vm: * (SEMVER-MINOR) use import attributes instead of import assertions (Antoine du Hamel) nodejs/node#50141 PR-URL: nodejs/node#50953

PR-URL: nodejs/node#49869 Backport-PR-URL: nodejs/node#50669 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

Notable changes: deps: * (SEMVER-MINOR) update uvwasi to 0.0.19 (Node.js GitHub Bot) nodejs/node#49908 * (SEMVER-MINOR) upgrade npm to 10.2.3 (npm team) nodejs/node#50531 doc: * move and rename loaders section (Geoffrey Booth) nodejs/node#49261 esm: * use import attributes instead of import assertions (Antoine du Hamel) nodejs/node#50140 * --experimental-default-type flag to flip module defaults (Geoffrey Booth) nodejs/node#49869 * unflag import.meta.resolve (Guy Bedford) nodejs/node#49028 * move hook execution to separate thread (Jacob Smith) nodejs/node#44710 * leverage loaders when resolving subsequent loaders (Maël Nison) nodejs/node#43772 lib: * (SEMVER-MINOR) add api to detect whether source-maps are enabled (翠 / green) nodejs/node#46391 * (SEMVER-MINOR) add tracing channel to diagnostics_channel (Stephen Belanger) nodejs/node#44943 src: * (SEMVER-MINOR) add cjs_module_lexer_version base64_version (Jithil P Ponnan) nodejs/node#45629 stream: * use bitmap in readable state (Benjamin Gruenbaum) nodejs/node#49745 test_runner: * (SEMVER-MINOR) accept `testOnly` in `run` (Moshe Atlow) nodejs/node#49753 * (SEMVER-MINOR) add junit reporter (Moshe Atlow) nodejs/node#49614 * (SEMVER-MINOR) expose location of tests (Colin Ihrig) nodejs/node#48975 * (SEMVER-MINOR) add shards support (Raz Luvaton) nodejs/node#48639 * (SEMVER-MINOR) add initial draft for fakeTimers (Erick Wendel) nodejs/node#47775 test_runner, cli: * (SEMVER-MINOR) add --test-concurrency flag (Colin Ihrig) nodejs/node#49996 tls: * (SEMVER-MINOR) add ALPNCallback server option for dynamic ALPN negotiation (Tim Perry) nodejs/node#45190 vm: * (SEMVER-MINOR) use import attributes instead of import assertions (Antoine du Hamel) nodejs/node#50141 PR-URL: nodejs/node#50953

GeoffreyBooth added module Issues and PRs related to the module subsystem. cli Issues and PRs related to the Node.js command line interface. esm Issues and PRs related to the ECMAScript Modules implementation. labels Sep 26, 2023

GeoffreyBooth requested review from JakobJingleheimer, LiviaMedeiros and bmeck September 26, 2023 00:53

nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Sep 26, 2023

ljharb reviewed Sep 26, 2023
View reviewed changes

Comment threadlib/internal/modules/esm/formats.js Outdated
Comment threadlib/internal/modules/esm/get_format.js Outdated

guybedford reviewed Sep 26, 2023
View reviewed changes

Comment threadlib/internal/modules/esm/get_format.js Outdated
Comment threadlib/internal/modules/esm/get_format.js Outdated

anonrig reviewed Sep 26, 2023
View reviewed changes

Comment threadlib/internal/modules/esm/formats.js Outdated
Comment threadlib/internal/modules/run_main.js Outdated
Comment threadsrc/node_options.h Outdated

This was referenced Sep 26, 2023
Discussion: Name of the flag for the “ESM by default” mode #49541
Closed
Add "type" to package.json MoLow/reporters#102
Merged
Node.js Technical Steering Committee (TSC) Meeting 2023-09-27 nodejs/TSC#1442
Closed

GeoffreyBooth force-pushed the flip-defaults-to-esm branch 2 times, most recently from e6069f0 to fbe28e2Compare September 26, 2023 04:34

MoLow mentioned this pull request Sep 26, 2023
tools: support updating @reporters/github manually #49871
Merged

bnoordhuis reviewed Sep 26, 2023
View reviewed changes

Comment threadlib/internal/modules/esm/formats.js Outdated
Comment threadlib/internal/modules/esm/formats.js Outdated

mcollina added the notable-change PRs with changes that should be highlighted in changelogs. label Sep 26, 2023

This comment was marked as resolved.
Sign in to view

mcollina reviewed Sep 26, 2023
View reviewed changes

Comment threaddoc/api/cli.md Outdated
Comment threadlib/internal/modules/esm/formats.js Outdated

targos reviewed Sep 26, 2023
View reviewed changes

Comment threadlib/internal/main/check_syntax.js Outdated
Comment threadlib/internal/main/eval_stdin.js Outdated
Comment threadlib/internal/main/eval_string.js Outdated

GeoffreyBooth force-pushed the flip-defaults-to-esm branch 2 times, most recently from bf5c677 to 6f0e209Compare September 26, 2023 16:15

aduh95 reviewed Sep 26, 2023
View reviewed changes

Comment threadtest/fixtures/es-modules/package-type-module/wasm-dep.mjs Outdated

aduh95 reviewed Sep 26, 2023
View reviewed changes

Comment threadlib/internal/modules/esm/get_format.js Outdated

guybedford approved these changes Sep 26, 2023
View reviewed changes

aduh95 reviewed Sep 26, 2023
View reviewed changes

Comment threadlib/internal/modules/run_main.js Outdated

GeoffreyBooth force-pushed the flip-defaults-to-esm branch from 1cb1578 to 62b4b09Compare September 26, 2023 22:06

targos mentioned this pull request Nov 11, 2023
[v18.x] Backport most ESM and customization hook changes #50669
Closed

targos mentioned this pull request Nov 12, 2023
v20.10.0 release proposal #50682
Merged

targos added the backported-to-v18.x label Nov 23, 2023

targos mentioned this pull request Nov 28, 2023
v18.19.0 release proposal #50953
Merged

juanarbol mentioned this pull request Dec 6, 2023
2023-12-06, N|Solid v5.0.1 Iron Release nodesource/nsolid#41
Merged

merceyz mentioned this pull request Dec 23, 2023
test: update Node.js range for extensionless files yarnpkg/berry#6035
Merged
3 tasks

coderabbitaiBot mentioned this pull request Jul 3, 2026
Extend AIW-45 to mixed GitHub and GitLab deployments Blazity/ai-workflow#107
Merged

Navigation Menu

Uh oh!

Uh oh!

esm: `--experimental-default-type` flag to flip module defaults - #49869

esm: `--experimental-default-type` flag to flip module defaults#49869
nodejs-github-bot merged 24 commits into
nodejs:mainfrom
GeoffreyBooth:flip-defaults-to-esm

GeoffreyBooth commented Sep 26, 2023•
edited
Loading

Uh oh!

nodejs-github-bot commented Sep 26, 2023

Uh oh!

Uh oh!

Uh oh!

guybedford left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

guybedford left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

12 participants

Uh oh!

Uh oh!

Conversation

GeoffreyBooth commented Sep 26, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Notable change

Uh oh!

nodejs-github-bot commented Sep 26, 2023

Uh oh!

Uh oh!

Uh oh!

guybedford left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

guybedford left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

12 participants

GeoffreyBooth commented Sep 26, 2023•
edited
Loading