build: bundle the npm CLI itself for release by zkat · Pull Request #4 · npm/cli

zkat · 2018-07-06T00:10:21Z

This makes it so we can bundle the npm CLI into a handful of files, instead of hundreds of thousands, with some advantages:

Smaller distribution and installed size
Faster use through npx
We'll be able to use scoped dependencies again (currently only needed for legacy upgrades)

So far, I've tried browserify, rollup, and even parcel, and webpack seems to be the closest thing to working, but it's not quite there yet.

Still TODO:

There are a number of dynamic requires throughout npm and its dependencies. I'm not sure how to resolve this.
On that topic, update-notifier uses require-lazy for its dependencies, and that doesn't get picked up by webpack at all.
There are three workers that get loaded through child_process.spawn/exec. Those need to be standalone entry points. The workers are the extraction worker and the metrics worker on the npm installer, and the extraction worker in libcipm (a dependency).
It's unclear whether this will seriously impact startup performance. Big slowdowns are no good.

Daniel15 · 2018-07-14T06:42:00Z

It's unclear whether this will seriously impact startup performance.

In Yarn we actually had a improvement overall when we switched to a JS bundle + v8-compile-cache. Fewer files to read means fewer stat() calls which means less disk activity. It also significantly reduced the amount of code in our downloads, as modules tend to have a lot of junk (tests, docs, unused files) that are all excluded when you build a bundle. 😃

mkg20001 · 2018-07-14T08:32:17Z

Maybe creating a fork of https://github.com/zeit/pkg that would create a js-bundle instead of an executable would be an idea. pkg already does things such as creating a virtual filesystem, mocking child_process.fork(), etc. so this would probably be the best solution that requires the least changes to npm itself (if deps are missing or not getting picked up by pkg one only has to add a glob to the pkg.scripts field in the package.json instead of creating a more complex build process). also it would be a lot more reusable.

zkat · 2018-08-13T21:25:57Z

As it stands, this feels like a fool's errand and a massive time sink. I think we should give it another whirl in the future when npm's gotten refactored a bit more and split into more modern/webpackable components (and maybe webpack has time to mature its node-building support?).

Until then, I'm gonna close this. I'll leave the branch up on the repo for future reference in case another adventurous soul decides this would be worth their time. In the meantime, I have way better things to do and this won't get any attention from me.

Trued up committer:get-all, update message, npm publishing commands

zkat · 2018-07-06T00:10:21Z

This makes it so we can bundle the npm CLI into a handful of files, instead of hundreds of thousands, with some advantages:

Smaller distribution and installed size
Faster use through npx
We'll be able to use scoped dependencies again (currently only needed for legacy upgrades)

So far, I've tried browserify, rollup, and even parcel, and webpack seems to be the closest thing to working, but it's not quite there yet.

Still TODO:

There are a number of dynamic requires throughout npm and its dependencies. I'm not sure how to resolve this.
On that topic, update-notifier uses require-lazy for its dependencies, and that doesn't get picked up by webpack at all.
There are three workers that get loaded through child_process.spawn/exec. Those need to be standalone entry points. The workers are the extraction worker and the metrics worker on the npm installer, and the extraction worker in libcipm (a dependency).
It's unclear whether this will seriously impact startup performance. Big slowdowns are no good.

Daniel15 · 2018-07-14T06:42:00Z

It's unclear whether this will seriously impact startup performance.

In Yarn we actually had a improvement overall when we switched to a JS bundle + v8-compile-cache. Fewer files to read means fewer stat() calls which means less disk activity. It also significantly reduced the amount of code in our downloads, as modules tend to have a lot of junk (tests, docs, unused files) that are all excluded when you build a bundle. 😃

mkg20001 · 2018-07-14T08:32:17Z

Maybe creating a fork of https://github.com/zeit/pkg that would create a js-bundle instead of an executable would be an idea. pkg already does things such as creating a virtual filesystem, mocking child_process.fork(), etc. so this would probably be the best solution that requires the least changes to npm itself (if deps are missing or not getting picked up by pkg one only has to add a glob to the pkg.scripts field in the package.json instead of creating a more complex build process). also it would be a lot more reusable.

zkat · 2018-08-13T21:25:57Z

As it stands, this feels like a fool's errand and a massive time sink. I think we should give it another whirl in the future when npm's gotten refactored a bit more and split into more modern/webpackable components (and maybe webpack has time to mature its node-building support?).

Until then, I'm gonna close this. I'll leave the branch up on the repo for future reference in case another adventurous soul decides this would be worth their time. In the meantime, I have way better things to do and this won't get any attention from me.

Trued up committer:get-all, update message, npm publishing commands

zkat · 2018-07-06T00:10:21Z

This makes it so we can bundle the npm CLI into a handful of files, instead of hundreds of thousands, with some advantages:

Smaller distribution and installed size
Faster use through npx
We'll be able to use scoped dependencies again (currently only needed for legacy upgrades)

So far, I've tried browserify, rollup, and even parcel, and webpack seems to be the closest thing to working, but it's not quite there yet.

Still TODO:

There are a number of dynamic requires throughout npm and its dependencies. I'm not sure how to resolve this.
On that topic, update-notifier uses require-lazy for its dependencies, and that doesn't get picked up by webpack at all.
There are three workers that get loaded through child_process.spawn/exec. Those need to be standalone entry points. The workers are the extraction worker and the metrics worker on the npm installer, and the extraction worker in libcipm (a dependency).
It's unclear whether this will seriously impact startup performance. Big slowdowns are no good.

Daniel15 · 2018-07-14T06:42:00Z

It's unclear whether this will seriously impact startup performance.

In Yarn we actually had a improvement overall when we switched to a JS bundle + v8-compile-cache. Fewer files to read means fewer stat() calls which means less disk activity. It also significantly reduced the amount of code in our downloads, as modules tend to have a lot of junk (tests, docs, unused files) that are all excluded when you build a bundle. 😃

mkg20001 · 2018-07-14T08:32:17Z

Maybe creating a fork of https://github.com/zeit/pkg that would create a js-bundle instead of an executable would be an idea. pkg already does things such as creating a virtual filesystem, mocking child_process.fork(), etc. so this would probably be the best solution that requires the least changes to npm itself (if deps are missing or not getting picked up by pkg one only has to add a glob to the pkg.scripts field in the package.json instead of creating a more complex build process). also it would be a lot more reusable.

zkat · 2018-08-13T21:25:57Z

As it stands, this feels like a fool's errand and a massive time sink. I think we should give it another whirl in the future when npm's gotten refactored a bit more and split into more modern/webpackable components (and maybe webpack has time to mature its node-building support?).

Until then, I'm gonna close this. I'll leave the branch up on the repo for future reference in case another adventurous soul decides this would be worth their time. In the meantime, I have way better things to do and this won't get any attention from me.

Trued up committer:get-all, update message, npm publishing commands

zkat · 2018-07-06T00:10:21Z

This makes it so we can bundle the npm CLI into a handful of files, instead of hundreds of thousands, with some advantages:

Smaller distribution and installed size
Faster use through npx
We'll be able to use scoped dependencies again (currently only needed for legacy upgrades)

So far, I've tried browserify, rollup, and even parcel, and webpack seems to be the closest thing to working, but it's not quite there yet.

Still TODO:

There are a number of dynamic requires throughout npm and its dependencies. I'm not sure how to resolve this.
On that topic, update-notifier uses require-lazy for its dependencies, and that doesn't get picked up by webpack at all.
There are three workers that get loaded through child_process.spawn/exec. Those need to be standalone entry points. The workers are the extraction worker and the metrics worker on the npm installer, and the extraction worker in libcipm (a dependency).
It's unclear whether this will seriously impact startup performance. Big slowdowns are no good.

Daniel15 · 2018-07-14T06:42:00Z

It's unclear whether this will seriously impact startup performance.

In Yarn we actually had a improvement overall when we switched to a JS bundle + v8-compile-cache. Fewer files to read means fewer stat() calls which means less disk activity. It also significantly reduced the amount of code in our downloads, as modules tend to have a lot of junk (tests, docs, unused files) that are all excluded when you build a bundle. 😃

mkg20001 · 2018-07-14T08:32:17Z

Maybe creating a fork of https://github.com/zeit/pkg that would create a js-bundle instead of an executable would be an idea. pkg already does things such as creating a virtual filesystem, mocking child_process.fork(), etc. so this would probably be the best solution that requires the least changes to npm itself (if deps are missing or not getting picked up by pkg one only has to add a glob to the pkg.scripts field in the package.json instead of creating a more complex build process). also it would be a lot more reusable.

zkat · 2018-08-13T21:25:57Z

As it stands, this feels like a fool's errand and a massive time sink. I think we should give it another whirl in the future when npm's gotten refactored a bit more and split into more modern/webpackable components (and maybe webpack has time to mature its node-building support?).

Until then, I'm gonna close this. I'll leave the branch up on the repo for future reference in case another adventurous soul decides this would be worth their time. In the meantime, I have way better things to do and this won't get any attention from me.

Trued up committer:get-all, update message, npm publishing commands

zkat · 2018-07-06T00:10:21Z

This makes it so we can bundle the npm CLI into a handful of files, instead of hundreds of thousands, with some advantages:

Smaller distribution and installed size
Faster use through npx
We'll be able to use scoped dependencies again (currently only needed for legacy upgrades)

So far, I've tried browserify, rollup, and even parcel, and webpack seems to be the closest thing to working, but it's not quite there yet.

Still TODO:

There are a number of dynamic requires throughout npm and its dependencies. I'm not sure how to resolve this.
On that topic, update-notifier uses require-lazy for its dependencies, and that doesn't get picked up by webpack at all.
There are three workers that get loaded through child_process.spawn/exec. Those need to be standalone entry points. The workers are the extraction worker and the metrics worker on the npm installer, and the extraction worker in libcipm (a dependency).
It's unclear whether this will seriously impact startup performance. Big slowdowns are no good.

Daniel15 · 2018-07-14T06:42:00Z

It's unclear whether this will seriously impact startup performance.

In Yarn we actually had a improvement overall when we switched to a JS bundle + v8-compile-cache. Fewer files to read means fewer stat() calls which means less disk activity. It also significantly reduced the amount of code in our downloads, as modules tend to have a lot of junk (tests, docs, unused files) that are all excluded when you build a bundle. 😃

mkg20001 · 2018-07-14T08:32:17Z

Maybe creating a fork of https://github.com/zeit/pkg that would create a js-bundle instead of an executable would be an idea. pkg already does things such as creating a virtual filesystem, mocking child_process.fork(), etc. so this would probably be the best solution that requires the least changes to npm itself (if deps are missing or not getting picked up by pkg one only has to add a glob to the pkg.scripts field in the package.json instead of creating a more complex build process). also it would be a lot more reusable.

zkat · 2018-08-13T21:25:57Z

As it stands, this feels like a fool's errand and a massive time sink. I think we should give it another whirl in the future when npm's gotten refactored a bit more and split into more modern/webpackable components (and maybe webpack has time to mature its node-building support?).

Until then, I'm gonna close this. I'll leave the branch up on the repo for future reference in case another adventurous soul decides this would be worth their time. In the meantime, I have way better things to do and this won't get any attention from me.

Trued up committer:get-all, update message, npm publishing commands

zkat · 2018-07-06T00:10:21Z

This makes it so we can bundle the npm CLI into a handful of files, instead of hundreds of thousands, with some advantages:

Smaller distribution and installed size
Faster use through npx
We'll be able to use scoped dependencies again (currently only needed for legacy upgrades)

So far, I've tried browserify, rollup, and even parcel, and webpack seems to be the closest thing to working, but it's not quite there yet.

Still TODO:

There are a number of dynamic requires throughout npm and its dependencies. I'm not sure how to resolve this.
On that topic, update-notifier uses require-lazy for its dependencies, and that doesn't get picked up by webpack at all.
There are three workers that get loaded through child_process.spawn/exec. Those need to be standalone entry points. The workers are the extraction worker and the metrics worker on the npm installer, and the extraction worker in libcipm (a dependency).
It's unclear whether this will seriously impact startup performance. Big slowdowns are no good.

Daniel15 · 2018-07-14T06:42:00Z

It's unclear whether this will seriously impact startup performance.

In Yarn we actually had a improvement overall when we switched to a JS bundle + v8-compile-cache. Fewer files to read means fewer stat() calls which means less disk activity. It also significantly reduced the amount of code in our downloads, as modules tend to have a lot of junk (tests, docs, unused files) that are all excluded when you build a bundle. 😃

mkg20001 · 2018-07-14T08:32:17Z

Maybe creating a fork of https://github.com/zeit/pkg that would create a js-bundle instead of an executable would be an idea. pkg already does things such as creating a virtual filesystem, mocking child_process.fork(), etc. so this would probably be the best solution that requires the least changes to npm itself (if deps are missing or not getting picked up by pkg one only has to add a glob to the pkg.scripts field in the package.json instead of creating a more complex build process). also it would be a lot more reusable.

zkat · 2018-08-13T21:25:57Z

As it stands, this feels like a fool's errand and a massive time sink. I think we should give it another whirl in the future when npm's gotten refactored a bit more and split into more modern/webpackable components (and maybe webpack has time to mature its node-building support?).

Until then, I'm gonna close this. I'll leave the branch up on the repo for future reference in case another adventurous soul decides this would be worth their time. In the meantime, I have way better things to do and this won't get any attention from me.

Trued up committer:get-all, update message, npm publishing commands

zkat · 2018-07-06T00:10:21Z

This makes it so we can bundle the npm CLI into a handful of files, instead of hundreds of thousands, with some advantages:

Smaller distribution and installed size
Faster use through npx
We'll be able to use scoped dependencies again (currently only needed for legacy upgrades)

So far, I've tried browserify, rollup, and even parcel, and webpack seems to be the closest thing to working, but it's not quite there yet.

Still TODO:

There are a number of dynamic requires throughout npm and its dependencies. I'm not sure how to resolve this.
On that topic, update-notifier uses require-lazy for its dependencies, and that doesn't get picked up by webpack at all.
There are three workers that get loaded through child_process.spawn/exec. Those need to be standalone entry points. The workers are the extraction worker and the metrics worker on the npm installer, and the extraction worker in libcipm (a dependency).
It's unclear whether this will seriously impact startup performance. Big slowdowns are no good.

Daniel15 · 2018-07-14T06:42:00Z

It's unclear whether this will seriously impact startup performance.

In Yarn we actually had a improvement overall when we switched to a JS bundle + v8-compile-cache. Fewer files to read means fewer stat() calls which means less disk activity. It also significantly reduced the amount of code in our downloads, as modules tend to have a lot of junk (tests, docs, unused files) that are all excluded when you build a bundle. 😃

mkg20001 · 2018-07-14T08:32:17Z

Maybe creating a fork of https://github.com/zeit/pkg that would create a js-bundle instead of an executable would be an idea. pkg already does things such as creating a virtual filesystem, mocking child_process.fork(), etc. so this would probably be the best solution that requires the least changes to npm itself (if deps are missing or not getting picked up by pkg one only has to add a glob to the pkg.scripts field in the package.json instead of creating a more complex build process). also it would be a lot more reusable.

zkat · 2018-08-13T21:25:57Z

As it stands, this feels like a fool's errand and a massive time sink. I think we should give it another whirl in the future when npm's gotten refactored a bit more and split into more modern/webpackable components (and maybe webpack has time to mature its node-building support?).

Until then, I'm gonna close this. I'll leave the branch up on the repo for future reference in case another adventurous soul decides this would be worth their time. In the meantime, I have way better things to do and this won't get any attention from me.

Trued up committer:get-all, update message, npm publishing commands

zkat · 2018-07-06T00:10:21Z

This makes it so we can bundle the npm CLI into a handful of files, instead of hundreds of thousands, with some advantages:

Smaller distribution and installed size
Faster use through npx
We'll be able to use scoped dependencies again (currently only needed for legacy upgrades)

So far, I've tried browserify, rollup, and even parcel, and webpack seems to be the closest thing to working, but it's not quite there yet.

Still TODO:

There are a number of dynamic requires throughout npm and its dependencies. I'm not sure how to resolve this.
On that topic, update-notifier uses require-lazy for its dependencies, and that doesn't get picked up by webpack at all.
There are three workers that get loaded through child_process.spawn/exec. Those need to be standalone entry points. The workers are the extraction worker and the metrics worker on the npm installer, and the extraction worker in libcipm (a dependency).
It's unclear whether this will seriously impact startup performance. Big slowdowns are no good.

Daniel15 · 2018-07-14T06:42:00Z

It's unclear whether this will seriously impact startup performance.

In Yarn we actually had a improvement overall when we switched to a JS bundle + v8-compile-cache. Fewer files to read means fewer stat() calls which means less disk activity. It also significantly reduced the amount of code in our downloads, as modules tend to have a lot of junk (tests, docs, unused files) that are all excluded when you build a bundle. 😃

mkg20001 · 2018-07-14T08:32:17Z

Maybe creating a fork of https://github.com/zeit/pkg that would create a js-bundle instead of an executable would be an idea. pkg already does things such as creating a virtual filesystem, mocking child_process.fork(), etc. so this would probably be the best solution that requires the least changes to npm itself (if deps are missing or not getting picked up by pkg one only has to add a glob to the pkg.scripts field in the package.json instead of creating a more complex build process). also it would be a lot more reusable.

zkat · 2018-08-13T21:25:57Z

As it stands, this feels like a fool's errand and a massive time sink. I think we should give it another whirl in the future when npm's gotten refactored a bit more and split into more modern/webpackable components (and maybe webpack has time to mature its node-building support?).

Until then, I'm gonna close this. I'll leave the branch up on the repo for future reference in case another adventurous soul decides this would be worth their time. In the meantime, I have way better things to do and this won't get any attention from me.

Trued up committer:get-all, update message, npm publishing commands

zkatand others added 6 commits July 5, 2018 15:16

experiment: try with webpack
7e7dcec

scripts: add a bundle script to do the webpack thing
a3c4c34

fix(format): fix stupid formatting mistakes
83697bb

fixes to patch
3247dfc

meh
a30670a

ignore webpack cache
37097bd

zkat added the in-progress label Jul 6, 2018

zkat requested a review from a team as a code owner July 6, 2018 00:10

zkat mentioned this pull request Jul 9, 2018
build: bundle the npm CLI itself for release npm/npm#20495
Closed
4 tasks

zkat force-pushed the release-next branch from 614c234 to 322d9c2Compare July 13, 2018 17:46

zkat added the semver:major backwards-incompatible breaking changes label Jul 18, 2018

zkat closed this Jul 25, 2018

zkat reopened this Jul 26, 2018

zkat closed this Jul 26, 2018

zkat reopened this Jul 26, 2018

zkat force-pushed the release-next branch from 6e68d60 to dbbb75cCompare August 9, 2018 04:06

zkat closed this Aug 13, 2018

masx200 mentioned this pull request Aug 9, 2020
build: bundle the cnpm CLI itself for release cnpm/cnpm#329
Open

darcyclarke mentioned this pull request Mar 1, 2022
[BUG] npm ignores package.json if package-lock.json or node_modules folder exists, dependency resolution terribly broken #4422
Closed
2 tasks

marco-ippolito pushed a commit to marco-ippolito/cli that referenced this pull request Nov 21, 2024
Merge pull requestnpm#4from jeremymwells/update-message
301b9f1
Trued up committer:get-all, update message, npm publishing commands

Paul75 mentioned this pull request Jul 10, 2025
[BUG] Docker npm install failed #8428
Closed
2 tasks

koralle pushed a commit to koralle/npm-cli that referenced this pull request Feb 11, 2026
chore(package.json): fix the typo (npm#4)
67d943d

github-actionsBot referenced this pull request in Kevinlee7250/cli Jun 30, 2026
🤖 봇 응답 (#4)
14cb626

github-actionsBot referenced this pull request in Kevinlee7250/cli Jul 1, 2026
✅ 수동 게시 완료 (#4)
8a15593

github-actionsBot referenced this pull request in Kevinlee7250/cli Jul 1, 2026
🔧 자동 수리 완료 (#4)
33ac579