Isaacs/install finish by isaacs · Pull Request #1245 · npm/cli

isaacs · 2020-05-05T00:32:00Z

Big red diff day!

Sorry for the massive drop. The main things going on here:

All of the commands that perform an arborist.reify() have been ported to actually use Arborist (except for prune and dedupe that are blocked by Arborist lacking those features for the moment.) That is: install, ci, rm, update, audit, audit fix.
A util/reify-output.js module is added, which all of these call to produce consistent output after operation.
npm audit is almost where it needs to be, but Arborist.audit() and npm-audit-report need to be made aware that bundled deps can't be fixed with npm audit fix.
Interesting change: if there's still an audit issue after running npm audit fix, it'll print a full audit, so you don't have the annoying behavior of running npm audit fix and having it tell you to run npm audit fix again. (Also: it'll only suggest running npm audit fix if there's actually something that can be fixed, and npm audit fix can fix transitive meta-dependency issues no matter how deep!)
npm shrinkwrap is updated to use Arborist, so that it will always update the shrinkwrap to the latest and greatest lockfileVersion.

Unfortunately, we still can't fully remove the lib/install/ folder, because it's being used by ls, fund, and util/error-message.js. But almost!

Next up for pieces keeping lib/install/*.js and lib/fetch-package-metadata.js around:

implement prune in Arborist
implement dedupe in Arborist
get npm outdated with arborist #1208 landed
implement rebuild in Arborist (or, if not a top-level arborist method, implement it using Arborist.loadActual() and bin-links instead; but it really feels like it might be more under Arborist's charter than the cli's.)
port ls to use Arborist.loadActual()
port fund to use Arborist.loadActual()
remove lib/build.js
remove lib/unbuild.js
remove or refactor the diff detection in utils/error-message.js so it doens't rely on install/read-shrinkwrap.js

isaacs · 2020-05-05T01:17:29Z

implement rebuild in Arborist (or, if not a top-level arborist method, implement it using Arborist.loadActual() and bin-links instead; but it really feels like it might be more under Arborist's charter than the cli's.)

Ok, just had a "how hard could it be? 🤷‍♂️" moment, and took a crack at doing this in cli.

Yeah, no. It needs to be an arborist thing. The thing that's there is kind of broken/incomplete anyway, and I don't want to reimplement half of reify outside of Arborist, that sense makes not any.

ruyadorno · 2020-05-07T21:07:51Z

ah great, I see it fixes #1234

ruyadorno

Just a small comment, otherwise everything LGTM 👍

Notes:

Let's add tests later, make sure we also have 100% coverage across the cli
Some of the builds are currently broken with an exit signal on install 🤔 is that expected?

ruyadorno · 2020-05-07T21:19:15Z

+
+ if (npm.flatOptions.depth !== Infinity) {
+ log.warn('update', 'The --depth option no longer has any effect. See RFC0019.\n' +
+ 'https://github.com/npm/rfcs/blob/latest/accepted/0019-remove-update-depth-option.md')


I'd prefer to be even more cautious and link to: https://github.com/npm/rfcs/blob/ea2d3024e6e149cd8c6366ed18373c9a566b1124/accepted/0019-remove-update-depth-option.md instead

Hm! I considered that, but I figured if we ever update it or something, we'd want to have the user see the latest and greatest version, no?

I would still prefer linking to the commit as I can see many possible problematic things happening:
what if us (or a future team working on this repo) decides to rename branches? latest to master, etc
what if someone at a point in future decides to rename the accepted folder?
what if that rfc gets withdrawn and/or moved to a diff folder?
Although it seems unlikely in the near-future, I had been bitten by these sorts of changes in the past and I grew to really appreciate linking to the commit blob links instead, just by seeing these things actually happening 😊
but then again I also see that the chance of this message changing before any of the aforementioned happening is also really high 😄

ruyadorno · 2020-05-07T21:25:56Z

ooohh actually might be a good idea to rebase release/v7.0.0-beta to master, I have tweaked the GH actions since to get all tests running again, including windows builds

This adds support for Arborist.audit()

This adds a 'reify-output.js' util, which can be passed any Arborist object after it reifies a tree. Consistent output is printed in all cases, showing the number of packages added/removed/changed, packages needing funding, and a minimal (but always actionable and relevant) audit summary. The only code using the Installer class now is in lib/outdated.js, which is has a pending update coming soon. Prune and dedupe commands are awaiting top-level Arborist methods, so that they can be similarly tightened up. (For now, this commit just has them fail with a 'coming soon' message.) The last piece holding the 'install/*.js' code in this repo is that it is used in 'ls', 'fund', 'shrinkwrap', and the error-message util.

We buffer the output for scripts, and throw it away if the failure is not something we have to care about. But if we DO have to care about it, it's important to show it. This is a bare-minimum approach. The error handling stuff here could use a careful refactor, and it'd be nice if @npmcli/promise-spawn put something more definitive on the error it returns, so that we didn't have to duck-type it like this.

@isaacs

PR-URL: #1245 Credit: @isaacsClose: #1245 Reviewed-by: @ruyadorno

isaacs · 2020-05-08T01:22:45Z

Landed on release/v7.0.0-beta. Thanks!

isaacs · 2020-05-05T00:32:00Z

Big red diff day!

Sorry for the massive drop. The main things going on here:

All of the commands that perform an arborist.reify() have been ported to actually use Arborist (except for prune and dedupe that are blocked by Arborist lacking those features for the moment.) That is: install, ci, rm, update, audit, audit fix.
A util/reify-output.js module is added, which all of these call to produce consistent output after operation.
npm audit is almost where it needs to be, but Arborist.audit() and npm-audit-report need to be made aware that bundled deps can't be fixed with npm audit fix.
Interesting change: if there's still an audit issue after running npm audit fix, it'll print a full audit, so you don't have the annoying behavior of running npm audit fix and having it tell you to run npm audit fix again. (Also: it'll only suggest running npm audit fix if there's actually something that can be fixed, and npm audit fix can fix transitive meta-dependency issues no matter how deep!)
npm shrinkwrap is updated to use Arborist, so that it will always update the shrinkwrap to the latest and greatest lockfileVersion.

Unfortunately, we still can't fully remove the lib/install/ folder, because it's being used by ls, fund, and util/error-message.js. But almost!

Next up for pieces keeping lib/install/*.js and lib/fetch-package-metadata.js around:

implement prune in Arborist
implement dedupe in Arborist
get npm outdated with arborist #1208 landed
implement rebuild in Arborist (or, if not a top-level arborist method, implement it using Arborist.loadActual() and bin-links instead; but it really feels like it might be more under Arborist's charter than the cli's.)
port ls to use Arborist.loadActual()
port fund to use Arborist.loadActual()
remove lib/build.js
remove lib/unbuild.js
remove or refactor the diff detection in utils/error-message.js so it doens't rely on install/read-shrinkwrap.js

isaacs · 2020-05-05T01:17:29Z

implement rebuild in Arborist (or, if not a top-level arborist method, implement it using Arborist.loadActual() and bin-links instead; but it really feels like it might be more under Arborist's charter than the cli's.)

Ok, just had a "how hard could it be? 🤷‍♂️" moment, and took a crack at doing this in cli.

Yeah, no. It needs to be an arborist thing. The thing that's there is kind of broken/incomplete anyway, and I don't want to reimplement half of reify outside of Arborist, that sense makes not any.

ruyadorno · 2020-05-07T21:07:51Z

ah great, I see it fixes #1234

ruyadorno

Just a small comment, otherwise everything LGTM 👍

Notes:

Let's add tests later, make sure we also have 100% coverage across the cli
Some of the builds are currently broken with an exit signal on install 🤔 is that expected?

ruyadorno · 2020-05-07T21:19:15Z

+
+ if (npm.flatOptions.depth !== Infinity) {
+ log.warn('update', 'The --depth option no longer has any effect. See RFC0019.\n' +
+ 'https://github.com/npm/rfcs/blob/latest/accepted/0019-remove-update-depth-option.md')


I'd prefer to be even more cautious and link to: https://github.com/npm/rfcs/blob/ea2d3024e6e149cd8c6366ed18373c9a566b1124/accepted/0019-remove-update-depth-option.md instead

Hm! I considered that, but I figured if we ever update it or something, we'd want to have the user see the latest and greatest version, no?

I would still prefer linking to the commit as I can see many possible problematic things happening:
what if us (or a future team working on this repo) decides to rename branches? latest to master, etc
what if someone at a point in future decides to rename the accepted folder?
what if that rfc gets withdrawn and/or moved to a diff folder?
Although it seems unlikely in the near-future, I had been bitten by these sorts of changes in the past and I grew to really appreciate linking to the commit blob links instead, just by seeing these things actually happening 😊
but then again I also see that the chance of this message changing before any of the aforementioned happening is also really high 😄

ruyadorno · 2020-05-07T21:25:56Z

ooohh actually might be a good idea to rebase release/v7.0.0-beta to master, I have tweaked the GH actions since to get all tests running again, including windows builds

This adds support for Arborist.audit()

This adds a 'reify-output.js' util, which can be passed any Arborist object after it reifies a tree. Consistent output is printed in all cases, showing the number of packages added/removed/changed, packages needing funding, and a minimal (but always actionable and relevant) audit summary. The only code using the Installer class now is in lib/outdated.js, which is has a pending update coming soon. Prune and dedupe commands are awaiting top-level Arborist methods, so that they can be similarly tightened up. (For now, this commit just has them fail with a 'coming soon' message.) The last piece holding the 'install/*.js' code in this repo is that it is used in 'ls', 'fund', 'shrinkwrap', and the error-message util.

We buffer the output for scripts, and throw it away if the failure is not something we have to care about. But if we DO have to care about it, it's important to show it. This is a bare-minimum approach. The error handling stuff here could use a careful refactor, and it'd be nice if @npmcli/promise-spawn put something more definitive on the error it returns, so that we didn't have to duck-type it like this.

@isaacs

PR-URL: #1245 Credit: @isaacsClose: #1245 Reviewed-by: @ruyadorno

isaacs · 2020-05-08T01:22:45Z

Landed on release/v7.0.0-beta. Thanks!

isaacs · 2020-05-05T00:32:00Z

Big red diff day!

Sorry for the massive drop. The main things going on here:

All of the commands that perform an arborist.reify() have been ported to actually use Arborist (except for prune and dedupe that are blocked by Arborist lacking those features for the moment.) That is: install, ci, rm, update, audit, audit fix.
A util/reify-output.js module is added, which all of these call to produce consistent output after operation.
npm audit is almost where it needs to be, but Arborist.audit() and npm-audit-report need to be made aware that bundled deps can't be fixed with npm audit fix.
Interesting change: if there's still an audit issue after running npm audit fix, it'll print a full audit, so you don't have the annoying behavior of running npm audit fix and having it tell you to run npm audit fix again. (Also: it'll only suggest running npm audit fix if there's actually something that can be fixed, and npm audit fix can fix transitive meta-dependency issues no matter how deep!)
npm shrinkwrap is updated to use Arborist, so that it will always update the shrinkwrap to the latest and greatest lockfileVersion.

Unfortunately, we still can't fully remove the lib/install/ folder, because it's being used by ls, fund, and util/error-message.js. But almost!

Next up for pieces keeping lib/install/*.js and lib/fetch-package-metadata.js around:

implement prune in Arborist
implement dedupe in Arborist
get npm outdated with arborist #1208 landed
implement rebuild in Arborist (or, if not a top-level arborist method, implement it using Arborist.loadActual() and bin-links instead; but it really feels like it might be more under Arborist's charter than the cli's.)
port ls to use Arborist.loadActual()
port fund to use Arborist.loadActual()
remove lib/build.js
remove lib/unbuild.js
remove or refactor the diff detection in utils/error-message.js so it doens't rely on install/read-shrinkwrap.js

isaacs · 2020-05-05T01:17:29Z

implement rebuild in Arborist (or, if not a top-level arborist method, implement it using Arborist.loadActual() and bin-links instead; but it really feels like it might be more under Arborist's charter than the cli's.)

Ok, just had a "how hard could it be? 🤷‍♂️" moment, and took a crack at doing this in cli.

Yeah, no. It needs to be an arborist thing. The thing that's there is kind of broken/incomplete anyway, and I don't want to reimplement half of reify outside of Arborist, that sense makes not any.

ruyadorno · 2020-05-07T21:07:51Z

ah great, I see it fixes #1234

ruyadorno

Just a small comment, otherwise everything LGTM 👍

Notes:

Let's add tests later, make sure we also have 100% coverage across the cli
Some of the builds are currently broken with an exit signal on install 🤔 is that expected?

ruyadorno · 2020-05-07T21:19:15Z

+
+ if (npm.flatOptions.depth !== Infinity) {
+ log.warn('update', 'The --depth option no longer has any effect. See RFC0019.\n' +
+ 'https://github.com/npm/rfcs/blob/latest/accepted/0019-remove-update-depth-option.md')


I'd prefer to be even more cautious and link to: https://github.com/npm/rfcs/blob/ea2d3024e6e149cd8c6366ed18373c9a566b1124/accepted/0019-remove-update-depth-option.md instead

Hm! I considered that, but I figured if we ever update it or something, we'd want to have the user see the latest and greatest version, no?

I would still prefer linking to the commit as I can see many possible problematic things happening:
what if us (or a future team working on this repo) decides to rename branches? latest to master, etc
what if someone at a point in future decides to rename the accepted folder?
what if that rfc gets withdrawn and/or moved to a diff folder?
Although it seems unlikely in the near-future, I had been bitten by these sorts of changes in the past and I grew to really appreciate linking to the commit blob links instead, just by seeing these things actually happening 😊
but then again I also see that the chance of this message changing before any of the aforementioned happening is also really high 😄

ruyadorno · 2020-05-07T21:25:56Z

ooohh actually might be a good idea to rebase release/v7.0.0-beta to master, I have tweaked the GH actions since to get all tests running again, including windows builds

This adds support for Arborist.audit()

This adds a 'reify-output.js' util, which can be passed any Arborist object after it reifies a tree. Consistent output is printed in all cases, showing the number of packages added/removed/changed, packages needing funding, and a minimal (but always actionable and relevant) audit summary. The only code using the Installer class now is in lib/outdated.js, which is has a pending update coming soon. Prune and dedupe commands are awaiting top-level Arborist methods, so that they can be similarly tightened up. (For now, this commit just has them fail with a 'coming soon' message.) The last piece holding the 'install/*.js' code in this repo is that it is used in 'ls', 'fund', 'shrinkwrap', and the error-message util.

We buffer the output for scripts, and throw it away if the failure is not something we have to care about. But if we DO have to care about it, it's important to show it. This is a bare-minimum approach. The error handling stuff here could use a careful refactor, and it'd be nice if @npmcli/promise-spawn put something more definitive on the error it returns, so that we didn't have to duck-type it like this.

@isaacs

PR-URL: #1245 Credit: @isaacsClose: #1245 Reviewed-by: @ruyadorno

isaacs · 2020-05-08T01:22:45Z

Landed on release/v7.0.0-beta. Thanks!

isaacs · 2020-05-05T00:32:00Z

Big red diff day!

Sorry for the massive drop. The main things going on here:

All of the commands that perform an arborist.reify() have been ported to actually use Arborist (except for prune and dedupe that are blocked by Arborist lacking those features for the moment.) That is: install, ci, rm, update, audit, audit fix.
A util/reify-output.js module is added, which all of these call to produce consistent output after operation.
npm audit is almost where it needs to be, but Arborist.audit() and npm-audit-report need to be made aware that bundled deps can't be fixed with npm audit fix.
Interesting change: if there's still an audit issue after running npm audit fix, it'll print a full audit, so you don't have the annoying behavior of running npm audit fix and having it tell you to run npm audit fix again. (Also: it'll only suggest running npm audit fix if there's actually something that can be fixed, and npm audit fix can fix transitive meta-dependency issues no matter how deep!)
npm shrinkwrap is updated to use Arborist, so that it will always update the shrinkwrap to the latest and greatest lockfileVersion.

Unfortunately, we still can't fully remove the lib/install/ folder, because it's being used by ls, fund, and util/error-message.js. But almost!

Next up for pieces keeping lib/install/*.js and lib/fetch-package-metadata.js around:

implement prune in Arborist
implement dedupe in Arborist
get npm outdated with arborist #1208 landed
implement rebuild in Arborist (or, if not a top-level arborist method, implement it using Arborist.loadActual() and bin-links instead; but it really feels like it might be more under Arborist's charter than the cli's.)
port ls to use Arborist.loadActual()
port fund to use Arborist.loadActual()
remove lib/build.js
remove lib/unbuild.js
remove or refactor the diff detection in utils/error-message.js so it doens't rely on install/read-shrinkwrap.js

isaacs · 2020-05-05T01:17:29Z

implement rebuild in Arborist (or, if not a top-level arborist method, implement it using Arborist.loadActual() and bin-links instead; but it really feels like it might be more under Arborist's charter than the cli's.)

Ok, just had a "how hard could it be? 🤷‍♂️" moment, and took a crack at doing this in cli.

Yeah, no. It needs to be an arborist thing. The thing that's there is kind of broken/incomplete anyway, and I don't want to reimplement half of reify outside of Arborist, that sense makes not any.

ruyadorno · 2020-05-07T21:07:51Z

ah great, I see it fixes #1234

ruyadorno

Just a small comment, otherwise everything LGTM 👍

Notes:

Let's add tests later, make sure we also have 100% coverage across the cli
Some of the builds are currently broken with an exit signal on install 🤔 is that expected?

ruyadorno · 2020-05-07T21:19:15Z

+
+ if (npm.flatOptions.depth !== Infinity) {
+ log.warn('update', 'The --depth option no longer has any effect. See RFC0019.\n' +
+ 'https://github.com/npm/rfcs/blob/latest/accepted/0019-remove-update-depth-option.md')


I'd prefer to be even more cautious and link to: https://github.com/npm/rfcs/blob/ea2d3024e6e149cd8c6366ed18373c9a566b1124/accepted/0019-remove-update-depth-option.md instead

Hm! I considered that, but I figured if we ever update it or something, we'd want to have the user see the latest and greatest version, no?

I would still prefer linking to the commit as I can see many possible problematic things happening:
what if us (or a future team working on this repo) decides to rename branches? latest to master, etc
what if someone at a point in future decides to rename the accepted folder?
what if that rfc gets withdrawn and/or moved to a diff folder?
Although it seems unlikely in the near-future, I had been bitten by these sorts of changes in the past and I grew to really appreciate linking to the commit blob links instead, just by seeing these things actually happening 😊
but then again I also see that the chance of this message changing before any of the aforementioned happening is also really high 😄

ruyadorno · 2020-05-07T21:25:56Z

ooohh actually might be a good idea to rebase release/v7.0.0-beta to master, I have tweaked the GH actions since to get all tests running again, including windows builds

This adds support for Arborist.audit()

This adds a 'reify-output.js' util, which can be passed any Arborist object after it reifies a tree. Consistent output is printed in all cases, showing the number of packages added/removed/changed, packages needing funding, and a minimal (but always actionable and relevant) audit summary. The only code using the Installer class now is in lib/outdated.js, which is has a pending update coming soon. Prune and dedupe commands are awaiting top-level Arborist methods, so that they can be similarly tightened up. (For now, this commit just has them fail with a 'coming soon' message.) The last piece holding the 'install/*.js' code in this repo is that it is used in 'ls', 'fund', 'shrinkwrap', and the error-message util.

We buffer the output for scripts, and throw it away if the failure is not something we have to care about. But if we DO have to care about it, it's important to show it. This is a bare-minimum approach. The error handling stuff here could use a careful refactor, and it'd be nice if @npmcli/promise-spawn put something more definitive on the error it returns, so that we didn't have to duck-type it like this.

@isaacs

PR-URL: #1245 Credit: @isaacsClose: #1245 Reviewed-by: @ruyadorno

isaacs · 2020-05-08T01:22:45Z

Landed on release/v7.0.0-beta. Thanks!

isaacs · 2020-05-05T00:32:00Z

Big red diff day!

Sorry for the massive drop. The main things going on here:

All of the commands that perform an arborist.reify() have been ported to actually use Arborist (except for prune and dedupe that are blocked by Arborist lacking those features for the moment.) That is: install, ci, rm, update, audit, audit fix.
A util/reify-output.js module is added, which all of these call to produce consistent output after operation.
npm audit is almost where it needs to be, but Arborist.audit() and npm-audit-report need to be made aware that bundled deps can't be fixed with npm audit fix.
Interesting change: if there's still an audit issue after running npm audit fix, it'll print a full audit, so you don't have the annoying behavior of running npm audit fix and having it tell you to run npm audit fix again. (Also: it'll only suggest running npm audit fix if there's actually something that can be fixed, and npm audit fix can fix transitive meta-dependency issues no matter how deep!)
npm shrinkwrap is updated to use Arborist, so that it will always update the shrinkwrap to the latest and greatest lockfileVersion.

Unfortunately, we still can't fully remove the lib/install/ folder, because it's being used by ls, fund, and util/error-message.js. But almost!

Next up for pieces keeping lib/install/*.js and lib/fetch-package-metadata.js around:

implement prune in Arborist
implement dedupe in Arborist
get npm outdated with arborist #1208 landed
implement rebuild in Arborist (or, if not a top-level arborist method, implement it using Arborist.loadActual() and bin-links instead; but it really feels like it might be more under Arborist's charter than the cli's.)
port ls to use Arborist.loadActual()
port fund to use Arborist.loadActual()
remove lib/build.js
remove lib/unbuild.js
remove or refactor the diff detection in utils/error-message.js so it doens't rely on install/read-shrinkwrap.js

isaacs · 2020-05-05T01:17:29Z

implement rebuild in Arborist (or, if not a top-level arborist method, implement it using Arborist.loadActual() and bin-links instead; but it really feels like it might be more under Arborist's charter than the cli's.)

Ok, just had a "how hard could it be? 🤷‍♂️" moment, and took a crack at doing this in cli.

Yeah, no. It needs to be an arborist thing. The thing that's there is kind of broken/incomplete anyway, and I don't want to reimplement half of reify outside of Arborist, that sense makes not any.

ruyadorno · 2020-05-07T21:07:51Z

ah great, I see it fixes #1234

ruyadorno

Just a small comment, otherwise everything LGTM 👍

Notes:

Let's add tests later, make sure we also have 100% coverage across the cli
Some of the builds are currently broken with an exit signal on install 🤔 is that expected?

ruyadorno · 2020-05-07T21:19:15Z

+
+ if (npm.flatOptions.depth !== Infinity) {
+ log.warn('update', 'The --depth option no longer has any effect. See RFC0019.\n' +
+ 'https://github.com/npm/rfcs/blob/latest/accepted/0019-remove-update-depth-option.md')


I'd prefer to be even more cautious and link to: https://github.com/npm/rfcs/blob/ea2d3024e6e149cd8c6366ed18373c9a566b1124/accepted/0019-remove-update-depth-option.md instead

Hm! I considered that, but I figured if we ever update it or something, we'd want to have the user see the latest and greatest version, no?

I would still prefer linking to the commit as I can see many possible problematic things happening:
what if us (or a future team working on this repo) decides to rename branches? latest to master, etc
what if someone at a point in future decides to rename the accepted folder?
what if that rfc gets withdrawn and/or moved to a diff folder?
Although it seems unlikely in the near-future, I had been bitten by these sorts of changes in the past and I grew to really appreciate linking to the commit blob links instead, just by seeing these things actually happening 😊
but then again I also see that the chance of this message changing before any of the aforementioned happening is also really high 😄

ruyadorno · 2020-05-07T21:25:56Z

ooohh actually might be a good idea to rebase release/v7.0.0-beta to master, I have tweaked the GH actions since to get all tests running again, including windows builds

This adds support for Arborist.audit()

This adds a 'reify-output.js' util, which can be passed any Arborist object after it reifies a tree. Consistent output is printed in all cases, showing the number of packages added/removed/changed, packages needing funding, and a minimal (but always actionable and relevant) audit summary. The only code using the Installer class now is in lib/outdated.js, which is has a pending update coming soon. Prune and dedupe commands are awaiting top-level Arborist methods, so that they can be similarly tightened up. (For now, this commit just has them fail with a 'coming soon' message.) The last piece holding the 'install/*.js' code in this repo is that it is used in 'ls', 'fund', 'shrinkwrap', and the error-message util.

We buffer the output for scripts, and throw it away if the failure is not something we have to care about. But if we DO have to care about it, it's important to show it. This is a bare-minimum approach. The error handling stuff here could use a careful refactor, and it'd be nice if @npmcli/promise-spawn put something more definitive on the error it returns, so that we didn't have to duck-type it like this.

@isaacs

PR-URL: #1245 Credit: @isaacsClose: #1245 Reviewed-by: @ruyadorno

isaacs · 2020-05-08T01:22:45Z

Landed on release/v7.0.0-beta. Thanks!

isaacs · 2020-05-05T00:32:00Z

Big red diff day!

Sorry for the massive drop. The main things going on here:

All of the commands that perform an arborist.reify() have been ported to actually use Arborist (except for prune and dedupe that are blocked by Arborist lacking those features for the moment.) That is: install, ci, rm, update, audit, audit fix.
A util/reify-output.js module is added, which all of these call to produce consistent output after operation.
npm audit is almost where it needs to be, but Arborist.audit() and npm-audit-report need to be made aware that bundled deps can't be fixed with npm audit fix.
Interesting change: if there's still an audit issue after running npm audit fix, it'll print a full audit, so you don't have the annoying behavior of running npm audit fix and having it tell you to run npm audit fix again. (Also: it'll only suggest running npm audit fix if there's actually something that can be fixed, and npm audit fix can fix transitive meta-dependency issues no matter how deep!)
npm shrinkwrap is updated to use Arborist, so that it will always update the shrinkwrap to the latest and greatest lockfileVersion.

Unfortunately, we still can't fully remove the lib/install/ folder, because it's being used by ls, fund, and util/error-message.js. But almost!

Next up for pieces keeping lib/install/*.js and lib/fetch-package-metadata.js around:

implement prune in Arborist
implement dedupe in Arborist
get npm outdated with arborist #1208 landed
implement rebuild in Arborist (or, if not a top-level arborist method, implement it using Arborist.loadActual() and bin-links instead; but it really feels like it might be more under Arborist's charter than the cli's.)
port ls to use Arborist.loadActual()
port fund to use Arborist.loadActual()
remove lib/build.js
remove lib/unbuild.js
remove or refactor the diff detection in utils/error-message.js so it doens't rely on install/read-shrinkwrap.js

isaacs · 2020-05-05T01:17:29Z

implement rebuild in Arborist (or, if not a top-level arborist method, implement it using Arborist.loadActual() and bin-links instead; but it really feels like it might be more under Arborist's charter than the cli's.)

Ok, just had a "how hard could it be? 🤷‍♂️" moment, and took a crack at doing this in cli.

Yeah, no. It needs to be an arborist thing. The thing that's there is kind of broken/incomplete anyway, and I don't want to reimplement half of reify outside of Arborist, that sense makes not any.

ruyadorno · 2020-05-07T21:07:51Z

ah great, I see it fixes #1234

ruyadorno

Just a small comment, otherwise everything LGTM 👍

Notes:

Let's add tests later, make sure we also have 100% coverage across the cli
Some of the builds are currently broken with an exit signal on install 🤔 is that expected?

ruyadorno · 2020-05-07T21:19:15Z

+
+ if (npm.flatOptions.depth !== Infinity) {
+ log.warn('update', 'The --depth option no longer has any effect. See RFC0019.\n' +
+ 'https://github.com/npm/rfcs/blob/latest/accepted/0019-remove-update-depth-option.md')


I'd prefer to be even more cautious and link to: https://github.com/npm/rfcs/blob/ea2d3024e6e149cd8c6366ed18373c9a566b1124/accepted/0019-remove-update-depth-option.md instead

Hm! I considered that, but I figured if we ever update it or something, we'd want to have the user see the latest and greatest version, no?

I would still prefer linking to the commit as I can see many possible problematic things happening:
what if us (or a future team working on this repo) decides to rename branches? latest to master, etc
what if someone at a point in future decides to rename the accepted folder?
what if that rfc gets withdrawn and/or moved to a diff folder?
Although it seems unlikely in the near-future, I had been bitten by these sorts of changes in the past and I grew to really appreciate linking to the commit blob links instead, just by seeing these things actually happening 😊
but then again I also see that the chance of this message changing before any of the aforementioned happening is also really high 😄

ruyadorno · 2020-05-07T21:25:56Z

ooohh actually might be a good idea to rebase release/v7.0.0-beta to master, I have tweaked the GH actions since to get all tests running again, including windows builds

This adds support for Arborist.audit()

This adds a 'reify-output.js' util, which can be passed any Arborist object after it reifies a tree. Consistent output is printed in all cases, showing the number of packages added/removed/changed, packages needing funding, and a minimal (but always actionable and relevant) audit summary. The only code using the Installer class now is in lib/outdated.js, which is has a pending update coming soon. Prune and dedupe commands are awaiting top-level Arborist methods, so that they can be similarly tightened up. (For now, this commit just has them fail with a 'coming soon' message.) The last piece holding the 'install/*.js' code in this repo is that it is used in 'ls', 'fund', 'shrinkwrap', and the error-message util.

We buffer the output for scripts, and throw it away if the failure is not something we have to care about. But if we DO have to care about it, it's important to show it. This is a bare-minimum approach. The error handling stuff here could use a careful refactor, and it'd be nice if @npmcli/promise-spawn put something more definitive on the error it returns, so that we didn't have to duck-type it like this.

@isaacs

PR-URL: #1245 Credit: @isaacsClose: #1245 Reviewed-by: @ruyadorno

isaacs · 2020-05-08T01:22:45Z

Landed on release/v7.0.0-beta. Thanks!

isaacs · 2020-05-05T00:32:00Z

Big red diff day!

Sorry for the massive drop. The main things going on here:

All of the commands that perform an arborist.reify() have been ported to actually use Arborist (except for prune and dedupe that are blocked by Arborist lacking those features for the moment.) That is: install, ci, rm, update, audit, audit fix.
A util/reify-output.js module is added, which all of these call to produce consistent output after operation.
npm audit is almost where it needs to be, but Arborist.audit() and npm-audit-report need to be made aware that bundled deps can't be fixed with npm audit fix.
Interesting change: if there's still an audit issue after running npm audit fix, it'll print a full audit, so you don't have the annoying behavior of running npm audit fix and having it tell you to run npm audit fix again. (Also: it'll only suggest running npm audit fix if there's actually something that can be fixed, and npm audit fix can fix transitive meta-dependency issues no matter how deep!)
npm shrinkwrap is updated to use Arborist, so that it will always update the shrinkwrap to the latest and greatest lockfileVersion.

Unfortunately, we still can't fully remove the lib/install/ folder, because it's being used by ls, fund, and util/error-message.js. But almost!

Next up for pieces keeping lib/install/*.js and lib/fetch-package-metadata.js around:

implement prune in Arborist
implement dedupe in Arborist
get npm outdated with arborist #1208 landed
implement rebuild in Arborist (or, if not a top-level arborist method, implement it using Arborist.loadActual() and bin-links instead; but it really feels like it might be more under Arborist's charter than the cli's.)
port ls to use Arborist.loadActual()
port fund to use Arborist.loadActual()
remove lib/build.js
remove lib/unbuild.js
remove or refactor the diff detection in utils/error-message.js so it doens't rely on install/read-shrinkwrap.js

isaacs · 2020-05-05T01:17:29Z

implement rebuild in Arborist (or, if not a top-level arborist method, implement it using Arborist.loadActual() and bin-links instead; but it really feels like it might be more under Arborist's charter than the cli's.)

Ok, just had a "how hard could it be? 🤷‍♂️" moment, and took a crack at doing this in cli.

Yeah, no. It needs to be an arborist thing. The thing that's there is kind of broken/incomplete anyway, and I don't want to reimplement half of reify outside of Arborist, that sense makes not any.

ruyadorno · 2020-05-07T21:07:51Z

ah great, I see it fixes #1234

ruyadorno

Just a small comment, otherwise everything LGTM 👍

Notes:

Let's add tests later, make sure we also have 100% coverage across the cli
Some of the builds are currently broken with an exit signal on install 🤔 is that expected?

ruyadorno · 2020-05-07T21:19:15Z

+
+ if (npm.flatOptions.depth !== Infinity) {
+ log.warn('update', 'The --depth option no longer has any effect. See RFC0019.\n' +
+ 'https://github.com/npm/rfcs/blob/latest/accepted/0019-remove-update-depth-option.md')


I'd prefer to be even more cautious and link to: https://github.com/npm/rfcs/blob/ea2d3024e6e149cd8c6366ed18373c9a566b1124/accepted/0019-remove-update-depth-option.md instead

Hm! I considered that, but I figured if we ever update it or something, we'd want to have the user see the latest and greatest version, no?

I would still prefer linking to the commit as I can see many possible problematic things happening:
what if us (or a future team working on this repo) decides to rename branches? latest to master, etc
what if someone at a point in future decides to rename the accepted folder?
what if that rfc gets withdrawn and/or moved to a diff folder?
Although it seems unlikely in the near-future, I had been bitten by these sorts of changes in the past and I grew to really appreciate linking to the commit blob links instead, just by seeing these things actually happening 😊
but then again I also see that the chance of this message changing before any of the aforementioned happening is also really high 😄

ruyadorno · 2020-05-07T21:25:56Z

ooohh actually might be a good idea to rebase release/v7.0.0-beta to master, I have tweaked the GH actions since to get all tests running again, including windows builds

This adds support for Arborist.audit()

This adds a 'reify-output.js' util, which can be passed any Arborist object after it reifies a tree. Consistent output is printed in all cases, showing the number of packages added/removed/changed, packages needing funding, and a minimal (but always actionable and relevant) audit summary. The only code using the Installer class now is in lib/outdated.js, which is has a pending update coming soon. Prune and dedupe commands are awaiting top-level Arborist methods, so that they can be similarly tightened up. (For now, this commit just has them fail with a 'coming soon' message.) The last piece holding the 'install/*.js' code in this repo is that it is used in 'ls', 'fund', 'shrinkwrap', and the error-message util.

We buffer the output for scripts, and throw it away if the failure is not something we have to care about. But if we DO have to care about it, it's important to show it. This is a bare-minimum approach. The error handling stuff here could use a careful refactor, and it'd be nice if @npmcli/promise-spawn put something more definitive on the error it returns, so that we didn't have to duck-type it like this.

@isaacs

PR-URL: #1245 Credit: @isaacsClose: #1245 Reviewed-by: @ruyadorno

isaacs · 2020-05-08T01:22:45Z

Landed on release/v7.0.0-beta. Thanks!

isaacs · 2020-05-05T00:32:00Z

Big red diff day!

Sorry for the massive drop. The main things going on here:

All of the commands that perform an arborist.reify() have been ported to actually use Arborist (except for prune and dedupe that are blocked by Arborist lacking those features for the moment.) That is: install, ci, rm, update, audit, audit fix.
A util/reify-output.js module is added, which all of these call to produce consistent output after operation.
npm audit is almost where it needs to be, but Arborist.audit() and npm-audit-report need to be made aware that bundled deps can't be fixed with npm audit fix.
Interesting change: if there's still an audit issue after running npm audit fix, it'll print a full audit, so you don't have the annoying behavior of running npm audit fix and having it tell you to run npm audit fix again. (Also: it'll only suggest running npm audit fix if there's actually something that can be fixed, and npm audit fix can fix transitive meta-dependency issues no matter how deep!)
npm shrinkwrap is updated to use Arborist, so that it will always update the shrinkwrap to the latest and greatest lockfileVersion.

Unfortunately, we still can't fully remove the lib/install/ folder, because it's being used by ls, fund, and util/error-message.js. But almost!

Next up for pieces keeping lib/install/*.js and lib/fetch-package-metadata.js around:

implement prune in Arborist
implement dedupe in Arborist
get npm outdated with arborist #1208 landed
implement rebuild in Arborist (or, if not a top-level arborist method, implement it using Arborist.loadActual() and bin-links instead; but it really feels like it might be more under Arborist's charter than the cli's.)
port ls to use Arborist.loadActual()
port fund to use Arborist.loadActual()
remove lib/build.js
remove lib/unbuild.js
remove or refactor the diff detection in utils/error-message.js so it doens't rely on install/read-shrinkwrap.js

isaacs · 2020-05-05T01:17:29Z

implement rebuild in Arborist (or, if not a top-level arborist method, implement it using Arborist.loadActual() and bin-links instead; but it really feels like it might be more under Arborist's charter than the cli's.)

Ok, just had a "how hard could it be? 🤷‍♂️" moment, and took a crack at doing this in cli.

Yeah, no. It needs to be an arborist thing. The thing that's there is kind of broken/incomplete anyway, and I don't want to reimplement half of reify outside of Arborist, that sense makes not any.

ruyadorno · 2020-05-07T21:07:51Z

ah great, I see it fixes #1234

ruyadorno

Just a small comment, otherwise everything LGTM 👍

Notes:

Let's add tests later, make sure we also have 100% coverage across the cli
Some of the builds are currently broken with an exit signal on install 🤔 is that expected?

ruyadorno · 2020-05-07T21:19:15Z

+
+ if (npm.flatOptions.depth !== Infinity) {
+ log.warn('update', 'The --depth option no longer has any effect. See RFC0019.\n' +
+ 'https://github.com/npm/rfcs/blob/latest/accepted/0019-remove-update-depth-option.md')


I'd prefer to be even more cautious and link to: https://github.com/npm/rfcs/blob/ea2d3024e6e149cd8c6366ed18373c9a566b1124/accepted/0019-remove-update-depth-option.md instead

Hm! I considered that, but I figured if we ever update it or something, we'd want to have the user see the latest and greatest version, no?

I would still prefer linking to the commit as I can see many possible problematic things happening:
what if us (or a future team working on this repo) decides to rename branches? latest to master, etc
what if someone at a point in future decides to rename the accepted folder?
what if that rfc gets withdrawn and/or moved to a diff folder?
Although it seems unlikely in the near-future, I had been bitten by these sorts of changes in the past and I grew to really appreciate linking to the commit blob links instead, just by seeing these things actually happening 😊
but then again I also see that the chance of this message changing before any of the aforementioned happening is also really high 😄

ruyadorno · 2020-05-07T21:25:56Z

ooohh actually might be a good idea to rebase release/v7.0.0-beta to master, I have tweaked the GH actions since to get all tests running again, including windows builds

This adds support for Arborist.audit()

This adds a 'reify-output.js' util, which can be passed any Arborist object after it reifies a tree. Consistent output is printed in all cases, showing the number of packages added/removed/changed, packages needing funding, and a minimal (but always actionable and relevant) audit summary. The only code using the Installer class now is in lib/outdated.js, which is has a pending update coming soon. Prune and dedupe commands are awaiting top-level Arborist methods, so that they can be similarly tightened up. (For now, this commit just has them fail with a 'coming soon' message.) The last piece holding the 'install/*.js' code in this repo is that it is used in 'ls', 'fund', 'shrinkwrap', and the error-message util.

We buffer the output for scripts, and throw it away if the failure is not something we have to care about. But if we DO have to care about it, it's important to show it. This is a bare-minimum approach. The error handling stuff here could use a careful refactor, and it'd be nice if @npmcli/promise-spawn put something more definitive on the error it returns, so that we didn't have to duck-type it like this.

@isaacs

PR-URL: #1245 Credit: @isaacsClose: #1245 Reviewed-by: @ruyadorno

isaacs · 2020-05-08T01:22:45Z

Landed on release/v7.0.0-beta. Thanks!

isaacs requested a review from a team as a code owner May 5, 2020 00:32

ruyadorno approved these changes May 7, 2020
View reviewed changes

isaacs force-pushed the release/v7.0.0-beta branch from cd6840b to e57a991Compare May 8, 2020 01:14

isaacs added 10 commits May 7, 2020 18:18

update @npmcli/arborist, dedupe/organize metadeps
5e940d9
This adds support for Arborist.audit()

remove libcipm, correct bundleDependencies
59cd29d

dedupe npm-pick-manifest up to 6.1.0
4f9f144

npm-audit-report@2.1.0
e1943ac

update arborist, dedupe pacote/make-fetch-happen
873665e

shrinkwrap: port to using Arborist
df53085

@npmcli/arborist@0.0.0-pre.19
87a1559

add the --legacy-peer-deps config
56a688a

isaacs force-pushed the isaacs/install-finish branch from 01300dd to 56a688aCompare May 8, 2020 01:20

isaacs added a commit that referenced this pull request May 8, 2020
add the --legacy-peer-deps config
5721d50
PR-URL: #1245 Credit: @isaacsClose: #1245 Reviewed-by: @ruyadorno

isaacs closed this May 8, 2020

This was referenced May 8, 2020
[v7] Update output of npm install command #1017
Closed
[v7] Complete npm install command #995
Closed

isaacs deleted the isaacs/install-finish branch October 2, 2020 21:40

hojatA1 mentioned this pull request Jun 20, 2023
[Snyk] Security upgrade nyc from 11.9.0 to 15.0.0 hojatA1/npm.cli#9
Open

Billywildhig mentioned this pull request Nov 30, 2023
[Snyk] Fix for 1 vulnerabilities Billywildhig/cli#19
Open

EMWickd mentioned this pull request Nov 30, 2023
[Snyk] Security upgrade nyc from 11.9.0 to 15.0.0 EMWickd/cli#47
Open

This was referenced Nov 30, 2023
[Snyk] Fix for 1 vulnerabilities 311821105/cli#33
Open
[Snyk] Security upgrade nyc from 11.9.0 to 15.0.0 311821105/cli#35
Open

Billywildhig mentioned this pull request Nov 30, 2023
[Snyk] Security upgrade nyc from 11.9.0 to 15.0.0 Billywildhig/cli#20
Open

EMWickd mentioned this pull request Nov 30, 2023
[Snyk] Fix for 1 vulnerabilities EMWickd/cli#54
Open

This was referenced Dec 1, 2023
[Snyk] Fix for 1 vulnerabilities hojatA1/npm.cli#62
Open
[Snyk] Security upgrade nyc from 11.9.0 to 15.0.0 hojatA1/npm.cli#63
Open

EMWickd mentioned this pull request Dec 1, 2023
[Snyk] Security upgrade nyc from 11.9.0 to 15.0.0 EMWickd/cli#71
Open

hojatA1 mentioned this pull request Dec 1, 2023
[Snyk] Security upgrade nyc from 11.9.0 to 15.0.0 hojatA1/npm.cli#82
Open

Billywildhig mentioned this pull request Dec 2, 2023
[Snyk] Security upgrade nyc from 11.9.0 to 15.0.0 Billywildhig/cli#49
Open