src: cache negative results in GetPackageJSON

[dmichon-msft]

Fixes#56821

Ensure that GetPackageJSON caches which package.json files do not exist, not just the contents of the ones that do. In packages with at least one subfolder, it is normal for most folders in a package not to contain a package.json file, so this will significantly reduce the number of file system reads performed during GetNearestParentPackageJSON.

[ljharb]

What happens if a package.json file is created after this is cached?

[dmichon-msft]

What happens if a package.json file is created after this is cached?

The same thing that happens if you change the content of an existing package.json file after it gets read (and cached). The state of the world as of the first read of the folder is preserved.

[ljharb]

How does one clear the cache, then, after making a change?

[dmichon-msft]

How does one clear the cache, then, after making a change?

Usually by shutting down and restarting the process, since this cache is used during module loading.

[dmichon-msft]

Missing package.json files were cached in Node <= 20:

node/lib/internal/modules/package_json_reader.js

Line 132 in 93447ab

cache.set(jsonPath,result);

As near as I can tell, this behavior change was simply an oversight when migrating readPackageScope from the JS layer to the native layer.

Node <= 20 also does not provide any means for cache entries regarding package.json to be cleared.

[jasnell]

Usually by shutting down and restarting the process, since this cache is used during module loading.

Hmm...as a follow up, should we consider making it possible to clear both the missing and found caches at runtime to allow for hot reloading of these? Would that even make sense? Not something to worry about in this PR but something worth exploring separately.

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/64886/

[dmichon-msft]

Usually by shutting down and restarting the process, since this cache is used during module loading.

Hmm...as a follow up, should we consider making it possible to clear both the missing and found caches at runtime to allow for hot reloading of these? Would that even make sense? Not something to worry about in this PR but something worth exploring separately.

The entries in the cache will be there because they impacted the loading of already-loaded modules, so you could get some really weird results if we purge the cache and not all of the modules that were loaded from folders whose package.json files were checked. Ultimately, I think any runtime change that affects resolution of already-loaded code is heavily in the "use at your own peril" category.

I think we'd be better served ensuring the hits and misses also find their way into the metadata for node --watch so that it can restart if a relevant package.json changes, is deleted, or becomes available (assuming they aren't already?).

[aduh95]

It looks like build is failing on CI

[jasnell]

Appears to need #56846 to land...

[bnoordhuis]

For historic background: I'm the one who first moved package.json reading to C++ land (commits fdbb6dd and 0fdd88a from Nov 2017) and at least back then, missing files were not recorded in the cache.

I'm somewhat surprised to learn that has changed because I remember bug reports from people trying to install npm packages at runtime, and, however unwise, we did in fact accommodate such users, IIRC.

[dmichon-msft]

Here's the commit where negative caching was added:
4396beb

[joyeecheung]

#56846 has landed, so I take that it's time to restart the CI..

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/65570/

[github-actions]

This pull request has been marked as stale due to 210 days of inactivity.
It will be automatically closed in 30 days if no further activity occurs. If this is still relevant, please leave a comment or update it to keep it open.

[github-actions]

This pull request has been marked as stale due to 90 days of inactivity.
It will be automatically closed in 30 days if no further activity occurs. If this is still relevant, please leave a comment or update it to keep it open.