fs,module: add module-loader-only realpath cache by addaleax · Pull Request #8100 · nodejs/node

addaleax · 2016-08-14T17:22:09Z

Checklist

make -j4 test (UNIX), or vcbuild test nosign (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

Affected core subsystem(s)

fs, module

Description of change

Reintroduce a realpath cache with the same mechanisms which existed before b488b19 (fs: optimize realpath using uv_fs_realpath()), but only for the synchronous version and with the cache being passed as a hidden option to make sure it is only used internally.

The cache is hidden from userland applications because it has been decided that fully reintroducing as part of the public API might stand in the way of future optimizations.

/cc @nodejs/fs @bzoz

mscdex · 2016-08-14T17:40:30Z

Would it be faster to just do cache.get() and check if it returned undefined?

@mscdex Maybe a bit, yes. I’ve updated the PR with only calls to cache.get().

addaleax · 2016-08-14T18:35:29Z

CI: https://ci.nodejs.org/job/node-test-commit/4567/

jasnell · 2016-08-15T16:05:05Z

LGTM . There's Red in CI that looks unrelated.

bzoz · 2016-08-16T09:51:25Z

LGTM. The CI failure is 99% unrelated, but still - one more run: https://ci.nodejs.org/job/node-test-pull-request/3692/

jasnell · 2016-08-18T01:33:06Z

CITGM: https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/361/

Because this touches module, I'd like a bit more signoff from @nodejs/ctc

vkurchatkin · 2016-08-18T01:39:49Z

Can't this be a const?

@vkurchatkin Yes, makes sense. Done!

jasnell · 2016-08-18T17:26:58Z

Marking this semver-minor given that it's technically additive. (I sure will be happy when we have all this fs module stuff settled)

jasnell · 2016-08-22T21:22:15Z

ping @nodejs/ctc

addaleax · 2016-08-27T05:42:21Z

hmm… anybody from @nodejs/ctc @nodejs/collaborators up for more reviews/thoughts on this?

This is not really urgent, but having it land in the next couple of days would be nice because it’s in a similar situation as Rod’s #8277, and should get a follow-up PR moving the cache symbol to internal/fs.

addaleax · 2016-09-09T21:37:01Z

hmm… bump again? would anybody feel more comfortable signing off on this if it’s marked dont-land-on-v6.x?

cjihrig · 2016-09-10T00:11:41Z

Should this be "an empty Map"?

Yes, thanks for noticing! Done!

Reintroduce a realpath cache with the same mechanisms which existed before b488b19 (`fs: optimize realpath using uv_fs_realpath()`), but only for the synchronous version and with the cache being passed as a hidden option to make sure it is only used internally. The cache is hidden from userland applications because it has been decided that fully reintroducing as part of the public API might stand in the way of future optimizations.

addaleax · 2016-09-10T10:21:39Z

Rebased, new CI: https://ci.nodejs.org/job/node-test-commit/4990/

jasnell · 2016-09-12T04:03:33Z

Still LGTM

addaleax · 2016-09-25T14:41:37Z

Okay, so, unless anybody objects or something else comes up, I’d like to land this sometime next week. Right now it looks like this doesn’t get any more review/feedback than @bzoz’ and @jasnell’s LGTMs.

If anybody would feel more comfortable with it, they can add a dont-land-on-v6.x label.

Edit: One more CI: https://ci.nodejs.org/job/node-test-commit/5375/

addaleax · 2016-09-30T14:01:21Z

I’ll land this:

Two LGTMs
No objections or indications of further review
Last two CIs add up as green, and the last CI was green up to a build failure and unrelated flaky tests.

addaleax · 2016-09-30T14:11:11Z

Landed in c084287

Reintroduce a realpath cache with the same mechanisms which existed before b488b19 (`fs: optimize realpath using uv_fs_realpath()`), but only for the synchronous version and with the cache being passed as a hidden option to make sure it is only used internally. The cache is hidden from userland applications because it has been decided that fully reintroducing as part of the public API might stand in the way of future optimizations. PR-URL: #8100 Reviewed-By: Bartosz Sosnowski <bartosz@janeasystems.com> Reviewed-By: James M Snell <jasnell@gmail.com>

* fs: - `SyncWriteStream` now inherits from `Stream.Writable`. (Anna Henningsen) #8830 - Practically, this means that when stdio is piped to a file, stdout and stderr will still be `Writable` streams. - `fs.existsSync()` has been undeprecated. `fs.exists()` remains deprecated. (Dan Fabulich) #8364 * http: `http.request()` now accepts a `timeout` option. (Rene Weber) #8101 * module: The module loader now maintains its own realpath cache. (Anna Henningsen) #8100 * npm: Upgraded to 3.10.8 (Kat Marchán) #8706 * stream: `Duplex` streams now show proper `instanceof Stream.Writable`. (Anna Henningsen) #8834 * timers: Improved `setTimeout`/`Interval` performance by up to 22%. (Brian White) #8661 PR-URL: #9034

* fs: - `SyncWriteStream` now inherits from `Stream.Writable`. (Anna Henningsen) nodejs/node#8830 - Practically, this means that when stdio is piped to a file, stdout and stderr will still be `Writable` streams. - `fs.existsSync()` has been undeprecated. `fs.exists()` remains deprecated. (Dan Fabulich) nodejs/node#8364 * http: `http.request()` now accepts a `timeout` option. (Rene Weber) nodejs/node#8101 * module: The module loader now maintains its own realpath cache. (Anna Henningsen) nodejs/node#8100 * npm: Upgraded to 3.10.8 (Kat Marchan) nodejs/node#8706 * stream: `Duplex` streams now show proper `instanceof Stream.Writable`. (Anna Henningsen) nodejs/node#8834 * timers: Improved `setTimeout`/`Interval` performance by up to 22%. (Brian White) nodejs/node#8661 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>

addaleax added fs Issues and PRs related to the fs subsystem / file system. module Issues and PRs related to the module subsystem. labels Aug 14, 2016

mscdex reviewed Aug 14, 2016
View reviewed changes

Fishrock123 added semver-minor PRs that contain new features and should be released in the next minor version. and removed semver-minor PRs that contain new features and should be released in the next minor version. labels Aug 14, 2016

vkurchatkin reviewed Aug 18, 2016
View reviewed changes

jasnell added the semver-minor PRs that contain new features and should be released in the next minor version. label Aug 18, 2016

addaleax added the dont-land-on-v4.x label Aug 27, 2016

cjihrig reviewed Sep 10, 2016
View reviewed changes

addaleax added 5 commits September 10, 2016 12:06

Replace cache.has with .get() & check.
434424f

Use local const instead of property on Module object
7f0c224

Address review comments
f3e8373

fixup: id cannot be const
c9954f4

addaleax force-pushed the fs-module-only-rpcache branch from c7efb61 to c9954f4Compare September 10, 2016 10:20

Trott force-pushed the master branch from b0df363 to c5ce7f4Compare September 21, 2016 00:09

addaleax self-assigned this Sep 30, 2016

addaleax closed this Sep 30, 2016

addaleax deleted the fs-module-only-rpcache branch September 30, 2016 14:11

CurryKitten mentioned this pull request Nov 23, 2016
Perf issue: Increased footprint in benchmark on PPC64 on 6.8.0 #9764
Closed

Uh oh!

Uh oh!

Conversation

addaleax commented Aug 14, 2016

Checklist

Affected core subsystem(s)

Description of change

Uh oh!

mscdexAug 14, 2016

Choose a reason for hiding this comment

Uh oh!

addaleaxAug 14, 2016

Choose a reason for hiding this comment

Uh oh!

addaleax commented Aug 14, 2016

Uh oh!

jasnell commented Aug 15, 2016

Uh oh!

bzoz commented Aug 16, 2016

Uh oh!

jasnell commented Aug 18, 2016

Uh oh!

vkurchatkinAug 18, 2016

Choose a reason for hiding this comment

Uh oh!

addaleaxAug 18, 2016

Choose a reason for hiding this comment

Uh oh!

jasnell commented Aug 18, 2016

Uh oh!

jasnell commented Aug 22, 2016

Uh oh!

addaleax commented Aug 27, 2016

Uh oh!

addaleax commented Sep 9, 2016

Uh oh!

cjihrigSep 10, 2016

Choose a reason for hiding this comment

Uh oh!

addaleaxSep 10, 2016

Choose a reason for hiding this comment

Uh oh!

addaleax commented Sep 10, 2016

Uh oh!

jasnell commented Sep 12, 2016

Uh oh!

addaleax commented Sep 25, 2016• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

addaleax commented Sep 30, 2016

Uh oh!

addaleax commented Sep 30, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

addaleax commented Sep 25, 2016•
edited
Loading