fs: refactor "options" processing as a function by thefourtheye · Pull Request #7165 · nodejs/node

thefourtheye · 2016-06-05T21:37:06Z

Checklist

tests and code linting passes
the commit message follows commit guidelines

Affected core subsystem(s)

fs

Description of change

As it is the "options" processing is repeated in all the functions
which need it. That introduces checks which are inconsistent with
similar functions and produces slightly different error messages.

This patch moves the basic "options" validation and processing to a
separate function.

cc @nodejs/fs @nodejs/collaborators

Marking this as major, as this might break some code in the wild.

thefourtheye · 2016-06-05T21:38:30Z

CI Run: https://ci.nodejs.org/job/node-test-pull-request/2928/

mscdex · 2016-06-05T21:39:43Z

FWIW v8 still does not (IIRC) optimize storing typeof foo in a variable. It's faster to just inline the typeof without using a variable (e.g. typeof actualType === 'function').

That's how I understand it too.

benjamingr · 2016-06-05T22:02:40Z

Generally +1 and LGTM, left a nit.

I think this can safely be a minor - this should only break code that parses error messages - right?

mscdex · 2016-06-05T22:22:32Z

AFAIK changing error message(s) still makes it a semver-major change.

seishun · 2016-06-06T05:40:45Z

While we're at it, either the error should say "object that isn't a function", or functions should be accepted as option bags too.

thefourtheye · 2016-06-06T12:24:11Z

@seishun The default options is used because, the functions which call getOptions handle the "options passed which are actually functions" case.

trevnorris · 2016-06-13T22:50:07Z

Can we please place this in an if (). no sense abusing JS's logical operators to run code.

Ya sure... I changed it to an if block.

silverwind · 2016-06-15T18:47:30Z

Why not keep the other tests and adapt the string?

@silverwind Now, I just changed the strings and allowed only null.

silverwind · 2016-06-15T18:48:07Z

LGTM with question.

thefourtheye · 2016-06-30T17:23:45Z

Bump!

trevnorris · 2016-06-30T19:25:03Z

Just want to double check that using == null is intentional?

Yes, @trevnorris. It is intentional.

Sometimes people pass null and sometimes undefined. That is why I wrote it like this. Eg: https://github.com/isaacs/node-graceful-fs/blob/master/graceful-fs.js#L89

(a bit late, sorry)
My only beef with this is that next time a developer sees this, they may become eager to "fix" this, causing either a bug or a new discussion about this pattern. My preference is to just be explicit and check for null and undefined explicitly with ===. Just my 2 cents.

@ronkorving Fair Enough. Changed it.

trevnorris · 2016-06-30T19:25:48Z

One question, otherwise LGTM

thefourtheye · 2016-07-12T02:27:59Z

Okay, added a commit to fix #7655 as well. PTAL.

Fishrock123 · 2016-07-12T09:53:46Z

We should probably only do this where we need to modify it, otherwise there will be a hit to each call

@Fishrock123 You mean if we don't actually modify options, we don't want to create this copy?

Hmmm, that's what I am afraid of. Then the actual logic of options processing will not at the same place. Perhaps we can have a second parameter in the function which can be used to flag if a copy has to be made. What do you think?

@Fishrock123 I changed it to util._extend from Object.assign. Hope the hit will not be that much now, at least as per #7655 (comment)

How does util._extend solve this problem? By the way, Object.assign should've become quite a bit faster since V8 5.1 (see http://v8project.blogspot.jp/2016/04/v8-release-51.html). Now I do believe someone was adding a benchmark to compare the two, so I'm not making the assertion that util._extends should be avoided (yet). But a copy is a copy, and is probably more than we need in quite a few cases.

To think of it, it just copies things to defaultOptions. That object is constructed already in memory. Would this still be a problem?

micnic · 2016-07-13T10:44:32Z

LGTM
I'm just wondering if we should also fix #7655 for v4.x or not

MylesBorins · 2016-07-13T14:48:41Z

@micnic how do you think it could be done in a non semver major way?

micnic · 2016-07-15T09:15:04Z

@thealphanerd something like options = util._extend({}, options); as the first expression in the affected methods would do the job, I don't think it is a semver major because it's an expected behavior and everywhere in the core modules except for some fs methods the external objects are not modified. In any case the impact of this change would be minimal, I could discover this only by using Object.freeze(), most of the users are using inline defined objects without reusing or freezing them.

thefourtheye · 2016-07-21T23:13:19Z

Rebased. CI Run: https://ci.nodejs.org/job/node-test-pull-request/3380/

addaleax · 2016-08-27T11:02:34Z

@thealphanerd started a comparison CITGM run with v6.3.1 yesterday with the same vinyl-fs failures: https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/371/ (from #8253 (comment)).

That shows the same failures, so the failures here are likely unrelated…

MylesBorins · 2016-08-27T11:34:16Z

/cc @phated

On Sat, Aug 27, 2016, 11:02 AM Anna Henningsen notifications@github.com
wrote:

@thealphanerd https://github.com/TheAlphaNerd started a comparison
CITGM run with v6.3.1 yesterday with the same vinyl-fs failures:
https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/371/ (from #8253
(comment)
#8253 (comment)).
That shows the same failures, so the failures here are likely unrelated…
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#7165 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAecVyWwn5yzmRmF-aUFNFUWrqXpBP_iks5qkBlSgaJpZM4IudDT
.

thefourtheye · 2016-09-02T06:34:26Z

Squashed and Rebased.

thefourtheye · 2016-09-28T11:09:26Z

New CI Run: https://ci.nodejs.org/job/node-test-pull-request/4305/

jasnell · 2016-09-30T14:55:08Z

CI looks good other than a known flaky failure. @thefourtheye, do you want to go ahead and get this landed?

As it is, the "options" processing is repeated in all the functions which need it. That introduces checks which are inconsistent with other functions and produces slightly different error messages. This patch moves the basic "options" validation and processing to a seperate function.

thefourtheye · 2016-10-05T18:50:46Z

Landed in e8e969a. Thanks for the follow-up @jasnell :-)

As it is, the "options" processing is repeated in all the functions which need it. That introduces checks which are inconsistent with other functions and produces slightly different error messages. This patch moves the basic "options" validation and processing to a seperate function. PR-URL: #7165 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Roman Reiss <me@silverwind.io> Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: Nicu Micleușanu <micnic90@gmail.com> Reviewed-By: Yorkie Liu <yorkiefixer@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

addaleax · 2016-10-05T18:54:59Z

Is this only semver-major due to the changed error messages? If so, would it make sense to pull this into v7.0.0? To me the diff looks like it could likely cause conflicts when backporting to v7 otherwise…

thefourtheye · 2016-10-05T20:18:12Z

@addaleax The encoding assertion happens for all the functions now. That also would qualify as a major change I believe.

addaleax · 2016-10-05T20:21:05Z

Okay then, that’s a bit more than just changed messages. I’m not sure, but maybe it’s still worth to ping @nodejs/ctc and see if anybody feels strongly about it?

Fishrock123 · 2016-10-05T21:10:52Z

yeah, prefer major.

If you want it in v7 we should discuss this now

jasnell · 2016-10-06T18:55:26Z

I'm ok with pulling this into v7. @nodejs/ctc ... thoughts?

addaleax · 2016-10-06T20:39:57Z

Seems okay to me, yes. 👍

jasnell · 2016-10-06T21:32:26Z

If there are no objections from @nodejs/ctc by Monday, I'll pull this in to v7.x-staging

As it is, the "options" processing is repeated in all the functions which need it. That introduces checks which are inconsistent with other functions and produces slightly different error messages. This patch moves the basic "options" validation and processing to a seperate function. PR-URL: #7165 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Roman Reiss <me@silverwind.io> Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: Nicu Micleușanu <micnic90@gmail.com> Reviewed-By: Yorkie Liu <yorkiefixer@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

As it is, the "options" processing is repeated in all the functions which need it. That introduces checks which are inconsistent with other functions and produces slightly different error messages. This patch moves the basic "options" validation and processing to a seperate function. PR-URL: nodejs/node#7165 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Roman Reiss <me@silverwind.io> Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: Nicu Micleușanu <micnic90@gmail.com> Reviewed-By: Yorkie Liu <yorkiefixer@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

thefourtheye added fs Issues and PRs related to the fs subsystem / file system. semver-major PRs that contain breaking changes and should be released in the next major version. labels Jun 5, 2016

mscdex reviewed Jun 5, 2016
View reviewed changes

trevnorris reviewed Jun 13, 2016
View reviewed changes

thefourtheye force-pushed the refactor-fs-options branch from 211453d to 8de37d0Compare June 15, 2016 10:50

silverwind reviewed Jun 15, 2016
View reviewed changes

thefourtheye force-pushed the refactor-fs-options branch 2 times, most recently from ffd1fb8 to 706316cCompare June 25, 2016 18:35

trevnorris reviewed Jun 30, 2016
View reviewed changes

thefourtheye force-pushed the refactor-fs-options branch from 706316c to 59ceb95Compare July 11, 2016 14:16

thefourtheye mentioned this pull request Jul 11, 2016
External objects should not be modified #7655
Closed

Fishrock123 reviewed Jul 12, 2016
View reviewed changes

thefourtheye force-pushed the refactor-fs-options branch from 8608551 to 2c279edCompare July 21, 2016 23:05

thefourtheye force-pushed the refactor-fs-options branch 3 times, most recently from 5c0feb3 to 845eeafCompare July 22, 2016 13:00

thefourtheye force-pushed the refactor-fs-options branch from f502293 to 1b13514Compare September 2, 2016 06:33

Trott force-pushed the master branch from b0df363 to c5ce7f4Compare September 21, 2016 00:09

silverwind approved these changes Sep 28, 2016
View reviewed changes

thefourtheye force-pushed the refactor-fs-options branch from 1b13514 to 4693ee5Compare October 5, 2016 18:42

thefourtheye closed this Oct 5, 2016

thefourtheye deleted the refactor-fs-options branch October 5, 2016 18:51

jasnell added this to the 7.0.0 milestone Oct 6, 2016

jasnell mentioned this pull request Oct 10, 2016
fs: don't alter user provided options object #7831
Closed
3 tasks

mrsimicsak mentioned this pull request Dec 20, 2016
OpenJSCAD throws fs.js error under node.js 7.0.0 jscad/OpenJSCAD.org#198
Closed

thefourtheye mentioned this pull request Mar 16, 2017
fs: getOption to populate passed in default values #11630
Closed
2 tasks

gibfahn mentioned this pull request Jun 15, 2017
Auditing for 6.11.1 nodejs/Release#230
Closed
3 tasks

corhere mentioned this pull request Aug 20, 2018
Normalize null options correctly jvilk/BrowserFS#254
Merged

Uh oh!

Uh oh!

Conversation

thefourtheye commented Jun 5, 2016

Checklist

Affected core subsystem(s)

Description of change

Uh oh!

thefourtheye commented Jun 5, 2016

Uh oh!

mscdexJun 5, 2016• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benjamingr commented Jun 5, 2016

Uh oh!

mscdex commented Jun 5, 2016

Uh oh!

seishun commented Jun 6, 2016

Uh oh!

thefourtheye commented Jun 6, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

silverwind commented Jun 15, 2016

Uh oh!

thefourtheye commented Jun 30, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

trevnorris commented Jun 30, 2016

Uh oh!

thefourtheye commented Jul 12, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thefourtheyeJul 13, 2016• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

micnic commented Jul 13, 2016

Uh oh!

MylesBorins commented Jul 13, 2016

Uh oh!

micnic commented Jul 15, 2016

Uh oh!

thefourtheye commented Jul 21, 2016

Uh oh!

addaleax commented Aug 27, 2016

Uh oh!

MylesBorins commented Aug 27, 2016

Uh oh!

mscdexJun 5, 2016•
edited
Loading

thefourtheyeJul 13, 2016•
edited
Loading