Skip to content

util: Adding warnings when NODE_DEBUG is set as http/http2 - #21914

Closed
antsmartian wants to merge 1 commit into
nodejs:masterfrom
antsmartian:21774
Closed

util: Adding warnings when NODE_DEBUG is set as http/http2#21914
antsmartian wants to merge 1 commit into
nodejs:masterfrom
antsmartian:21774

Conversation

@antsmartian

@antsmartianantsmartian commented Jul 20, 2018

Copy link
Copy Markdown
Contributor

I have fixed the issue #21774 (related to NODE_DEBUG=http part). There are couple of things I wanted to understand here:

  1. Not sure util.js is the right place for inserting this check. This should be fine I guess. Please let me know otherwise.
  2. The test case, that I had written is actually failing. I debugged it, looks like for some reason debugEnvRegextest is not working when we set the NODE_DEBUG from code, so warning is not getting triggered. I did make -j4 and tested manually with build, and I could able to see warning like the following:
(node:75788) Warning: Setting the NODE_DEBUG environment variable to 'http' can exposes sensitive data of your application.

So somewhere, I'm making a mistake in my test.

Thanks for your time on this.

  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

ChALkeR
ChALkeR previously requested changes Jul 23, 2018

@ChALkeRChALkeR left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See inline comment — we need to make sure that the warning does not get emitted multiple times.

@ChALkeR
ChALkeR dismissed their stale reviewJuly 23, 2018 17:13

Disregard my comment, the flag is not required here, it already gets cached.

Comment threadlib/util.js Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can exposes

I am not a native English speaker, but this needs rewording.

@antsmartian

antsmartian commented Jul 24, 2018

Copy link
Copy Markdown
ContributorAuthor

@ChALkeR Thanks, have made the correction. As I have mentioned on my PR, the test case is failing. Not sure where is the mistake. Thanks for your help on this.

Comment threadlib/util.js Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fwiw, this is likely also true of http2

@antsmartian

Copy link
Copy Markdown
ContributorAuthor

@jasnell Thanks for reviewing the code. Taken care for http2 too. Should be good too go I guess.

Also made the test case pass for http logging. Just for my education purpose, I tried to test these changes for http debug log like:

// doesn't work
process.env.NODE_DEBUG = 'http';
common.expectWarning(
'Warning',
'Setting the NODE_DEBUG ' +
'environment variable to \'http\' can expose sensitive data ' +
'of your application.',
common.noWarnCode
);

another method:

// doesn't work
process.env.NODE_DEBUG = 'http';
process.on('warning', ({ name, message }) => {
// assert here
});

I tried adding these assertions in test-http-client-get-url. But no luck. Finally I went ahead and created a separate test case as seen in the PR.

Comment threadlib/util.js Outdated

@ChALkeRChALkeRJul 27, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Imo, this needs to be made more explicit, something along «… can expose sensitive data (including passwords, tokens, authentication headers) in the resulting log.»

@richardlau

Copy link
Copy Markdown
Member

The test case, that I had written is actually failing. I debugged it, looks like for some reason debugEnvRegextest is not working when we set the NODE_DEBUG from code,

debugEnvRegex is set when util is loaded, so NODE_DEBUG would need to be set before any require('util'); calls (including those from other required modules, e.g. require('http');).

node/lib/util.js

Lines 298 to 307 in 28a3e28

constdebugs={};
letdebugEnvRegex=/^$/;
if(process.env.NODE_DEBUG){
letdebugEnv=process.env.NODE_DEBUG;
debugEnv=debugEnv.replace(/[|\\{}()[\]^$+?.]/g,'\\$&')
.replace(/\*/g,'.*')
.replace(/,/g,'$|^')
.toUpperCase();
debugEnvRegex=newRegExp(`^${debugEnv}$`,'i');
}

@antsmartianantsmartian changed the title util: Adding warnings when NODE_DEBUG is set as httputil: Adding warnings when NODE_DEBUG is set as http/http2Jul 27, 2018
@antsmartian

Copy link
Copy Markdown
ContributorAuthor

@richardlau Thanks, that make sense.
@ChALkeR ping, addressed your comments.

Let me know if all good.

@ChALkeRChALkeR left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rubber-stump LGTM. I have not tested this locally, though, but code and message look good to me.

I would prefer to wait a bit more for more comments from others (re: message and other things) though.

@ChALkeR

Copy link
Copy Markdown
Member

/cc @nodejs/collaborators

Comment threadlib/util.js Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional suggested text changes:

  • s/including/such as/
  • s/tokens, authentication/tokens, and authentication/

Tests will need to be updated to reflect any changes here, of course.

@Trott

Copy link
Copy Markdown
Member

@nodejs/tsc What's the semver-ness of adding a warning? We treat runtime deprecation warnings as semver-major and I'm not sure there's a good justification for treating the introduction of other runtime warnings as any less severe.

Adding semver-major label out of caution, but feel free to remove it if I'm the only person who feels that way.

@TrottTrott added semver-major PRs that contain breaking changes and should be released in the next major version. util Issues and PRs related to the built-in util module. http Issues or PRs related to the http subsystem. http2 Issues or PRs related to the http2 subsystem. labels Jul 27, 2018
@ChALkeR

Copy link
Copy Markdown
Member

@antsmartian Won't this trigger the warning twice on NODE_DEBUG=http,http2? If yes, perhaps there needs to be a flag to check if the warning has been already emitted.

@antsmartian

antsmartian commented Jul 28, 2018

Copy link
Copy Markdown
ContributorAuthor

@ChALkeR: Yes it will if the same code imports both http and http2:

require('http')
require('http2')

will give us two warnings:

(node:87392) Warning: Setting the NODE_DEBUG environment variable to 'http' can expose sensitive data (including passwords, tokens, authentication headers) in the resulting log.
(node:87392) Warning: Setting the NODE_DEBUG environment variable to 'http2' can expose sensitive data (including passwords, tokens, authentication headers) in the resulting log.

one for http and another for http2. I guess it should be ok for showing the warning for http and http2 if used as above. Let me know if you feel otherwise.

@ChALkeR

Copy link
Copy Markdown
Member

Ah, in fact it's ok if the message is different in that aspect.
@antsmartian Thanks! Disregard my comment above.

@antsmartian

Copy link
Copy Markdown
ContributorAuthor

@Trott Addressed your comments. Also thanks for fixing minor punctuation issues in comment section.

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mcollina

Copy link
Copy Markdown
Member

CI: https://ci.nodejs.org/job/node-test-pull-request/16079/

@SomeoneWeirdSomeoneWeird left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Much needed. Have seen this go very wrong before.

@mcollina

Copy link
Copy Markdown
Member

I'm planning to land next Monday morning (CEST), please comment before then.

@mcollina

Copy link
Copy Markdown
Member

Landed in 980877f

@mcollinamcollina closed this Aug 6, 2018
mcollina pushed a commit that referenced this pull request Aug 6, 2018
PR-URL: #21914
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
@ChALkeR

Copy link
Copy Markdown
Member

@mcollina Was that landed manually or with help of node-core-utils?

@antsmartian

Copy link
Copy Markdown
ContributorAuthor

Thanks everyone !

@mcollina

Copy link
Copy Markdown
Member

I’ve used ncu.

@ChALkeR

Copy link
Copy Markdown
Member

@mcollina Thanks!

@richardlaurichardlau mentioned this pull request Aug 9, 2018
4 tasks
@antsmartian
antsmartian deleted the 21774 branch September 4, 2018 05:02
jasnell added a commit that referenced this pull request Oct 2, 2018
Notable changes:
* Build
* FreeBSD 10 is no longer supported. [#22617](#22617)
* `child_process`
* The default value of the `windowsHide` option has been changed to `true`. [#21316](#21316)
* `console`
* `console.countReset()` will emit a warning if the timer being reset does not exist. [#21649](#21649)
* `console.time()` will no longer reset a timer if it already exists. [#20442](#20442)
* `crypto`
* PEM-level encryption is now supported. [#23151](#23151)
* An API for key pair generation has been added. [#22660](#22660)
* Dependencies
* V8 has been updated to 7.0. [#22754](#22754)
* `fs`
* The `fs.read()` method now requires a callback. [#22146](#22146)
* The previously deprecated `fs.SyncWriteStream` utility has been removed.[#20735](#20735)
* `http`
* The `http`, `https`, and `tls` modules now use the WHATWG URL parser by default. [#20270](#20270)
* `http2`
* An event will be emitted when a `PING` frame is received. [#23009](#23009)
* Support for the `ORIGIN` frame has been added. [#22956](#22956)
* General
* Use of `process.binding()` has been deprecated. Userland code using `process.binding()` should re-evaluate that use and begin migrating.
* An experimental implementation of `queueMicrotask()` has been added. [#22951](#22951)
* Internal
* Windows performance-counter support has been removed. [#22485](#22485)
* The `--expose-http2` command-line option has been removed. [#20887](#20887)
* Promises
* A new `multipleResolves` event will be emitted when a Promise is resolved (or rejected) more than once. [#22218](#22218)
* Timers
* Interval timers will be rescheduled even if previous interval threw an error. [#20002](#20002)
* `util`
* The WHATWG `TextEncoder` and `TextDecoder` are now globals. [#22281](#22281)
* `util.inspect()` output size is limited to 128 MB by default. [#22756](#22756)
* A runtime warning will be emitted when `NODE_DEBUG` is set for either `http` or `http2`. [#21914](#21914)
@jasnelljasnell mentioned this pull request Oct 2, 2018
4 tasks
jasnell added a commit that referenced this pull request Oct 17, 2018
Notable changes:
* Build
* FreeBSD 10 is no longer supported.[#22617](#22617)
* `child_process`
* The default value of the `windowsHide` option has been changed
to `true`. [#21316](#21316)
* `console`
* `console.countReset()` will emit a warning if the timer
being reset does not exist. [#21649](#21649)
* `console.time()` will no longer reset a timer if it already
exists. [#20442](#20442)
* Dependencies
* V8 has been updated to 7.0.
[#22754](#22754)
* `fs`
* The `fs.read()` method now requires a callback.
[#22146](#22146)
* The previously deprecated `fs.SyncWriteStream` utility has been
removed.[#20735](#20735)
* `http`
* The `http`, `https`, and `tls` modules now use the WHATWG URL parser
by default. [#20270](#20270)
* General
* Use of `process.binding()` has been deprecated. Userland code using
`process.binding()` should re-evaluate that use and begin migrating. If
there are no supported API alternatives, please open an issue in the
Node.js GitHub repository so that a suitable alternative may be discussed.
* An experimental implementation of `queueMicrotask()` has been added.
[#22951](#22951)
* Internal
* Windows performance-counter support has been removed.
[#22485](#22485)
* The `--expose-http2` command-line option has been removed.
[#20887](#20887)
* Timers
* Interval timers will be rescheduled even if previous interval threw
an error. [#20002](#20002)
* `util`
* The WHATWG `TextEncoder` and `TextDecoder` are now globals.
[#22281](#22281)
* `util.inspect()` output size is limited to 128 MB by default.
[#22756](#22756)
* A runtime warning will be emitted when `NODE_DEBUG` is set for
either `http` or `http2`. [#21914](#21914)
jasnell added a commit that referenced this pull request Oct 17, 2018
Notable changes:
* Build
* FreeBSD 10 is no longer supported.[#22617](#22617)
* `child_process`
* The default value of the `windowsHide` option has been changed
to `true`. [#21316](#21316)
* `console`
* `console.countReset()` will emit a warning if the timer
being reset does not exist. [#21649](#21649)
* `console.time()` will no longer reset a timer if it already
exists. [#20442](#20442)
* Dependencies
* V8 has been updated to 7.0.
[#22754](#22754)
* `fs`
* The `fs.read()` method now requires a callback.
[#22146](#22146)
* The previously deprecated `fs.SyncWriteStream` utility has been
removed.[#20735](#20735)
* `http`
* The `http`, `https`, and `tls` modules now use the WHATWG URL parser
by default. [#20270](#20270)
* General
* Use of `process.binding()` has been deprecated. Userland code using
`process.binding()` should re-evaluate that use and begin migrating. If
there are no supported API alternatives, please open an issue in the
Node.js GitHub repository so that a suitable alternative may be discussed.
* An experimental implementation of `queueMicrotask()` has been added.
[#22951](#22951)
* Internal
* Windows performance-counter support has been removed.
[#22485](#22485)
* The `--expose-http2` command-line option has been removed.
[#20887](#20887)
* Timers
* Interval timers will be rescheduled even if previous interval threw
an error. [#20002](#20002)
* `util`
* The WHATWG `TextEncoder` and `TextDecoder` are now globals.
[#22281](#22281)
* `util.inspect()` output size is limited to 128 MB by default.
[#22756](#22756)
* A runtime warning will be emitted when `NODE_DEBUG` is set for
either `http` or `http2`. [#21914](#21914)
jasnell added a commit that referenced this pull request Oct 21, 2018
Notable changes:
* Build
* FreeBSD 10 is no longer supported.[#22617](#22617)
* `child_process`
* The default value of the `windowsHide` option has been changed
to `true`. [#21316](#21316)
* `console`
* `console.countReset()` will emit a warning if the timer
being reset does not exist. [#21649](#21649)
* `console.time()` will no longer reset a timer if it already
exists. [#20442](#20442)
* Dependencies
* V8 has been updated to 7.0.
[#22754](#22754)
* `fs`
* The `fs.read()` method now requires a callback.
[#22146](#22146)
* The previously deprecated `fs.SyncWriteStream` utility has been
removed.[#20735](#20735)
* `http`
* The `http`, `https`, and `tls` modules now use the WHATWG URL parser
by default. [#20270](#20270)
* General
* Use of `process.binding()` has been deprecated. Userland code using
`process.binding()` should re-evaluate that use and begin migrating. If
there are no supported API alternatives, please open an issue in the
Node.js GitHub repository so that a suitable alternative may be discussed.
* An experimental implementation of `queueMicrotask()` has been added.
[#22951](#22951)
* Internal
* Windows performance-counter support has been removed.
[#22485](#22485)
* The `--expose-http2` command-line option has been removed.
[#20887](#20887)
* Timers
* Interval timers will be rescheduled even if previous interval threw
an error. [#20002](#20002)
* `util`
* The WHATWG `TextEncoder` and `TextDecoder` are now globals.
[#22281](#22281)
* `util.inspect()` output size is limited to 128 MB by default.
[#22756](#22756)
* A runtime warning will be emitted when `NODE_DEBUG` is set for
either `http` or `http2`. [#21914](#21914)
jasnell added a commit that referenced this pull request Oct 22, 2018
Notable changes:
* Build
* FreeBSD 10 is no longer supported.[#22617](#22617)
* `child_process`
* The default value of the `windowsHide` option has been changed
to `true`. [#21316](#21316)
* `console`
* `console.countReset()` will emit a warning if the timer
being reset does not exist. [#21649](#21649)
* `console.time()` will no longer reset a timer if it already
exists. [#20442](#20442)
* Dependencies
* V8 has been updated to 7.0.
[#22754](#22754)
* `fs`
* The `fs.read()` method now requires a callback.
[#22146](#22146)
* The previously deprecated `fs.SyncWriteStream` utility has been
removed.[#20735](#20735)
* `http`
* The `http`, `https`, and `tls` modules now use the WHATWG URL parser
by default. [#20270](#20270)
* General
* Use of `process.binding()` has been deprecated. Userland code using
`process.binding()` should re-evaluate that use and begin migrating. If
there are no supported API alternatives, please open an issue in the
Node.js GitHub repository so that a suitable alternative may be discussed.
* An experimental implementation of `queueMicrotask()` has been added.
[#22951](#22951)
* Internal
* Windows performance-counter support has been removed.
[#22485](#22485)
* The `--expose-http2` command-line option has been removed.
[#20887](#20887)
* Timers
* Interval timers will be rescheduled even if previous interval threw
an error. [#20002](#20002)
* `util`
* The WHATWG `TextEncoder` and `TextDecoder` are now globals.
[#22281](#22281)
* `util.inspect()` output size is limited to 128 MB by default.
[#22756](#22756)
* A runtime warning will be emitted when `NODE_DEBUG` is set for
either `http` or `http2`. [#21914](#21914)
devsnek pushed a commit to devsnek/node that referenced this pull request Oct 23, 2018
Notable changes:
* Build
* FreeBSD 10 is no longer supported.[nodejs#22617](nodejs#22617)
* `child_process`
* The default value of the `windowsHide` option has been changed
to `true`. [nodejs#21316](nodejs#21316)
* `console`
* `console.countReset()` will emit a warning if the timer
being reset does not exist. [nodejs#21649](nodejs#21649)
* `console.time()` will no longer reset a timer if it already
exists. [nodejs#20442](nodejs#20442)
* Dependencies
* V8 has been updated to 7.0.
[nodejs#22754](nodejs#22754)
* `fs`
* The `fs.read()` method now requires a callback.
[nodejs#22146](nodejs#22146)
* The previously deprecated `fs.SyncWriteStream` utility has been
removed.[nodejs#20735](nodejs#20735)
* `http`
* The `http`, `https`, and `tls` modules now use the WHATWG URL parser
by default. [nodejs#20270](nodejs#20270)
* General
* Use of `process.binding()` has been deprecated. Userland code using
`process.binding()` should re-evaluate that use and begin migrating. If
there are no supported API alternatives, please open an issue in the
Node.js GitHub repository so that a suitable alternative may be discussed.
* An experimental implementation of `queueMicrotask()` has been added.
[nodejs#22951](nodejs#22951)
* Internal
* Windows performance-counter support has been removed.
[nodejs#22485](nodejs#22485)
* The `--expose-http2` command-line option has been removed.
[nodejs#20887](nodejs#20887)
* Timers
* Interval timers will be rescheduled even if previous interval threw
an error. [nodejs#20002](nodejs#20002)
* `util`
* The WHATWG `TextEncoder` and `TextDecoder` are now globals.
[nodejs#22281](nodejs#22281)
* `util.inspect()` output size is limited to 128 MB by default.
[nodejs#22756](nodejs#22756)
* A runtime warning will be emitted when `NODE_DEBUG` is set for
either `http` or `http2`. [nodejs#21914](nodejs#21914)
deepak1556 pushed a commit to electron/node that referenced this pull request Dec 10, 2018
Notable changes:
* Build
* FreeBSD 10 is no longer supported.[#22617](nodejs/node#22617)
* `child_process`
* The default value of the `windowsHide` option has been changed
to `true`. [#21316](nodejs/node#21316)
* `console`
* `console.countReset()` will emit a warning if the timer
being reset does not exist. [#21649](nodejs/node#21649)
* `console.time()` will no longer reset a timer if it already
exists. [#20442](nodejs/node#20442)
* Dependencies
* V8 has been updated to 7.0.
[#22754](nodejs/node#22754)
* `fs`
* The `fs.read()` method now requires a callback.
[#22146](nodejs/node#22146)
* The previously deprecated `fs.SyncWriteStream` utility has been
removed.[#20735](nodejs/node#20735)
* `http`
* The `http`, `https`, and `tls` modules now use the WHATWG URL parser
by default. [#20270](nodejs/node#20270)
* General
* Use of `process.binding()` has been deprecated. Userland code using
`process.binding()` should re-evaluate that use and begin migrating. If
there are no supported API alternatives, please open an issue in the
Node.js GitHub repository so that a suitable alternative may be discussed.
* An experimental implementation of `queueMicrotask()` has been added.
[#22951](nodejs/node#22951)
* Internal
* Windows performance-counter support has been removed.
[#22485](nodejs/node#22485)
* The `--expose-http2` command-line option has been removed.
[#20887](nodejs/node#20887)
* Timers
* Interval timers will be rescheduled even if previous interval threw
an error. [#20002](nodejs/node#20002)
* `util`
* The WHATWG `TextEncoder` and `TextDecoder` are now globals.
[#22281](nodejs/node#22281)
* `util.inspect()` output size is limited to 128 MB by default.
[#22756](nodejs/node#22756)
* A runtime warning will be emitted when `NODE_DEBUG` is set for
either `http` or `http2`. [#21914](nodejs/node#21914)
deepak1556 pushed a commit to electron/node that referenced this pull request Dec 19, 2018
Notable changes:
* Build
* FreeBSD 10 is no longer supported.[#22617](nodejs/node#22617)
* `child_process`
* The default value of the `windowsHide` option has been changed
to `true`. [#21316](nodejs/node#21316)
* `console`
* `console.countReset()` will emit a warning if the timer
being reset does not exist. [#21649](nodejs/node#21649)
* `console.time()` will no longer reset a timer if it already
exists. [#20442](nodejs/node#20442)
* Dependencies
* V8 has been updated to 7.0.
[#22754](nodejs/node#22754)
* `fs`
* The `fs.read()` method now requires a callback.
[#22146](nodejs/node#22146)
* The previously deprecated `fs.SyncWriteStream` utility has been
removed.[#20735](nodejs/node#20735)
* `http`
* The `http`, `https`, and `tls` modules now use the WHATWG URL parser
by default. [#20270](nodejs/node#20270)
* General
* Use of `process.binding()` has been deprecated. Userland code using
`process.binding()` should re-evaluate that use and begin migrating. If
there are no supported API alternatives, please open an issue in the
Node.js GitHub repository so that a suitable alternative may be discussed.
* An experimental implementation of `queueMicrotask()` has been added.
[#22951](nodejs/node#22951)
* Internal
* Windows performance-counter support has been removed.
[#22485](nodejs/node#22485)
* The `--expose-http2` command-line option has been removed.
[#20887](nodejs/node#20887)
* Timers
* Interval timers will be rescheduled even if previous interval threw
an error. [#20002](nodejs/node#20002)
* `util`
* The WHATWG `TextEncoder` and `TextDecoder` are now globals.
[#22281](nodejs/node#22281)
* `util.inspect()` output size is limited to 128 MB by default.
[#22756](nodejs/node#22756)
* A runtime warning will be emitted when `NODE_DEBUG` is set for
either `http` or `http2`. [#21914](nodejs/node#21914)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

httpIssues or PRs related to the http subsystem.http2Issues or PRs related to the http2 subsystem.securityIssues and PRs related to security.semver-majorPRs that contain breaking changes and should be released in the next major version.utilIssues and PRs related to the built-in util module.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@antsmartian@richardlau@ChALkeR@Trott@mcollina@jasnell@SomeoneWeird