no_proxy support - #1176

Closed
ballwood wants to merge 3 commits into
nodejs:masterfrom
ballwood:master
Closed

no_proxy support#1176
ballwood wants to merge 3 commits into
nodejs:masterfrom
ballwood:master

Conversation

@ballwood

@ballwoodballwood commented Apr 14, 2017

Copy link
Copy Markdown

Ported code across from the request library, which I know to work correctly - https://github.com/request/request.

Unsure of where to put the unit/integration tests, if someone could guide me I'll write them. There were 11 tests failing when running npm test when I forked and the same amount after I made the code change.

@ballwoodballwood mentioned this pull request Apr 14, 2017
@bnoordhuis

Copy link
Copy Markdown
Member

Sorry for the delay. This needs some tests but I'm kind of surprised at how complex it still is. Is npm_config_no_proxy a thing?

@vbfox

Copy link
Copy Markdown

I'd really like to see that PR get merged.

For npm config exporting npm_config_no_proxy there is a PR open but I don't know if it'll ever get merged: npm/npm#14248 (Currently it can't be specified in npm config)

@ballwood

Copy link
Copy Markdown
Author

Hi, sorry I thought I replied to this last week, must have not pressed send. Yeah I noticed that PR was open and included it to future proof it but their work on it seems to have gone by the wayside.

I can have a look at the tests I'm just trying to understand them at the moment and work out something that works correctly.

@ballwood

Copy link
Copy Markdown
Author

@bnoordhuis wrote some small unit tests as I got some free time

@bnoordhuisbnoordhuis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments. Thanks for following up.

Comment threadlib/install.js

function formatHostname (hostname) {
// canonicalize the hostname, so that 'oogle.com' won't match 'google.com'
return hostname.replace(/^\.*/, '.').toLowerCase()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This transforms google.com into .google.com. I infer that is intentional but can you update the comment to make that more explicit? This behavior also seems only lightly tested.

Aside: can you capitalize and punctuate comments?

Comment threadlib/install.js
var zoneParts = zone.split(':', 2)
var zoneHost = formatHostname(zoneParts[0])
var zonePort = zoneParts[1]
var hasPort = zone.indexOf(':') > -1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not wrong per se but can also be derived without re-scanning from zonePort !== undefined or zoneParts.length === 2.

Comment threadlib/install.js
var isMatchedAt = hostname.indexOf(noProxyZone.hostname)
var hostnameMatched = (
isMatchedAt > -1 &&
(isMatchedAt === hostname.length - noProxyZone.hostname.length)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct me if I'm wrong but isn't this just a circuitous way of saying hostname === noProxyZone.hostname?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it's a suffix match

Comment threadlib/install.js
var noProxyUrl = gyp.opts.no_proxy
|| env.no_proxy
|| env.NO_PROXY
|| env.npm_config_no_proxy

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably go; the npm pull request wasn't merged.

Comment threadlib/install.js
|| env.npm_config_no_proxy
|| ''

if (noProxyUrl === '*' || (noProxyUrl !== '' && uriInNoProxy(url, noProxyUrl))) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Long line; can you keep it under 80 columns? The parens around the && aren't necessary.

Comment threadlib/install.js
log.verbose('download', 'using proxy url: "%s"', proxyUrl)
requestOpts.proxy = proxyUrl
} else {
log.warn('download', 'ignoring invalid "proxy" config setting: "%s"', proxyUrl)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto, long line.

@@ -0,0 +1,63 @@
var test = require('tape')

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add 'use strict';?


t.equal(req.opts.proxy, gyp.opts.proxy)
t.equal(req.opts.uri, url)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you drop the blank line here and below?

@bnoordhuisbnoordhuis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wups sorry, meant to request changes. Only had one cup of coffee so far.

@maclover7

Copy link
Copy Markdown
Contributor

ping @ballwood

@dlangerenken

Copy link
Copy Markdown

@bnoordhuis is someone taking care of this PR? Seems like @ballwood is no longer working on it. It would be really nice to have in master.

@rvagg

Copy link
Copy Markdown
Member

This looks pretty good and we have HTTPS_PROXY support about to land too but it's going to need a new champion. If someone wants this in, please take this code, clean it up as per the suggestions in here and make it work against master. Tests would be good too if you can come up with some that are meaningful.

@rvaggrvagg closed this Jun 21, 2019
@imatlopez

imatlopez commented Nov 27, 2019

Copy link
Copy Markdown
Contributor

@rvagg I was looking at doing just this given this is quite a pain point inside our corporate network, however is of this approach to make it independent of requests? Would it be simpler to set the environment variables from the cli flags as needed and let requests do its job?

@rvagg

Copy link
Copy Markdown
Member

@imatlopez you mean request not requests right? Apparently obeys no_proxy these days but I don't know what we would need to do here, if anything, for that support to flow through. If it's obeying environment variables then I'd assume we don't have to do anything on our end, just set the env var before calling. If you need gyp.opts to be able to carry it then I guess some additional work could go in here and hopefully that'd be simple. You're welcome to open a PR if you'd like it added.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@ballwood@bnoordhuis@vbfox@maclover7@dlangerenken@rvagg@imatlopez@alsotang
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

no_proxy support - #1176

Closed
ballwood wants to merge 3 commits into
nodejs:masterfrom
ballwood:master
Closed

no_proxy support#1176
ballwood wants to merge 3 commits into
nodejs:masterfrom
ballwood:master

Conversation

@ballwood

@ballwoodballwood commented Apr 14, 2017

Copy link
Copy Markdown

Ported code across from the request library, which I know to work correctly - https://github.com/request/request.

Unsure of where to put the unit/integration tests, if someone could guide me I'll write them. There were 11 tests failing when running npm test when I forked and the same amount after I made the code change.

@ballwoodballwood mentioned this pull request Apr 14, 2017
@bnoordhuis

Copy link
Copy Markdown
Member

Sorry for the delay. This needs some tests but I'm kind of surprised at how complex it still is. Is npm_config_no_proxy a thing?

@vbfox

Copy link
Copy Markdown

I'd really like to see that PR get merged.

For npm config exporting npm_config_no_proxy there is a PR open but I don't know if it'll ever get merged: npm/npm#14248 (Currently it can't be specified in npm config)

@ballwood

Copy link
Copy Markdown
Author

Hi, sorry I thought I replied to this last week, must have not pressed send. Yeah I noticed that PR was open and included it to future proof it but their work on it seems to have gone by the wayside.

I can have a look at the tests I'm just trying to understand them at the moment and work out something that works correctly.

@ballwood

Copy link
Copy Markdown
Author

@bnoordhuis wrote some small unit tests as I got some free time

@bnoordhuisbnoordhuis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments. Thanks for following up.

Comment threadlib/install.js

function formatHostname (hostname) {
// canonicalize the hostname, so that 'oogle.com' won't match 'google.com'
return hostname.replace(/^\.*/, '.').toLowerCase()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This transforms google.com into .google.com. I infer that is intentional but can you update the comment to make that more explicit? This behavior also seems only lightly tested.

Aside: can you capitalize and punctuate comments?

Comment threadlib/install.js
var zoneParts = zone.split(':', 2)
var zoneHost = formatHostname(zoneParts[0])
var zonePort = zoneParts[1]
var hasPort = zone.indexOf(':') > -1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not wrong per se but can also be derived without re-scanning from zonePort !== undefined or zoneParts.length === 2.

Comment threadlib/install.js
var isMatchedAt = hostname.indexOf(noProxyZone.hostname)
var hostnameMatched = (
isMatchedAt > -1 &&
(isMatchedAt === hostname.length - noProxyZone.hostname.length)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct me if I'm wrong but isn't this just a circuitous way of saying hostname === noProxyZone.hostname?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it's a suffix match

Comment threadlib/install.js
var noProxyUrl = gyp.opts.no_proxy
|| env.no_proxy
|| env.NO_PROXY
|| env.npm_config_no_proxy

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably go; the npm pull request wasn't merged.

Comment threadlib/install.js
|| env.npm_config_no_proxy
|| ''

if (noProxyUrl === '*' || (noProxyUrl !== '' && uriInNoProxy(url, noProxyUrl))) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Long line; can you keep it under 80 columns? The parens around the && aren't necessary.

Comment threadlib/install.js
log.verbose('download', 'using proxy url: "%s"', proxyUrl)
requestOpts.proxy = proxyUrl
} else {
log.warn('download', 'ignoring invalid "proxy" config setting: "%s"', proxyUrl)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto, long line.

@@ -0,0 +1,63 @@
var test = require('tape')

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add 'use strict';?


t.equal(req.opts.proxy, gyp.opts.proxy)
t.equal(req.opts.uri, url)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you drop the blank line here and below?

@bnoordhuisbnoordhuis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wups sorry, meant to request changes. Only had one cup of coffee so far.

@maclover7

Copy link
Copy Markdown
Contributor

ping @ballwood

@dlangerenken

Copy link
Copy Markdown

@bnoordhuis is someone taking care of this PR? Seems like @ballwood is no longer working on it. It would be really nice to have in master.

@rvagg

Copy link
Copy Markdown
Member

This looks pretty good and we have HTTPS_PROXY support about to land too but it's going to need a new champion. If someone wants this in, please take this code, clean it up as per the suggestions in here and make it work against master. Tests would be good too if you can come up with some that are meaningful.

@rvaggrvagg closed this Jun 21, 2019
@imatlopez

imatlopez commented Nov 27, 2019

Copy link
Copy Markdown
Contributor

@rvagg I was looking at doing just this given this is quite a pain point inside our corporate network, however is of this approach to make it independent of requests? Would it be simpler to set the environment variables from the cli flags as needed and let requests do its job?

@rvagg

Copy link
Copy Markdown
Member

@imatlopez you mean request not requests right? Apparently obeys no_proxy these days but I don't know what we would need to do here, if anything, for that support to flow through. If it's obeying environment variables then I'd assume we don't have to do anything on our end, just set the env var before calling. If you need gyp.opts to be able to carry it then I guess some additional work could go in here and hopefully that'd be simple. You're welcome to open a PR if you'd like it added.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@ballwood@bnoordhuis@vbfox@maclover7@dlangerenken@rvagg@imatlopez@alsotang
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

no_proxy support - #1176

Closed
ballwood wants to merge 3 commits into
nodejs:masterfrom
ballwood:master
Closed

no_proxy support#1176
ballwood wants to merge 3 commits into
nodejs:masterfrom
ballwood:master

Conversation

@ballwood

@ballwoodballwood commented Apr 14, 2017

Copy link
Copy Markdown

Ported code across from the request library, which I know to work correctly - https://github.com/request/request.

Unsure of where to put the unit/integration tests, if someone could guide me I'll write them. There were 11 tests failing when running npm test when I forked and the same amount after I made the code change.

@ballwoodballwood mentioned this pull request Apr 14, 2017
@bnoordhuis

Copy link
Copy Markdown
Member

Sorry for the delay. This needs some tests but I'm kind of surprised at how complex it still is. Is npm_config_no_proxy a thing?

@vbfox

Copy link
Copy Markdown

I'd really like to see that PR get merged.

For npm config exporting npm_config_no_proxy there is a PR open but I don't know if it'll ever get merged: npm/npm#14248 (Currently it can't be specified in npm config)

@ballwood

Copy link
Copy Markdown
Author

Hi, sorry I thought I replied to this last week, must have not pressed send. Yeah I noticed that PR was open and included it to future proof it but their work on it seems to have gone by the wayside.

I can have a look at the tests I'm just trying to understand them at the moment and work out something that works correctly.

@ballwood

Copy link
Copy Markdown
Author

@bnoordhuis wrote some small unit tests as I got some free time

@bnoordhuisbnoordhuis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments. Thanks for following up.

Comment threadlib/install.js

function formatHostname (hostname) {
// canonicalize the hostname, so that 'oogle.com' won't match 'google.com'
return hostname.replace(/^\.*/, '.').toLowerCase()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This transforms google.com into .google.com. I infer that is intentional but can you update the comment to make that more explicit? This behavior also seems only lightly tested.

Aside: can you capitalize and punctuate comments?

Comment threadlib/install.js
var zoneParts = zone.split(':', 2)
var zoneHost = formatHostname(zoneParts[0])
var zonePort = zoneParts[1]
var hasPort = zone.indexOf(':') > -1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not wrong per se but can also be derived without re-scanning from zonePort !== undefined or zoneParts.length === 2.

Comment threadlib/install.js
var isMatchedAt = hostname.indexOf(noProxyZone.hostname)
var hostnameMatched = (
isMatchedAt > -1 &&
(isMatchedAt === hostname.length - noProxyZone.hostname.length)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct me if I'm wrong but isn't this just a circuitous way of saying hostname === noProxyZone.hostname?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it's a suffix match

Comment threadlib/install.js
var noProxyUrl = gyp.opts.no_proxy
|| env.no_proxy
|| env.NO_PROXY
|| env.npm_config_no_proxy

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably go; the npm pull request wasn't merged.

Comment threadlib/install.js
|| env.npm_config_no_proxy
|| ''

if (noProxyUrl === '*' || (noProxyUrl !== '' && uriInNoProxy(url, noProxyUrl))) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Long line; can you keep it under 80 columns? The parens around the && aren't necessary.

Comment threadlib/install.js
log.verbose('download', 'using proxy url: "%s"', proxyUrl)
requestOpts.proxy = proxyUrl
} else {
log.warn('download', 'ignoring invalid "proxy" config setting: "%s"', proxyUrl)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto, long line.

@@ -0,0 +1,63 @@
var test = require('tape')

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add 'use strict';?


t.equal(req.opts.proxy, gyp.opts.proxy)
t.equal(req.opts.uri, url)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you drop the blank line here and below?

@bnoordhuisbnoordhuis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wups sorry, meant to request changes. Only had one cup of coffee so far.

@maclover7

Copy link
Copy Markdown
Contributor

ping @ballwood

@dlangerenken

Copy link
Copy Markdown

@bnoordhuis is someone taking care of this PR? Seems like @ballwood is no longer working on it. It would be really nice to have in master.

@rvagg

Copy link
Copy Markdown
Member

This looks pretty good and we have HTTPS_PROXY support about to land too but it's going to need a new champion. If someone wants this in, please take this code, clean it up as per the suggestions in here and make it work against master. Tests would be good too if you can come up with some that are meaningful.

@rvaggrvagg closed this Jun 21, 2019
@imatlopez

imatlopez commented Nov 27, 2019

Copy link
Copy Markdown
Contributor

@rvagg I was looking at doing just this given this is quite a pain point inside our corporate network, however is of this approach to make it independent of requests? Would it be simpler to set the environment variables from the cli flags as needed and let requests do its job?

@rvagg

Copy link
Copy Markdown
Member

@imatlopez you mean request not requests right? Apparently obeys no_proxy these days but I don't know what we would need to do here, if anything, for that support to flow through. If it's obeying environment variables then I'd assume we don't have to do anything on our end, just set the env var before calling. If you need gyp.opts to be able to carry it then I guess some additional work could go in here and hopefully that'd be simple. You're welcome to open a PR if you'd like it added.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@ballwood@bnoordhuis@vbfox@maclover7@dlangerenken@rvagg@imatlopez@alsotang
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

no_proxy support - #1176

Closed
ballwood wants to merge 3 commits into
nodejs:masterfrom
ballwood:master
Closed

no_proxy support#1176
ballwood wants to merge 3 commits into
nodejs:masterfrom
ballwood:master

Conversation

@ballwood

@ballwoodballwood commented Apr 14, 2017

Copy link
Copy Markdown

Ported code across from the request library, which I know to work correctly - https://github.com/request/request.

Unsure of where to put the unit/integration tests, if someone could guide me I'll write them. There were 11 tests failing when running npm test when I forked and the same amount after I made the code change.

@ballwoodballwood mentioned this pull request Apr 14, 2017
@bnoordhuis

Copy link
Copy Markdown
Member

Sorry for the delay. This needs some tests but I'm kind of surprised at how complex it still is. Is npm_config_no_proxy a thing?

@vbfox

Copy link
Copy Markdown

I'd really like to see that PR get merged.

For npm config exporting npm_config_no_proxy there is a PR open but I don't know if it'll ever get merged: npm/npm#14248 (Currently it can't be specified in npm config)

@ballwood

Copy link
Copy Markdown
Author

Hi, sorry I thought I replied to this last week, must have not pressed send. Yeah I noticed that PR was open and included it to future proof it but their work on it seems to have gone by the wayside.

I can have a look at the tests I'm just trying to understand them at the moment and work out something that works correctly.

@ballwood

Copy link
Copy Markdown
Author

@bnoordhuis wrote some small unit tests as I got some free time

@bnoordhuisbnoordhuis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments. Thanks for following up.

Comment threadlib/install.js

function formatHostname (hostname) {
// canonicalize the hostname, so that 'oogle.com' won't match 'google.com'
return hostname.replace(/^\.*/, '.').toLowerCase()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This transforms google.com into .google.com. I infer that is intentional but can you update the comment to make that more explicit? This behavior also seems only lightly tested.

Aside: can you capitalize and punctuate comments?

Comment threadlib/install.js
var zoneParts = zone.split(':', 2)
var zoneHost = formatHostname(zoneParts[0])
var zonePort = zoneParts[1]
var hasPort = zone.indexOf(':') > -1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not wrong per se but can also be derived without re-scanning from zonePort !== undefined or zoneParts.length === 2.

Comment threadlib/install.js
var isMatchedAt = hostname.indexOf(noProxyZone.hostname)
var hostnameMatched = (
isMatchedAt > -1 &&
(isMatchedAt === hostname.length - noProxyZone.hostname.length)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct me if I'm wrong but isn't this just a circuitous way of saying hostname === noProxyZone.hostname?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it's a suffix match

Comment threadlib/install.js
var noProxyUrl = gyp.opts.no_proxy
|| env.no_proxy
|| env.NO_PROXY
|| env.npm_config_no_proxy

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably go; the npm pull request wasn't merged.

Comment threadlib/install.js
|| env.npm_config_no_proxy
|| ''

if (noProxyUrl === '*' || (noProxyUrl !== '' && uriInNoProxy(url, noProxyUrl))) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Long line; can you keep it under 80 columns? The parens around the && aren't necessary.

Comment threadlib/install.js
log.verbose('download', 'using proxy url: "%s"', proxyUrl)
requestOpts.proxy = proxyUrl
} else {
log.warn('download', 'ignoring invalid "proxy" config setting: "%s"', proxyUrl)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto, long line.

@@ -0,0 +1,63 @@
var test = require('tape')

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add 'use strict';?


t.equal(req.opts.proxy, gyp.opts.proxy)
t.equal(req.opts.uri, url)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you drop the blank line here and below?

@bnoordhuisbnoordhuis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wups sorry, meant to request changes. Only had one cup of coffee so far.

@maclover7

Copy link
Copy Markdown
Contributor

ping @ballwood

@dlangerenken

Copy link
Copy Markdown

@bnoordhuis is someone taking care of this PR? Seems like @ballwood is no longer working on it. It would be really nice to have in master.

@rvagg

Copy link
Copy Markdown
Member

This looks pretty good and we have HTTPS_PROXY support about to land too but it's going to need a new champion. If someone wants this in, please take this code, clean it up as per the suggestions in here and make it work against master. Tests would be good too if you can come up with some that are meaningful.

@rvaggrvagg closed this Jun 21, 2019
@imatlopez

imatlopez commented Nov 27, 2019

Copy link
Copy Markdown
Contributor

@rvagg I was looking at doing just this given this is quite a pain point inside our corporate network, however is of this approach to make it independent of requests? Would it be simpler to set the environment variables from the cli flags as needed and let requests do its job?

@rvagg

Copy link
Copy Markdown
Member

@imatlopez you mean request not requests right? Apparently obeys no_proxy these days but I don't know what we would need to do here, if anything, for that support to flow through. If it's obeying environment variables then I'd assume we don't have to do anything on our end, just set the env var before calling. If you need gyp.opts to be able to carry it then I guess some additional work could go in here and hopefully that'd be simple. You're welcome to open a PR if you'd like it added.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@ballwood@bnoordhuis@vbfox@maclover7@dlangerenken@rvagg@imatlopez@alsotang
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

no_proxy support - #1176

Closed
ballwood wants to merge 3 commits into
nodejs:masterfrom
ballwood:master
Closed

no_proxy support#1176
ballwood wants to merge 3 commits into
nodejs:masterfrom
ballwood:master

Conversation

@ballwood

@ballwoodballwood commented Apr 14, 2017

Copy link
Copy Markdown

Ported code across from the request library, which I know to work correctly - https://github.com/request/request.

Unsure of where to put the unit/integration tests, if someone could guide me I'll write them. There were 11 tests failing when running npm test when I forked and the same amount after I made the code change.

@ballwoodballwood mentioned this pull request Apr 14, 2017
@bnoordhuis

Copy link
Copy Markdown
Member

Sorry for the delay. This needs some tests but I'm kind of surprised at how complex it still is. Is npm_config_no_proxy a thing?

@vbfox

Copy link
Copy Markdown

I'd really like to see that PR get merged.

For npm config exporting npm_config_no_proxy there is a PR open but I don't know if it'll ever get merged: npm/npm#14248 (Currently it can't be specified in npm config)

@ballwood

Copy link
Copy Markdown
Author

Hi, sorry I thought I replied to this last week, must have not pressed send. Yeah I noticed that PR was open and included it to future proof it but their work on it seems to have gone by the wayside.

I can have a look at the tests I'm just trying to understand them at the moment and work out something that works correctly.

@ballwood

Copy link
Copy Markdown
Author

@bnoordhuis wrote some small unit tests as I got some free time

@bnoordhuisbnoordhuis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments. Thanks for following up.

Comment threadlib/install.js

function formatHostname (hostname) {
// canonicalize the hostname, so that 'oogle.com' won't match 'google.com'
return hostname.replace(/^\.*/, '.').toLowerCase()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This transforms google.com into .google.com. I infer that is intentional but can you update the comment to make that more explicit? This behavior also seems only lightly tested.

Aside: can you capitalize and punctuate comments?

Comment threadlib/install.js
var zoneParts = zone.split(':', 2)
var zoneHost = formatHostname(zoneParts[0])
var zonePort = zoneParts[1]
var hasPort = zone.indexOf(':') > -1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not wrong per se but can also be derived without re-scanning from zonePort !== undefined or zoneParts.length === 2.

Comment threadlib/install.js
var isMatchedAt = hostname.indexOf(noProxyZone.hostname)
var hostnameMatched = (
isMatchedAt > -1 &&
(isMatchedAt === hostname.length - noProxyZone.hostname.length)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct me if I'm wrong but isn't this just a circuitous way of saying hostname === noProxyZone.hostname?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it's a suffix match

Comment threadlib/install.js
var noProxyUrl = gyp.opts.no_proxy
|| env.no_proxy
|| env.NO_PROXY
|| env.npm_config_no_proxy

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably go; the npm pull request wasn't merged.

Comment threadlib/install.js
|| env.npm_config_no_proxy
|| ''

if (noProxyUrl === '*' || (noProxyUrl !== '' && uriInNoProxy(url, noProxyUrl))) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Long line; can you keep it under 80 columns? The parens around the && aren't necessary.

Comment threadlib/install.js
log.verbose('download', 'using proxy url: "%s"', proxyUrl)
requestOpts.proxy = proxyUrl
} else {
log.warn('download', 'ignoring invalid "proxy" config setting: "%s"', proxyUrl)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto, long line.

@@ -0,0 +1,63 @@
var test = require('tape')

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add 'use strict';?


t.equal(req.opts.proxy, gyp.opts.proxy)
t.equal(req.opts.uri, url)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you drop the blank line here and below?

@bnoordhuisbnoordhuis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wups sorry, meant to request changes. Only had one cup of coffee so far.

@maclover7

Copy link
Copy Markdown
Contributor

ping @ballwood

@dlangerenken

Copy link
Copy Markdown

@bnoordhuis is someone taking care of this PR? Seems like @ballwood is no longer working on it. It would be really nice to have in master.

@rvagg

Copy link
Copy Markdown
Member

This looks pretty good and we have HTTPS_PROXY support about to land too but it's going to need a new champion. If someone wants this in, please take this code, clean it up as per the suggestions in here and make it work against master. Tests would be good too if you can come up with some that are meaningful.

@rvaggrvagg closed this Jun 21, 2019
@imatlopez

imatlopez commented Nov 27, 2019

Copy link
Copy Markdown
Contributor

@rvagg I was looking at doing just this given this is quite a pain point inside our corporate network, however is of this approach to make it independent of requests? Would it be simpler to set the environment variables from the cli flags as needed and let requests do its job?

@rvagg

Copy link
Copy Markdown
Member

@imatlopez you mean request not requests right? Apparently obeys no_proxy these days but I don't know what we would need to do here, if anything, for that support to flow through. If it's obeying environment variables then I'd assume we don't have to do anything on our end, just set the env var before calling. If you need gyp.opts to be able to carry it then I guess some additional work could go in here and hopefully that'd be simple. You're welcome to open a PR if you'd like it added.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@ballwood@bnoordhuis@vbfox@maclover7@dlangerenken@rvagg@imatlopez@alsotang
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

no_proxy support - #1176

Closed
ballwood wants to merge 3 commits into
nodejs:masterfrom
ballwood:master
Closed

no_proxy support#1176
ballwood wants to merge 3 commits into
nodejs:masterfrom
ballwood:master

Conversation

@ballwood

@ballwoodballwood commented Apr 14, 2017

Copy link
Copy Markdown

Ported code across from the request library, which I know to work correctly - https://github.com/request/request.

Unsure of where to put the unit/integration tests, if someone could guide me I'll write them. There were 11 tests failing when running npm test when I forked and the same amount after I made the code change.

@ballwoodballwood mentioned this pull request Apr 14, 2017
@bnoordhuis

Copy link
Copy Markdown
Member

Sorry for the delay. This needs some tests but I'm kind of surprised at how complex it still is. Is npm_config_no_proxy a thing?

@vbfox

Copy link
Copy Markdown

I'd really like to see that PR get merged.

For npm config exporting npm_config_no_proxy there is a PR open but I don't know if it'll ever get merged: npm/npm#14248 (Currently it can't be specified in npm config)

@ballwood

Copy link
Copy Markdown
Author

Hi, sorry I thought I replied to this last week, must have not pressed send. Yeah I noticed that PR was open and included it to future proof it but their work on it seems to have gone by the wayside.

I can have a look at the tests I'm just trying to understand them at the moment and work out something that works correctly.

@ballwood

Copy link
Copy Markdown
Author

@bnoordhuis wrote some small unit tests as I got some free time

@bnoordhuisbnoordhuis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments. Thanks for following up.

Comment threadlib/install.js

function formatHostname (hostname) {
// canonicalize the hostname, so that 'oogle.com' won't match 'google.com'
return hostname.replace(/^\.*/, '.').toLowerCase()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This transforms google.com into .google.com. I infer that is intentional but can you update the comment to make that more explicit? This behavior also seems only lightly tested.

Aside: can you capitalize and punctuate comments?

Comment threadlib/install.js
var zoneParts = zone.split(':', 2)
var zoneHost = formatHostname(zoneParts[0])
var zonePort = zoneParts[1]
var hasPort = zone.indexOf(':') > -1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not wrong per se but can also be derived without re-scanning from zonePort !== undefined or zoneParts.length === 2.

Comment threadlib/install.js
var isMatchedAt = hostname.indexOf(noProxyZone.hostname)
var hostnameMatched = (
isMatchedAt > -1 &&
(isMatchedAt === hostname.length - noProxyZone.hostname.length)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct me if I'm wrong but isn't this just a circuitous way of saying hostname === noProxyZone.hostname?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it's a suffix match

Comment threadlib/install.js
var noProxyUrl = gyp.opts.no_proxy
|| env.no_proxy
|| env.NO_PROXY
|| env.npm_config_no_proxy

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably go; the npm pull request wasn't merged.

Comment threadlib/install.js
|| env.npm_config_no_proxy
|| ''

if (noProxyUrl === '*' || (noProxyUrl !== '' && uriInNoProxy(url, noProxyUrl))) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Long line; can you keep it under 80 columns? The parens around the && aren't necessary.

Comment threadlib/install.js
log.verbose('download', 'using proxy url: "%s"', proxyUrl)
requestOpts.proxy = proxyUrl
} else {
log.warn('download', 'ignoring invalid "proxy" config setting: "%s"', proxyUrl)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto, long line.

@@ -0,0 +1,63 @@
var test = require('tape')

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add 'use strict';?


t.equal(req.opts.proxy, gyp.opts.proxy)
t.equal(req.opts.uri, url)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you drop the blank line here and below?

@bnoordhuisbnoordhuis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wups sorry, meant to request changes. Only had one cup of coffee so far.

@maclover7

Copy link
Copy Markdown
Contributor

ping @ballwood

@dlangerenken

Copy link
Copy Markdown

@bnoordhuis is someone taking care of this PR? Seems like @ballwood is no longer working on it. It would be really nice to have in master.

@rvagg

Copy link
Copy Markdown
Member

This looks pretty good and we have HTTPS_PROXY support about to land too but it's going to need a new champion. If someone wants this in, please take this code, clean it up as per the suggestions in here and make it work against master. Tests would be good too if you can come up with some that are meaningful.

@rvaggrvagg closed this Jun 21, 2019
@imatlopez

imatlopez commented Nov 27, 2019

Copy link
Copy Markdown
Contributor

@rvagg I was looking at doing just this given this is quite a pain point inside our corporate network, however is of this approach to make it independent of requests? Would it be simpler to set the environment variables from the cli flags as needed and let requests do its job?

@rvagg

Copy link
Copy Markdown
Member

@imatlopez you mean request not requests right? Apparently obeys no_proxy these days but I don't know what we would need to do here, if anything, for that support to flow through. If it's obeying environment variables then I'd assume we don't have to do anything on our end, just set the env var before calling. If you need gyp.opts to be able to carry it then I guess some additional work could go in here and hopefully that'd be simple. You're welcome to open a PR if you'd like it added.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@ballwood@bnoordhuis@vbfox@maclover7@dlangerenken@rvagg@imatlopez@alsotang
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

no_proxy support - #1176

Closed
ballwood wants to merge 3 commits into
nodejs:masterfrom
ballwood:master
Closed

no_proxy support#1176
ballwood wants to merge 3 commits into
nodejs:masterfrom
ballwood:master

Conversation

@ballwood

@ballwoodballwood commented Apr 14, 2017

Copy link
Copy Markdown

Ported code across from the request library, which I know to work correctly - https://github.com/request/request.

Unsure of where to put the unit/integration tests, if someone could guide me I'll write them. There were 11 tests failing when running npm test when I forked and the same amount after I made the code change.

@ballwoodballwood mentioned this pull request Apr 14, 2017
@bnoordhuis

Copy link
Copy Markdown
Member

Sorry for the delay. This needs some tests but I'm kind of surprised at how complex it still is. Is npm_config_no_proxy a thing?

@vbfox

Copy link
Copy Markdown

I'd really like to see that PR get merged.

For npm config exporting npm_config_no_proxy there is a PR open but I don't know if it'll ever get merged: npm/npm#14248 (Currently it can't be specified in npm config)

@ballwood

Copy link
Copy Markdown
Author

Hi, sorry I thought I replied to this last week, must have not pressed send. Yeah I noticed that PR was open and included it to future proof it but their work on it seems to have gone by the wayside.

I can have a look at the tests I'm just trying to understand them at the moment and work out something that works correctly.

@ballwood

Copy link
Copy Markdown
Author

@bnoordhuis wrote some small unit tests as I got some free time

@bnoordhuisbnoordhuis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments. Thanks for following up.

Comment threadlib/install.js

function formatHostname (hostname) {
// canonicalize the hostname, so that 'oogle.com' won't match 'google.com'
return hostname.replace(/^\.*/, '.').toLowerCase()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This transforms google.com into .google.com. I infer that is intentional but can you update the comment to make that more explicit? This behavior also seems only lightly tested.

Aside: can you capitalize and punctuate comments?

Comment threadlib/install.js
var zoneParts = zone.split(':', 2)
var zoneHost = formatHostname(zoneParts[0])
var zonePort = zoneParts[1]
var hasPort = zone.indexOf(':') > -1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not wrong per se but can also be derived without re-scanning from zonePort !== undefined or zoneParts.length === 2.

Comment threadlib/install.js
var isMatchedAt = hostname.indexOf(noProxyZone.hostname)
var hostnameMatched = (
isMatchedAt > -1 &&
(isMatchedAt === hostname.length - noProxyZone.hostname.length)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct me if I'm wrong but isn't this just a circuitous way of saying hostname === noProxyZone.hostname?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it's a suffix match

Comment threadlib/install.js
var noProxyUrl = gyp.opts.no_proxy
|| env.no_proxy
|| env.NO_PROXY
|| env.npm_config_no_proxy

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably go; the npm pull request wasn't merged.

Comment threadlib/install.js
|| env.npm_config_no_proxy
|| ''

if (noProxyUrl === '*' || (noProxyUrl !== '' && uriInNoProxy(url, noProxyUrl))) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Long line; can you keep it under 80 columns? The parens around the && aren't necessary.

Comment threadlib/install.js
log.verbose('download', 'using proxy url: "%s"', proxyUrl)
requestOpts.proxy = proxyUrl
} else {
log.warn('download', 'ignoring invalid "proxy" config setting: "%s"', proxyUrl)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto, long line.

@@ -0,0 +1,63 @@
var test = require('tape')

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add 'use strict';?


t.equal(req.opts.proxy, gyp.opts.proxy)
t.equal(req.opts.uri, url)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you drop the blank line here and below?

@bnoordhuisbnoordhuis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wups sorry, meant to request changes. Only had one cup of coffee so far.

@maclover7

Copy link
Copy Markdown
Contributor

ping @ballwood

@dlangerenken

Copy link
Copy Markdown

@bnoordhuis is someone taking care of this PR? Seems like @ballwood is no longer working on it. It would be really nice to have in master.

@rvagg

Copy link
Copy Markdown
Member

This looks pretty good and we have HTTPS_PROXY support about to land too but it's going to need a new champion. If someone wants this in, please take this code, clean it up as per the suggestions in here and make it work against master. Tests would be good too if you can come up with some that are meaningful.

@rvaggrvagg closed this Jun 21, 2019
@imatlopez

imatlopez commented Nov 27, 2019

Copy link
Copy Markdown
Contributor

@rvagg I was looking at doing just this given this is quite a pain point inside our corporate network, however is of this approach to make it independent of requests? Would it be simpler to set the environment variables from the cli flags as needed and let requests do its job?

@rvagg

Copy link
Copy Markdown
Member

@imatlopez you mean request not requests right? Apparently obeys no_proxy these days but I don't know what we would need to do here, if anything, for that support to flow through. If it's obeying environment variables then I'd assume we don't have to do anything on our end, just set the env var before calling. If you need gyp.opts to be able to carry it then I guess some additional work could go in here and hopefully that'd be simple. You're welcome to open a PR if you'd like it added.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@ballwood@bnoordhuis@vbfox@maclover7@dlangerenken@rvagg@imatlopez@alsotang
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

no_proxy support - #1176

Closed
ballwood wants to merge 3 commits into
nodejs:masterfrom
ballwood:master
Closed

no_proxy support#1176
ballwood wants to merge 3 commits into
nodejs:masterfrom
ballwood:master

Conversation

@ballwood

@ballwoodballwood commented Apr 14, 2017

Copy link
Copy Markdown

Ported code across from the request library, which I know to work correctly - https://github.com/request/request.

Unsure of where to put the unit/integration tests, if someone could guide me I'll write them. There were 11 tests failing when running npm test when I forked and the same amount after I made the code change.

@ballwoodballwood mentioned this pull request Apr 14, 2017
@bnoordhuis

Copy link
Copy Markdown
Member

Sorry for the delay. This needs some tests but I'm kind of surprised at how complex it still is. Is npm_config_no_proxy a thing?

@vbfox

Copy link
Copy Markdown

I'd really like to see that PR get merged.

For npm config exporting npm_config_no_proxy there is a PR open but I don't know if it'll ever get merged: npm/npm#14248 (Currently it can't be specified in npm config)

@ballwood

Copy link
Copy Markdown
Author

Hi, sorry I thought I replied to this last week, must have not pressed send. Yeah I noticed that PR was open and included it to future proof it but their work on it seems to have gone by the wayside.

I can have a look at the tests I'm just trying to understand them at the moment and work out something that works correctly.

@ballwood

Copy link
Copy Markdown
Author

@bnoordhuis wrote some small unit tests as I got some free time

@bnoordhuisbnoordhuis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments. Thanks for following up.

Comment threadlib/install.js

function formatHostname (hostname) {
// canonicalize the hostname, so that 'oogle.com' won't match 'google.com'
return hostname.replace(/^\.*/, '.').toLowerCase()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This transforms google.com into .google.com. I infer that is intentional but can you update the comment to make that more explicit? This behavior also seems only lightly tested.

Aside: can you capitalize and punctuate comments?

Comment threadlib/install.js
var zoneParts = zone.split(':', 2)
var zoneHost = formatHostname(zoneParts[0])
var zonePort = zoneParts[1]
var hasPort = zone.indexOf(':') > -1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not wrong per se but can also be derived without re-scanning from zonePort !== undefined or zoneParts.length === 2.

Comment threadlib/install.js
var isMatchedAt = hostname.indexOf(noProxyZone.hostname)
var hostnameMatched = (
isMatchedAt > -1 &&
(isMatchedAt === hostname.length - noProxyZone.hostname.length)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct me if I'm wrong but isn't this just a circuitous way of saying hostname === noProxyZone.hostname?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it's a suffix match

Comment threadlib/install.js
var noProxyUrl = gyp.opts.no_proxy
|| env.no_proxy
|| env.NO_PROXY
|| env.npm_config_no_proxy

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably go; the npm pull request wasn't merged.

Comment threadlib/install.js
|| env.npm_config_no_proxy
|| ''

if (noProxyUrl === '*' || (noProxyUrl !== '' && uriInNoProxy(url, noProxyUrl))) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Long line; can you keep it under 80 columns? The parens around the && aren't necessary.

Comment threadlib/install.js
log.verbose('download', 'using proxy url: "%s"', proxyUrl)
requestOpts.proxy = proxyUrl
} else {
log.warn('download', 'ignoring invalid "proxy" config setting: "%s"', proxyUrl)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto, long line.

@@ -0,0 +1,63 @@
var test = require('tape')

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add 'use strict';?


t.equal(req.opts.proxy, gyp.opts.proxy)
t.equal(req.opts.uri, url)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you drop the blank line here and below?

@bnoordhuisbnoordhuis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wups sorry, meant to request changes. Only had one cup of coffee so far.

@maclover7

Copy link
Copy Markdown
Contributor

ping @ballwood

@dlangerenken

Copy link
Copy Markdown

@bnoordhuis is someone taking care of this PR? Seems like @ballwood is no longer working on it. It would be really nice to have in master.

@rvagg

Copy link
Copy Markdown
Member

This looks pretty good and we have HTTPS_PROXY support about to land too but it's going to need a new champion. If someone wants this in, please take this code, clean it up as per the suggestions in here and make it work against master. Tests would be good too if you can come up with some that are meaningful.

@rvaggrvagg closed this Jun 21, 2019
@imatlopez

imatlopez commented Nov 27, 2019

Copy link
Copy Markdown
Contributor

@rvagg I was looking at doing just this given this is quite a pain point inside our corporate network, however is of this approach to make it independent of requests? Would it be simpler to set the environment variables from the cli flags as needed and let requests do its job?

@rvagg

Copy link
Copy Markdown
Member

@imatlopez you mean request not requests right? Apparently obeys no_proxy these days but I don't know what we would need to do here, if anything, for that support to flow through. If it's obeying environment variables then I'd assume we don't have to do anything on our end, just set the env var before calling. If you need gyp.opts to be able to carry it then I guess some additional work could go in here and hopefully that'd be simple. You're welcome to open a PR if you'd like it added.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@ballwood@bnoordhuis@vbfox@maclover7@dlangerenken@rvagg@imatlopez@alsotang