module: prefer async/await in https imports by benjamingr · Pull Request #41950 · nodejs/node

benjamingr · 2022-02-12T18:50:10Z

cc @bmeck @nodejs/modules

This PR refactors the fetch_module fetchWithRedirects function to use the new modern facilities for streams:

compose instead of pipe for simplified and "more easily correct" error handling on the body stream cc @ronag .
async/await over new Promise.
toArray() on the response body instead of manually.

Note this can probably be further simplified by always caching the promise (and never the value) and by moving the "memoize" functionality to a helper).

Fixes: #41950

nodejs-github-bot · 2022-02-12T18:50:14Z

Review requested:

@nodejs/modules

GeoffreyBooth · 2022-02-13T00:04:00Z

cc @JakobJingleheimer

GeoffreyBooth

Some minor style notes, I haven’t reviewed in depth.

benjamingr · 2022-02-13T10:26:33Z

@GeoffreyBooth fixed + added http 404 handling

bmeck · 2022-02-13T13:52:11Z

Note this can probably be further simplified by always caching the promise (and never the value) and by moving the "memoize" functionality to a helper).

This is actually a bit complicated since import.meta.url needs the value synchronously for the response location after all redirects so it actually relies on storing the value for now.

benjamingr · 2022-02-13T14:51:59Z

This is actually a bit complicated since import.meta.url needs the value synchronously for the response location after all redirects so it actually relies on storing the value for now.

Ah, mind if I add a comment to the code explaining that around the cacheForGET code?

bmeck · 2022-02-13T15:26:44Z

Ah, mind if I add a comment to the code explaining that around the cacheForGET code?

I'm all for comments

benjamingr · 2022-02-13T16:25:02Z

Added both comments.

JakobJingleheimer

Oow, fancy. Much cleaner than .on(…) 🙌

JakobJingleheimer · 2022-02-15T18:55:08Z

Suggested change
*Itcacheseitherthepromiseorthecahceentrysinceimport.meta.urlneeds
*Itcacheseitherthepromiseorthecacheentrysinceimport.meta.urlneeds

benjamingr · 2022-02-16T07:53:02Z

@bmeck @nodejs/modules this could use some reviews :)

guybedford

Can we get a 404 test though to land?

benjamingr · 2022-02-16T18:05:30Z

Added test :)

nodejs-github-bot · 2022-02-16T18:13:36Z

CI: https://ci.nodejs.org/job/node-test-pull-request/42625/

nodejs-github-bot · 2022-02-18T11:26:22Z

CI: https://ci.nodejs.org/job/node-test-pull-request/42649/

nodejs-github-bot · 2022-02-23T10:39:31Z

CI: https://ci.nodejs.org/job/node-test-pull-request/42731/

nodejs-github-bot · 2022-02-23T11:20:41Z

CI: https://ci.nodejs.org/job/node-test-pull-request/42735/

nodejs-github-bot · 2022-02-23T16:36:26Z

CI: https://ci.nodejs.org/job/node-test-pull-request/42742/

nodejs-github-bot · 2022-02-23T18:10:07Z

CI: https://ci.nodejs.org/job/node-test-pull-request/42748/

nodejs-github-bot · 2022-02-23T20:24:29Z

Commit Queue failed

- Loading data for nodejs/node/pull/41950
✔ Done loading data for nodejs/node/pull/41950
----------------------------------- PR info ------------------------------------
Title module: prefer async/await in https imports (#41950)
Author Benjamin Gruenbaum (@benjamingr)
Branch benjamingr:use-async-await-in-fetch-module -> nodejs:master
Labels esm, needs-ci
Commits 1
- module: prefer async/await in https imports
Committers 1
- Benjamin Gruenbaum PR-URL: https://github.com/nodejs/node/pull/41950
Fixes: https://github.com/nodejs/node/issues/41950
Reviewed-By: Guy Bedford Reviewed-By: James M Snell ------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/41950
Fixes: https://github.com/nodejs/node/issues/41950
Reviewed-By: Guy Bedford Reviewed-By: James M Snell --------------------------------------------------------------------------------
⚠ Commits were pushed since the last review:
⚠ - module: prefer async/await in https imports
ℹ This PR was created on Sat, 12 Feb 2022 18:50:10 GMT
✔ Approvals: 2
✔ - Guy Bedford (@guybedford): https://github.com/nodejs/node/pull/41950#pullrequestreview-884778590
✔ - James M Snell (@jasnell) (TSC): https://github.com/nodejs/node/pull/41950#pullrequestreview-889949951
✔ Last GitHub CI successful
ℹ Last Full PR CI on 2022-02-23T18:10:07Z: https://ci.nodejs.org/job/node-test-pull-request/42748/
- Querying data for job/node-test-pull-request/42748/
✔ Last Jenkins CI successful
--------------------------------------------------------------------------------
✔ Aborted `git node land` session in /home/runner/work/node/node/.ncu

https://github.com/nodejs/node/actions/runs/1889467631

benjamingr · 2022-02-23T22:00:03Z

Landed in 7efef74 🎉

PR-URL: #41950Fixes: #41950 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

PR-URL: nodejs#41950Fixes: nodejs#41950 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

juanarbol · 2022-05-30T21:22:21Z

This is failing in the v16.x branch; let me work in a backport for this:

/node--inspect--experimental-network-imports--dns-result-order=ipv4firsttest/es-module/test-http-imports.mjs Debugger listeningonws://127.0.0.1:9229/924045ba-ef66-4869-84bf-085849cbf26b Forhelp,see: https://nodejs.org/en/docs/inspector 
node:internal/errors:465ErrorCaptureStackTrace(err);^TypeError[ERR_INVALID_ARG_TYPE]: The"list"argumentmustbeaninstanceofArray.ReceivedaninstanceofBufferatconcat(node:buffer:537:3) at node:internal/modules/esm/fetch_module:165:24atprocessTicksAndRejections(node:internal/process/task_queues:96:5)atasync node:internal/modules/esm/fetch_module:171:7atasyncdefaultLoad(node:internal/modules/esm/load:21:14)atasyncESMLoader.load(node:internal/modules/esm/loader:407:20)atasyncESMLoader.moduleProvider(node:internal/modules/esm/loader:326:11)atasynclink(node:internal/modules/esm/module_job:70:21){ code: 'ERR_INVALID_ARG_TYPE'}

Looks like the toArray() method is returning the Buffer not the array, see: https://github.com/nodejs/node/pull/41553/files#diff-00d8d040a3fe90e3052af39cb55dc9bcac70539d5d17791449cbfe7e3b55fb90R1903-R1904

PR-URL: #41950Fixes: #41950 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

PR-URL: nodejs/node#41950Fixes: nodejs/node#41950 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

nodejs-github-bot added esm Issues and PRs related to the ECMAScript Modules implementation. needs-ci PRs that need a full CI run. labels Feb 12, 2022

benjamingr mentioned this pull request Feb 12, 2022
HTTP imports not found error #41948
Closed

GeoffreyBooth requested a review from bmeck February 13, 2022 00:03

GeoffreyBooth reviewed Feb 13, 2022
View reviewed changes

Comment threadlib/internal/modules/esm/fetch_module.js Outdated
Comment threadlib/internal/modules/esm/fetch_module.js Outdated
Comment threadlib/internal/modules/esm/fetch_module.js Outdated

bmeck reviewed Feb 13, 2022
View reviewed changes

Comment threadlib/internal/modules/esm/fetch_module.js Outdated

bmeck reviewed Feb 13, 2022
View reviewed changes

Comment threadlib/internal/modules/esm/fetch_module.js Outdated

bmeck reviewed Feb 13, 2022
View reviewed changes

Comment threadlib/internal/modules/esm/fetch_module.js Outdated

bmeck reviewed Feb 13, 2022
View reviewed changes

Comment threadlib/internal/modules/esm/fetch_module.js Outdated

JakobJingleheimer reviewed Feb 15, 2022
View reviewed changes

guybedford approved these changes Feb 16, 2022
View reviewed changes

benjamingr force-pushed the use-async-await-in-fetch-module branch 2 times, most recently from 67713f1 to ace0812Compare February 16, 2022 18:05

benjamingr added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 16, 2022

github-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 16, 2022

This was referenced Feb 17, 2022
CI Reliability 2022-02-17 nodejs/reliability#205
Open
CI Reliability 2022-02-18 nodejs/reliability#206
Open

github-actionsBot mentioned this pull request Feb 19, 2022
CI Reliability 2022-02-19 nodejs/reliability#207
Open
26 tasks

benjamingr added the commit-queue Add this label to land a pull request using GitHub Actions. label Feb 19, 2022

benjamingr added request-ci Add this label to start a Jenkins CI on a PR. and removed commit-queue-failed An error occurred while landing this pull request using GitHub Actions. labels Feb 23, 2022

github-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 23, 2022

module: prefer async/await in https imports
b575c6d

benjamingr force-pushed the use-async-await-in-fetch-module branch from b3ef5cb to b575c6dCompare February 23, 2022 11:18

benjamingr added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 23, 2022

github-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 23, 2022

benjamingr added the commit-queue Add this label to land a pull request using GitHub Actions. label Feb 23, 2022

nodejs-github-bot added commit-queue-failed An error occurred while landing this pull request using GitHub Actions. and removed commit-queue Add this label to land a pull request using GitHub Actions. labels Feb 23, 2022

benjamingr closed this Feb 23, 2022

benjamingr added a commit that referenced this pull request Feb 23, 2022
module: prefer async/await in https imports
7efef74
PR-URL: #41950Fixes: #41950 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

sxa pushed a commit to sxa/node that referenced this pull request Mar 7, 2022
module: prefer async/await in https imports
96510b3
PR-URL: nodejs#41950Fixes: nodejs#41950 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

sxa mentioned this pull request Mar 8, 2022
v17.7.0 release proposal #42254
Merged

xtx1130 pushed a commit to xtx1130/node that referenced this pull request Apr 25, 2022
module: prefer async/await in https imports
47d4289
PR-URL: nodejs#41950Fixes: nodejs#41950 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

juanarbol added the backport-blocked-v16.x label May 30, 2022

juanarbol added backport-requested-v16.x and removed backport-blocked-v16.x labels May 30, 2022

targos removed the backport-requested-v16.x label Jul 19, 2022

targos pushed a commit that referenced this pull request Jul 19, 2022
module: prefer async/await in https imports
cb18e8e
PR-URL: #41950Fixes: #41950 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

targos pushed a commit that referenced this pull request Jul 31, 2022
module: prefer async/await in https imports
abda05a
PR-URL: #41950Fixes: #41950 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

targos mentioned this pull request Aug 3, 2022
v16.17.0 release proposal #44098
Merged

guangwong pushed a commit to noslate-project/node that referenced this pull request Oct 10, 2022
module: prefer async/await in https imports
b48c310
PR-URL: nodejs/node#41950Fixes: nodejs/node#41950 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

	*Itcacheseitherthepromiseorthecahceentrysinceimport.meta.urlneeds
	*Itcacheseitherthepromiseorthecacheentrysinceimport.meta.urlneeds

Uh oh!

Uh oh!

Conversation

benjamingr commented Feb 12, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nodejs-github-bot commented Feb 12, 2022

Uh oh!

GeoffreyBooth commented Feb 13, 2022

Uh oh!

GeoffreyBooth left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

benjamingr commented Feb 13, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bmeck commented Feb 13, 2022

Uh oh!

benjamingr commented Feb 13, 2022

Uh oh!

bmeck commented Feb 13, 2022

Uh oh!

benjamingr commented Feb 13, 2022

Uh oh!

JakobJingleheimer left a comment

Choose a reason for hiding this comment

Uh oh!

JakobJingleheimerFeb 15, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

benjamingr commented Feb 16, 2022

Uh oh!

guybedford left a comment

Choose a reason for hiding this comment

Uh oh!

benjamingr commented Feb 16, 2022

Uh oh!

nodejs-github-bot commented Feb 16, 2022

Uh oh!

nodejs-github-bot commented Feb 18, 2022

Uh oh!

nodejs-github-bot commented Feb 23, 2022

Uh oh!

nodejs-github-bot commented Feb 23, 2022

Uh oh!

nodejs-github-bot commented Feb 23, 2022

Uh oh!

nodejs-github-bot commented Feb 23, 2022

Uh oh!

nodejs-github-bot commented Feb 23, 2022

Uh oh!

benjamingr commented Feb 23, 2022

Uh oh!

juanarbol commented May 30, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

benjamingr commented Feb 12, 2022•
edited
Loading