fix: extract tarball to temp directory on Windows

[dsanders11]

Checklist

• npm install && npm test passes
• tests are included
• documentation is changed or added
• commit message follows commit guidelines

Description of change

Fixes#2482.
Fixes#2484.
Fixes#2584.
Fixes#2683.
Fixes#2751.
Fixes#2832.

NOTE: This PR contains a temporary initial commit which reverts a PR which causes CI workflow failures on main, so that the tests for this PR can run clean. It should be removed before merging this PR. #2837 is currently trying to fix that issue.

node-tar unlinks file paths before writing to them on Windows, which causes errors when there are parallel installs going on which are extracting the headers tarball to devDir. This currently leads to random corruption of the cache at devDir with missing files and hanging files with <filename>.DELETE.<random> names. See more in-depth discussion about the underlying issue here.

This PR fixes the situation by:

• Fixing usage of node-tar to watch for errors during extraction
• Extracting the headers tarball on Windows to a temp directory and then copies the files over
• Using the exponential-backoff package in case of collision (EBUSY) when copying the files to devDir
• Bumps installVersion to force clearing of existing cached versions which might be corrupt on Windows

This is set up in commits which were pushed one at a time to this PR so that the CI workflows for each commit can be seen, confirming that the added tests fail until the final fix is applied.

Unfortunately the new tests, despite being simple, are a bit flaky, due to network requests to https://nodejs.org/download/release/. The tests also take significant longer (greater than 2x) on Windows with Node.js 18.x and 20.x - it's not clear why, but since it repros consistently with those versions, it seems likely to be some upstream regression in Node.js, possibly something to do with the fs module.

[MarshallOfSound]

Is there a reason we wouldn't want to rename / move here instead of copy? This could still race condition if it's not atomic?

[dsanders11]

Is there a reason we wouldn't want to rename / move here instead of copy?

Since there are parallel installs happening, there's no way to atomically rename to devDir. That directory is created (if it doesn't already exist) at the start of the install process to confirm permissions, so a direct rename of tarExtractDir to devDir would fail. Any rename process would need to first rename devDir, which opens a race condition where devDir disappears momentarily during an ongoing build, causing a failure. That was my original approach for fixing this issue, but I switched to the current implementation to close that race condition.

This could still race condition if it's not atomic?

The copyFile approach stays similar to the existing implementation, with node-tar overwriting the existing files when there are parallel installs going on. The main difference is that on Windows node-tar always unlinks the destination file path before writing to it to avoid certain conditions (same path multiple times in an archive) which don't exist for this usage.

I've stress tested this change (on CI and locally with a real package) with 35 parallel installs and didn't trip over any race conditions in practice.

[StefanStojanovic]

LGTM. Since there are already 2 PRs addressing GitHub Action issues, we should make sure 6bb7cd6 is removed from here before landing it. Ideally, we'd first land a GitHub Actions fix, then rebase this PR, remove the mentioned commit and push changes.

[StefanStojanovic]

@rvagg@cclauss I'm changing tests from tap to mocha in #2851. Since this PR adds some tests and it's already approved, what are your thoughts on landing it (without 6bb7cd6) so I can change those tests in my PR as well?

[cclauss]

I have no objection to these changes but I am not a Windows user so I lack the skills to be a reviewer.

[StefanStojanovic]

Hey, @dsanders11 can you prepare your branch for merging by removing 6bb7cd6 and potentially addressing #2846 (comment) and squashing some commits together (especially the ones addressing PR comments)?

[dsanders11]

@StefanStojanovic, done! Removed 6bb7cd6 and squashed it back down to the original three commits. For CI reference, 2bb914b was the last commit before removing 6bb7cd6, and it went green.

[dsanders11]

Checklist

• npm install && npm test passes
• tests are included
• documentation is changed or added
• commit message follows commit guidelines

Description of change

Fixes#2482.
Fixes#2484.
Fixes#2584.
Fixes#2683.
Fixes#2751.
Fixes#2832.

NOTE: This PR contains a temporary initial commit which reverts a PR which causes CI workflow failures on main, so that the tests for this PR can run clean. It should be removed before merging this PR. #2837 is currently trying to fix that issue.

node-tar unlinks file paths before writing to them on Windows, which causes errors when there are parallel installs going on which are extracting the headers tarball to devDir. This currently leads to random corruption of the cache at devDir with missing files and hanging files with <filename>.DELETE.<random> names. See more in-depth discussion about the underlying issue here.

This PR fixes the situation by:

• Fixing usage of node-tar to watch for errors during extraction
• Extracting the headers tarball on Windows to a temp directory and then copies the files over
• Using the exponential-backoff package in case of collision (EBUSY) when copying the files to devDir
• Bumps installVersion to force clearing of existing cached versions which might be corrupt on Windows

This is set up in commits which were pushed one at a time to this PR so that the CI workflows for each commit can be seen, confirming that the added tests fail until the final fix is applied.

Unfortunately the new tests, despite being simple, are a bit flaky, due to network requests to https://nodejs.org/download/release/. The tests also take significant longer (greater than 2x) on Windows with Node.js 18.x and 20.x - it's not clear why, but since it repros consistently with those versions, it seems likely to be some upstream regression in Node.js, possibly something to do with the fs module.

[MarshallOfSound]

Is there a reason we wouldn't want to rename / move here instead of copy? This could still race condition if it's not atomic?

[dsanders11]

Is there a reason we wouldn't want to rename / move here instead of copy?

Since there are parallel installs happening, there's no way to atomically rename to devDir. That directory is created (if it doesn't already exist) at the start of the install process to confirm permissions, so a direct rename of tarExtractDir to devDir would fail. Any rename process would need to first rename devDir, which opens a race condition where devDir disappears momentarily during an ongoing build, causing a failure. That was my original approach for fixing this issue, but I switched to the current implementation to close that race condition.

This could still race condition if it's not atomic?

The copyFile approach stays similar to the existing implementation, with node-tar overwriting the existing files when there are parallel installs going on. The main difference is that on Windows node-tar always unlinks the destination file path before writing to it to avoid certain conditions (same path multiple times in an archive) which don't exist for this usage.

I've stress tested this change (on CI and locally with a real package) with 35 parallel installs and didn't trip over any race conditions in practice.

[StefanStojanovic]

LGTM. Since there are already 2 PRs addressing GitHub Action issues, we should make sure 6bb7cd6 is removed from here before landing it. Ideally, we'd first land a GitHub Actions fix, then rebase this PR, remove the mentioned commit and push changes.

[StefanStojanovic]

@rvagg@cclauss I'm changing tests from tap to mocha in #2851. Since this PR adds some tests and it's already approved, what are your thoughts on landing it (without 6bb7cd6) so I can change those tests in my PR as well?

[cclauss]

I have no objection to these changes but I am not a Windows user so I lack the skills to be a reviewer.

[StefanStojanovic]

Hey, @dsanders11 can you prepare your branch for merging by removing 6bb7cd6 and potentially addressing #2846 (comment) and squashing some commits together (especially the ones addressing PR comments)?

[dsanders11]

@StefanStojanovic, done! Removed 6bb7cd6 and squashed it back down to the original three commits. For CI reference, 2bb914b was the last commit before removing 6bb7cd6, and it went green.

[dsanders11]

Checklist

• npm install && npm test passes
• tests are included
• documentation is changed or added
• commit message follows commit guidelines

Description of change

Fixes#2482.
Fixes#2484.
Fixes#2584.
Fixes#2683.
Fixes#2751.
Fixes#2832.

NOTE: This PR contains a temporary initial commit which reverts a PR which causes CI workflow failures on main, so that the tests for this PR can run clean. It should be removed before merging this PR. #2837 is currently trying to fix that issue.

node-tar unlinks file paths before writing to them on Windows, which causes errors when there are parallel installs going on which are extracting the headers tarball to devDir. This currently leads to random corruption of the cache at devDir with missing files and hanging files with <filename>.DELETE.<random> names. See more in-depth discussion about the underlying issue here.

This PR fixes the situation by:

• Fixing usage of node-tar to watch for errors during extraction
• Extracting the headers tarball on Windows to a temp directory and then copies the files over
• Using the exponential-backoff package in case of collision (EBUSY) when copying the files to devDir
• Bumps installVersion to force clearing of existing cached versions which might be corrupt on Windows

This is set up in commits which were pushed one at a time to this PR so that the CI workflows for each commit can be seen, confirming that the added tests fail until the final fix is applied.

Unfortunately the new tests, despite being simple, are a bit flaky, due to network requests to https://nodejs.org/download/release/. The tests also take significant longer (greater than 2x) on Windows with Node.js 18.x and 20.x - it's not clear why, but since it repros consistently with those versions, it seems likely to be some upstream regression in Node.js, possibly something to do with the fs module.

[MarshallOfSound]

Is there a reason we wouldn't want to rename / move here instead of copy? This could still race condition if it's not atomic?

[dsanders11]

Is there a reason we wouldn't want to rename / move here instead of copy?

Since there are parallel installs happening, there's no way to atomically rename to devDir. That directory is created (if it doesn't already exist) at the start of the install process to confirm permissions, so a direct rename of tarExtractDir to devDir would fail. Any rename process would need to first rename devDir, which opens a race condition where devDir disappears momentarily during an ongoing build, causing a failure. That was my original approach for fixing this issue, but I switched to the current implementation to close that race condition.

This could still race condition if it's not atomic?

The copyFile approach stays similar to the existing implementation, with node-tar overwriting the existing files when there are parallel installs going on. The main difference is that on Windows node-tar always unlinks the destination file path before writing to it to avoid certain conditions (same path multiple times in an archive) which don't exist for this usage.

I've stress tested this change (on CI and locally with a real package) with 35 parallel installs and didn't trip over any race conditions in practice.

[StefanStojanovic]

LGTM. Since there are already 2 PRs addressing GitHub Action issues, we should make sure 6bb7cd6 is removed from here before landing it. Ideally, we'd first land a GitHub Actions fix, then rebase this PR, remove the mentioned commit and push changes.

[StefanStojanovic]

@rvagg@cclauss I'm changing tests from tap to mocha in #2851. Since this PR adds some tests and it's already approved, what are your thoughts on landing it (without 6bb7cd6) so I can change those tests in my PR as well?

[cclauss]

I have no objection to these changes but I am not a Windows user so I lack the skills to be a reviewer.

[StefanStojanovic]

Hey, @dsanders11 can you prepare your branch for merging by removing 6bb7cd6 and potentially addressing #2846 (comment) and squashing some commits together (especially the ones addressing PR comments)?

[dsanders11]

@StefanStojanovic, done! Removed 6bb7cd6 and squashed it back down to the original three commits. For CI reference, 2bb914b was the last commit before removing 6bb7cd6, and it went green.

[dsanders11]

Checklist

• npm install && npm test passes
• tests are included
• documentation is changed or added
• commit message follows commit guidelines

Description of change

Fixes#2482.
Fixes#2484.
Fixes#2584.
Fixes#2683.
Fixes#2751.
Fixes#2832.

NOTE: This PR contains a temporary initial commit which reverts a PR which causes CI workflow failures on main, so that the tests for this PR can run clean. It should be removed before merging this PR. #2837 is currently trying to fix that issue.

node-tar unlinks file paths before writing to them on Windows, which causes errors when there are parallel installs going on which are extracting the headers tarball to devDir. This currently leads to random corruption of the cache at devDir with missing files and hanging files with <filename>.DELETE.<random> names. See more in-depth discussion about the underlying issue here.

This PR fixes the situation by:

• Fixing usage of node-tar to watch for errors during extraction
• Extracting the headers tarball on Windows to a temp directory and then copies the files over
• Using the exponential-backoff package in case of collision (EBUSY) when copying the files to devDir
• Bumps installVersion to force clearing of existing cached versions which might be corrupt on Windows

This is set up in commits which were pushed one at a time to this PR so that the CI workflows for each commit can be seen, confirming that the added tests fail until the final fix is applied.

Unfortunately the new tests, despite being simple, are a bit flaky, due to network requests to https://nodejs.org/download/release/. The tests also take significant longer (greater than 2x) on Windows with Node.js 18.x and 20.x - it's not clear why, but since it repros consistently with those versions, it seems likely to be some upstream regression in Node.js, possibly something to do with the fs module.

[MarshallOfSound]

Is there a reason we wouldn't want to rename / move here instead of copy? This could still race condition if it's not atomic?

[dsanders11]

Is there a reason we wouldn't want to rename / move here instead of copy?

Since there are parallel installs happening, there's no way to atomically rename to devDir. That directory is created (if it doesn't already exist) at the start of the install process to confirm permissions, so a direct rename of tarExtractDir to devDir would fail. Any rename process would need to first rename devDir, which opens a race condition where devDir disappears momentarily during an ongoing build, causing a failure. That was my original approach for fixing this issue, but I switched to the current implementation to close that race condition.

This could still race condition if it's not atomic?

The copyFile approach stays similar to the existing implementation, with node-tar overwriting the existing files when there are parallel installs going on. The main difference is that on Windows node-tar always unlinks the destination file path before writing to it to avoid certain conditions (same path multiple times in an archive) which don't exist for this usage.

I've stress tested this change (on CI and locally with a real package) with 35 parallel installs and didn't trip over any race conditions in practice.

[StefanStojanovic]

LGTM. Since there are already 2 PRs addressing GitHub Action issues, we should make sure 6bb7cd6 is removed from here before landing it. Ideally, we'd first land a GitHub Actions fix, then rebase this PR, remove the mentioned commit and push changes.

[StefanStojanovic]

@rvagg@cclauss I'm changing tests from tap to mocha in #2851. Since this PR adds some tests and it's already approved, what are your thoughts on landing it (without 6bb7cd6) so I can change those tests in my PR as well?

[cclauss]

I have no objection to these changes but I am not a Windows user so I lack the skills to be a reviewer.

[StefanStojanovic]

Hey, @dsanders11 can you prepare your branch for merging by removing 6bb7cd6 and potentially addressing #2846 (comment) and squashing some commits together (especially the ones addressing PR comments)?

[dsanders11]

@StefanStojanovic, done! Removed 6bb7cd6 and squashed it back down to the original three commits. For CI reference, 2bb914b was the last commit before removing 6bb7cd6, and it went green.

[dsanders11]

Checklist

• npm install && npm test passes
• tests are included
• documentation is changed or added
• commit message follows commit guidelines

Description of change

Fixes#2482.
Fixes#2484.
Fixes#2584.
Fixes#2683.
Fixes#2751.
Fixes#2832.

NOTE: This PR contains a temporary initial commit which reverts a PR which causes CI workflow failures on main, so that the tests for this PR can run clean. It should be removed before merging this PR. #2837 is currently trying to fix that issue.

node-tar unlinks file paths before writing to them on Windows, which causes errors when there are parallel installs going on which are extracting the headers tarball to devDir. This currently leads to random corruption of the cache at devDir with missing files and hanging files with <filename>.DELETE.<random> names. See more in-depth discussion about the underlying issue here.

This PR fixes the situation by:

• Fixing usage of node-tar to watch for errors during extraction
• Extracting the headers tarball on Windows to a temp directory and then copies the files over
• Using the exponential-backoff package in case of collision (EBUSY) when copying the files to devDir
• Bumps installVersion to force clearing of existing cached versions which might be corrupt on Windows

This is set up in commits which were pushed one at a time to this PR so that the CI workflows for each commit can be seen, confirming that the added tests fail until the final fix is applied.

Unfortunately the new tests, despite being simple, are a bit flaky, due to network requests to https://nodejs.org/download/release/. The tests also take significant longer (greater than 2x) on Windows with Node.js 18.x and 20.x - it's not clear why, but since it repros consistently with those versions, it seems likely to be some upstream regression in Node.js, possibly something to do with the fs module.

[MarshallOfSound]

Is there a reason we wouldn't want to rename / move here instead of copy? This could still race condition if it's not atomic?

[dsanders11]

Is there a reason we wouldn't want to rename / move here instead of copy?

Since there are parallel installs happening, there's no way to atomically rename to devDir. That directory is created (if it doesn't already exist) at the start of the install process to confirm permissions, so a direct rename of tarExtractDir to devDir would fail. Any rename process would need to first rename devDir, which opens a race condition where devDir disappears momentarily during an ongoing build, causing a failure. That was my original approach for fixing this issue, but I switched to the current implementation to close that race condition.

This could still race condition if it's not atomic?

The copyFile approach stays similar to the existing implementation, with node-tar overwriting the existing files when there are parallel installs going on. The main difference is that on Windows node-tar always unlinks the destination file path before writing to it to avoid certain conditions (same path multiple times in an archive) which don't exist for this usage.

I've stress tested this change (on CI and locally with a real package) with 35 parallel installs and didn't trip over any race conditions in practice.

[StefanStojanovic]

LGTM. Since there are already 2 PRs addressing GitHub Action issues, we should make sure 6bb7cd6 is removed from here before landing it. Ideally, we'd first land a GitHub Actions fix, then rebase this PR, remove the mentioned commit and push changes.

[StefanStojanovic]

@rvagg@cclauss I'm changing tests from tap to mocha in #2851. Since this PR adds some tests and it's already approved, what are your thoughts on landing it (without 6bb7cd6) so I can change those tests in my PR as well?

[cclauss]

I have no objection to these changes but I am not a Windows user so I lack the skills to be a reviewer.

[StefanStojanovic]

Hey, @dsanders11 can you prepare your branch for merging by removing 6bb7cd6 and potentially addressing #2846 (comment) and squashing some commits together (especially the ones addressing PR comments)?

[dsanders11]

@StefanStojanovic, done! Removed 6bb7cd6 and squashed it back down to the original three commits. For CI reference, 2bb914b was the last commit before removing 6bb7cd6, and it went green.

[dsanders11]

Checklist

• npm install && npm test passes
• tests are included
• documentation is changed or added
• commit message follows commit guidelines

Description of change

Fixes#2482.
Fixes#2484.
Fixes#2584.
Fixes#2683.
Fixes#2751.
Fixes#2832.

NOTE: This PR contains a temporary initial commit which reverts a PR which causes CI workflow failures on main, so that the tests for this PR can run clean. It should be removed before merging this PR. #2837 is currently trying to fix that issue.

node-tar unlinks file paths before writing to them on Windows, which causes errors when there are parallel installs going on which are extracting the headers tarball to devDir. This currently leads to random corruption of the cache at devDir with missing files and hanging files with <filename>.DELETE.<random> names. See more in-depth discussion about the underlying issue here.

This PR fixes the situation by:

• Fixing usage of node-tar to watch for errors during extraction
• Extracting the headers tarball on Windows to a temp directory and then copies the files over
• Using the exponential-backoff package in case of collision (EBUSY) when copying the files to devDir
• Bumps installVersion to force clearing of existing cached versions which might be corrupt on Windows

This is set up in commits which were pushed one at a time to this PR so that the CI workflows for each commit can be seen, confirming that the added tests fail until the final fix is applied.

Unfortunately the new tests, despite being simple, are a bit flaky, due to network requests to https://nodejs.org/download/release/. The tests also take significant longer (greater than 2x) on Windows with Node.js 18.x and 20.x - it's not clear why, but since it repros consistently with those versions, it seems likely to be some upstream regression in Node.js, possibly something to do with the fs module.

[MarshallOfSound]

Is there a reason we wouldn't want to rename / move here instead of copy? This could still race condition if it's not atomic?

[dsanders11]

Is there a reason we wouldn't want to rename / move here instead of copy?

Since there are parallel installs happening, there's no way to atomically rename to devDir. That directory is created (if it doesn't already exist) at the start of the install process to confirm permissions, so a direct rename of tarExtractDir to devDir would fail. Any rename process would need to first rename devDir, which opens a race condition where devDir disappears momentarily during an ongoing build, causing a failure. That was my original approach for fixing this issue, but I switched to the current implementation to close that race condition.

This could still race condition if it's not atomic?

The copyFile approach stays similar to the existing implementation, with node-tar overwriting the existing files when there are parallel installs going on. The main difference is that on Windows node-tar always unlinks the destination file path before writing to it to avoid certain conditions (same path multiple times in an archive) which don't exist for this usage.

I've stress tested this change (on CI and locally with a real package) with 35 parallel installs and didn't trip over any race conditions in practice.

[StefanStojanovic]

LGTM. Since there are already 2 PRs addressing GitHub Action issues, we should make sure 6bb7cd6 is removed from here before landing it. Ideally, we'd first land a GitHub Actions fix, then rebase this PR, remove the mentioned commit and push changes.

[StefanStojanovic]

@rvagg@cclauss I'm changing tests from tap to mocha in #2851. Since this PR adds some tests and it's already approved, what are your thoughts on landing it (without 6bb7cd6) so I can change those tests in my PR as well?

[cclauss]

I have no objection to these changes but I am not a Windows user so I lack the skills to be a reviewer.

[StefanStojanovic]

Hey, @dsanders11 can you prepare your branch for merging by removing 6bb7cd6 and potentially addressing #2846 (comment) and squashing some commits together (especially the ones addressing PR comments)?

[dsanders11]

@StefanStojanovic, done! Removed 6bb7cd6 and squashed it back down to the original three commits. For CI reference, 2bb914b was the last commit before removing 6bb7cd6, and it went green.

[dsanders11]

Checklist

• npm install && npm test passes
• tests are included
• documentation is changed or added
• commit message follows commit guidelines

Description of change

Fixes#2482.
Fixes#2484.
Fixes#2584.
Fixes#2683.
Fixes#2751.
Fixes#2832.

NOTE: This PR contains a temporary initial commit which reverts a PR which causes CI workflow failures on main, so that the tests for this PR can run clean. It should be removed before merging this PR. #2837 is currently trying to fix that issue.

node-tar unlinks file paths before writing to them on Windows, which causes errors when there are parallel installs going on which are extracting the headers tarball to devDir. This currently leads to random corruption of the cache at devDir with missing files and hanging files with <filename>.DELETE.<random> names. See more in-depth discussion about the underlying issue here.

This PR fixes the situation by:

• Fixing usage of node-tar to watch for errors during extraction
• Extracting the headers tarball on Windows to a temp directory and then copies the files over
• Using the exponential-backoff package in case of collision (EBUSY) when copying the files to devDir
• Bumps installVersion to force clearing of existing cached versions which might be corrupt on Windows

This is set up in commits which were pushed one at a time to this PR so that the CI workflows for each commit can be seen, confirming that the added tests fail until the final fix is applied.

Unfortunately the new tests, despite being simple, are a bit flaky, due to network requests to https://nodejs.org/download/release/. The tests also take significant longer (greater than 2x) on Windows with Node.js 18.x and 20.x - it's not clear why, but since it repros consistently with those versions, it seems likely to be some upstream regression in Node.js, possibly something to do with the fs module.

[MarshallOfSound]

Is there a reason we wouldn't want to rename / move here instead of copy? This could still race condition if it's not atomic?

[dsanders11]

Is there a reason we wouldn't want to rename / move here instead of copy?

Since there are parallel installs happening, there's no way to atomically rename to devDir. That directory is created (if it doesn't already exist) at the start of the install process to confirm permissions, so a direct rename of tarExtractDir to devDir would fail. Any rename process would need to first rename devDir, which opens a race condition where devDir disappears momentarily during an ongoing build, causing a failure. That was my original approach for fixing this issue, but I switched to the current implementation to close that race condition.

This could still race condition if it's not atomic?

The copyFile approach stays similar to the existing implementation, with node-tar overwriting the existing files when there are parallel installs going on. The main difference is that on Windows node-tar always unlinks the destination file path before writing to it to avoid certain conditions (same path multiple times in an archive) which don't exist for this usage.

I've stress tested this change (on CI and locally with a real package) with 35 parallel installs and didn't trip over any race conditions in practice.

[StefanStojanovic]

LGTM. Since there are already 2 PRs addressing GitHub Action issues, we should make sure 6bb7cd6 is removed from here before landing it. Ideally, we'd first land a GitHub Actions fix, then rebase this PR, remove the mentioned commit and push changes.

[StefanStojanovic]

@rvagg@cclauss I'm changing tests from tap to mocha in #2851. Since this PR adds some tests and it's already approved, what are your thoughts on landing it (without 6bb7cd6) so I can change those tests in my PR as well?

[cclauss]

I have no objection to these changes but I am not a Windows user so I lack the skills to be a reviewer.

[StefanStojanovic]

Hey, @dsanders11 can you prepare your branch for merging by removing 6bb7cd6 and potentially addressing #2846 (comment) and squashing some commits together (especially the ones addressing PR comments)?

[dsanders11]

@StefanStojanovic, done! Removed 6bb7cd6 and squashed it back down to the original three commits. For CI reference, 2bb914b was the last commit before removing 6bb7cd6, and it went green.

[dsanders11]

Checklist

• npm install && npm test passes
• tests are included
• documentation is changed or added
• commit message follows commit guidelines

Description of change

Fixes#2482.
Fixes#2484.
Fixes#2584.
Fixes#2683.
Fixes#2751.
Fixes#2832.

NOTE: This PR contains a temporary initial commit which reverts a PR which causes CI workflow failures on main, so that the tests for this PR can run clean. It should be removed before merging this PR. #2837 is currently trying to fix that issue.

node-tar unlinks file paths before writing to them on Windows, which causes errors when there are parallel installs going on which are extracting the headers tarball to devDir. This currently leads to random corruption of the cache at devDir with missing files and hanging files with <filename>.DELETE.<random> names. See more in-depth discussion about the underlying issue here.

This PR fixes the situation by:

• Fixing usage of node-tar to watch for errors during extraction
• Extracting the headers tarball on Windows to a temp directory and then copies the files over
• Using the exponential-backoff package in case of collision (EBUSY) when copying the files to devDir
• Bumps installVersion to force clearing of existing cached versions which might be corrupt on Windows

This is set up in commits which were pushed one at a time to this PR so that the CI workflows for each commit can be seen, confirming that the added tests fail until the final fix is applied.

Unfortunately the new tests, despite being simple, are a bit flaky, due to network requests to https://nodejs.org/download/release/. The tests also take significant longer (greater than 2x) on Windows with Node.js 18.x and 20.x - it's not clear why, but since it repros consistently with those versions, it seems likely to be some upstream regression in Node.js, possibly something to do with the fs module.

[MarshallOfSound]

Is there a reason we wouldn't want to rename / move here instead of copy? This could still race condition if it's not atomic?

[dsanders11]

Is there a reason we wouldn't want to rename / move here instead of copy?

Since there are parallel installs happening, there's no way to atomically rename to devDir. That directory is created (if it doesn't already exist) at the start of the install process to confirm permissions, so a direct rename of tarExtractDir to devDir would fail. Any rename process would need to first rename devDir, which opens a race condition where devDir disappears momentarily during an ongoing build, causing a failure. That was my original approach for fixing this issue, but I switched to the current implementation to close that race condition.

This could still race condition if it's not atomic?

The copyFile approach stays similar to the existing implementation, with node-tar overwriting the existing files when there are parallel installs going on. The main difference is that on Windows node-tar always unlinks the destination file path before writing to it to avoid certain conditions (same path multiple times in an archive) which don't exist for this usage.

I've stress tested this change (on CI and locally with a real package) with 35 parallel installs and didn't trip over any race conditions in practice.

[StefanStojanovic]

LGTM. Since there are already 2 PRs addressing GitHub Action issues, we should make sure 6bb7cd6 is removed from here before landing it. Ideally, we'd first land a GitHub Actions fix, then rebase this PR, remove the mentioned commit and push changes.

[StefanStojanovic]

@rvagg@cclauss I'm changing tests from tap to mocha in #2851. Since this PR adds some tests and it's already approved, what are your thoughts on landing it (without 6bb7cd6) so I can change those tests in my PR as well?

[cclauss]

I have no objection to these changes but I am not a Windows user so I lack the skills to be a reviewer.

[StefanStojanovic]

Hey, @dsanders11 can you prepare your branch for merging by removing 6bb7cd6 and potentially addressing #2846 (comment) and squashing some commits together (especially the ones addressing PR comments)?

[dsanders11]

@StefanStojanovic, done! Removed 6bb7cd6 and squashed it back down to the original three commits. For CI reference, 2bb914b was the last commit before removing 6bb7cd6, and it went green.