Using mocha for running some tests by StefanStojanovic · Pull Request #2837 · nodejs/node-gyp

StefanStojanovic · 2023-05-04T06:17:51Z

Checklist

npm install && npm test passes
tests are included
commit message follows commit guidelines

Description of change

Constant GitHub Action failures started after landing 02480f6. One way to fix this is reverting that commit as proposed in #2849. This PR uses a different approach by keeping the make-fetch-happen version and making changes in testing modules.

The first thing updated is the tap module, from its legacy version 12.7.0 to the latest version 16.3.4. This change fixed the failing tests, but afterward, test-addon.js started failing randomly in GitHub Actions. Rerunning failed jobs would sometimes fix that, but in some cases, mostly on Windows, they stood broken after many reruns. There was no consistency in this and no pattern was to be found. On the other hand, running the npm test locally always passed for me and I tested with various node/python combinations.

Since I couldn't reproduce any issues locally, I decided to try changing the testing framework (at least for the failing tests). That brings me to my second change - using mocha to run test-addon.js which had to be adapted to the new framework. After doing this everything passed in the GitHub Actions without any issues.

Other ideas

At this time, the solution isn't pretty but it unblocks the CI, which was my main goal. Potentially, all tests can be migrated to mocha eventually, thus tap can be removed completely then. Another idea (proposed in #2851) is to simply use mocha to run tests we already have (eg. mocha test/test-*) instead of tap, and keep tests written in tap.

legobeat · 2023-05-17T00:18:55Z

@StefanStojanovic this may be relevant: #2848

This reverts commit 02480f6, thereby rolling back dependency make-fetch-happen from ^11.0.3 to ^10.0.3. The upgrade is breaking for node-fetch users as it has transitive dependencies with syntax incompatible with supported Node.js versions. Related: - nodejs#2770 - nodejs#2837 - nodejs#2816 - nodejs#2848 - nodejs#2827 - nodejs#2796

After make-fetch-happen update GitHub Actions started failing. Updating tap to the latest version and migrating to mocha for addon tests fixes GitHub Action failures.

StefanStojanovic · 2023-05-17T12:35:47Z

Closing in favor of #2851

This reverts commit 02480f6, thereby rolling back dependency make-fetch-happen from ^11.0.3 to ^10.0.3. The upgrade is breaking for node-fetch users as it has transitive dependencies with syntax incompatible with supported Node.js versions. Related: - nodejs#2770 - nodejs#2837 - nodejs#2816 - nodejs#2848 - nodejs#2827 - nodejs#2796

StefanStojanovic · 2023-05-04T06:17:51Z

Checklist

npm install && npm test passes
tests are included
commit message follows commit guidelines

Description of change

Constant GitHub Action failures started after landing 02480f6. One way to fix this is reverting that commit as proposed in #2849. This PR uses a different approach by keeping the make-fetch-happen version and making changes in testing modules.

The first thing updated is the tap module, from its legacy version 12.7.0 to the latest version 16.3.4. This change fixed the failing tests, but afterward, test-addon.js started failing randomly in GitHub Actions. Rerunning failed jobs would sometimes fix that, but in some cases, mostly on Windows, they stood broken after many reruns. There was no consistency in this and no pattern was to be found. On the other hand, running the npm test locally always passed for me and I tested with various node/python combinations.

Since I couldn't reproduce any issues locally, I decided to try changing the testing framework (at least for the failing tests). That brings me to my second change - using mocha to run test-addon.js which had to be adapted to the new framework. After doing this everything passed in the GitHub Actions without any issues.

Other ideas

At this time, the solution isn't pretty but it unblocks the CI, which was my main goal. Potentially, all tests can be migrated to mocha eventually, thus tap can be removed completely then. Another idea (proposed in #2851) is to simply use mocha to run tests we already have (eg. mocha test/test-*) instead of tap, and keep tests written in tap.

legobeat · 2023-05-17T00:18:55Z

@StefanStojanovic this may be relevant: #2848

This reverts commit 02480f6, thereby rolling back dependency make-fetch-happen from ^11.0.3 to ^10.0.3. The upgrade is breaking for node-fetch users as it has transitive dependencies with syntax incompatible with supported Node.js versions. Related: - nodejs#2770 - nodejs#2837 - nodejs#2816 - nodejs#2848 - nodejs#2827 - nodejs#2796

After make-fetch-happen update GitHub Actions started failing. Updating tap to the latest version and migrating to mocha for addon tests fixes GitHub Action failures.

StefanStojanovic · 2023-05-17T12:35:47Z

Closing in favor of #2851

This reverts commit 02480f6, thereby rolling back dependency make-fetch-happen from ^11.0.3 to ^10.0.3. The upgrade is breaking for node-fetch users as it has transitive dependencies with syntax incompatible with supported Node.js versions. Related: - nodejs#2770 - nodejs#2837 - nodejs#2816 - nodejs#2848 - nodejs#2827 - nodejs#2796

StefanStojanovic · 2023-05-04T06:17:51Z

Checklist

npm install && npm test passes
tests are included
commit message follows commit guidelines

Description of change

Constant GitHub Action failures started after landing 02480f6. One way to fix this is reverting that commit as proposed in #2849. This PR uses a different approach by keeping the make-fetch-happen version and making changes in testing modules.

The first thing updated is the tap module, from its legacy version 12.7.0 to the latest version 16.3.4. This change fixed the failing tests, but afterward, test-addon.js started failing randomly in GitHub Actions. Rerunning failed jobs would sometimes fix that, but in some cases, mostly on Windows, they stood broken after many reruns. There was no consistency in this and no pattern was to be found. On the other hand, running the npm test locally always passed for me and I tested with various node/python combinations.

Since I couldn't reproduce any issues locally, I decided to try changing the testing framework (at least for the failing tests). That brings me to my second change - using mocha to run test-addon.js which had to be adapted to the new framework. After doing this everything passed in the GitHub Actions without any issues.

Other ideas

At this time, the solution isn't pretty but it unblocks the CI, which was my main goal. Potentially, all tests can be migrated to mocha eventually, thus tap can be removed completely then. Another idea (proposed in #2851) is to simply use mocha to run tests we already have (eg. mocha test/test-*) instead of tap, and keep tests written in tap.

legobeat · 2023-05-17T00:18:55Z

@StefanStojanovic this may be relevant: #2848

This reverts commit 02480f6, thereby rolling back dependency make-fetch-happen from ^11.0.3 to ^10.0.3. The upgrade is breaking for node-fetch users as it has transitive dependencies with syntax incompatible with supported Node.js versions. Related: - nodejs#2770 - nodejs#2837 - nodejs#2816 - nodejs#2848 - nodejs#2827 - nodejs#2796

After make-fetch-happen update GitHub Actions started failing. Updating tap to the latest version and migrating to mocha for addon tests fixes GitHub Action failures.

StefanStojanovic · 2023-05-17T12:35:47Z

Closing in favor of #2851

This reverts commit 02480f6, thereby rolling back dependency make-fetch-happen from ^11.0.3 to ^10.0.3. The upgrade is breaking for node-fetch users as it has transitive dependencies with syntax incompatible with supported Node.js versions. Related: - nodejs#2770 - nodejs#2837 - nodejs#2816 - nodejs#2848 - nodejs#2827 - nodejs#2796

StefanStojanovic · 2023-05-04T06:17:51Z

Checklist

npm install && npm test passes
tests are included
commit message follows commit guidelines

Description of change

Constant GitHub Action failures started after landing 02480f6. One way to fix this is reverting that commit as proposed in #2849. This PR uses a different approach by keeping the make-fetch-happen version and making changes in testing modules.

The first thing updated is the tap module, from its legacy version 12.7.0 to the latest version 16.3.4. This change fixed the failing tests, but afterward, test-addon.js started failing randomly in GitHub Actions. Rerunning failed jobs would sometimes fix that, but in some cases, mostly on Windows, they stood broken after many reruns. There was no consistency in this and no pattern was to be found. On the other hand, running the npm test locally always passed for me and I tested with various node/python combinations.

Since I couldn't reproduce any issues locally, I decided to try changing the testing framework (at least for the failing tests). That brings me to my second change - using mocha to run test-addon.js which had to be adapted to the new framework. After doing this everything passed in the GitHub Actions without any issues.

Other ideas

At this time, the solution isn't pretty but it unblocks the CI, which was my main goal. Potentially, all tests can be migrated to mocha eventually, thus tap can be removed completely then. Another idea (proposed in #2851) is to simply use mocha to run tests we already have (eg. mocha test/test-*) instead of tap, and keep tests written in tap.

legobeat · 2023-05-17T00:18:55Z

@StefanStojanovic this may be relevant: #2848

This reverts commit 02480f6, thereby rolling back dependency make-fetch-happen from ^11.0.3 to ^10.0.3. The upgrade is breaking for node-fetch users as it has transitive dependencies with syntax incompatible with supported Node.js versions. Related: - nodejs#2770 - nodejs#2837 - nodejs#2816 - nodejs#2848 - nodejs#2827 - nodejs#2796

After make-fetch-happen update GitHub Actions started failing. Updating tap to the latest version and migrating to mocha for addon tests fixes GitHub Action failures.

StefanStojanovic · 2023-05-17T12:35:47Z

Closing in favor of #2851

This reverts commit 02480f6, thereby rolling back dependency make-fetch-happen from ^11.0.3 to ^10.0.3. The upgrade is breaking for node-fetch users as it has transitive dependencies with syntax incompatible with supported Node.js versions. Related: - nodejs#2770 - nodejs#2837 - nodejs#2816 - nodejs#2848 - nodejs#2827 - nodejs#2796

StefanStojanovic · 2023-05-04T06:17:51Z

Checklist

npm install && npm test passes
tests are included
commit message follows commit guidelines

Description of change

Constant GitHub Action failures started after landing 02480f6. One way to fix this is reverting that commit as proposed in #2849. This PR uses a different approach by keeping the make-fetch-happen version and making changes in testing modules.

The first thing updated is the tap module, from its legacy version 12.7.0 to the latest version 16.3.4. This change fixed the failing tests, but afterward, test-addon.js started failing randomly in GitHub Actions. Rerunning failed jobs would sometimes fix that, but in some cases, mostly on Windows, they stood broken after many reruns. There was no consistency in this and no pattern was to be found. On the other hand, running the npm test locally always passed for me and I tested with various node/python combinations.

Since I couldn't reproduce any issues locally, I decided to try changing the testing framework (at least for the failing tests). That brings me to my second change - using mocha to run test-addon.js which had to be adapted to the new framework. After doing this everything passed in the GitHub Actions without any issues.

Other ideas

At this time, the solution isn't pretty but it unblocks the CI, which was my main goal. Potentially, all tests can be migrated to mocha eventually, thus tap can be removed completely then. Another idea (proposed in #2851) is to simply use mocha to run tests we already have (eg. mocha test/test-*) instead of tap, and keep tests written in tap.

legobeat · 2023-05-17T00:18:55Z

@StefanStojanovic this may be relevant: #2848

This reverts commit 02480f6, thereby rolling back dependency make-fetch-happen from ^11.0.3 to ^10.0.3. The upgrade is breaking for node-fetch users as it has transitive dependencies with syntax incompatible with supported Node.js versions. Related: - nodejs#2770 - nodejs#2837 - nodejs#2816 - nodejs#2848 - nodejs#2827 - nodejs#2796

After make-fetch-happen update GitHub Actions started failing. Updating tap to the latest version and migrating to mocha for addon tests fixes GitHub Action failures.

StefanStojanovic · 2023-05-17T12:35:47Z

Closing in favor of #2851

This reverts commit 02480f6, thereby rolling back dependency make-fetch-happen from ^11.0.3 to ^10.0.3. The upgrade is breaking for node-fetch users as it has transitive dependencies with syntax incompatible with supported Node.js versions. Related: - nodejs#2770 - nodejs#2837 - nodejs#2816 - nodejs#2848 - nodejs#2827 - nodejs#2796

StefanStojanovic · 2023-05-04T06:17:51Z

Checklist

npm install && npm test passes
tests are included
commit message follows commit guidelines

Description of change

Constant GitHub Action failures started after landing 02480f6. One way to fix this is reverting that commit as proposed in #2849. This PR uses a different approach by keeping the make-fetch-happen version and making changes in testing modules.

The first thing updated is the tap module, from its legacy version 12.7.0 to the latest version 16.3.4. This change fixed the failing tests, but afterward, test-addon.js started failing randomly in GitHub Actions. Rerunning failed jobs would sometimes fix that, but in some cases, mostly on Windows, they stood broken after many reruns. There was no consistency in this and no pattern was to be found. On the other hand, running the npm test locally always passed for me and I tested with various node/python combinations.

Since I couldn't reproduce any issues locally, I decided to try changing the testing framework (at least for the failing tests). That brings me to my second change - using mocha to run test-addon.js which had to be adapted to the new framework. After doing this everything passed in the GitHub Actions without any issues.

Other ideas

At this time, the solution isn't pretty but it unblocks the CI, which was my main goal. Potentially, all tests can be migrated to mocha eventually, thus tap can be removed completely then. Another idea (proposed in #2851) is to simply use mocha to run tests we already have (eg. mocha test/test-*) instead of tap, and keep tests written in tap.

legobeat · 2023-05-17T00:18:55Z

@StefanStojanovic this may be relevant: #2848

This reverts commit 02480f6, thereby rolling back dependency make-fetch-happen from ^11.0.3 to ^10.0.3. The upgrade is breaking for node-fetch users as it has transitive dependencies with syntax incompatible with supported Node.js versions. Related: - nodejs#2770 - nodejs#2837 - nodejs#2816 - nodejs#2848 - nodejs#2827 - nodejs#2796

After make-fetch-happen update GitHub Actions started failing. Updating tap to the latest version and migrating to mocha for addon tests fixes GitHub Action failures.

StefanStojanovic · 2023-05-17T12:35:47Z

Closing in favor of #2851

This reverts commit 02480f6, thereby rolling back dependency make-fetch-happen from ^11.0.3 to ^10.0.3. The upgrade is breaking for node-fetch users as it has transitive dependencies with syntax incompatible with supported Node.js versions. Related: - nodejs#2770 - nodejs#2837 - nodejs#2816 - nodejs#2848 - nodejs#2827 - nodejs#2796

StefanStojanovic · 2023-05-04T06:17:51Z

Checklist

npm install && npm test passes
tests are included
commit message follows commit guidelines

Description of change

Constant GitHub Action failures started after landing 02480f6. One way to fix this is reverting that commit as proposed in #2849. This PR uses a different approach by keeping the make-fetch-happen version and making changes in testing modules.

The first thing updated is the tap module, from its legacy version 12.7.0 to the latest version 16.3.4. This change fixed the failing tests, but afterward, test-addon.js started failing randomly in GitHub Actions. Rerunning failed jobs would sometimes fix that, but in some cases, mostly on Windows, they stood broken after many reruns. There was no consistency in this and no pattern was to be found. On the other hand, running the npm test locally always passed for me and I tested with various node/python combinations.

Since I couldn't reproduce any issues locally, I decided to try changing the testing framework (at least for the failing tests). That brings me to my second change - using mocha to run test-addon.js which had to be adapted to the new framework. After doing this everything passed in the GitHub Actions without any issues.

Other ideas

At this time, the solution isn't pretty but it unblocks the CI, which was my main goal. Potentially, all tests can be migrated to mocha eventually, thus tap can be removed completely then. Another idea (proposed in #2851) is to simply use mocha to run tests we already have (eg. mocha test/test-*) instead of tap, and keep tests written in tap.

legobeat · 2023-05-17T00:18:55Z

@StefanStojanovic this may be relevant: #2848

This reverts commit 02480f6, thereby rolling back dependency make-fetch-happen from ^11.0.3 to ^10.0.3. The upgrade is breaking for node-fetch users as it has transitive dependencies with syntax incompatible with supported Node.js versions. Related: - nodejs#2770 - nodejs#2837 - nodejs#2816 - nodejs#2848 - nodejs#2827 - nodejs#2796

After make-fetch-happen update GitHub Actions started failing. Updating tap to the latest version and migrating to mocha for addon tests fixes GitHub Action failures.

StefanStojanovic · 2023-05-17T12:35:47Z

Closing in favor of #2851

This reverts commit 02480f6, thereby rolling back dependency make-fetch-happen from ^11.0.3 to ^10.0.3. The upgrade is breaking for node-fetch users as it has transitive dependencies with syntax incompatible with supported Node.js versions. Related: - nodejs#2770 - nodejs#2837 - nodejs#2816 - nodejs#2848 - nodejs#2827 - nodejs#2796

StefanStojanovic · 2023-05-04T06:17:51Z

Checklist

npm install && npm test passes
tests are included
commit message follows commit guidelines

Description of change

Constant GitHub Action failures started after landing 02480f6. One way to fix this is reverting that commit as proposed in #2849. This PR uses a different approach by keeping the make-fetch-happen version and making changes in testing modules.

The first thing updated is the tap module, from its legacy version 12.7.0 to the latest version 16.3.4. This change fixed the failing tests, but afterward, test-addon.js started failing randomly in GitHub Actions. Rerunning failed jobs would sometimes fix that, but in some cases, mostly on Windows, they stood broken after many reruns. There was no consistency in this and no pattern was to be found. On the other hand, running the npm test locally always passed for me and I tested with various node/python combinations.

Since I couldn't reproduce any issues locally, I decided to try changing the testing framework (at least for the failing tests). That brings me to my second change - using mocha to run test-addon.js which had to be adapted to the new framework. After doing this everything passed in the GitHub Actions without any issues.

Other ideas

At this time, the solution isn't pretty but it unblocks the CI, which was my main goal. Potentially, all tests can be migrated to mocha eventually, thus tap can be removed completely then. Another idea (proposed in #2851) is to simply use mocha to run tests we already have (eg. mocha test/test-*) instead of tap, and keep tests written in tap.

legobeat · 2023-05-17T00:18:55Z

@StefanStojanovic this may be relevant: #2848

This reverts commit 02480f6, thereby rolling back dependency make-fetch-happen from ^11.0.3 to ^10.0.3. The upgrade is breaking for node-fetch users as it has transitive dependencies with syntax incompatible with supported Node.js versions. Related: - nodejs#2770 - nodejs#2837 - nodejs#2816 - nodejs#2848 - nodejs#2827 - nodejs#2796

After make-fetch-happen update GitHub Actions started failing. Updating tap to the latest version and migrating to mocha for addon tests fixes GitHub Action failures.

StefanStojanovic · 2023-05-17T12:35:47Z

Closing in favor of #2851

This reverts commit 02480f6, thereby rolling back dependency make-fetch-happen from ^11.0.3 to ^10.0.3. The upgrade is breaking for node-fetch users as it has transitive dependencies with syntax incompatible with supported Node.js versions. Related: - nodejs#2770 - nodejs#2837 - nodejs#2816 - nodejs#2848 - nodejs#2827 - nodejs#2796

StefanStojanovic marked this pull request as draft May 4, 2023 11:43

StefanStojanovic force-pushed the mefi-github-actions-test branch from f3031da to 1a438f3Compare May 5, 2023 13:36

StefanStojanovic changed the title ~~Test~~Github actions constant failure investigationMay 8, 2023

cclauss changed the title ~~Github actions constant failure investigation~~GitHub Actions constant failure investigationMay 9, 2023

StefanStojanovic force-pushed the mefi-github-actions-test branch 4 times, most recently from e186ce0 to 1247646Compare May 11, 2023 16:02

dsanders11 mentioned this pull request May 11, 2023
fix: extract tarball to temp directory on Windows #2846
Merged
4 tasks

StefanStojanovic force-pushed the mefi-github-actions-test branch 10 times, most recently from d639602 to 52ae417Compare May 15, 2023 14:45

cclauss mentioned this pull request May 15, 2023
update make-fetch-happen to 11.0.3 #2795
Closed

StefanStojanovic force-pushed the mefi-github-actions-test branch 4 times, most recently from a7e339a to 1a2445cCompare May 16, 2023 15:44

legobeat mentioned this pull request May 17, 2023
Revert "update make-fetch-happen to 11.0.3 (#2796)" #2849
Closed
4 tasks

migrate from tap to mocha
b26ab05
After make-fetch-happen update GitHub Actions started failing. Updating tap to the latest version and migrating to mocha for addon tests fixes GitHub Action failures.

StefanStojanovic force-pushed the mefi-github-actions-test branch from 4994631 to b26ab05Compare May 17, 2023 10:24

StefanStojanovic changed the title ~~GitHub Actions constant failure investigation~~Using mocha for running some testsMay 17, 2023

StefanStojanovic mentioned this pull request May 17, 2023
Migration from tap to mocha #2851
Merged
3 tasks

StefanStojanovic closed this May 17, 2023