Skip to content

test: reduce impact of flaky HTTP server tests - #42926

Merged
aduh95 merged 1 commit into
nodejs:masterfrom
tniessen:flaky-test-http-server-x-timeout-keepalive
May 2, 2022
Merged

test: reduce impact of flaky HTTP server tests#42926
aduh95 merged 1 commit into
nodejs:masterfrom
tniessen:flaky-test-http-server-x-timeout-keepalive

Conversation

@tniessen

@tniessentniessen commented Apr 30, 2022

Copy link
Copy Markdown
Member

Mark the tests as flaky on macOS and increase the timeout for other platforms (namely, FreeBSD).

Context: #42741 (comment)

Median success rate (refer to patch and separate tables above):

osx11freebsd12-x64
master (2000ms)91.0 %83.8 %
patched (5000ms)71.3 %100 %

Refs: #42741

@tniessentniessen added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 30, 2022
@nodejs-github-botnodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. labels Apr 30, 2022
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 30, 2022
@nodejs-github-bot

This comment was marked as outdated.

Sign in to view
@tniessen

tniessen commented Apr 30, 2022•
edited
Loading

Copy link
Copy Markdown
MemberAuthor

Is PASS,FLAKY not the right configuration? node-test-commit-osx still fails with this. Should it be SKIP?

Removed the file extensions, maybe that works.

@tniessen
tniessenforce-pushed the flaky-test-http-server-x-timeout-keepalive branch from 35014d7 to 06c1abeCompareApril 30, 2022 14:06
@targos

targos commented Apr 30, 2022

Copy link
Copy Markdown
Member

Removed the file extensions, maybe that works.

Yep:

node/test/parallel/parallel.status

Lines 3 to 5 in 02e0c17

# To mark a test as flaky, list the test name in the appropriate section
# below, without ".js", followed by ": PASS,FLAKY". Example:
# sample-test : PASS,FLAKY

@tniessentniessen added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 30, 2022
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 30, 2022
@nodejs-github-bot

nodejs-github-bot commented Apr 30, 2022

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/43796/

aduh95
aduh95 approved these changes Apr 30, 2022
View reviewed changes

@aduh95aduh95 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The flakiness is also annoying on the GHA runner, and unfortunately this PR won't help with that I'm afraid. Oh well, at least it's improving things for the Jenkins CI, so LGTM.

@github-actionsgithub-actionsBot mentioned this pull request May 1, 2022
Open
29 tasks
@tniessen

tniessen commented May 1, 2022

Copy link
Copy Markdown
MemberAuthor

The flakiness is also annoying on the GHA runner, and unfortunately this PR won't help with that I'm afraid.

@aduh95 I am not sure about that. #42741 (comment) suggested that GitHub Actions might respect the list of flaky tests.

@nodejs-github-bot

nodejs-github-bot commented May 1, 2022

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/43804/

@tniessen

tniessen commented May 1, 2022

Copy link
Copy Markdown
MemberAuthor

CI is failing because of nodejs/build#2917.

@tniessentniessen added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label May 1, 2022
@github-actionsgithub-actionsBot mentioned this pull request May 2, 2022
Open
30 tasks
@nodejs-github-bot

nodejs-github-bot commented May 2, 2022

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/43810/

@tniessentniessen added the commit-queue Add this label to land a pull request using GitHub Actions. label May 2, 2022
@tniessen

tniessen commented May 2, 2022

Copy link
Copy Markdown
MemberAuthor

CI is green but GitHub seems to be thinking that Jenkins is still running, and the commit queue refuses to run...

@nodejs-github-bot

nodejs-github-bot commented May 2, 2022

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/43826/

@aduh95aduh95 removed the commit-queue Add this label to land a pull request using GitHub Actions. label May 2, 2022
@aduh95
aduh95 merged commit 0fc455a into nodejs:masterMay 2, 2022
@aduh95

aduh95 commented May 2, 2022

Copy link
Copy Markdown
Contributor

Landed in 0fc455a

@tniessen

tniessen commented May 2, 2022

Copy link
Copy Markdown
MemberAuthor

Thank you @aduh95 :)

This was referenced May 3, 2022
Open
Open
Open
RafaelGSS pushed a commit that referenced this pull request May 10, 2022
@tniessen@RafaelGSS
test: reduce impact of flaky HTTP server tests
72b90fd
Refs: #42741
PR-URL: #42926
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
@RafaelGSSRafaelGSS mentioned this pull request May 10, 2022
Merged
@daeyeondaeyeon mentioned this pull request May 23, 2022
Merged
@danielleadams

danielleadams commented Jun 26, 2022

Copy link
Copy Markdown
Contributor

This PR is dependent on #42846, so marking as blocked.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lpincalpincalpinca approved these changes
@targostargostargos approved these changes
@aduh95aduh95aduh95 approved these changes
@richardlaurichardlauAwaiting requested review from richardlau

Assignees

No one assigned

Labels

author readyPRs that have at least one approval, no pending requests for changes, and a CI started.needs-ciPRs that need a full CI run.testIssues and PRs related to the tests.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@tniessen@nodejs-github-bot@targos@aduh95@danielleadams@lpinca