Skip to content

test: make test-perf-hooks more robust and work with workers - #49197

Merged
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
joyeecheung:perf-hook-test
Aug 18, 2023
Merged

test: make test-perf-hooks more robust and work with workers#49197
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
joyeecheung:perf-hook-test

Conversation

@joyeecheung

@joyeecheungjoyeecheung commented Aug 16, 2023•
edited
Loading

Copy link
Copy Markdown
Member

Previously the test makes several assumptions about the absolute values of the nodeTiming fields, which can make the test flaky on slow machines. This patch rewrites the test to check the relative values instead. It also updates the test to make it work with workers instead of directly skipping in workers.

Refs: nodejs/reliability#638

@joyeecheung
test: make test-perf-hooks more robust and work with workers
3ecbef5
Previously the test makes several assumptions about the absolute
values of the nodeTiming fields, which can make the test flaky
on slow machines. This patch rewrites the test to check the
relative values instead. It also updates the test to make it
work with workers instead of directly skipping in workers.
@joyeecheungjoyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 16, 2023
@nodejs-github-botnodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. labels Aug 16, 2023
@joyeecheung

joyeecheung commented Aug 16, 2023

Copy link
Copy Markdown
MemberAuthor

From nodejs/reliability#638 this test has been failing 6 recent PRs, hopefully this makes the flake go away..

Reasonsequential/test-perf-hooks
TypeJS_TEST_FAILURE
Failed PR6 (#49104, #49127, #49149, #49162, #49164, #46391)
Appearedtest-equinix-ubuntu1804_container-arm64-2, test-osuosl-ubuntu2004_sharedlibs_container-arm64-1, test-osuosl-ubuntu2004_container-armv7l-1, test-equinix-ubuntu2004_container-armv7l-2, test-azure_msft-win11_vs2022-arm64-4, test-equinix-ubuntu2004_container-arm64-2
First CIhttps://ci.nodejs.org/job/node-test-pull-request/53267/
Last CIhttps://ci.nodejs.org/job/node-test-pull-request/53336/
Example
not ok 3928 sequential/test-perf-hooks
---
duration_ms: 426.79600
severity: fail
exitcode: 1
stack: |-
{
name: 'node',
entryType: 'node',
startTime: 0,
duration: { around: 285.6393041610718 },
nodeStart: { around: 0 },
v8Start: { around: 0 },
bootstrapComplete: { around: 285.5992240905762, delay: 2500 },
environment: { around: 0 },
loopStart: -1,
loopExit: -1
}
node:assert:399
throw err;
^
AssertionError [ERR_ASSERTION]: environment: 252.57520198822021 >= 250
at checkNodeTiming (/home/iojs/build/workspace/node-test-commit-arm/test/sequential/test-perf-hooks.js:31:7)
at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-arm/test/sequential/test-perf-hooks.js:43:1)
at Module._compile (node:internal/modules/cjs/loader:1241:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1295:10)
at Module.load (node:internal/modules/cjs/loader:1091:32)
...

@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 16, 2023
@nodejs-github-bot

nodejs-github-bot commented Aug 16, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/53373/

debadree25
debadree25 reviewed Aug 16, 2023
View reviewed changes
Comment threadtest/sequential/test-perf-hooks.js
debadree25
debadree25 approved these changes Aug 16, 2023
View reviewed changes

@debadree25debadree25 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!
Thank you!

@debadree25debadree25 added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Aug 16, 2023
@debadree25

debadree25 commented Aug 16, 2023

Copy link
Copy Markdown
Contributor

i tried stress testing this too locally with python3 tools/test.py test/sequential/test-perf-hooks.js --repeat 50 -j10 it passed well! so maybe could graduate to parallel too?

@joyeecheung

joyeecheung commented Aug 16, 2023•
edited
Loading

Copy link
Copy Markdown
MemberAuthor

I am not sure why it was in sequential in the first place, but I think on the safe side we can just keep it there in this PR and if it looks well in the CI, move it to parallel afterwards (otherwise one doesn't know which is causing a regression if there is a regression).

@debadree25

debadree25 commented Aug 16, 2023

Copy link
Copy Markdown
Contributor

Yeah makes sense!

aduh95
aduh95 approved these changes Aug 16, 2023
View reviewed changes
Comment threadtest/sequential/test-perf-hooks.js
// Use a fairly large epsilon value, since we can only guarantee that the node
// process started up in 15 seconds.
assert(Math.abs(performance.timeOrigin - Date.now()) < 15000);
assert(testStartTime < 15000, `${testStartTime} >= 15000`);

@aduh95aduh95Aug 16, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit

Suggested change
assert(testStartTime<15000,`${testStartTime}>= 15000`);
assert(testStartTime<15000,`${testStartTime}≥ 15000`);

@joyeecheungjoyeecheungAug 17, 2023

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in general we use >= in this codebase instead?

@joyeecheungjoyeecheung added the commit-queue Add this label to land a pull request using GitHub Actions. label Aug 18, 2023
@joyeecheung

joyeecheung commented Aug 18, 2023•
edited
Loading

Copy link
Copy Markdown
MemberAuthor

By the way the failure also shows up on v20.x-staging https://ci.nodejs.org/job/node-test-binary-armv7l/7027/RUN_SUBSET=js,nodes=ubuntu2004-armv7l/testReport/junit/(root)/sequential/test_perf_hooks/

@nodejs-github-botnodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Aug 18, 2023
@nodejs-github-bot
nodejs-github-bot merged commit ecde9d9 into nodejs:mainAug 18, 2023
@nodejs-github-bot

nodejs-github-bot commented Aug 18, 2023

Copy link
Copy Markdown
Collaborator

Landed in ecde9d9

UlisesGascon pushed a commit that referenced this pull request Sep 10, 2023
@joyeecheung@UlisesGascon
test: make test-perf-hooks more robust and work with workers
9610008
Previously the test makes several assumptions about the absolute
values of the nodeTiming fields, which can make the test flaky
on slow machines. This patch rewrites the test to check the
relative values instead. It also updates the test to make it
work with workers instead of directly skipping in workers.
PR-URL: #49197
Refs: nodejs/reliability#638
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
@UlisesGasconUlisesGascon mentioned this pull request Sep 10, 2023
Merged
targos pushed a commit that referenced this pull request Nov 27, 2023
@joyeecheung@targos
test: make test-perf-hooks more robust and work with workers
0261c5d
Previously the test makes several assumptions about the absolute
values of the nodeTiming fields, which can make the test flaky
on slow machines. This patch rewrites the test to check the
relative values instead. It also updates the test to make it
work with workers instead of directly skipping in workers.
PR-URL: #49197
Refs: nodejs/reliability#638
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
@targostargos mentioned this pull request Nov 28, 2023
Merged
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
@sercher
test: make test-perf-hooks more robust and work with workers
e952ee9
Previously the test makes several assumptions about the absolute
values of the nodeTiming fields, which can make the test flaky
on slow machines. This patch rewrites the test to check the
relative values instead. It also updates the test to make it
work with workers instead of directly skipping in workers.
PR-URL: nodejs/node#49197
Refs: nodejs/reliability#638
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
@sercher
test: make test-perf-hooks more robust and work with workers
3cee2c0
Previously the test makes several assumptions about the absolute
values of the nodeTiming fields, which can make the test flaky
on slow machines. This patch rewrites the test to check the
relative values instead. It also updates the test to make it
work with workers instead of directly skipping in workers.
PR-URL: nodejs/node#49197
Refs: nodejs/reliability#638
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@aduh95aduh95aduh95 approved these changes
+1 more reviewer
@debadree25debadree25debadree25 approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

author readyPRs that have at least one approval, no pending requests for changes, and a CI started.needs-ciPRs that need a full CI run.testIssues and PRs related to the tests.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@joyeecheung@nodejs-github-bot@debadree25@aduh95