Skip to content

fs: improve error performance for fsyncSync - #49880

Merged
nodejs-github-bot merged 9 commits into
nodejs:mainfrom
pluris:perf/fsync
Oct 20, 2023
Merged

fs: improve error performance for fsyncSync#49880
nodejs-github-bot merged 9 commits into
nodejs:mainfrom
pluris:perf/fsync

Conversation

@pluris

@plurispluris commented Sep 26, 2023

Copy link
Copy Markdown
Contributor
 confidence improvement accuracy (*) (**) (***)
fs/bench-fsyncSync.js n=10000 type='existing' *** 3.37 % ±1.49% ±1.99% ±2.60%
fs/bench-fsyncSync.js n=10000 type='non-existing' *** 116.95 % ±6.64% ±8.87% ±11.64%
Be aware that when doing many comparisons the risk of a false-positive result increases.
In this case, there are 2 comparisons, you can thus expect the following amount of false-positive results:
0.10 false positives, when considering a 5% risk acceptance (*, **, ***),
0.02 false positives, when considering a 1% risk acceptance (**, ***),
0.00 false positives, when considering a 0.1% risk acceptance (***)

Refs: nodejs/performance#106

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to the fs subsystem / file system. needs-ci PRs that need a full CI run. labels Sep 26, 2023
@plurispluris changed the title fs: improve error preformance fori fsyncSyncfs: improve error performance for fsyncSyncSep 26, 2023
Comment threadbenchmark/fs/bench-fsyncSync.js Outdated
Comment threadbenchmark/fs/bench-fsyncSync.js Outdated
Comment threadbenchmark/fs/bench-fsyncSync.js Outdated
Comment threadsrc/node_file.cc Outdated
Comment threadlib/internal/fs/sync.js Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: If you moved this function to c++, it would improve the happy path (existing file)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good opinion!
I think you just need to add CHECK_GE(fd,0) to C++.
Do you have a better idea?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to replicate the behavior of getValidatedFd in C++ and throw the correct error.

const getValidatedFd = hideStackFrames((fd, propName = 'fd') => {
if (ObjectIs(fd, -0)) {
return 0;
}
validateInt32(fd, propName, 0);
return fd;
});

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the implementation in C++.
I'm not sure if this implementation is correct.
also I updated the benchmark.
When I check it locally, the performance is strange.

Please review again.

Comment threadsrc/node_file.cc Outdated
@anonriganonrig added the performance Issues and PRs related to the performance of Node.js. label Sep 26, 2023
Comment threadtypings/internalBinding/fs.d.ts Outdated
@pluris
plurisforce-pushed the perf/fsync branch 2 times, most recently from e1ebe02 to c710f94CompareSeptember 27, 2023 02:43
@pluris
pluris requested a review from anonrigSeptember 27, 2023 07:57
@pluris
plurisforce-pushed the perf/fsync branch 3 times, most recently from 764386f to ad76d06CompareSeptember 27, 2023 08:09
Comment threadsrc/node_file.cc Outdated
Comment threadsrc/node_file.cc Outdated
@anonriganonrig added the commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. label Sep 27, 2023
@anonriganonrig added the request-ci Add this label to start a Jenkins CI on a PR. label Sep 27, 2023
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Sep 27, 2023
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@joyeecheungjoyeecheung left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the diff can be a lot simpler if you just change the sync branch of the original implementations instead of repeating the code in a new binding..(and if you only introduce new bindings, the original sync branch would be dead code..) also I think this breaks --trace-sync-io?

@anonrig

Copy link
Copy Markdown
Member

Hey @pluris can you rebase this pull request? You'll see that sync.js file is now removed and merged into lib/fs.js

@anonrig
anonrig requested a review from tniessenOctober 1, 2023 16:50
Comment threadsrc/node_file.cc Outdated

inline int GetValidatedFd(Environment* env, Local<Value> value) {
if (!value->IsInt32()) {
env->isolate()->ThrowException(ERR_INVALID_ARG_TYPE(

@joyeecheungjoyeecheungOct 2, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be a ValidateInt32 if we want to start moving the validation to C++. Also this is not entirely on-par with the original implementation - the original implementation would print the value in a readable format in the case it's not a number, and this does not give any information about the invalid argument, which would make it harder for users to fix the error. We could also just don't move the error validation code now and leave it in JS if it's not ready to take on implementing proper argument validation + printing in C++.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joyeecheung Thank you for your comment.
You are right. As you said, I think we need to think more about the implementation of GetValidateFd(). First, I will remove this addition to C++ and change it to the original implementation.

@pluris
plurisforce-pushed the perf/fsync branch 2 times, most recently from cfd9354 to fcffde8CompareOctober 2, 2023 14:01
Comment threadtypings/internalBinding/fs.d.ts Outdated
@anonriganonrig added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 4, 2023
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 4, 2023
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Co-authored-by: Joyee Cheung <joyeec9h3@gmail.com>
@anonriganonrig added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 4, 2023
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 4, 2023
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@anonriganonrig added the commit-queue Add this label to land a pull request using GitHub Actions. label Oct 20, 2023
@nodejs-github-botnodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Oct 20, 2023
@nodejs-github-bot
nodejs-github-bot merged commit fbd08ec into nodejs:mainOct 20, 2023
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in fbd08ec

targos pushed a commit that referenced this pull request Oct 23, 2023
PR-URL: #49880
Refs: nodejs/performance#106
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
@targostargos mentioned this pull request Oct 23, 2023
@pluris
pluris deleted the perf/fsync branch November 8, 2023 02:24
targos pushed a commit that referenced this pull request Nov 11, 2023
PR-URL: #49880
Refs: nodejs/performance#106
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
@targostargos mentioned this pull request Nov 12, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++Issues and PRs that require attention from people who are familiar with C++.commit-queue-squashAdd this label to instruct the Commit Queue to squash all the PR commits into the first one.fsIssues and PRs related to the fs subsystem / file system.needs-ciPRs that need a full CI run.performanceIssues and PRs related to the performance of Node.js.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@pluris@nodejs-github-bot@anonrig@joyeecheung