fs: improve error performance for `fsyncSync` #49880

pluris · 2023-09-26T15:30:53Z

                                                  confidence improvement accuracy (*)   (**)   (***)
fs/bench-fsyncSync.js n=10000 type='existing'            ***      3.37 %       ±1.49% ±1.99%  ±2.60%
fs/bench-fsyncSync.js n=10000 type='non-existing'        ***    116.95 %       ±6.64% ±8.87% ±11.64%

Be aware that when doing many comparisons the risk of a false-positive result increases.
In this case, there are 2 comparisons, you can thus expect the following amount of false-positive results:
  0.10 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.02 false positives, when considering a   1% risk acceptance (**, ***),
  0.00 false positives, when considering a 0.1% risk acceptance (***)

Refs: nodejs/performance#106

benchmark/fs/bench-fsyncSync.js

src/node_file.cc

anonrig · 2023-09-26T15:40:16Z

lib/internal/fs/sync.js

@@ -88,6 +88,11 @@ function close(fd) {
  return binding.closeSync(fd);
 }

+function fsync(fd) {
+  fd = getValidatedFd(fd);


Nit: If you moved this function to c++, it would improve the happy path (existing file)

That's a good opinion!
I think you just need to add CHECK_GE(fd,0) to C++.
Do you have a better idea?

We need to replicate the behavior of getValidatedFd in C++ and throw the correct error.

const getValidatedFd = hideStackFrames((fd, propName = 'fd') => { if (ObjectIs(fd, -0)) { return 0; } validateInt32(fd, propName, 0); return fd; });

I added the implementation in C++.
I'm not sure if this implementation is correct.
also I updated the benchmark.
When I check it locally, the performance is strange.

Please review again.

src/node_file.cc

typings/internalBinding/fs.d.ts

src/node_file.cc

nodejs-github-bot · 2023-09-27T15:14:58Z

CI: https://ci.nodejs.org/job/node-test-pull-request/54293/

joyeecheung

I think the diff can be a lot simpler if you just change the sync branch of the original implementations instead of repeating the code in a new binding..(and if you only introduce new bindings, the original sync branch would be dead code..) also I think this breaks --trace-sync-io?

anonrig · 2023-09-30T19:39:29Z

Hey @pluris can you rebase this pull request? You'll see that sync.js file is now removed and merged into lib/fs.js

joyeecheung · 2023-10-02T13:38:01Z

src/node_file.cc

@@ -116,6 +116,24 @@ inline int64_t GetOffset(Local<Value> value) {
  return IsSafeJsInt(value) ? value.As<Integer>()->Value() : -1;
 }

+inline int GetValidatedFd(Environment* env, Local<Value> value) {
+  if (!value->IsInt32()) {
+    env->isolate()->ThrowException(ERR_INVALID_ARG_TYPE(


I think this should be a ValidateInt32 if we want to start moving the validation to C++. Also this is not entirely on-par with the original implementation - the original implementation would print the value in a readable format in the case it's not a number, and this does not give any information about the invalid argument, which would make it harder for users to fix the error. We could also just don't move the error validation code now and leave it in JS if it's not ready to take on implementing proper argument validation + printing in C++.

@joyeecheung Thank you for your comment.
You are right. As you said, I think we need to think more about the implementation of GetValidateFd(). First, I will remove this addition to C++ and change it to the original implementation.

typings/internalBinding/fs.d.ts

nodejs-github-bot · 2023-10-04T14:37:01Z

CI: https://ci.nodejs.org/job/node-test-pull-request/54530/

Co-authored-by: Joyee Cheung <[email protected]>

nodejs-github-bot · 2023-10-04T15:30:49Z

CI: https://ci.nodejs.org/job/node-test-pull-request/54533/

nodejs-github-bot · 2023-10-09T21:22:26Z

CI: https://ci.nodejs.org/job/node-test-pull-request/54637/

nodejs-github-bot · 2023-10-12T01:40:40Z

CI: https://ci.nodejs.org/job/node-test-pull-request/54732/

nodejs-github-bot · 2023-10-20T23:39:16Z

Landed in fbd08ec

PR-URL: #49880 Refs: nodejs/performance#106 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Joyee Cheung <[email protected]>

PR-URL: nodejs#49880 Refs: nodejs/performance#106 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Joyee Cheung <[email protected]>

PR-URL: #49880 Refs: nodejs/performance#106 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Joyee Cheung <[email protected]>

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to the fs subsystem / file system. needs-ci PRs that need a full CI run. labels Sep 26, 2023

pluris changed the title ~~fs: improve error preformance fori fsyncSync~~ fs: improve error performance for fsyncSync Sep 26, 2023

pluris force-pushed the perf/fsync branch from e6a80ee to fb0eeb8 Compare September 26, 2023 15:35

anonrig mentioned this pull request Sep 26, 2023

Performance of node:fs nodejs/performance#106

Open

anonrig reviewed Sep 26, 2023

View reviewed changes

anonrig added the performance Issues and PRs related to the performance of Node.js. label Sep 26, 2023

pluris force-pushed the perf/fsync branch from 05cce21 to e0702cf Compare September 26, 2023 16:15

anonrig reviewed Sep 26, 2023

View reviewed changes

typings/internalBinding/fs.d.ts Outdated Show resolved Hide resolved

pluris force-pushed the perf/fsync branch 2 times, most recently from e1ebe02 to c710f94 Compare September 27, 2023 02:43

pluris requested a review from anonrig September 27, 2023 07:57

pluris force-pushed the perf/fsync branch 3 times, most recently from 764386f to ad76d06 Compare September 27, 2023 08:09

anonrig reviewed Sep 27, 2023

View reviewed changes

src/node_file.cc Outdated Show resolved Hide resolved

src/node_file.cc Outdated Show resolved Hide resolved

anonrig added the commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. label Sep 27, 2023

pluris force-pushed the perf/fsync branch from 09b5de3 to e769fc5 Compare September 27, 2023 14:17

anonrig approved these changes Sep 27, 2023

View reviewed changes

anonrig added the request-ci Add this label to start a Jenkins CI on a PR. label Sep 27, 2023

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Sep 27, 2023

This was referenced Sep 27, 2023

fs: improve error performance for fdatasyncSync #49898

Merged

src: add GetValidatedFd() to node_file #49903

Closed

joyeecheung reviewed Sep 27, 2023

View reviewed changes

This was referenced Sep 28, 2023

CI Reliability 2023-09-28 nodejs/reliability#677

Open

CI Reliability 2023-09-30 nodejs/reliability#679

Open

anonrig requested a review from tniessen October 1, 2023 16:50

joyeecheung reviewed Oct 2, 2023

View reviewed changes

src: remove GetValidatedFd

fcffde8

pluris force-pushed the perf/fsync branch 2 times, most recently from cfd9354 to fcffde8 Compare October 2, 2023 14:01

pluris requested a review from joyeecheung October 2, 2023 14:11

joyeecheung approved these changes Oct 4, 2023

View reviewed changes

typings/internalBinding/fs.d.ts Outdated Show resolved Hide resolved

anonrig added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 4, 2023

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 4, 2023

Update typings/internalBinding/fs.d.ts

6396e80

Co-authored-by: Joyee Cheung <[email protected]>

anonrig added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 4, 2023

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 4, 2023

This was referenced Oct 10, 2023

CI Reliability 2023-10-10 nodejs/reliability#689

Open

CI Reliability 2023-10-11 nodejs/reliability#690

Open

anonrig approved these changes Oct 12, 2023

View reviewed changes

anonrig added the commit-queue Add this label to land a pull request using GitHub Actions. label Oct 20, 2023

nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Oct 20, 2023

nodejs-github-bot merged commit fbd08ec into nodejs:main Oct 20, 2023
27 checks passed

targos pushed a commit that referenced this pull request Oct 23, 2023

fs: improve error performance for fsyncSync

5942edb

PR-URL: #49880 Refs: nodejs/performance#106 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Joyee Cheung <[email protected]>

targos mentioned this pull request Oct 23, 2023

v21.1.0 release proposal #50335

Merged

alexfernandez pushed a commit to alexfernandez/node that referenced this pull request Nov 1, 2023

fs: improve error performance for fsyncSync

02dff20

PR-URL: nodejs#49880 Refs: nodejs/performance#106 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Joyee Cheung <[email protected]>

pluris deleted the perf/fsync branch November 8, 2023 02:24

targos pushed a commit that referenced this pull request Nov 11, 2023

fs: improve error performance for fsyncSync

eb33f70

PR-URL: #49880 Refs: nodejs/performance#106 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Joyee Cheung <[email protected]>

targos mentioned this pull request Nov 12, 2023

v20.10.0 release proposal #50682

Merged

juanarbol mentioned this pull request Dec 6, 2023

2023-12-06, N|Solid v5.0.1 Iron Release nodesource/nsolid#41

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fs: improve error performance for `fsyncSync` #49880

fs: improve error performance for `fsyncSync` #49880

pluris commented Sep 26, 2023 •

edited

Loading

anonrig Sep 26, 2023

pluris Sep 26, 2023

anonrig Sep 26, 2023

pluris Sep 27, 2023

nodejs-github-bot commented Sep 27, 2023

joyeecheung left a comment

anonrig commented Sep 30, 2023

joyeecheung Oct 2, 2023 •

edited

Loading

pluris Oct 2, 2023

nodejs-github-bot commented Oct 4, 2023

nodejs-github-bot commented Oct 4, 2023

nodejs-github-bot commented Oct 9, 2023

nodejs-github-bot commented Oct 12, 2023

nodejs-github-bot commented Oct 20, 2023

fs: improve error performance for fsyncSync #49880

fs: improve error performance for fsyncSync #49880

Conversation

pluris commented Sep 26, 2023 • edited Loading

anonrig Sep 26, 2023

Choose a reason for hiding this comment

pluris Sep 26, 2023

Choose a reason for hiding this comment

anonrig Sep 26, 2023

Choose a reason for hiding this comment

pluris Sep 27, 2023

Choose a reason for hiding this comment

nodejs-github-bot commented Sep 27, 2023

joyeecheung left a comment

Choose a reason for hiding this comment

anonrig commented Sep 30, 2023

joyeecheung Oct 2, 2023 • edited Loading

Choose a reason for hiding this comment

pluris Oct 2, 2023

Choose a reason for hiding this comment

nodejs-github-bot commented Oct 4, 2023

nodejs-github-bot commented Oct 4, 2023

nodejs-github-bot commented Oct 9, 2023

nodejs-github-bot commented Oct 12, 2023

nodejs-github-bot commented Oct 20, 2023

fs: improve error performance for `fsyncSync` #49880

fs: improve error performance for `fsyncSync` #49880

pluris commented Sep 26, 2023 •

edited

Loading

joyeecheung Oct 2, 2023 •

edited

Loading