fs: add mkstemp functions #33549

targos · 2020-05-25T07:20:32Z

fs.mkstemp and fs.mkstempSync return path and fd.
fsPromises.mkstemp returns path and handle.

@nodejs/fs

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

* fs.mkstemp and fs.mkstempSync return `path` and `fd`. * fsPromises.mkstemp returns `path` and `handle`.

targos · 2020-05-25T10:47:22Z

/cc @addaleax

src/node_file.cc

vdeturckheim

LGTM

nodejs-github-bot · 2020-05-25T11:58:51Z

CI: https://ci.nodejs.org/job/node-test-pull-request/31552/

doc/api/fs.md

test/parallel/test-fs-mkstemp.js

benjamingr · 2020-05-25T15:29:29Z

I think a disposer API for the promises side would have been nicer. I maintain tmp-promise and in my experience that's the more popular API.

I'd also steal some tests from tmp but that can be in a follow up PR :]

doc/api/fs.md

Co-authored-by: Benjamin Gruenbaum <[email protected]>

doc/api/fs.md

targos · 2020-05-25T15:35:59Z

I think a disposer API for the promises side would have been nicer. I maintain tmp-promise and in my experience that's the more popular API.

I'd also steal some tests from tmp but that can be in a follow up PR :]

I don't understand what you are suggesting, sorry.

benjamingr · 2020-05-25T15:37:50Z

@targos in tmp-promise there are two APIs, one is similar to the one outlined here and the other looks a bit like

fs.promises.withMkstemp(async (file) => {
  // when the promise returned from this async function settles, file is automatically closed
});

This makes for better error handling most of the time. I am not saying we should ship this sort of API or that if we do we shouldn't ship the one already in this PR - only that in my (anecdotal) experience as a maintainer of tmp-promise it's the more popular one.

doc/api/fs.md

targos · 2020-05-25T17:05:43Z

Test failed on AIX: https://ci.nodejs.org/job/node-test-commit-aix/30706/nodes=aix71-ppc64/testReport/junit/(root)/test/parallel_test_fs_error_messages/

Any idea why?

nodejs-github-bot · 2020-05-29T19:04:43Z

CI: https://ci.nodejs.org/job/node-test-pull-request/31634/

targos · 2020-05-30T09:02:03Z

@nodejs/platform-aix PTAL

richardlau · 2020-05-30T11:30:26Z

I can take a look on Monday but my suspicion would be that the value of the template (i.e. req->path) passed into the mkstemp C call on completion is undefined in the error case.

bnoordhuis

LGTM. The AIX failure looks like it's happening on the libuv or libc level. Needs further investigation but I don't think it's caused by this PR.

bnoordhuis · 2020-05-30T11:53:16Z

That sounds plausible. POSIX allows it to be unspecified. I'll open a libuv PR to clobber req->path on error to flush out such applications bugs.

richardlau · 2020-06-01T10:25:19Z

I can take a look on Monday but my suspicion would be that the value of the template (i.e. req->path) passed into the mkstemp C call on completion is undefined in the error case.

Yes, looks like that's the case here:

diff --git a/deps/uv/src/unix/fs.c b/deps/uv/src/unix/fs.c
index f5b2b94207..ce9683cb51 100644
--- a/deps/uv/src/unix/fs.c
+++ b/deps/uv/src/unix/fs.c
@@ -336,7 +336,9 @@ static int uv__fs_mkstemp(uv_fs_t* req) {
   if (req->cb != NULL)
     uv_rwlock_rdlock(&req->loop->cloexec_lock);

+printf("before mkstemp, path: %s\n", path);
   r = mkstemp(path);
+printf("after mkstemp, path: %s\n", path);

   /* In case of failure `uv__cloexec` will leave error in `errno`,
    * so it is enough to just set `r` to `-1`.

bash-4.4$ ./tools/test.py parallel/test-fs-error-messages
=== release test-fs-error-messages ===
Path: parallel/test-fs-error-messages
before mkstemp, path: /home/riclau/sandbox/github/nodejs/test/.tmp.0/non-existent/foo/barXXXXXX
after mkstemp, path:
before mkstemp, path: /home/riclau/sandbox/github/nodejs/test/.tmp.0/non-existent/foo/barXXXXXX
after mkstemp, path:
(node:13172966) internal/test/binding: These APIs are for internal testing only. Do not use them.
(Use `node --trace-warnings ...` to show where the warning was created)
assert.js:920
    throw err;
    ^

AssertionError [ERR_ASSERTION]: The input did not match the regular expression /^\/home\/riclau\/sandbox\/github\/nodejs\/test\/\.tmp\.0\/non\-existent\/foo\/bar/. Input:

''

    at validateError (/home/riclau/sandbox/github/nodejs/test/parallel/test-fs-error-messages.js:670:12)
    at /home/riclau/sandbox/github/nodejs/test/common/index.js:363:15
    at fs.js:163:23
    at FSReqCallback.oncomplete (fs.js:1893:14) {
  generatedMessage: true,
  code: 'ERR_ASSERTION',
  actual: '',
  expected: /^\/home\/riclau\/sandbox\/github\/nodejs\/test\/\.tmp\.0\/non\-existent\/foo\/bar/,
  operator: 'match'
}
Command: out/Release/node --expose-internals /home/riclau/sandbox/github/nodejs/test/parallel/test-fs-error-messages.js
[00:00|% 100|+   0|-   1]: Done
bash-4.4$

POSIX doesn't specify what happens to the template string in the error case: https://pubs.opengroup.org/onlinepubs/9699919799/functions/mkdtemp.html

The Linux documentation also points out that the template may be undefined for some errors (EEXIST): https://www.man7.org/linux/man-pages/man3/mkstemp.3.html

So it's not safe to assume the contents of the template (req->path) if the mkstemp call fails as the test is currently doing. If we deem it useful from a debugging/errors point of view in Node.js we should probably store a copy of the template before making the call into libuv.

targos · 2020-07-14T07:57:29Z

@bnoordhuis I'm not sure anymore. Is this blocked on work that should be done on the libuv side, or should I do something here to fix the issue?

bnoordhuis · 2020-07-14T09:56:07Z

@targos There's still some work to be done in libuv but nothing critical, it doesn't have to hold up this PR. I'd go with Richard's suggestion:

we should probably store a copy of the template before making the call into libuv

targos · 2020-07-14T12:52:55Z

I'd go with Richard's suggestion:

we should probably store a copy of the template before making the call into libuv

Okay, can you help me do it? I don't know how I could access the value from the AfterMkstemp callback.

Contents of template variable passed for posix call mkstemp on error code EINVAL is unknown. On AIX platform, template will get clobbered on EINVAL and any attempt to read template might result in error. In libuv, req->path is passed directly to the mkstemp call and behavior of this string on error is platform dependent. To avoid portability issues, it's better to have a common behavior on all platform. For both unix and windows platform libuv will rewrite path with an empty string on all error cases. Fixes: #2913 Refs: nodejs/node#33549 Refs: #2933 PR-URL: #2938 Reviewed-By: Jameson Nash <[email protected]>

aduh95 · 2020-10-16T20:39:41Z

@targos does the change made to libuv by @vtjnash help unblock this?

targos · 2020-10-17T10:30:07Z

It doesn't. I still need help for #33549 (comment)

Contents of template variable passed for posix call mkstemp on error code EINVAL is unknown. On AIX platform, template will get clobbered on EINVAL and any attempt to read template might result in error. In libuv, req->path is passed directly to the mkstemp call and behavior of this string on error is platform dependent. To avoid portability issues, it's better to have a common behavior on all platform. For both unix and windows platform libuv will rewrite path with an empty string on all error cases. Fixes: libuv#2913 Refs: nodejs/node#33549 Refs: libuv#2933 PR-URL: libuv#2938 Reviewed-By: Jameson Nash <[email protected]>

mschfh · 2023-08-22T03:20:51Z

it doesn't have to hold up this PR

If we deem it useful from a debugging/errors point of view in Node.js we should probably store a copy of the template before making the call into libuv.

Could this PR be merged and the enhancement be added later on?

BrianJDrake · 2023-10-14T06:23:58Z

This pull request would complete the old feature request #5332.

targos · 2023-10-17T09:11:12Z

Anyone feel free to take this over. I'm not able to fix the problems and I don't need the feature myself.

fs: add mkstemp functions

8ea7bb6

* fs.mkstemp and fs.mkstempSync return `path` and `fd`. * fsPromises.mkstemp returns `path` and `handle`.

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. labels May 25, 2020

targos added fs Issues and PRs related to the fs subsystem / file system. semver-minor PRs that contain new features and should be released in the next minor version. and removed lib / src Issues and PRs related to general changes in the lib or src directory. labels May 25, 2020

addaleax approved these changes May 25, 2020

View reviewed changes

src/node_file.cc Outdated Show resolved Hide resolved

vdeturckheim approved these changes May 25, 2020

View reviewed changes

targos added 2 commits May 25, 2020 13:45

fixup! fs: add mkstemp functions

308b6f9

fixup! fixup! fs: add mkstemp functions

e047588

This comment has been minimized.

Sign in to view

fixup! fixup! fixup! fs: add mkstemp functions

744a476

This comment has been minimized.

Sign in to view

mscdex reviewed May 25, 2020

View reviewed changes

doc/api/fs.md Outdated Show resolved Hide resolved

mscdex reviewed May 25, 2020

View reviewed changes

doc/api/fs.md Outdated Show resolved Hide resolved

targos commented May 25, 2020

View reviewed changes

test/parallel/test-fs-mkstemp.js Outdated Show resolved Hide resolved

cjihrig approved these changes May 25, 2020

View reviewed changes

devnexen approved these changes May 25, 2020

View reviewed changes

fixup! fixup! fixup! fixup! fs: add mkstemp functions

bf9ce89

benjamingr closed this May 25, 2020

benjamingr reopened this May 25, 2020

benjamingr reviewed May 25, 2020

View reviewed changes

doc/api/fs.md Outdated Show resolved Hide resolved

Update doc/api/fs.md

f57b726

Co-authored-by: Benjamin Gruenbaum <[email protected]>

benjamingr reviewed May 25, 2020

View reviewed changes

doc/api/fs.md Show resolved Hide resolved

targos commented May 25, 2020

View reviewed changes

doc/api/fs.md Show resolved Hide resolved

jasnell approved these changes May 25, 2020

View reviewed changes

bnoordhuis approved these changes May 30, 2020

View reviewed changes

targos added the blocked PRs that are blocked by other issues or PRs. label May 31, 2020

BridgeAR force-pushed the master branch 2 times, most recently from 8ae28ff to 2935f72 Compare May 31, 2020 12:18

juanarbol approved these changes Jun 1, 2020

View reviewed changes

targos mentioned this pull request Jun 15, 2020

add fs.mkstemp (lib_uv recently landed uv_fs_mkstemp #21) #33890

Closed

addaleax removed the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Jun 19, 2020

bnoordhuis mentioned this pull request Jul 14, 2020

fs: clobber req->path on uv_fs_mkstemp() error libuv/libuv#2913

Closed

targos added the help wanted Issues that need assistance from volunteers or PRs that need help to proceed. label Jul 17, 2020

guybedford force-pushed the master branch from dc5a5da to 8e46568 Compare March 29, 2021 21:33

Trott force-pushed the main branch from 2d76238 to ca3ed36 Compare November 12, 2022 01:49

BrianJDrake mentioned this pull request Oct 14, 2023

Securely create temporary directories and files using fsPromises.mkdtemp() bytecodealliance/jco#191

Merged

targos closed this Oct 17, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fs: add mkstemp functions #33549

fs: add mkstemp functions #33549

targos commented May 25, 2020

targos commented May 25, 2020

vdeturckheim left a comment

This comment has been minimized.

This comment has been minimized.

nodejs-github-bot commented May 25, 2020

benjamingr commented May 25, 2020

targos commented May 25, 2020

benjamingr commented May 25, 2020

targos commented May 25, 2020

nodejs-github-bot commented May 29, 2020

targos commented May 30, 2020

richardlau commented May 30, 2020

bnoordhuis left a comment

bnoordhuis commented May 30, 2020

richardlau commented Jun 1, 2020

targos commented Jul 14, 2020

bnoordhuis commented Jul 14, 2020

targos commented Jul 14, 2020

aduh95 commented Oct 16, 2020

targos commented Oct 17, 2020

mschfh commented Aug 22, 2023

BrianJDrake commented Oct 14, 2023

targos commented Oct 17, 2023

fs: add mkstemp functions #33549

fs: add mkstemp functions #33549

Conversation

targos commented May 25, 2020

Checklist

targos commented May 25, 2020

vdeturckheim left a comment

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

nodejs-github-bot commented May 25, 2020

benjamingr commented May 25, 2020

targos commented May 25, 2020

benjamingr commented May 25, 2020

targos commented May 25, 2020

nodejs-github-bot commented May 29, 2020

targos commented May 30, 2020

richardlau commented May 30, 2020

bnoordhuis left a comment

Choose a reason for hiding this comment

bnoordhuis commented May 30, 2020

richardlau commented Jun 1, 2020

targos commented Jul 14, 2020

bnoordhuis commented Jul 14, 2020

targos commented Jul 14, 2020

aduh95 commented Oct 16, 2020

targos commented Oct 17, 2020

mschfh commented Aug 22, 2023

BrianJDrake commented Oct 14, 2023

targos commented Oct 17, 2023