Proposal: changes to rimraf retry options #30580

cjihrig · 2019-11-21T16:23:17Z

The recursive rmdir() currently supports two options:

emfileWait - When EMFILE (and soon ENFILE) is encountered, the operation is retried. The retry logic involves creating a series of timers, where the delay increases by 1ms until emfileWait is reached. This is only used in the asynchronous rimraf, and once it supports ENFILE, the name isn't really accurate anymore. This currently defaults to 1000.
maxBusyTries - This option is used when EBUSY, ENOTEMPTY, or EPERM is encountered. This retry logic involves creating a series of maxBusyTries timers, where the delay increases by 100ms. This is also only used in the asynchronous rimraf. This currently defaults to 3.

I'm proposing the following changes (note that rimraf is still experimental):

Rename maxBusyTries to maxRetries and make the default 0. By making the default 0, I think we can start applying this to the synchronous rimraf. I think retries should be opt-in, particularly for synchronous operations, which can block the event loop for a while. I also think that the async and sync versions should have consistent handling of retry logic.
Remove emfileWait completely, and use the same retry logic consistently.
Introduce a retryDelay option that specifies the amount of time to wait between retry attempts.

The text was updated successfully, but these errors were encountered:

This is part of reworking the rimraf retry logic. Refs: nodejs#30580 PR-URL: nodejs#30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

This commit makes retries an opt-in feature by defaulting to no automatic retries. This will be particularly important once synchronous operations can sleep between attempts. Refs: nodejs#30580 PR-URL: nodejs#30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

This commit removes the emfileWait option. EMFILE errors are now handled the same as any other retriable error. Refs: nodejs#30580 PR-URL: nodejs#30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

This commit adds a retryDelay option to rimraf which configures the amount of time between retry operations. Refs: nodejs#30580 PR-URL: nodejs#30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

Co-authored-by: Thang Tran <[email protected]> Fixes: nodejs#30482 Refs: nodejs#30499 Refs: nodejs#30580 PR-URL: nodejs#30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

This is part of reworking the rimraf retry logic. Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

This commit makes retries an opt-in feature by defaulting to no automatic retries. This will be particularly important once synchronous operations can sleep between attempts. Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

This commit removes the emfileWait option. EMFILE errors are now handled the same as any other retriable error. Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

This commit adds a retryDelay option to rimraf which configures the amount of time between retry operations. Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

Co-authored-by: Thang Tran <[email protected]> Fixes: #30482 Refs: #30499 Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

rimraf should only retry if certain errors are encountered. Additionally, there is no point sleeping if an error occurs on the last try. PR-URL: #30785 Fixes: #30580 Refs: #30569 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Richard Lau <[email protected]>

This commit gives the synchronous version of rimraf the same linear retry logic as the asynchronous version. Prior to this commit, sync rimraf kept retrying the operation as soon as possible until maxRetries was reached. PR-URL: #30785 Fixes: #30580 Refs: #30569 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Richard Lau <[email protected]>

rimraf should only retry if certain errors are encountered. Additionally, there is no point sleeping if an error occurs on the last try. PR-URL: #30785 Fixes: #30580 Refs: #30569 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Richard Lau <[email protected]>

This is part of reworking the rimraf retry logic. Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

This commit makes retries an opt-in feature by defaulting to no automatic retries. This will be particularly important once synchronous operations can sleep between attempts. Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

This commit removes the emfileWait option. EMFILE errors are now handled the same as any other retriable error. Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

This commit adds a retryDelay option to rimraf which configures the amount of time between retry operations. Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

Co-authored-by: Thang Tran <[email protected]> Fixes: #30482 Refs: #30499 Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

This commit gives the synchronous version of rimraf the same linear retry logic as the asynchronous version. Prior to this commit, sync rimraf kept retrying the operation as soon as possible until maxRetries was reached. PR-URL: #30785 Fixes: #30580 Refs: #30569 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Richard Lau <[email protected]>

rimraf should only retry if certain errors are encountered. Additionally, there is no point sleeping if an error occurs on the last try. PR-URL: #30785 Fixes: #30580 Refs: #30569 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Richard Lau <[email protected]>

This is part of reworking the rimraf retry logic. Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

This commit makes retries an opt-in feature by defaulting to no automatic retries. This will be particularly important once synchronous operations can sleep between attempts. Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

This commit removes the emfileWait option. EMFILE errors are now handled the same as any other retriable error. Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

This commit adds a retryDelay option to rimraf which configures the amount of time between retry operations. Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

Co-authored-by: Thang Tran <[email protected]> Fixes: #30482 Refs: #30499 Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

This commit gives the synchronous version of rimraf the same linear retry logic as the asynchronous version. Prior to this commit, sync rimraf kept retrying the operation as soon as possible until maxRetries was reached. PR-URL: #30785 Fixes: #30580 Refs: #30569 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Richard Lau <[email protected]>

rimraf should only retry if certain errors are encountered. Additionally, there is no point sleeping if an error occurs on the last try. PR-URL: #30785 Fixes: #30580 Refs: #30569 Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Richard Lau <[email protected]>

cjihrig mentioned this issue Nov 25, 2019

test: use fs rimraf to refresh tmpdir #30569

Merged

2 tasks

cjihrig mentioned this issue Nov 25, 2019

fs: refactor rimraf retry options #30644

Merged

4 tasks

addaleax pushed a commit that referenced this issue Nov 30, 2019

fs: rename rimraf's maxBusyTries to maxRetries

fa1f87b

This is part of reworking the rimraf retry logic. Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

cjihrig mentioned this issue Dec 4, 2019

fs: add synchronous retries to rimraf #30785

Closed

2 tasks

danbev closed this as completed in 26991d0 Dec 9, 2019

targos pushed a commit that referenced this issue Jan 13, 2020

fs: rename rimraf's maxBusyTries to maxRetries

dfb86f9

This is part of reworking the rimraf retry logic. Refs: #30580 PR-URL: #30644 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Proposal: changes to rimraf retry options #30580

Proposal: changes to rimraf retry options #30580

cjihrig commented Nov 21, 2019

Proposal: changes to rimraf retry options #30580

Proposal: changes to rimraf retry options #30580

Comments

cjihrig commented Nov 21, 2019