timers: introduce setInterval async iterator #37153

Linkgoron · 2021-01-31T00:03:52Z

Added setInterval async generator to timers\promises. Utilises async generators to provide an iterator compatible with
for await. This is a continuation of initial work by Fabian Cook #35841 (who is also marked as co-author in this PR). The async-generator throws an AbortError when aborted.

Note that there are some decisions I've made regarding (so-called) backpressure. If multiple iterations were completed before a value is yielded, the next iteration won't wait delay long, but yield the next value in the next tick. In addition, if there was backpressure before the controller was aborted, the generator will emit all of the passed intervals, and only then finish the iteration. I'd be happy to change this behavior if this is not what's expected (i.e. change the generator to stop producing immediately).

See the previous PR for more information and discussions.

Example:

const {
  setInterval
} = require('timers/promises');

const ac = new AbortController();
const { signal } = ac;
const interval = 10;
setTimeout(() => ac.abort(), 500);

let i=0;
for await (const value of setInterval(interval, undefined, { ref: false, signal })) {
  console.log(Date.now(), i);
  if(i++>=5) {
     break;
  }
}

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Linkgoron · 2021-01-31T00:52:12Z

cc @benjamingr

lib/timers.js

lib/timers/promises.js

test/parallel/test-timers-promisified.js

lib/timers/promises.js

aduh95 · 2021-01-31T01:42:56Z

lib/timers/promises.js

+      if (abortCallback) {
+        abortCallback(new AbortError());
+        passCallback = undefined;
+        abortCallback = undefined;
+      }


Assigning passCallback and abortCallback seems unnecessary here.

Suggested change

if (abortCallback) {

abortCallback(new AbortError());

passCallback = undefined;

abortCallback = undefined;

}

abortCallback?.(new AbortError());

the undefinds are there because otherwise we sometimes get an error during the tests because of multiple resolves (as we could resolve the promise and then someone aborts the iterator).

Could you move those to after the await new Promise to avoid the repetition? No strong feeling though, if you prefer as it is now, feel free to disregard, I'd personally find it easier to follow if it were down there.

Actually, that's where I originally put it. However, it's incorrect because you could still resolve and reject before the await yields, and get multiple resolves.

aduh95 · 2021-01-31T01:44:38Z

lib/timers/promises.js

+      // eslint-disable-next-line no-undef
+      clearInterval(interval);


Shouldn't we import this from timers?

I could, I just saw that clearTimeout also wasn't imported (I actually copied the exact same comment from that one)

aduh95 · 2021-01-31T01:46:30Z

lib/timers/promises.js

+    if (passCallback) {
+      passCallback();
+      passCallback = undefined;
+      abortCallback = undefined;
+    }


Again, assigning the variables to undefined seems unnecessary

Suggested change

if (passCallback) {

passCallback();

passCallback = undefined;

abortCallback = undefined;

}

passCallback?.();

the undefinds are there because otherwise we sometimes get an error during the tests because of multiple resolves (as we could resolve the promise and then someone aborts the iterator)

aduh95 · 2021-01-31T01:49:13Z

lib/timers/promises.js

+  }
+
+  while (!signal?.aborted) {
+    if (notYielded === 0) {


Can we get rid of the if statement by moving the for loop above the await new Promise?

I think that we would get an issue like the following:
suppose that the promise was fulfilled, and before we got back to it the interval was somehow aborted. we'll go back to the while condition and get out of the while loop, instead of yielding the value.

doc/api/timers.md

lib/timers/promises.js

benjamingr

Good job, left a few comments!

targos · 2021-01-31T09:45:20Z

Is it possible to break out of the loop to stop the interval (there doesn't seem to be a test for that case)?

benjamingr · 2021-01-31T09:59:29Z

Is it possible to break out of the loop to stop the interval

A test for that case is a good idea, I think indeed a finally clause clearing the interval is missing here.

benjamingr · 2021-01-31T15:59:42Z

@Linkgoron I think what Michael meant is that you can break inside a for await loop and that calls .return on the async generator.

aduh95 · 2021-01-31T16:18:14Z

lib/timers/promises.js

+        await new Promise((resolve, reject) => {
+          passCallback = resolve;
+          abortCallback = reject;
+        });


Suggested change

});

});

passCallback = undefined;

abortCallback = undefined;

aduh95 · 2021-01-31T16:20:45Z

lib/timers/promises.js

+      if (abortCallback) {
+        abortCallback(new AbortError());
+        passCallback = undefined;
+        abortCallback = undefined;
+      }


Could you move those to after the await new Promise to avoid the repetition? No strong feeling though, if you prefer as it is now, feel free to disregard, I'd personally find it easier to follow if it were down there.

Linkgoron · 2021-01-31T22:06:53Z

@Linkgoron I think what Michael meant is that you can break inside a for await loop and that calls .return on the async generator.

Oh. I didn't know that! Anyway, I'll update the PR soon.

benjamingr

Thanks, good job. I think this is good enough to land (the module is experimental) and iron out small nits later :]

test/parallel/test-timers-promisified.js

doc/api/timers.md

targos · 2021-02-01T10:30:59Z

Just a thought: could we easily make the iterated values resolve to the number of iterations?

benjamingr · 2021-02-01T10:39:22Z

@targos I would prefer that as well - but James raised the point in the original PR that setTimeout from timers/promises already does this with the value parameter.

nodejs-github-bot · 2021-02-01T14:00:23Z

CI: https://ci.nodejs.org/job/node-test-pull-request/35856/

nodejs-github-bot · 2021-02-02T17:48:35Z

CI: https://ci.nodejs.org/job/node-test-pull-request/35892/

nodejs-github-bot · 2021-02-02T18:47:34Z

CI: https://ci.nodejs.org/job/node-test-pull-request/35898/

nodejs-github-bot · 2021-02-02T21:59:05Z

CI: https://ci.nodejs.org/job/node-test-pull-request/35903/

nodejs-github-bot · 2021-02-03T08:39:09Z

CI: https://ci.nodejs.org/job/node-test-pull-request/35915/

benjamingr · 2021-02-03T11:35:17Z

Ping @devsnek @ronag @jasnell @aduh95 any chance this could get a second review to not have to wait a week :]?

benjamingr · 2021-02-03T17:28:22Z

Landed in fefc639 🎉

Added setInterval async generator to timers\promises. Utilises async generators to provide an iterator compatible with `for await`. Co-Authored-By: Fabian Cook <[email protected]> fix message PR-URL: #37153 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: James M Snell <[email protected]>

Trott · 2021-02-04T13:15:57Z

test/parallel/test-timers-promisified.js

+  {
+    // Check that if we abort when we have some callbacks left,
+    // we actually call them.
+    const controller = new AbortController();
+    const { signal } = controller;
+    const delay = 10;
+    let totalIterations = 0;
+    const timeoutLoop = runInterval(async (iterationNumber) => {
+      if (iterationNumber === 2) {
+        await setTimeout(delay * 2);
+        controller.abort();
+      }
+      if (iterationNumber > totalIterations) {
+        totalIterations = iterationNumber;
+      }
+    }, delay, signal);
+
+    timeoutLoop.catch(common.mustCall(() => {
+      assert.ok(totalIterations >= 3, `iterations was ${totalIterations} < 3`);
+    }));
+  }
+}


Unfortunately, this test is unreliable and is failing in CI on Raspberry Pi devices. I also can make it fail trivially on my macOS laptop by running tools/test.py -j96 --repeat=192 test/parallel/test-timers-promisified.js. I haven't looked closely (yet) but in my experience, having a magic number delay variable like this in a timers test indicates an assumption that the host isn't so slow that (say) a 10ms second timer won't fire 50ms later. This assumption is, of course, incorrect if the machine has low CPU/memory (like a Raspberry Pi device) or if there are a lot of other things competing for the machine's resources (-j96 --repeat=192).

Sorry for causing an issue - I'll take a look and fix it

When implementing this, I was under the assumption that if we I have an interval and a timeout where the interval starts before a timeout and is supposed to execute before the timeout executes, the interval's callback would always execute before the timeout's callback. Either this assumption was wrong, or something in my setInterval implementation is incorrect. (i.e. I thought that the setTimeout would always run after the internal setInterval)

Added setInterval async generator to timers\promises. Utilises async generators to provide an iterator compatible with `for await`. Co-Authored-By: Fabian Cook <[email protected]> fix message PR-URL: #37153 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: James M Snell <[email protected]>

Notable Changes: * crypto: * add keyObject.export() 'jwk' format option (Filip Skokan) #37081 * deps: * upgrade to libuv 1.41.0 (Colin Ihrig) #37360 * doc: * add dmabupt to collaborators (Xu Meng) #37377 * refactor fs docs structure (James M Snell) #37170 * fs: * add fsPromises.watch() (James M Snell) #37179 * use a default callback for fs.close() (James M Snell) #37174 * add AbortSignal support to watch (Benjamin Gruenbaum) #37190 * perf_hooks: * introduce createHistogram (James M Snell) #37155 * stream: * improve Readable.from error handling (Benjamin Gruenbaum) #37158 * timers: * introduce setInterval async iterator (linkgoron) #37153 * tls: * add ability to get cert/peer cert as X509Certificate object (James M Snell) #37070

Notable Changes: * crypto: * add keyObject.export() jwk format option (Filip Skokan) #37081 * deps: * upgrade to libuv 1.41.0 (Colin Ihrig) #37360 * doc: * add dmabupt to collaborators (Xu Meng) #37377 * refactor fs docs structure (James M Snell) #37170 * fs: * add fsPromises.watch() (James M Snell) #37179 * use a default callback for fs.close() (James M Snell) #37174 * add AbortSignal support to watch (Benjamin Gruenbaum) #37190 * perf_hooks: * introduce createHistogram (James M Snell) #37155 * stream: * improve Readable.from error handling (Benjamin Gruenbaum) #37158 * timers: * introduce setInterval async iterator (linkgoron) #37153 * tls: * add ability to get cert/peer cert as X509Certificate object (James M Snell) #37070

PR-URL: #37406 Notable Changes: * crypto: * add keyObject.export() jwk format option (Filip Skokan) #37081 * deps: * upgrade to libuv 1.41.0 (Colin Ihrig) #37360 * doc: * add dmabupt to collaborators (Xu Meng) #37377 * refactor fs docs structure (James M Snell) #37170 * fs: * add fsPromises.watch() (James M Snell) #37179 * use a default callback for fs.close() (James M Snell) #37174 * add AbortSignal support to watch (Benjamin Gruenbaum) #37190 * perf_hooks: * introduce createHistogram (James M Snell) #37155 * stream: * improve Readable.from error handling (Benjamin Gruenbaum) #37158 * timers: * introduce setInterval async iterator (linkgoron) #37153 * tls: * add ability to get cert/peer cert as X509Certificate object (James M Snell) #37070

targos · 2021-07-19T13:42:56Z

I'm marking this dont-land-on-v14.x because timers/promises is not available on v14.x.

nodejs-github-bot added the timers Issues and PRs related to the timers subsystem / setImmediate, setInterval, setTimeout. label Jan 31, 2021

Linkgoron force-pushed the timers-set-interval-async branch 3 times, most recently from 45584a1 to 42a7f52 Compare January 31, 2021 00:32

aduh95 reviewed Jan 31, 2021

View reviewed changes

Linkgoron force-pushed the timers-set-interval-async branch 2 times, most recently from 2cb80b2 to 1f8b21c Compare January 31, 2021 03:06