core.Promise should not fall back to native Promise when native implementation suffers from V8 bug 4162

[Arnavion]

https://code.google.com/p/v8/issues/detail?id=4162

---

Quoted from that bug:

https://people.mozilla.org/~jorendorff/es6-draft.html#sec-promise-resolve-functions states that when resolving a Promise with a value, it must try to see immediately if the value is a thenable or not, and if it is, invoke it. So the following code:

var resolve; var p = new Promise(function () { resolve = arguments[0]; });
resolve({ then: function() { console.log("Invoking then..."); throw new Error(""); } });

must print "Invoking then..." immediately (not synchronously, but next tick or so).

On Chrome dev (V8 4.5.16) / io.js (V8 4.2.77.20), this prints nothing.

The same thing happens if it was instead resolved with a thenable where even accessing the getter should print something.

resolve(Object.defineProperty({}, "then", { get: function() { console.log("Getting then..."); throw new Error(""); } }));

This will also print nothing (in this case it should be printed synchronously).

I'm guessing V8 is deferring checking whether it's a thenable or not since there are no reactions registered to p with p.then(...).

If I then do

p.then(console.log.bind(console), console.error.bind(console));

now "Invoking then..." / "Getting then..." get printed, and console.error is invoked with the error thrown by the thenable's body. Unfortunately this leads to the second (more serious) bug - that multiple such calls to p.then will print "Invoking then..." / "Getting then..." multiple times! Effectively the thenable's code is getting run once each time a reaction is registered using p.then, which is very bad.

---

(Promise.resolve() has the same bug.)

core-js's own Promise implementation is fine, but since it falls back to native Promise if that is found, it becomes vulnerable to this.

This affects Babel too. For example, compare this example's output in a browser that doesn't support native Promise with Chrome / node 0.12. (The spec requires that "A" and "D" be printed only once, that "D" be printed before "Sync end", and that all the comparisons be true).

---

For the fix, the check for useNative could do this in addition to what it already does:

if (works) {
    var thenableThenGotten = false;
    P.resolve(Object.defineProperty({}, "then", {
        get: function () {
            thenableThenGotten = true;
        }
    }));
    works = thenableThenGotten;
}

[stefanpenner]

Doesn't the spec say to explicitly enqueue the then resolve task? It does say to access the then property synchronously and detect if it is callable or not. But the enquement is deferred.

I may be wrong, just responding from memory and from my phone between flights.

Cc @domenic as he will shed a definitive light on the situation.

[Arnavion]

Yes, getting the then (D) is synchronous. Invoking the then (A) for the purpose of following its resolution is not.

8. Let then be Get(resolution, "then").
9. If then is an abrupt completion, then
9a. Return RejectPromise(promise, then.[[value]]).
10. Let thenAction be then.[[value]].
11. If IsCallable(thenAction) is false, then
11a. Return FulfillPromise(promise, resolution).
12. Perform EnqueueJob ("PromiseJobs", PromiseResolveThenableJob, «‍promise, resolution, thenAction»)

That's why I only said "D" must happen before "Sync end".

(And A or D being printed multiple times, once for each reaction registered with .then(), is the much more serious result of the bug.)

[Arnavion]

For reference, this is Chrome's native Promise's output for that example:

E count2 === 1 : false
Sync end
A Invoking then...
A Invoking then...
D Getting then...
D Getting then...
B count === 1 : false
C count === 1 : false
F count2 === 1 : false
G count2 === 1 : false

Note that all the comparisons are false - both count and count2 are two instead of one (one for each call to p.then() / p.catch()), which is confirmed by how A and D got printed twice. Furthermore, D is printed after "Sync end"

Here's the output from FF Nightly (native Promise):

D Getting then...
E count2 === 1 : true
Sync end
A Invoking then...
F count2 === 1 : true
G count2 === 1 : true
B count === 1 : true
C count === 1 : true

This is the output from core-js 0.9.15:

A Invoking then...
D Getting then...
E count2 === 1 : true
Sync end
B count === 1 : true
C count === 1 : true
F count2 === 1 : true
G count2 === 1 : true

As you can see, A is being printed synchronously which is a bug in core-js (but separate from this one). (This is actually correct behavior per Promises/A+ spec which is perhaps why core-js does it? I don't know.)

[stefanpenner]

Ah. Then misread your bug, and my comment seems to confirm what you describe. So feel free to ignore me :p

[zloirock]

@Arnavion

As you can see, A is being printed synchronously which is a bug in core-js

This is correct behavior by Promises/A+, 2.3.3.1 - 2.3.3.3, looks like you rigth - ES6 Promise uses EnqueueJob here. Initially, core-js Promise core was build on native-promise-only, I fixes most problems, but missed this.

\cc @getify this bug still in native-promise-only

@stefanpenner can you help adapt es6-promise test case for core-js? Maybe I missed something else.

About v8 bug - proposed test based on getter and will not work in ES3 engines, but core-js should use alternative Promise libraries. I think, possible ignore it for old engines, what do you think?

[Arnavion]

This is correct behavior by Promises/A+

Yeah, I figured that was the reason core-js was doing that.

[Arnavion]

(Also, the fact that the order of reactions is different between FF and core-js - F-G-B-C vs B-C-F-G - is also probably because A is getting run too soon, so the fix to call thenable.then asynchronously will probably also fix the wrong order of reactions.)

[Arnavion]

About v8 bug - proposed test based on getter and will not work in ES3 engines

If Object.defineProperty doesn't exist, then it wouldn't be V8, right? I'm okay that the common case of running in V8 with the bug is detected and handled, so I don't think core-js needs to try hard to detect the bug in other runtimes.

Native global Promise shouldn't exist in that case either, so works = isFunction(P) ... would already be false and this code won't run anyway. Even if third-party global Promise is in fact available (maybe because some other Promise library is providing it) then I think it's okay to trust it doesn't have this bug.

[getify]

I'm confused by this thread. Is the assertion that there's a conflict between Promises/A+ and ES6 Promises? If so, that's a pretty significant problem.

Can you please explain a simpler test case of what you think is different between Promises/A+ compliance and ES6 Promises compliance?

[getify]

OK, I think I am digging in and starting to glimpse a little bit what this bug is about. Let me make some observations to check if I understand the concern:

1. "A ... Sync end" is called for by Promises/A+, but there's an assertion that ES6 requires "Sync end ... A" because of its EnqueueJob(..)?

   I'm not sure I'm convinced yet that's exactly what the intent is of the ES6 spec. But further, I'm quite surprised if there is indeed an intentional difference between Promises/A+ (which clearly calls for it to happen synchronously) and ES6. If I change native-promise-only to async call then(..) on a thenable, I'm fairly certain it will break a ton of other Promises/A+ tests. That sequencing is fairly sensitive and I recall it was quite delicate to get all tests to pass.

   Why would ES6 intentionally create a conflict with Promises/A+ compliance? AFAIK, nearly all promises libs conform to Promises/A+. I'm not positive, but I don't see how you can possibly be both Promises/A+ and ES6 compliant, if this is indeed the intended and correct interpretation.

2. "F-G-B-C vs B-C-F-G" -- I'm not sure I understand any reasoning why "F-G-B-C" is seen as the more correct ordering? AFAICT, "B" and "C" are enqueued before "F" and "G", so the order I'd expect is "B-C-F-G". Am I missing something?

   Moreover, other than the fact that the spec implies that "B-C" go into the same internal queue ("PromiseJobs") as "F-G", I don't believe there's any spec required guarantee of inter-promise job ordering. That is, I think an implementation could service the "F-G" first or the "B-C" first, and either way would still be consistent with the spirit of the spec.

   This topic of inter-promise ordering has been contentious in the past on several occasions. Some parties have asserted that ES6 does guarantee this by its naming of the queue, and others say that the queue concept is abstract and up to implementations, and that two separate promises are not necessarily required to use the same queue. "B" has to come before "C", and "F" has to come before "G", but that's because all jobs inside a single promise must be ordered as specified.

   I think it's a really terrible idea to rely on inter-promise ordering.

[zloirock]

After @getify comment I'm confused. FF and V8 Promise invokes .then async, but passes Promises/A+ test suite.

[getify]

Well, I'm confused too. I'm not positive what's supposed to happen here. I do know that I've now tried to defer the then(..) call as suggested, and that causes NPO to fail a bunch of tests.

If I defer it with process.nextTick(..) in node, I get about 20 test failures. If I defer it with setTimeout(..0), I get about 80 test failures.

[getify]

NPO as currently released is Promises/A+ compliant. And it invokes then(..) synchronously, as demonstrated, and as called for in Promises/A+. v8 observably calls then(..) asynchronously. This leaves a couple of possible conclusions:

1. v8 is not actually A+ compliant. Does anyone have a link to how I can run the A+ test suite in v8 (chrome browser) browser? The last I knew (a long while back) there wasn't a solution to needing to be able to do job/microtask scheduling since such an API is not available to user-land code. /cc @smikes
2. Promises/A+ is not testing this sequencing like I thought it was (and thus I'm misunderstanding how I'm reading the tests). Which also means both:
   • A+ should be updated to adhere to ES6 async semantics, and should test for it completely. A divergence here is unacceptable.
   • NPO has some other latent bug/incompat that's being exposed when I try to defer, which causes the test failures instead.

[stefanpenner]

I'm pretty sure rsvp and es6 satisfy both a+ and the spec here. I'll verify when I'm at the office

[zloirock]

@getify https://github.com/zloirock/core-js/tree/master/tests/promises-aplus ?

[getify]

Looking more closely at the A+ test suite starting here, I don't see it testing the sequencing of whether then is called synchronously or asynchronously. I find that fact rather surprising.

So is this that Promises/A+ doesn't care if you call it synchronously or not? Or is it that Promises/A+ expects synchronous calling (as the wording implies) but has no tests to verify it?

@domenic, can you please comment specifically on that?

When I first wrote NPO, I was deeply concerned that there were additional ES6 requirements (the whole Promise API) that Promises/A+ tests were not covering. I enlisted @smikes to suffer through the pain of figuring out a more comprehensive test suite. I don't know what happened with that, but as I said earlier, I thought I recalled it stalled at an impasse around lack of a reliable scheduling API.

However, this issue at present is not one I anticipated: that there is wording in Promises/A+ which is contradicted by ES6, and moreover, that there's wording in Promises/A+ which was not tested.

[Arnavion]

(It is not my intent to turn this bug into a discussion about whether the thenable's then is to be invoked synchronously or asyncronously. V8's Promise is broken because it invokes the thenable's then multiple times, once for each reaction registered to the promise, and that is how I discovered it.)

I'm confused by this thread. Is the assertion that there's a conflict between Promises/A+ and ES6 Promises?

It would seem so. The former asks thenable's then to be gotten and invoked synchronously. The latter asks thenable's then to be gotten synchronously and invoked asynchronously.

v8 is not actually A+ compliant.

As I said in the V8 bug linked in the OP, without looking at the implementation, I'm assuming it defers checking what value a promise was resolved with until evaluating reactions. Such an algorithm would lead to the same observed behavior as the ES6 spec if the resolution was not a thenable, but for thenables it leads to different behavior. Note again that I'm talking about the bug of invoking the thenable's then multiple times. I'm more concerned about that one.

"F-G-B-C vs B-C-F-G" -- I'm not sure I understand any reasoning why "F-G-B-C" is seen as the more correct ordering?

If you believe that p is to invoke the thenable's then asynchronously to follow its resolution, that pushes the resolution of its reactions down by a tick, putting them after F and G's resolutions via p2. p2 was rejected while trying to get the thenable's then, so it happened synchronously, and so F and G's resolutions are queued to the next tick, the same one as where p invokes its thenable's then.

---

I wrote a small Promise implementation overnight by mostly following the ES6 spec verbatim (I ignored some of the subclassing-specific / interop stuff but promises-aplus-tests also doesn't care about that). It gives the same result with the babel example in the OP as Firefox, so there's that. It also passes all promises-aplus-tests. I guess NPO has some other bug that causes 10 tests to fail for you?

[smikes]

One issue was that in order to land more tests of NPO we'd be testing things (internals) NPO is explicitly not compatible with. 

The other side is that when we looked at including all of promises/A+ (over 800 tests if I recall correctly) it didn't look like it was worth the trouble, and we had concerns about bootstrapping. /cc @bterlson @domenic for their input

On Tue, Jun 9, 2015 at 10:22 AM, Kyle Simpson [email protected]
wrote:

Looking more closely at the A+ test suite starting here, I don't see it testing the sequencing of whether then is called synchronously or asynchronously. I find that fact rather surprising.
So is this that Promises/A+ doesn't care if you call it synchronously or not? Or is that Promises/A+ expects synchronous calling (as the wording implies) but has no tests to verify it?
When I first wrote NPO, I was deeply concerned that there were additional ES6 requirements (the whole Promise API) that Promises/A+ tests were not covering. I enlisted @smikes to suffer through the pain of figuring out a more comprehensive test suite. I don't know what happened with that, but as I said earlier, I thought I recalled it stalled at an impasse around lack of a reliable scheduling API.

However, this issue at present is not one I anticipated: that there is wording in Promises/A+ which is contradicted by ES6, and moreover, that there's wording in Promises/A+ which was not tested.

Reply to this email directly or view it on GitHub:
#78 (comment)

[getify]

I've spent all day trying to get NPO to run then(..) async and still be A+ compliant, and I've failed. That doesn't mean it's impossible, it just means I don't know how to resolve the two.

I'm also getting different test suite failures every single time I run it, so that makes it basically impossible to figure out what's wrong. Test suites with timers are bullshit.

Anyway, I guess at this point NPO has a bug but I don't know how to fix it. Perhaps someone could file an issue at the NPO repo about it. But unless someone else can figure out how to fix it, I don't expect there to be any fix any time soon.

[stefanpenner]

I've spent all day trying to get NPO to run then(..) async and still be A+ compliant, and I've failed. That doesn't mean it's impossible, it just means I don't know how to resolve the two.

skimming over either of these two repos, maybe help inspire a solution for you:

• https://github.com/tildeio/rsvp.js
• https://github.com/jakearchibald/es6-promise

[stefanpenner]

The other side is that when we looked at including all of promises/A+ (over 800 tests if I recall correctly) it didn't look like it was worth the trouble, and we had concerns about bootstrapping. /cc @bterlson @domenic for their input

Maybe its worth a collective effort to port promise/a+ to something more test262 friendly? I suspect it may be worth it, as continuing to grow and audit a single compliance suite has nice value.

That being said, it won't be a trivial task.

[domenic]

Maybe its worth a collective effort to port promise/a+ to something more test262 friendly?

The issue is that all the browsers are already running the Promises/A+ tests as part of their test suite and so duplicating that into test262 isn't terribly useful. It has theoretical benefits for e.g. new JS engines but in practice it's not worth the work.

[stefanpenner]

Ah ok. Thanks

[getify]

I don't understand how v8 can have the bug listed in the OP (resolving then twice) and still be A+ (and ES6) compliant? Either there are test failures that are being ignored, or the tests are not actually covering everything as they ostensibly purport to be.

The same way that v8 can be "compliant" but still have that bug seems to be how something like NPO can be compliant but have the sync/async then(..) bug.

Especially since my usage of the A+ test suite is totally falling over on itself every time I run it, I don't trust the tests (or the notion of "compliance") at all anymore. It's theater.

[getify]

It has theoretical benefits for e.g. new JS engines

It also would benefit any library/framework that's trying to actually be fully compliant with ES6 Promises if there weren't multiple scattered test suites to try to keep up with. And of course if the one test suite actually tested everything it was supposed to.

[stefanpenner]

weren't multiple scattered test suites to try to keep up with.

Ya, this would be prevented by a unified test. @domenic i do think it would be worth while, especially since the current state of things may indicate some gap.

Maybe someones intern can start the process?

[domenic]

This thread is pretty toxic so I'd appreciate not being @-mentioned in it any more.

[getify]

I'll bow out too.

[zloirock]

Fixed in 0.9.16.