Haskell-ish left arrow operator, async callbacks

[ghost]

Hello!

This is just a proof of concept for this idea I had. It's a bit like IcedCoffeeScript, but instead of using async/defer, it uses the <- operator to turn sequential code into CPS, just like Haskell. I think it looks "cleaner" and fits more with the rest of CoffeeScript, plus, <- makes a lot of sense, since it's the opposite of -> ;)

This is nowhere as advanced as @maxtaco 's fork, and the code quality isn't that good, but since it's just a proof of concept, here it goes...

So, it turns this:

test = ->
  id = (v,f) ->
    f(v)
  a <- id(1)
  b <- id(a)
  c <- id(b)
  console.log(c)

Into this (generated by CoffeeScript):

(function() {
  var test;

  test = function() {
    var id;
    id = function(v, f) {
      return f(v);
    };
    id(1, function(a) {
      id(a, function(b) {
        id(b, function(c) {
          return console.log(c);
        });
      });
    });
  };

  test();

}).call(this);

As I said, it's not production-quality code. Particularly, the algorithm that searches and replaces nodes downward on node.coffee is hack-ish and has a lot of duplication (the getSiblings should be on Base), but I wanted to keep it short, so...

Oh, you have to rebuild the parser for it to work, btw...

So, what do you folks think? Yay? Nay? Way too close to monad-land?

Thanks!

[MichaelBlume]

I would use the shit out of this syntax, the only thing I'm concerned about is how strongly it assumes that the final argument to the async function is the "standard" callback -- that is, if you had do_request(foo, bar, baz, success_callback, error_callback), you'd be SOL.

[paulmillr]

I say yay because i'm pretty tired of async callback shit in my code. And yes, the syntax is a lot more simplier than @maxtaco's. And it doesn't introduce new keywords.

Just one question — what about async loops?

As for "non-standard" callbacks — I think we should prohibit them.

[paulmillr]

Tests don't pass for me.

$ ./bin/coffee -bpe 'a <- b p e'
a < -b(p(e));

[ghost]

@MichaelBlume That's true. The code already accepts stuff like [a,b] <- request for multiple-parameter callbacks. I thought about accepting nested arrays, like [[req, res], [error_code]] <- do_request(foo, bar, baz) but it looks too ugly and unpractical... Or maybe we can just wrap the callers in something like a bind function to handle errors and provide a more uniform callback interface.

@paulmillr I think you have to rebuild the parser! I didn't include my .js files in it... It's ./bin/cake build:parser (you have to do cake build before and after that, I think)

About async loops, I don't really know how to handle them, you can use <- inside for blocks for async calls, but I don't really know if it's what IcedCoffeeScript does :(

[paulmillr]

Thanks, this works now.

As for multiple-param callbacks, i'm highly in favor of using rounded parentheses: (error, result) <- request

IcedCoffee:

  await 
    for k,i in keywords
      search k, defer out[i]
  doThings...

[michaelficarra]

This syntax reminds me of #1710

[satyr]

Duplicate of #1704?

[paulmillr]

@satyr yea, but I just proposed it and hadn't actually written any code for the problem.

[michaelficarra]

@paulmillr: well, if this is an implementation of that proposal, and the proposal was turned down, why should the implementation be accepted? I think it still might be appropriate to have another discussion about this feature on the grounds that it's probably good to periodically revisit this topic; it is extremely important to some people and would be a huge feature for CS if it could be done right.

[paulmillr]

@michaelficarra I agree with you, let's discuss the feature, especially cons of it. /cc @showell

[MichaelBlume]

This is really the first unobtrusive, simple implementation of this kind of feature that I've seen yet. Most importantly: if you hand me a page of code using this syntax, I can easily work out what the Javascript will look like.

[paulmillr]

@MichaelBlume especially when compared to IcedCoffee which compiles await readFile defer error, data to this

var data, error, __iced_deferrals, __iced_k_noop,
  _this = this;

__iced_k_noop = function() {};

__iced_deferrals = new iced.Deferrals(__iced_k, {});
readFile(__iced_deferrals.defer({
  assign_fn: (function() {
    return function() {
      error = arguments[0];
      return data = arguments[1];
    };
  })(),
  lineno: 0
}));
__iced_deferrals._fulfill();

and requires iced runtime to work.

[maxtaco]

@paulmillr A lot of that generated code is there for features that aren't covered in this new proposal, like async-aware stack-traces and parallel async calls. We use these features frequently in the app we're building.

I would argue this proposal introduces one new keyword, <-, which is analogous to await. If you adopt this current proposal, you'll wind up rewriting a lot of libraries, since the position of the callback across all node libraries varies. If you want to skip this rewrite, you'll wind up introducing a second keyword (like _ in streamline JS), that's analogous to defer.

Finally, if you want to handle rewriting of loops properly, and you don't get runtime support from V8 and/or node and/or your browsers of choice, you need to part with the notion of simple code output.

[csubagio]

I'm afraid that it isn't particularly clear what's going on here at all to me.

does a <- id(1) mean everything after this line is now going to be deferred until id returns? If so, how do you do parallel async calls? What if I wanted to fire 3 id calls and continue when they were all done?

a <- id(1)
b <- id(2)
c <- id(3)

Would that also result in 3 nested deferrals? If so, then the await/defer model is far more exciting to me than this minor sugar for your initial case

id 1, (a) ->
  id a, (b) ->
    id b, (c) ->
      console.log c

Am I right to say that all you've actually saved is a bit of indenting for the trivial case? Unless the syntax is meant to automatically figure out what it can and can't fire in parallel (which seems a daunting task, and ignores cases where you'd like to intentionally not fire things in parallel), then for me this is so trivial as to unfortunately constitute a waste of the interesting <- symbol.

Beyond being limiting that it only works where the callback is the last argument, it's also very unintuitive to me to see the last argument missing in the call to id all together. It just straight up looks obfuscatedly ugly. I understand the desire to make this statement look more like an assignment, but the whole thing just looks fishy now.

Also, pretty much +1 on everything @maxtaco said.

[ghost]

I believe that the main point of <- as suggested here is that it's just syntax sugar. It's not really a silver bullet, able to handle complex errors and optimized infrastructure code. It's just to make what would be async code look more like sequential code without doing crazy transformations. It's more of a end-user thing rather than something for library/framework builders...

I think it's just like postfix if and unless, or list comprehensions: yeah, they're limited (single-line), and you're pretty much just replacing a single level of indentation with a shorter form of the code. You don't absolutely need them to write quality CoffeeScript, but we appreciate expressive syntax...

About this being a duplicate, I had no idea. I tried searching but couldn't find those other issues. It's just something I thought that would be cool, and hacked in a few minutes.

[tcr]

To steal some of IcedCoffee's keywords, here's an attempt at writing multiple callbacks that resume at the same time, implemented as a pure Coffee library:

id 1, defer
id 2, defer
id 3, defer
(a, b, c) <- await

Assuming "defer()" queues callbacks, and "await(cb)" waits for these to finish, and returns all arguments in defer's calling order. This also gives you the benefit of being able to move the callback:

secondArgCallback 5, defer, "more arguments???"
(err, res) <- await

[csubagio]

@timcameronryan that's an interesting take, but that's really only the start of it.

getStuff tag, defer for tag in tags
( ...? ) <- await

It's also pretty badly unDRY: your a, b, c order is a repeat of the order of the defers: changing the defers requires you to reorder that list. That invokes all the ugliness of maintaining an unwieldy printf statement.

That defer and await have to deal with some global data there too, which could lead to bugs starting a long way off from where they break things, say removing an await, but missing one of the defers leading up to it, leaving it in the await stack for the next guy, thereby throwing off the count.

I'd be more inclined to say that in a pure coffescript lib, you might see:

await
  a: [id, 1]
  b: [id, 2]
  c: [id, 3]
, (res) ->
  { a, b, c } = res
  console.log a + b + c

Which looks hideous, because to make this a nice sealed block of stuff:

• to avoid working with globals, you can't automagic the defer keyword: you'll have to let the await function deal with building the function calls
• you want to avoid using positions in an array because you'd have to repeat the order in a brittle fashion in the continuation

The proposed sugar, I suppose could be applied to produce:

( res ) <- await  
  a: [id, 1]
  b: [id, 2]
  c: [id, 3]
{ a, b, c } = res
console.log a + b + c

But that's hardly attacking the problem, imho. The proposed syntax just barely shuffles things around with minimal benefit, and you'd still have to write different versions for dealing with cases where the number of deferred items was unknown.

The problem that the iced coffeescript syntax solves for me is that the await keyword actually produces a block in which:

• all calls to defer know which await block they belong to
• the defer keyword can be placed anywhere in said block, and will accept any number of variables to capture
• the end of the block signals where to actually start waiting

Beyond that though, they've actually started tackling bigger problems like debugging who called the deferred function in the first place when things go wrong, which is where the real area of interest for me is. Those things definitely require that the concept of a callback be more integrated into the language than just a syntactic slight of hand.

I guess what I'm saying is that while you reckon this is a problem on the postfix unless level of functionality, I reckon this is actually up at the super or class level. By upgrading await and defer to keyword status, you can write much clearer, much more maintainable code.

await 
  id 1, defer a 
  id 2, defer b
  id 3, defer c

await id 1, defer a
await id 2, defer b
await id 3, defer c

await 
  id 1, defer a 
  id 2, defer b
await id 3, defer c

As for what that produces in javascript terms, well I suppose that's negotiable. Looking at iced's output, I have no trouble buying why it's done what it's done: it broke down the problem step by step and has had to construct temporary functions and objects to translate what you've asked for into javascript, while adding its debugging features along for the ride. If you look at it, in principal it's just a grander, more flexible, more complete vision for what this pull request attempts to do.

[ghost]

@timcameronryan IMO, using <- for parallel async callbacks is complicated... it would completely take away the simplicity, require external libraries, complex code transforms and duh, IcedCoffeeScript already covers it perfectly, is already used in production, and is just a merge away.

I think the selling point of the <- proposal isn't the operator, it is just about generating simple code - even though the code that my patch generates is still a bit off and it's missing a couple return.

But since we're having fun bikeshedding and the <- syntax apparently gets a lot of love, you could just reserve another hypothetical operator (aaargh yes yes I know sorry) and write something like that:

a <~ id 1
b <~ id 2
c <~ id 3    # execute those three in parallel
d <- sum a b c
console.log d   # returns 6 I think

O_O

[tcr]

@csubagio @shf I agree. My point wasn't to poorly reimplement IcedCoffee, but see what complex behaviors could be trivially implemented when enabled by the <- operator. Of course after a point it's duplicate effort; but if the <- operator were (hypothetically) in Coffee, I might need parallelization code in situations I couldn't instead use the Iced runtime.

[meisl]

@shf so with <~ there'd be some kind of implicit barriers? Pretty high level of abstraction...
How would you have more than one barrier, e.g. a1 <~ foo a2 <~ bar b1 <~ qux b2 <~ qmbl # all independent up to here a <~ expensive a1 a2 # need not wait for b1 or b2 b <~ expensive b1 b2 # need not wait for a1 or a2, but should be paralleled with computation of a console.log (combine a b)
?

[ghost]

@meisl Well, this is why I think that the <- syntax is only suitable for simpler operations and IcedCoffeeScript is also suitable for complex stuff... plus, something like <~ would also need additional libraries, it would be just a poor re-implementations as tim said, etc etc...

[jashkenas]

I'm afraid that -- while it's great to revisit these issues from time to time -- this current conversation mostly re-treads earlier attempts to add simple async sugar to CoffeeScript ... and suffers from precisely the same issues, some of which @maxtaco mentions. For a more complete version of this sort of thing, see what Kaffeine does with !:

http://weepy.github.com/kaffeine/docs/unwrapping_async_calls.html