Async/Await

[kirsle]

This implements Async/Await support to make async stuff in RiveScript more awesome. This fixes #220.

Backwards Incompatible Changes

• rs.reply() now returns a Promise, not a string. In this regard it behaves like replyAsync()
• rs.replyAsync() is now deprecated. Update your code to call rs.reply() instead.
• rs.loadFile() and rs.loadDirectory() are to use Promises instead of callbacks.

These functions had to change because of how Async/Await works: if a function uses the await keyword, it needs to be an async function and async functions always return Promises. Even if you try to return a value like a string, it gets wrapped in a Promise automatically.

An example for how to update legacy code:

- var reply = rs.reply(username, message, this);
- console.log("Bot>", reply);
+ rs.reply(username, message, this).then(function(reply) {
+    console.log("Bot>", reply);
+ });

You can also update legacy code to await the reply, and continue using it as you always have. But you'll need to change your function signature to be an async function, and so-on up your call stack.

- function getReply(username, message) {
-    var reply = rs.reply(username, message);
+ async function getReply(username, message) {
+    var reply = await rs.reply(username, message);
      return reply;
  }

New Features

Async/Await allows a function to "pause" and wait for a promise to resolve before continuing execution. For RiveScript, this means we can use await all over the place in-line in functions to resolve promises, such as those that might be returned by object macros.

Before v2.0.0-final can be released, the following items must be completed:

• Async object macros should work in *Condition. The condition check will wait for it to resolve, test its truthiness, and continue to the next condition if false.
• Async macros work all over the place. Say goodbye to the [ERR: Using async routine with reply: use replyAsync instead] error messages!
• Pluggable session manager support. This will allow you to replace the in-memory User Variable store with one backed by an async database like MongoDB, MySQL or Redis.
  • This PR will contain at least the interface for session managers. The implementations may come in later pull requests. They'll probably ship on npm as separate packages similar to rivescript-contrib-coffeescript
• Replace all callback-based APIs with promise-based ones. loadDirectory(), loadFile(), etc. Since we have to break backward compatibility with reply(), may as well go all the way.

Some emergent properties that I didn't expect but were nice surprises:

• The BEGIN block now can post-process object macro outputs, i.e. to {uppercase} the entire reply. Previously the text that came from object macros were not affected by these tags.
• Errors thrown by object macro promises won't cause the top-level .reply() promise to error out. Instead, the macro's output in the reply is replaced with an "[ERR]" string.

I got to delete a lot of code for this. Since we can use the await keyword, I was able to gut out all of the clunky object macro substitution noise. The <call> tag is once again handled in the same processTags() function as all the other tags. No more ugly «__call__» and «__call_arg__» tags occasionally breaking responses.

Decaffeinated the Code

Initially I had upgraded the codebase to CoffeeScript 2 (a fork of the original), which had added support for the await keyword. But I saw that CoffeeScript 2 produced very nice ES2015 source code, preserving the comments and formatting with a relatively clean structure.

Having to compile from CoffeeScript -> ES2015 -> ES5 for backwards compatibility and web browsers was adding complexity, so I decided to just take the ES2015 output, clean it up and call that the new Source Code.

RiveScript is no longer written in CoffeeScript but has returned to native JavaScript (ES2015+). So we won't be held back from new EcmaScript features due to limitations in CoffeeScript.

Tooling Change: No More Grunt

I've replaced Grunt with a series of npm run scripts.

Browserify was replaced by Webpack out of popularity. Babel was brought in to produce backward compatible builds for older versions of Node and web browsers. RiveScript runs on Node versions >= 5.

The new relevant build commands are npm run build, npm run test and npm run dist. See the README.md on this branch for the rest of the Run Script options.

Testers Wanted

This branch will need some testing. It seems to work well and unit tests pass on Node 8, but things that need to be verified working:

• General regression testing. See if the new RiveScript has any bugs in a modern Node 7+ environment.
• Babel can build valid source for Node <= 6
• Webpack builds a bundle that works in the latest 2 versions of each major web browser

[dcsan]

wow this looks like a big change. but it will be nice to be able to reply on return values from JS <call> blocks.

How would you suggest testing this? I'd like to try it out in an app that has a lot of scripts, and see if things break. I believe I can install a module directly from an a github branch with:

npm install aichaos/rivescript-js#async-await

so then it's possible to switch between versions of rivescript. however changing branches with github won't change the module version since we usually ignore them in the repo, I'd need to delete inside node_modules to switch around.

[kirsle]

I think what I'll do is publish some pre-releases to npm for this branch, starting at v2.0.0-alpha.1

For now I've pushed a commit that updates the version in package.json and rivescript.js to v2.0.0-alpha.1, so if you install from the git branch you should hopefully get the right version number. If so the first npm release would be alpha.2 so it can update normally.

Before I publish to npm I wanna fix the branch either tonight or tomorrow morning, to put the Babel step immediately after CoffeeScript so that the lib/ folder contains ES5-compatible code, for Node <= 6. Since Node 6 is the LTS release, it should support it until the LTS becomes one with native async/await support. Currently, I only use Babel before running Webpack and the unit tests only pass on Node >= 7.

[josephrexme]

This looks great. Would be very happy to test

[kirsle]

I've published v2.0.0-alpha.2 to npm so you guys can start playing with it.

The main change so far is that replyAsync() has been renamed to reply() (meaning the old reply() now returns a Promise): so if you were already using replyAsync, just find/replace it to reply(), and if you were using reply(), refactor it to use promises.

-    var reply = bot.reply(username, message, this);
-    console.log("Bot>", reply);
+    rs.reply(username, message, this).then(function(reply) {
        console.log("Bot>", reply);
+    })

[kirsle]

I'm merging this to master and have published v2.0.0-alpha.4 to npm, which fixes a bug when <input> is used in triggers.

The current v1.19.0 version on master will be branched as v1 in case it needs any further work or backporting or anything.

Before the final 2.0.0 version, I need to clean up documentation and examples and such. See the v2.0.0 Milestone.