-
Notifications
You must be signed in to change notification settings - Fork 144
Asynchronous Support
Here is some information, backstory, and links to discussions about adding async support to RiveScript-JS and why it's difficult.
This is a lot of text but it fully covers all the background and ins-and-outs of the problem.
For the impatient:
- None of the RiveScript implementations supported anything async to begin with, because the original implementation was written in Perl which has no async at the language level built in, and all the other versions are derived from this version somehow.
- In PR #78, async was added for object macros only, but these were a special case in RiveScript anyway and is probably the only part that could be made async at all. This caused other issues like #111, #178 and #203, and it has some limitations due to the separation of when macros are actually resolved vs. when all the other tags get processed.
- Everything else
reply()/replyAsync()does is very CPU driven (looping over lists, compiling regular expressions) and adding support for hooks to make user variables go through an external database adapter would do nothing but add latency in all cases, especially for triggers containing any tag that would need such a database lookup to resolve.
Note: RiveScript.js v2.0.0 and onwards (since early 2019) now has full async/await support and the rest of this page just serves as a history lesson on the pain points and workarounds we used along the way.
- Examples of Async
- Backstory: Why RiveScript Wasn't Async
- Object Macros
- What Happens in
reply() - Object Macros in Conditions
- User Variable Session Adapters
To understand what I mean by async support, here are some examples of things people want to do that need to work asynchronously (e.g. by using JavaScript promises):
- Connecting to async web APIs, e.g. to look up weather or movie showtimes. These would be object macros that return a promise to resolve "in the near future", but you can't allow your entire chatbot to lock up while waiting for an API call to complete. (This is reasonably well supported now, see Object Macros below).
- Replacing the user data session driver with one backed by a cache or database. For example when
<set name=<formal>>is processed, put the user'snameinto a Redis cache, MySQL or MongoDB table rather than simply keeping it in an in-memory object map (and using methods likegetUservars()to export them and put them somewhere yourself).
The very first implementation of RiveScript was written in Perl around 2003, and Perl is not a language that emphasizes asynchronous programming patterns: Perl code is generally procedural by default, although other design patterns can be laid on top such as to make a program object-oriented or parallelized, e.g. with POE.
One of the design goals of RiveScript is to keep it as lightweight and self-contained as possible. Perl doesn't have any built-in (at the language level) async primitives, and there are cults around each of the async event modules in Perl, so if I had chosen to use POE it would be incompatible with those who prefer AnyEvent, or vice versa for any pair of Perl modules that provide parallel programming features. So, as the language itself didn't support async, RiveScript wasn't designed to work that way in Perl.
The later ports of RiveScript to Java, Python, JavaScript and Go were all based on the Perl version (or they were based on another implementation which was, itself, derived from the Perl version), and so none of the RiveScript implementations supported async at all: all object macros needed to return a string answer, and everything the code did from calling reply() to returning its answer was all done in a single threaded context and blocked until the answer was returned. RiveScript generally returns an answer pretty quickly so the single-threaded nature of it wasn't perceived to be a problem.
In Pull Request #78 (Feature/subroutines with promises), Philip Nuzhnyi contributed code to allow object macros (or subroutines) to return a promise instead of a string, which opened the door to creating object macros that deal with external APIs such as web servers or accessing the filesystem or querying a database -- all operations that typically are run asynchronously so they don't block your entire program while waiting on their answer.
Before this change, object macros had to return strings, and RiveScript needed their answers "now" so it would substitute the <call> tag in the reply for the string output of the macro. Any code that needed to run asynchronously couldn't work very easily. Some workarounds involved spawning a background task anyway (e.g. with setTimeout or calling a promise that you don't care to resolve), and this is where the second-reply example came from (the example was originally named async-object until the name became misleading with real async support for object macros).
To make the async object macros work, the processTags function had to be modified. Before the change, all tags got handled in this function, including <call> tags, which would have their macros called and their result substituted in place of the tags. After the change, the <call> tags get mangled out into other placeholder tags that were then processed "at the very end" -- right before the reply would have been given, RiveScript then calls all the object macros to gather their promises, waits on the resolution of all promises, and finally replaces these temporary placeholder tags with those results, and resolves the final reply for the caller.
This "two phase" tag processing logic caused some issues of its own:
-
In #111 (Execute object macros within conditionals), using an object macro on the left side of a conditional stopped working. Example:
+ is it sunny outside * <call>weather <get zipcode></call> == sunny => Yes it is! - No it's not.Because of the "two phase" processing of
<call>tags, the condition would be left with a bunch of mangled placeholder tags and not have the actual result of the macro, and the condition equality check would fail. #111 fixed this by calling the second phase immediately, but this only worked with purely synchronous object macros that returned strings and not promises. -
In #178 (Something very strange happened), using an example code from the RiveScript tutorial doesn't work quite right if the bot's answer includes an object macro that returned a promise.
After the "BEGIN Block" gets the initial reply, it tries to run any post-processing tags such as uppercasing the reply. However, the object macro with the promise hasn't been run at this point in the processing, so any manipulation it does to the intermediate reply won't affect the output of the macro. The macro resolves after this point in execution and the placeholder tags are put back in. So: you can't wrap a
<call>tag with something like{uppercase}from the BEGIN block.
Despite these down sides, async support in object macros is one of the best supported ways to do anything async in rivescript-js.
Before going on, I'd like to outline what all happens when you call the reply() function (it's mostly the same as replyAsync() until the very end, so these two functions work very similarly).
Everything that happens from the moment you call reply() until it returns a string answer is very CPU bound, with no logical hooks for adding any async code with the way it's currently structured.
To search for a match to the user's message, RiveScript loops over a sorted array (this is the array sortReplies() creates) of best matches for the message. At the top of the array are the "atomic" triggers (those that contain no tags or wildcards), ordered by the largest number of words, and later in the array are triggers containing optionals or wildcards and the very bottom has the triggers containing just one wildcard like *, so that the catch-all trigger * is the very last one tested (because it would match all user messages).
When a trigger is matched, what happens next is still very CPU driven: if there are conditions to check, RiveScript iterates over each condition in the order they originally appeared in the source. If there are multiple replies, RiveScript populates a temporary array containing each reply (with duplicates for replies with a {weight} tag) before randomly choosing a reply from that array.
Tag processing is also very synchronous: the code procedurally steps through different types of tags and does a "find and replace" operation on each, replacing the raw tag in the reply with the results of said tag. For a long time the <call> tag was also handled this way, replacing the tag with its string output.
Because of the CPU bound and synchronous nature of all these tasks, adding hooks for async support (such as allowing the developer to replace the in-memory user variable cache with an external driver) don't make any sense. For example a <set name=<formal>> tag would need to be fully handled (meaning: put the user name variable into your external database driver) "all at once", before the next tag, <get name> is handled. Otherwise, if you were to defer all writes via the <set> tag until "sometime in the future", then a <get> tag in the same reply that wants the newly written value would get the previous one, or worse.
If RiveScript were to be refactored to make every <get> and <set> tag go out to a database and wait for the resolution (using promises), then the latency of the module would be drastically harmed. A single reply that gets and sets two user variables would wait for at least four round-trip interactions with a database. This would be especially problematic for tags that appear inside triggers, such as + my name is <get name> or + <input1> -- you'd need database roundtrips to happen every time the trigger is even tested for a match, which would slow down the reply fetching process for all replies even if the user was going to match a different one.
The only thing different in replyAsync() is that object macros are allowed to return promises, and these get resolved at the end.
In the normal reply() function, if an object macro returns a promise, RiveScript will replace the <call> tag with a string error telling you that promises aren't supported unless you call replyAsync(). The macro still got executed, but its answer will be discarded because nobody is waiting to resolve it.
In replyAsync(), if an object macro returns a promise, RiveScript replaces its original <call> tag with some placeholders, and at the very end of the reply process, RiveScript waits to resolve all the macros and replaces the placeholder tags with the final outputs of each. When all promises are resolved, the replyAsync() promise itself resolves and the final reply text is given to the caller.
Everything else replyAsync() does is the same as reply() with regards to the matching process and how all other tags get handled.
As briefly mentioned in Object Macros above, object macros in conditions do not support asynchronous macros.
In the reply process, if a trigger has conditions attached, RiveScript needs to loop through each condition and test its truthiness to see if it can find a good reply to the user's message. Object macros inside these condition lines need to give their answers "now" so that RiveScript can check their equality and make a decision one way or another.
Making it support async macros in conditions would require a non-trivial refactor of the codebase, and would just add latency, which would be wasteful in the event that the condition isn't even truthy at the end and RiveScript goes on to check the next one instead. The latency wouldn't be as bad as supporting database drivers for user variables, but it would still be there.
The biggest issue blocking this support would be refactoring the whole reply process to give it the ability to "pause" in the middle and wait for an object macro to resolve. I'm not sure this is even a desirable change.
The other common request is to replace the in-memory user variable store with something replaceable by a database driver. This would make it so that any <get> or <set> tag (or their cousins), as well as any function calls like setUservar() or getUservar() would immediately go through a database driver of some sort (left as an exercise to the developer) so that MongoDB or MySQL or Redis or something could keep track of the user variables instead of them just sitting in an in-memory JavaScript object.
I had actually written some code to support this in #137 (Add support for third party session handlers), but then I realized it wasn't a useful code change with the way the entire RiveScript library works.
As mentioned above in What Happens in reply(), user variables have to be always on-hand at any given time so that tags and condition checks can be handled quickly. The default in-memory user variable store works for now because setting an object key is a synchronous procedure, and any replacement driver for the session manager would have to also work synchronously. This rules out all of the drivers that developers would have wanted: things like Redis, MySQL, MongoDB or even writes to the filesystem are generally asynchronous tasks, and the core of RiveScript can't support that. You'd only be able to replace the in-memory driver with another sync driver, which is much less useful.
The preferred way to deal with user variables is to use the getUservars() and setUservars() function and handle these out-of-band in your own app code, to export and re-import user variables on your own. In your own code's space you're free to use async modules all you want, but the core of RiveScript currently can't support async very well.
bot.replyAsync(username, message, this).then(function(reply) {
// After the reply, get all the user's data and store them somewhere.
var userdata = bot.getUservars(username);
redisClient.set(username, JSON.stringify(userdata), redisClient.print);
// And do whatever you want with the reply.
messengerDriver.sendMessage(username, reply);
});