loops and yield (and async?)

[gilch]

Hissp's target Python subset only has expressions. Python's comprehension syntax (being expressions) already works in theory.

Loop-equivalents (for/while) are fairly easy to implement using higher-order functions, though while gets awkward without assignment statements. (It may be less of a problem when we get the walrus in Python 3.8.)

Technically, yield is considered an expression in Python, but it does something weird to its surrounding function: It makes it return a generator instead, so they compose differently. Hebigo control-statement macros expand to thunks to be able to delay, repeat, or avoid evaluation, so any yield expression in an if or loop (the normal case) would transform the wrong function's return: the inner thunk, instead of the surrounding definition. Higher-order functions would now require a yield from to propagate this behavior. Even if we could add that everywhere it's needed, how do you tell if the thunk has a yield in it?

One possible solution is to have a parallel version of each control statement that has the yield from, and then the user has to pick the right one (or uses a macro that does it for them). I think this could get awkward fast. And then what about new user macros?

This lack of natural composability makes me think that yield expressions are the wrong approach.

Drython's solution was to create a background thread to save the execution state. This works, but has greater overhead than a native yield would. Perhaps that is acceptable, since using higher-order functions everywhere has a higher overhead to begin with. Performance was never really the point of Hissp. I'm also not sure if I ever got them garbage collected properly. Threaded code is notoriously hard to test due to nondeterministic behavior (race conditions).

Scheme's call/cc or Haskell's IO monad also look promising. They seem a lot more naturally composable than Python's yield. Unfortunately, I don't think I understand them well enough to tell, much less implement yield this way.

Another option might be to not support yield at all. Given itertools, genexprs, and statement-equivalent macros that work in lambdas, you'll probably rarely need them. But not having them means adding friction to the use of Python libraries. There are cases even in the standard library that require yield, like @contextlib.contextmanager.

A class with the right dunder methods can be a generator, though implementing them seems more difficult generally. This goes for awaitables as well.

[gilch]

This looks relevant: https://en.wikibooks.org/wiki/Haskell/Continuation_passing_style

[gilch]

Clojure's core.async is implemented as macros, and doesn't even require threading. The threadless option seems to be implemented as a "virtual virtual machine" that can interpret every special form. Hissp has only two special forms (I designed it that way in part to make this kind of macro easier), so this approach looks even easier than it was for Clojure. Yet still not easy. I don't understand how the virtual virtual machine works. CPS might be easier.

[gilch]

The latest Hissp prelude defines an Ensue class that pretty much handles this. It's not quite as convenient as yield and async for simple cases, but it is a lot more regular. A macro layer for Hebigo built on top of something like this could have a pretty good compact syntax.