CoroutinesLeveraging

Leveraging Coroutines in Hammerspoon

The examples presented in this document required Hammerspoon 0.9.79 or newer; or a development build of Hammerspoon built from https://github.com/Hammerspoon/hammerspoon/tree/ea693f1d25d3a5e793575ecc68f94452255e89e1 or newer.

Now that coroutines are formally being supported within Hammerspoon, this document will outline a way to utilize them to hopefully make your installation more responsive.

This is by no means the only way to utilize Coroutines within Hammerspoon, or within Lua itself. It is however a fairly simple one that will hopefully provide tangable results with a minimum of change to your existing code.

A Little Background

Lua, the scripting language Hammerspoon uses for user defined functionality, is not a multithreaded language. At its simplest, this means that the Lua engine or interpreter (used interchangably here for simplicity) can only do one thing at a time in a sequential manner. When a long running task is being performed, nothing else can occur until the task is completed.

One method that Lua offers to mitigate this is known as Coroutines. Again, simplifying a bit, coroutines are a way to pause a long running task to allow the Lua engine to perform another, usually shorter, task so that both tasks can progress in a close to simultaneous manner.

Complicating things a little bit is the way macOS applications, like Hammerspoon, are structured. A macOS application is comprised of multiple application threads, most of which perform tasks that the user, and often even the developers of the application don't have to worry about. The interesting parts happen on the primary or "main" application thread, such as user interface updates, event handling, and in the case of Hammerspoon, running the Lua engine. While the Lua engine is running lua code, however, the main application thread can't perform its other tasks -- like updating the UI or handling events, timers, delegate callbacks, hotkeys, etc.

What follows is a methodology for breaking up a long running task into a format that utilizes coroutines to allow the main application thread of Hammerspoon time to perform its updates and then quickly get back to working on our long running task.

It should be noted that this approach will work best when much of the processing time for the long running task is spent within the Lua engine processing Lua commands; if your slowdown is occuring within a single invocation of a function or method written in Objective-C provided by one of the Hammerspoon modules, then it will be necessary to figure out a way to fix or speed up that code instead, or break it up into smaller chunks which can be iterated through to invoke the Objective-C code multple times by the Lua engine which can then leverage the approach outlined here.

The Approach

For this discussion, consider this fairly generic function outline:

myFunction = function(...)

    -- initial setup for the task
    --    e.g. validate arguments, setup initial variables, etc.

    -- long running task, often, but not always within a loop

    -- clean up for the task
    --    possibly including using the results of long running task
end

It is important to note that for the approach outlined here, the function myFunction is not returning anything -- it is self contained, performing a specific task and acting on the results in whatever way is appropriate before returning from the function. Sharing Coroutine Results describes some possible approaches to consider if it truely makes more sense to code in such a way that another code section or block requires the results of myFunction rather than include such code within the clean-up portion of myFunction.

As described in this generic function outline, there are basically three sections. It is assumed that the initial setup and clean up sections are relatively streamlined and efficient -- if they aren't, you might need to reconsider how you have designed your function.

To leverage coroutines to make this code more responsive, we need to modify the code to something like this:

myFunction = function(...)

    -- initial setup for the task
    --    e.g. validate arguments, setup initial variables, etc.

    local longTask
    longTask = coroutine.wrap(function(...)
        -- if you refer to local variable defined within the setup section, they
        -- will become up-values to the coroutine; or if it makes more sense,
        -- pass them in as arguments

        local exitCondition = false
        -- long running task, often, but not always within a loop, e.g.
        while not exitCondition do
            -- do one iteration of the long running task, changing exitCondition to
            -- true when we have finished
            coroutine.applicationYield()
        end

        -- clean up for the task
        --    possibly including using the results of long running task

        longTask = nil -- by referencing longTask within the coroutine function
                       -- it becomes an up-value so it won't be collected
    end)
    longTask(...) -- this is where we pass arguments into the coroutine
end

The long running task and the clean up sections have been moved into the coroutine function. For the example here, a while loop is used to repeat the task until exitCondition becomes true.

An important line to note here is coroutine.applicationYield(). This function has been added to Hammerspoon (it is not part of the stock Lua coroutine library) and it performs a couple of tasks for us:

sets up a timer to restart the coroutine function, by default in hs.math.minFloat seconds
yields the coroutine with coroutine.yield

hs.math.minFloat is the smallest floating point number recognized by both Lua and hs.timer and represents the smallest timeslice we can assign to a timer. This ensures that our coroutine is restarted as quickly as possible, but still allows Hamemrspoon to respond to other events and perform the necessary application housekeeping that keeps it responsive. You can provide a floating point argument to coroutine.applicationYield(delay) if you know that your code doesn't need to be returned to immediately (for example if you're waiting on a response to an hs.http.asyncGet -- see further exmaple below) and want to give Hammerspoon more time to handle other events.

When iterating and performing whatever data manipulation you require, you don't have to invoke coroutine.applicationYield with every iteration; you could just as easily create a counter (or whatever other logic you want to use) and do something like:

        -- define as local count = 0 outside of the while loop; this segment is just
        -- to replace the lone coroutine.applicationYield line in the above example
        count = count + 1
        if count % 10 == 0 then -- only yield every 10 iterations
            count = 0
            coroutine.applicationYield()
        end

If your long running code is not in a loop but is actually just a lot of lua commands, you can intersperse coroutine.applicationYield at various points where it makes sense and the code will resume immediately after where the last coroutine.applicationYield was invoked.

Variants

As mentioned above, we can adjust the time coroutine.applicationYield will delay before resuming the coroutine. This may be useful when we know we're waiting on something that will take a while and don't want to waste cycles immediatly yielding again while waiting for the results. An obvious example is hs.http.asyncGet. An example of how we might handle this follows:

myFunction = function(...)

    -- initial setup for the task
    --    e.g. validate arguments, setup initial variables, etc.

    local longTask
    longTask = coroutine.wrap(function(...)
        -- if you refer to local variable defined within the setup section, they
        -- will become up-values to the coroutine; or if it makes more sense,
        -- pass them in as arguments

        -- predeclare these so they are upvalues to the get request callback
        local status, body, hdrs = nil, nil, nil
        local getDone = false
        hs.http.asyncGet("http://site.com", {}, function(s,b,h)
            status, body, hdrs = s, b, h
            getDone = true
        end)
        while not getDone do
            coroutine.applicationYield(2.0) -- no reason to jump back here immediatly
        end

        local exitCondition = false
        -- long running task, often, but not always within a loop, e.g.
        while not exitCondition do
            -- do one iteration of the long running task, changing exitCondition to
            -- true when we have finished
            coroutine.applicationYield() -- now resume as quickly as possible
        end

        -- clean up for the task
        --    possibly including using the results of long running task

        longTask = nil -- by referencing longTask within the coroutine function
                       -- it becomes an up-value so it won't be collected
    end)
    longTask(...) -- this is where we pass arguments into the coroutine
end

If your coroutine function should be resumed when some other activity occurs, rather than after a set amount of time, (for example when the user types a specific hotkey) then it may make more sense to replace coroutine.applicationYield with something else. Consider:

myFunction = function(...)

    -- initial setup for the task
    --    e.g. validate arguments, setup initial variables, etc.

    local hk

    local longTask
    longTask = coroutine.wrap(function(...)
        -- if you refer to local variable defined within the setup section, they
        -- will become up-values to the coroutine; or if it makes more sense,
        -- pass them in as arguments

        local exitCondition = false
        -- long running task, often, but not always within a loop, e.g.
        while not exitCondition do
            -- do one iteration of the long running task, changing exitCondition to
            -- true when we have finished

            -- make sure we don't recreate the hotkey each time it repeaats
            if not hk then
                -- resume each time the user taps or holds the spacebar
                local hk = hs.hotkey.bind({}, "space", function()
                    -- no arguments even if we initially supplied them
                    -- as we're just resuming
                    longTask()
                end, function()
                    hk:disable()
                    hk = nil
                end, function()
                    longTask()
                end)
            end
            coroutine.yield() -- instead of coroutine.applicationYield

        end

        -- clean up for the task
        --    possibly including using the results of long running task

        longTask = nil -- by referencing longTask within the coroutine function
                       -- it becomes an up-value so it won't be collected
    end)
    longTask(...) -- this is where we pass arguments into the coroutine
end

(It should be noted that you actually can pass new arguments into and out of an active coroutine, but such activity is beyond the scope of this document. If this is necessary for your setup, then consult the Lua Documentation or a Lua reference book -- I've often used the third and fourth edition of Programming in Lua to good effect. Note that if you refer to the online version of this manual, it only covers Lua 5.1 -- there may be some minor differences, though for coroutines, thus far I've only observed that coroutine.isyieldable is an addition to 5.3.)

Sharing Coroutine Results

For our purposes, the usage of coroutines as outlined in this document focuses on them being self contained and not passing data in or out while they are executing the long task. As noted in the Variants section, it is possible to pass arguments in and out of a coroutine using the built in coroutine library, but this usage is beyond the scope of this document.

The reason for this is because I wanted to show a simple way to make Hammerspoon more responsive and by scheduling the resuming of the coroutine with a timer created by a generic helper function, we don't know exactly when the coroutine will be resumed and thus don't know what to pass in or expect out from the coroutine. Passing arguments into and out of a coroutine requires additional thought and planning because you have to make sure that the coroutine.resume or function generated by coroutine.wrapis invoked only within another code block that knows what to expect out or pass in.

Of course, using global variables and checking them within the long task section of our coroutine, or checking that they've been set by our coroutine in other code blocks is always an option; however, as most programmers like to avoid using global variables to keep things cleaner and avoid possible name collisions, our options are more limited.

For full flexibility, consult the references suggested in the Variants section above.

However, passing data out of the coroutine as structured in this document can easily be handled by using callbacks, and with hs.watchable we can get bi-directional transfer of data, but we can't control when (e.g. at which iteration or point in the control flow) the data changes -- if you require that level of control, you'll have to consult the references given above and compose your own replacement of coroutine.applicationYield and possibly change how the coroutine is actually started (e.g. returning longTask from myFunction(...) instead of executing it immediately so some other code block can track it and pass values in and receive them at appropriate points).

With Callbacks -- this is the easiest, but is out only; you only need to implement the callback you require, but this example shows both

myFunction = function(cbInterim, cbAtEnd, ...)

    -- initial setup for the task
    --    e.g. validate arguments, setup initial variables, etc.

    local longTask
    longTask = coroutine.wrap(function(...)
        -- if you refer to local variable defined within the setup section, they
        -- will become up-values to the coroutine; or if it makes more sense,
        -- pass them in as arguments

        local exitCondition = false
        -- long running task, often, but not always within a loop, e.g.
        while not exitCondition do
            -- do one iteration of the long running task, changing exitCondition to
            -- true when we have finished
            cbInterim(...) -- pass whatever we need to the interim callback
            coroutine.applicationYield() -- then yield
        end

        cbAtEnd(...) -- pass whatever we need to at completion of the task

        -- clean up for the task
        --    possibly including using the results of long running task

        longTask = nil -- by referencing longTask within the coroutine function
                       -- it becomes an up-value so it won't be collected
    end)
    longTask(...) -- this is where we pass arguments into the coroutine
end

With hs.watchable -- bidirectional but more complex

myFunction = function(cbInterim, cbAtEnd, ...)

    -- initial setup for the task
    --    e.g. validate arguments, setup initial variables, etc.

    -- if we pass in true, then we can accept data from the outside
    local myFunctionWatchables = hs.watchable.new("myFunctionWatchables" [, true])

    -- only required if we set `true` when creating myFunctionWatchables:
        -- we set this so outside code can verify we exist and not throw an error
        -- for trying to change a non-existent (and athus ssumed unidirectional)
        -- watchable
        myFunctionWatchables.running = true

        -- used within long task to see if something has changed
        local dataChanged = false

        -- to reduce invoking this when assigning "output" values, specify
        -- specific keys we want to watch for. Repeat for as many "input" keys
        -- as required
        -- could specify "*" but then would have to check against things we're
        -- passing "out" (in this example `interimData` and `finalData`) and
        -- that would incur additional overhead
        local internalWatchers = {
            hs.watchable.watch("myFunctionWatchables.quit", function(w, p, k, o, n)
                    dataChanged == true
                end
            ),
        }

    local longTask
    longTask = coroutine.wrap(function(...)
        -- if you refer to local variable defined within the setup section, they
        -- will become up-values to the coroutine; or if it makes more sense,
        -- pass them in as arguments

        local exitCondition = false
        -- long running task, often, but not always within a loop, e.g.
        while not exitCondition do
            -- do one iteration of the long running task, changing exitCondition to
            -- true when we have finished

            -- only required if we set `true` when creating myFunctionWatchables:
                if dataChanged then -- something changed
                    if myFunctionWatchables.quit == true then -- exit early
                        exitCondition = true
                    end
                    dataChanged = false
                end

            myFunctionWatchables.interimData = someValue -- to pass out to outside watchers

            coroutine.applicationYield() -- then yield
        end

        myFunctionWatchables.finalData = someValue -- to pass out to outside watchers

        -- clean up for the task
        --    possibly including using the results of long running task

        longTask = nil -- by referencing longTask within the coroutine function
                       -- it becomes an up-value so it won't be collected

        -- only required if we set `true` when creating myFunctionWatchables:
            -- clean up so we don't leave things around that are no longer active
            for i,v in ipairs(internalWatchers) do
                v:release() -- it's done, so release our watchers
            end
            internalWatchers = nil

    end)
    longTask(...) -- this is where we pass arguments into the coroutine
end

Outside code can trigger an early termination with:

-- unfortunately these requires an empty function... may have to update `hs.watchable`
local emptyFn = function() end

-- we check to see if `running` has been set so that the change won't trigger an
-- error for trying to change a non-existent (and thus assumed unidirectional)
-- watchable
if hs.watchable.watch("myFunctionWatchables.running", emptyFn):value() then
    hs.watchable.watch("myFunctionWatchables.quit", emptyFn):change(true)
end

And to receive the updates to myFunctionWatchables.interimData and myFunctionWatchables.finalData, outside code blocks can use something along the lines of:

local outsideWatchers -- predeclare so we can use it inside ourselves
outsideWatchers = {
    hs.watchable.watch("myFunctionWatchables.interimData", function(w, p, k, o, n)
        -- see docs for `hs.watchable` for full details, but `n` will contain the new
        -- value, so do whatever you need to with it
    end),
    hs.watchable.watch("myFunctionWatchables.finalData", function(w, p, k, o, n)
        -- see docs for `hs.watchable` for full details, but `n` will contain the new
        -- value, so do whatever you need to with it

        -- clean up so we don't leave things around that are no longer active
        for i,v in ipairs(outsideWatchers) do
            v:release() -- it's done, so release our watchers
        end
        outsideWatchers = nil
    end),
}

Final Thoughts

This introduction by no means covers all of the possibilities, but should hopefully allow you to get started moving your long running lua code into more responsive and friendly coroutines with minimal changes.

At present, it is not possible to yield or resume within long running Objective-C code used by the Hammerspoon modules for accessing the macOS Objective-C runtime environment; hopefully most of these are already fast enough that by coding your Lua code appropriately these do not cause Hammerspoon to become unnecessarily unresponsive.

If you believe that a compiled function or method needs to be looked at, you are welcome to submit your own tweaks or code updates by submitting a pull request to https://github.com/Hammerspoon/hammerspoon or by submitting an issue to https://github.com/Hammerspoon/hammerspoon/issues and our developers will examine the specifics and determine if a different approach within your code might help or if code changes to Hammerspoon or its modules needs to be considered.

Home | Tutorials | Sample Configurations