AXPosition: Attempting to query position of text cursor in given text field returns top-left location

[kwvg]

Hello! I've been interested in Hammerspoon but my knowledge of its APIs (and by extension of AXUIElement) is extremely limited. I've tried to piece together a small script that finds the text cursor location of an active textbox and then overlays information over it (right now it's just right-clicking)

Expected Behavior

AXPosition should return the X,Y coordinates of a text cursor in the active text box

Actual Behaviour

AXPosition returns the X,Y coordinates of the top left extreme of the active text box

Code used:

local ax = require("hs.axuielement")
hs.hotkey.bindSpec(
    {{ "ctrl", "alt" }, "/"},
    function()
        local currentApp = hs.application.frontmostApplication()
        local axApp = ax.applicationElement(currentApp)
        axApp:setAttributeValue('AXEnhancedUserInterface', true)
        axApp:setAttributeValue('AXManualAccessibility', true)

        local systemElement = ax.systemWideElement()
        local currentElement = systemElement:attributeValue("AXFocusedUIElement")
        hs.eventtap.rightClick(currentElement:attributeValue("AXPosition"))
    end
)

Resources

For the code snippet written above

• Retrieving input field values and cursor position/selection with Hammerspoon

System Information

macOS Catalina 10.15.7 (19H114)
AXUIElement: Version 1.0.8
Hammerspoon: Version 0.9.82 (5614)

[atanasj]

@kittywhiskers, thanks for this… I have implemented this but am also keen to so something similar to what your intention is here, but my skills are beyond what is needed I'm afraid.

I posted a question about how I could do this on the hammerspoon discussion and was directed to this post on stackoverflow…

Not sure if you could make use of this.

[latenitefilms]

AFAIK AXPosition just returns "the global screen position of the top-left corner of an element". So, if you're working with an AXTextArea, it will return the top-left corner of that text area - not the position of where the cursor currently is.

The Stack Overflow post that @atanasj posted is however interesting... I wonder if @asmagill has any ideas?

[asmagill]

A literal implementation looks something like this:

local axuielement = require("hs.axuielement")
local canvas      = require("hs.canvas")

-- since this is used in both the keyDown and the keyUp functions, if we don't want it to be global,
-- it needs to be defined outside of both
local selectionRectangle = nil

hs.hotkey.bind({"cmd", "ctrl"}, "l", function()
    local systemWideElement = axuielement.systemWideElement()
    local focusedElement    = systemWideElement.AXFocusedUIElement
    if focusedElement then
        local selectedRange = focusedElement.AXSelectedTextRange
        if selectedRange then
            local selectionBounds = focusedElement:parameterizedAttributeValue("AXBoundsForRange", selectedRange)
            if selectionBounds then
                selectionRectangle = canvas.new(selectionBounds):show()
                selectionRectangle[#selectionRectangle + 1] = {
                    type        = "rectangle",
                    strokeWidth = 5,
                    action      = "stroke",
                    strokeColor = { green = 1, blue = 1 },
                }
                print(finspect(selectionBounds))
            end
        end
    end
end, function()
    if selectionRectangle then
        selectionRectangle:hide()
        selectionRectangle = nil
    end
end)

Only seems to work for some applications, not sure how to tell which ones or alternative methods for the applications that it doesn't work for, but maybe this will get you started.

[asmagill]

Oh, and axuielement is now part of the core application, you don't need to use this external module anymore. If I make additions or changes to it, I may use this external module repository for testing before updating core, but otherwise they should maintain parity.

[atanasj]

Thanks @asmagill. I got a chance to try your code, but I am getting the below error in the console:

2022-09-28 23:26:09: 23:26:09 ERROR:   LuaSkin: hs.hotkey callback: /Users/atanas/.hammerspoon/keyboard/mouse-clicker.lua:23: global 'finspect' is not callable (a nil value)
stack traceback:
	/Users/atanas/.hammerspoon/keyboard/mouse-clicker.lua:23: in function </Users/atanas/.hammerspoon/keyboard/mouse-clicker.lua:8>

Any idea what this means?

[asmagill]

Sorry, that's a diagnostic function I've written to "flatten" the output generated by hs.inspect... It's basically the following:

finspect = function(...)
    local stuff = { ... }
    if #stuff == 1 and type(stuff[1]) == "table" then stuff = stuff[1] end
    return hs.inspect(stuff, { newline = " ", indent = "" })
end

The function takes 1 or more arguments and then uses hs.inspect to return a "flattened table" representation of them (flattened means in one line, as opposed to the multi-line structure normally returned by hs.inspect for tables)

[muescha]

somehow this not work at a textfield in chrome (and obsidian)
only in IntelliJ or in Notes app

[muescha]

BTW my idea was with this mouse position to have an context menu like on windows where I can hit shift+F10.

On some keyboards there was also a key for context menu.

I miss this a lot :(