How to manipulate click and drag controls (e.g. davinci color wheels) with buttons and knobs?

[lxne]

Hi, I am trying to manipulate some click and drag controls (primary color wheels in Davinci Resolve) with buttons and knobs (in my case with a Loupedeck Live). For that, I try to emulate mouse commands (mouse position, click on a wheel and drag the mouse).

Select wheel to manipulate (working)

I thought of one dedicated button for each wheel to position the mouse over the wheel…

hs.eventtap.leftClick({x=150, y=770}, 100000)
hs.mouse.absolutePosition({x=120, y=920})

The first line makes sure the color wheels are active.
The second command places the mouse cursor on the first color wheel.
Off course, the values depend on the screen, but I don't know of any better way to position the mouse.

Change the color values (not working)

Now I am trying to make the click and drag commands with two knobs. One knob for horizontal and another for vertical adjustment/dragging. Unfortunately, I can't get it to work.
I thought, with each step of the knob, the control point of the wheel should be dragged into the assigned direction.
For example, to accomplish correction to the right, I tried the following (doesn't work).

pointNow = hs.mouse.absolutePosition()
hs.eventtap.event.newMouseEvent(hs.eventtap.event.types.leftMouseDown, {x=pointNow.x, y=pointNow.y}):post()
hs.eventtap.event.newMouseEvent(hs.eventtap.event.types.leftMouseDragged, {x=pointNow.x+1, y=pointNow.y}):post()
hs.eventtap.event.newMouseEvent(hs.eventtap.event.types.leftMouseUp, {x=pointNow.x, y=pointNow.y}):post()

One big problem with this is, the wheel gets reset with a double click. And that's what's happening with this snippet.
If I turn the knob pretty fast, I can move the mouse, but else this is not working.

Any ideas would be appreciated.

[latenitefilms]

You can use the Accessibility API to get positions of some UI elements.

I started playing with this a long time ago here:

https://github.com/CommandPost/CommandPost/blob/develop/src/extensions/cp/blackmagic/resolve/init.lua

Sadly, I never got around to doing much with DaVinci Resolve - it moves so fast, it's hard to keep up.

Rather than re-creating the wheel, you might want to have a look at what https://www.sideshowfx.net have already done?

[lxne]

Thank you for your reply.

Yes, over at sideshowfx they somehow managed to control the wheels.

I have a view issues with the loupedeck app (e.g. Blender Shortcuts not working) and would prefer to use commandPost only. I also don't want to spend nearly 100 bucks to control the wheels - if there's another option. Which probably neither Blackmagic nor Sideshowfx would be very happy about.

Unfortunately, I am not a programmer and this seems a bit too complex for me.

Probably an extension of some kind is needed to accomplish it?
Turning a knob sends positive or negative increments to the extension.
e.g. each step to the left -1 and each step to the right +1
the extension activates drag mode, keeps the drag mode active for a while and drags the mouse according to the steps.
Could that work?

Something like…
(not real code, only my thoughts expressed somehow)

pointNow = hs.mouse.absolutePosition()
if left mouse hs.eventtap.event.newMouseEvent(hs.eventtap.event:getButtonState(0)) is not down then
  hs.eventtap.event.newMouseEvent(hs.eventtap.event.types.leftMouseDown
end
hs.eventtap.event.newMouseEvent(hs.eventtap.event.types.leftMouseDragged, {x=pointNow.x+a, y=pointNow.y+b})
if some time has passed (enough to prevent double clicks) then
hs.eventtap.event.newMouseEvent(hs.eventtap.event.types.leftMouseUp
end