Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find all elements on the screen, not just those in the focused window of the frontmost app #14

Open
mjrusso opened this issue Jan 15, 2022 · 1 comment

Comments

@mjrusso
Copy link
Owner

mjrusso commented Jan 15, 2022

In element-based nav mode (first introduced in #1), Scoot will find all elements in the focused window of the frontmost app.

It would be better if Scoot could find all elements that are on all attached screen(s), across all apps.

Of course, windows can overlap, and we'll need to ignore any elements that are fully occluded by a window above it.

AXUIElementCopyElementAtPosition will find an element at the specific coordinate (if one exists), but you need to pass it an application reference, so it is not directly usable in this context (and, regardless, checking every possible x,y value on every screen seems wasteful from a performance perspective).

https://stackoverflow.com/a/56941105/ has some code that might be helpful (finding all windows on the screen, and ordering them relative to each other).

@mjrusso
Copy link
Owner Author

mjrusso commented Jan 15, 2022

It would be better if Scoot could find all elements that are on all attached screen(s), across all apps.

A stepping stone could be finding all elements in the window that is under the mouse cursor (if it differs from the focused window of the frontmost app). Of course, there's still occlusion issues to deal with.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant