Skip to content

JeffersonQin/Ayase

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

45 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

Ayase

็ฎ€ไฝ“ไธญๆ–‡ | For Simplified Chinese version, please click here

Ayase is a tool which can help you to control everything by keyboard. This is specifically designed for lazy people like me and blind people. It is known to all that in the most cases, we have to control the computer by both mouse and keyboard. However, the frequent switch for our hand between mice and keyboards would severely threaten the health of our wrists. Meanwhile, for blind people, though in most cases, it is convenient enough to use the screen readers to control elements on screen, most softwares cannot control both precisely and swiftly. You have to wait for the element or switch many times to find the element you want to control. With Ayase, these problems are all solved.

Screenshots

Use Ayase to control Netease Music UWP:

netease-demo

Use Ayase to surf on the Internet:

surf-internet-demo

About Naming (Ayase)

I love Mitsukasa Ayase forever!

https://zh.moegirl.org.cn/ไธ‰ๅธ็ปซๆฟ‘

Features

  • Support high DPI, different DPI across multiple monitors, cross monitor display
  • Support Chrome-based Apps (e.g. Github Desktop)
  • Support Electron Apps (e.g. Visual Studio Code)
  • Support WinForms, WPF, UWP (e.g. Netease Music UWP)
  • Support the self-developed UI frameworks (e.g. Wechat & QQ of Tencent Inc.)
  • Scanning of UI tree with high speed
  • Support pinyin for Chinese (e.g. ็ฝ‘ๆ˜“ไบ‘ โ†’ wyy / wangyiyun)
  • Ignore upper and lower case

For the discussion on accessibility development and speed optimization, please refer to my following articles. Finally, the implementation scheme I adopted is to use C++ to invoke Win32API / uiautomation.h and compile into dll (dynamic link library). Then, I used P/Invoke in the C# / WPF project to use the library. In most cases, the performance was improved for about 80%.

Note that I do not have additional time and effort to translate those articles into English. Using translators can also understand the articles well. If you still have any questions or advice, you can contact me directly. You can find ways to contact me in my GitHub homepage. This is the first time I write WPF / C# this formally, so if you have any advice, please don't hesitate to raise issues or pull requests.

Project Structure

.
โ”œโ”€โ”€ Ayase                                   # Ayase Project
โ”‚ย ย  โ”œโ”€โ”€ AccessibilityBridge                 # P/Invoke bridge for .dll
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ GNativeIUIAutomationManager.cs
โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ GNativeUIElement.cs
โ”‚ย ย  โ”œโ”€โ”€ App.xaml
โ”‚ย ย  โ”œโ”€โ”€ App.xaml.cs                         # Entrance
โ”‚ย ย  โ”œโ”€โ”€ AssemblyInfo.cs
โ”‚ย ย  โ”œโ”€โ”€ AutomationLib                       # Most of them are no longer used. Serve as alternatives
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ GAutomationManager.cs
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ GIUIAutomationManager.cs
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ GMSAAManager.cs
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ GRawUIManager.cs
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ GUIElement.cs
โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ Wrapper                         # Wrapper for Win32API used in this section
โ”‚ย ย  โ”‚ย ย      โ”œโ”€โ”€ ObjectIdentifiers.cs
โ”‚ย ย  โ”‚ย ย      โ”œโ”€โ”€ RECT.cs
โ”‚ย ย  โ”‚ย ย      โ”œโ”€โ”€ ReferenceIdentifiers.cs
โ”‚ย ย  โ”‚ย ย      โ””โ”€โ”€ Win32API.cs
โ”‚ย ย  โ”œโ”€โ”€ Ayase.csproj
โ”‚ย ย  โ”œโ”€โ”€ Ayase.csproj.user
โ”‚ย ย  โ”œโ”€โ”€ HotkeyLib                           # Global hotkey wrapping
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ GHotKey.cs
โ”‚ย ย  โ”‚ย ย  โ”œโ”€โ”€ GKeybdEvent.cs
โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ VirtualKeys.cs
โ”‚ย ย  โ”œโ”€โ”€ MouseLib                            # Mouse control wrapping
โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ GMouse.cs
โ”‚ย ย  โ”œโ”€โ”€ Properties
โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ launchSettings.json
โ”‚ย ย  โ”œโ”€โ”€ ScreenLib                           # Screen library wrapping
โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ PrimaryScreen.cs
โ”‚ย ย  โ”œโ”€โ”€ ThreadLib                           # Private thread pool
โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ GThreadPool.cs
โ”‚ย ย  โ””โ”€โ”€ UI                                  # UI
โ”‚ย ย   ย ย  โ”œโ”€โ”€ FormMaskWindow.xaml
โ”‚ย ย   ย ย  โ”œโ”€โ”€ FormMaskWindow.xaml.cs
โ”‚ย ย   ย ย  โ”œโ”€โ”€ MaskWindow.cs
โ”‚ย ย   ย ย  โ”œโ”€โ”€ NotationLabel.cs
โ”‚ย ย   ย ย  โ”œโ”€โ”€ ReuseWindow.cs
โ”‚ย ย   ย ย  โ”œโ”€โ”€ ScreenMaskWindow.xaml
โ”‚ย ย   ย ย  โ”œโ”€โ”€ ScreenMaskWindow.xaml.cs
โ”‚ย ย   ย ย  โ”œโ”€โ”€ SearchWindow.xaml
โ”‚ย ย   ย ย  โ”œโ”€โ”€ SearchWindow.xaml.cs
โ”‚ย ย   ย ย  โ”œโ”€โ”€ SettingsWindow.xaml
โ”‚ย ย   ย ย  โ”œโ”€โ”€ SettingsWindow.xaml.cs
โ”‚ย ย   ย ย  โ””โ”€โ”€ WindowManager.cs                # Windows scheduling
โ”œโ”€โ”€ Ayase.Accessibility                     # dll project
โ”‚ย ย  โ”œโ”€โ”€ Ayase.Accessibility.vcxproj
โ”‚ย ย  โ”œโ”€โ”€ Ayase.Accessibility.vcxproj.filters
โ”‚ย ย  โ”œโ”€โ”€ Ayase.Accessibility.vcxproj.user
โ”‚ย ย  โ”œโ”€โ”€ GConstant.h                         # Some constants
โ”‚ย ย  โ”œโ”€โ”€ GIUIAutomationManager.cpp           # Main implementation
โ”‚ย ย  โ”œโ”€โ”€ GIUIAutomationManager.h
โ”‚ย ย  โ”œโ”€โ”€ GUIElement.cpp                      # Wrapping for data structure
โ”‚ย ย  โ”œโ”€โ”€ GUIElement.h
โ”‚ย ย  โ”œโ”€โ”€ dllmain.cpp
โ”‚ย ย  โ”œโ”€โ”€ framework.h
โ”‚ย ย  โ”œโ”€โ”€ pch.cpp
โ”‚ย ย  โ””โ”€โ”€ pch.h
โ””โ”€โ”€ Ayase.sln

The TOC for generated project:

.
โ”œโ”€โ”€ ...
โ”œโ”€โ”€ Ayase.Accessibility.dll
โ””โ”€โ”€ <.net target>
    โ”œโ”€โ”€ ...
    โ””โ”€โ”€ Ayase.exe

Dependencies

  • Microsoft.Toolkit.Uwp.Notifications
  • Notifications.Wpf
  • ToolGood.Words

Usage

  • Start UI Scanning: Alt + CapLocks (We would adjust the status of CapLocks automatically)
  • Exit UI Scanning: Esc
  • Next Element: Tab
  • Previous Element: Shift + Tab
  • Click the Selected Element: Enter
  • Move the mouse to the Selected Element: Shift + Enter

TODO

Note: most modules have been completed but not released here due to license issues. I am now making effort to crafting new alternatives that are compatible with MIT license.

  • Add voice module
  • Add settings module
  • Add support for control by AWSD
  • Add pure mouse mode