Ctrl-[ does not behave as Esc and CTRL-] as jump to ctag on Czech keyboard on Windows

[zbyna]

Describe the bug

Ctrl-[ and Ctrl-] on Czech keyboard on Windows does not behave as in plain console Neovim a Neovim QT.
I borrowed the text from this vim issue (thanks @risa2000), which is exact the same: Ctrl-[ no longer behaves as Esc on Czech keyboard on Windows (it worked in g/vim 8)

To Reproduce

Pressing Ctrl-[ (or FWIW Ctrl-] ) no longer works as expected (as Esc, or goto tag). It writes the Czech letters instead:
Ctrl-[ writes ú,
Ctrl-] writes ).
which are the actual letters on the Czech layout at the corresponding keys.
I did not check other Ctrl-<??> combinations.

Expected behavior

From what I read in those and some other threads about the Ctrl handling I suppose there is a confusion among some people how Ctrl-[ works on non English layouts. I will try to explain on the Czech layout on Windows.

Note: For someone not familiar with Czech layouts, there are several "Czech" layouts with different degree of localization, but the one I will be talking about is the one named just Czech (the others are usually certain mix of English and Czech layout).

On the Czech layout the key right to P (which on the English layout is the one with [ ) has a Czech letter ú. The symbol [ is however written by AltGr+f on the Czech layout. For English key ] (which is right to [) the situation on the Czech keyboard is similar again - there is ) on Czech layout and ] is written by AltGr+g.

When Microsoft was localizing the keyboard they actually did a smart thing and did not map Ctrl-[ to Ctrl-AltGr-f (as happened on some linux distros), but realized that Ctrl-[ is indeed a control code (and not an actual letter) and that the usability should be more important than formal correctness. We may say they did not localize Ctrl-[ by the character, but by the key (i.e. kept the same key in the layout regardless its localized character).

I have been used to use Ctrl-[ for Esc and it worked in linux terminal, or Windows, regardless the keyboard layout for years. The only dire cases were GUIs in some linux distros where they indeed expected users to write Ctrl-AltGr-f.

There is an alternative deep explanation (another big thank @risa2000) which may be more appropriate for somebody:
vim/vim#12595 (comment)

Desktop (please complete the following information):

• OS: Windows 10 Pro 22H2
• Neovide Version 0.12.2
• Neovim Version v0.9.5

Please run neovide --log and paste the contents of the .log file created in the current directory here:

TRACE [neovide::window::keyboard_manager] KeyEvent {
    physical_key: Code(
        ControlLeft,
    ),
    logical_key: Named(
        Control,
    ),
    text: None,
    location: Left,
    state: Pressed,
    repeat: true,
    platform_specific: KeyEventExtra {
        text_with_all_modifers: None,
        key_without_modifiers: Named(
            Control,
        ),
    },
}
TRACE [neovide::window::keyboard_manager] KeyEvent {
    physical_key: Code(
        BracketRight,
    ),
    logical_key: Character(
        ")",
    ),
    text: Some(
        ")",
    ),
    location: Standard,
    state: Pressed,
    repeat: false,
    platform_specific: KeyEventExtra {
        text_with_all_modifers: Some(
            "\u{1d}",
        ),
        key_without_modifiers: Character(
            ")",
        ),
    },
}
TRACE [neovide::window::keyboard_manager] Key pressed <C-)> ModifiersState(CONTROL)
TRACE [neovide::channel_utils] UICommand Serial(Keyboard("<C-)>"))
TRACE [neovide::bridge::ui_commands] In Serial Command
TRACE [neovide::bridge::ui_commands] Keyboard Input Sent: <C-)>
TRACE [neovide::bridge::handler] Neovim notification: "redraw"
TRACE [neovide::channel_utils] neovim_handler GridLine { grid: 5, row: 0, column_start: 159, cells: [GridLineCell { text: ")", highlight_id: Some(130), repeat: None }] }
TRACE [neovide::channel_utils] neovim_handler Flush
TRACE [neovide::editor] Image flushed
TRACE [neovide::editor] send_batch
TRACE [neovide::renderer::fonts::caching_shaper] Shaping text: "/vimtut                                                                                                                                                 [1/1]  )"
TRACE [neovide::bridge::handler] Neovim notification: "redraw"
TRACE [neovide::channel_utils] neovim_handler GridLine { grid: 5, row: 0, column_start: 159, cells: [GridLineCell { text: " ", highlight_id: Some(130), repeat: None }, GridLineCell { text: " ", highlight_id: Some(0), repeat: Some(0) }] }
TRACE [neovide::channel_utils] neovim_handler HighlightAttributesDefine { id: 532, style: Style { colors: Colors { foreground: Some(Color4f { r: 1.0, g: 1.0, b: 0.8784314, a: 1.0 }), background: Some(Color4f { r: 0.15686275, g: 0.30980393, b: 0.15686275, a: 1.0 }), special: None }, reverse: false, italic: false, bold: false, strikethrough: false, blend: 0, underline: None, infos: [HighlightInfo { kind: Syntax, ui_name: "", hi_name: "Identifier", id: 139 }, HighlightInfo { kind: Ui, ui_name: "Search", hi_name: "Search", id: 438 }] } }
TRACE [neovide::channel_utils] neovim_handler GridLine { grid: 6, row: 8, column_start: 40, cells: [GridLineCell { text: "v", highlight_id: Some(532), repeat: None }, GridLineCell { text: "i", highlight_id: None, repeat: None }, GridLineCell { text: "m", highlight_id: None, repeat: None }, GridLineCell { text: "t", highlight_id: None, repeat: None }, GridLineCell { text: "u", highlight_id: None, repeat: None }, GridLineCell { text: "t", highlight_id: None, repeat: None }] }
TRACE [neovide::channel_utils] neovim_handler GridLine { grid: 1, row: 18, column_start: 158, cells: [GridLineCell { text: "6", highlight_id: Some(390), repeat: None }, GridLineCell { text: "/", highlight_id: None, repeat: None }, GridLineCell { text: "2", highlight_id: None, repeat: None }, GridLineCell { text: "0", highlight_id: None, repeat: None }, GridLineCell { text: "5", highlight_id: None, repeat: None }, GridLineCell { text: "☰", highlight_id: None, repeat: None }, GridLineCell { text: " ", highlight_id: None, repeat: None }, GridLineCell { text: "℅", highlight_id: None, repeat: None }, GridLineCell { text: ":", highlight_id: None, repeat: None }, GridLineCell { text: "2", highlight_id: None, repeat: None }, GridLineCell { text: "1", highlight_id: None, repeat: None }] }
TRACE [neovide::channel_utils] neovim_handler WindowViewport { grid: 6, top_line: 26.0, bottom_line: 44.0, current_line: 35.0, current_column: 6.0, line_count: Some(205.0), scroll_delta: Some(0.0) }
TRACE [neovide::channel_utils] neovim_handler CursorGoto { grid: 6, row: 9, column: 20 }
TRACE [neovide::channel_utils] neovim_handler Flush
TRACE [neovide::editor] Image flushed
TRACE [neovide::editor] send_batch
TRACE [neovide::renderer::rendered_window] Handling Viewport 6
TRACE [neovide::renderer::fonts::caching_shaper] Shaping text: "\u{e0a1}:36/205☰ ℅:21"
TRACE [neovide::renderer::fonts::caching_shaper] Shaping text: "R"
TRACE [neovide::window::keyboard_manager] KeyEvent {
    physical_key: Code(
        BracketRight,
    ),
    logical_key: Character(
        ")",
    ),
    text: None,
    location: Standard,
    state: Released,
    repeat: false,
    platform_specific: KeyEventExtra {
        text_with_all_modifers: None,
        key_without_modifiers: Character(
            ")",
        ),
    },
}
TRACE [neovide::window::keyboard_manager] Modifiers { state: ModifiersState(0x0), pressed_mods: ModifiersKeys(0x0) }
TRACE [neovide::window::keyboard_manager] KeyEvent {
    physical_key: Code(
        ControlLeft,
    ),
    logical_key: Named(
        Control,
    ),
    text: None,
    location: Left,
    state: Released,
    repeat: false,
    platform_specific: KeyEventExtra {
        text_with_all_modifers: None,
        key_without_modifiers: Named(
            Control,
        ),
    },
}
TRACE [neovide::channel_utils] UICommand Parallel(Quit)
TRACE [neovide::bridge::handler] Neovim request: "neovide.quit"
INFO [neovide::running_tracker] Quit with code 0: Quit from neovim
INFO [neovide::running_tracker] Quit neovim processed failed
DEBUG [neovide::settings::window_size] Saved Window Settings: {"window":{"Windowed":{"position":{"x":159,"y":150},"pixel_size":{"width":1708,"height":500},"grid_size":{"width":170,"height":22}}}}
DEBUG [neovide::settings::window_size] Saved Window Settings: {"window":{"Windowed":{"position":{"x":159,"y":150},"pixel_size":{"width":1708,"height":500},"grid_size":{"width":170,"height":22}}}}

Additional context
Add any other context about the problem here.

[risa2000]

@zbyna
Just for the record, this has been fixed in vim repo with patch 9.1.0050 here (vim/vim@68d9472). It may not be a problem with neovide, but rather with neovim, which merged the broken code from vim before. I do not know how neovim uses code from vim, but if they merge regularly vim updates, they may as well merge the patch soon.

[fredizzimo]

I don't think that patch is relevant since it applies to gvim only.

I'm not sure if we have enough information here to send the correct code to Neovim. The KeyEvent contains the information we get from https://github.com/rust-windowing/winit.

There is

        text_with_all_modifers: Some(
            "\u{1d}",
        ),

Which corresponds to ctrl-j, but that's generally only something a terminal emulator can use, because if we use that for other keys, then a lot of things will be unmappable.

I do think that winit reports wrong here. It should probably report [ and ], instead of ( and ú. Can you open an issue on the winit repository, so that this can be further discussed there.

[zbyna]

@risa2000
Probably I did not express clearly the fact that shortcuts Ctrl-[ and Ctrl-] work in plain Neovim and Neovim QT on Czech keyboard on Windows. When I use the same Neovim version with Neovide, above mentioned shortcuts do not work. I may be wrong, but in my opinion these facts indicate that the issue is on Neovide side.

[fredizzimo]

Neovim uses the terminal for the keyboard handling, and neovim-qt uses qt. We use winit, and this appears to be a bug in that library, for not providing us the correct information.

[zbyna]

@fredizzimo Thanks for explanation. I will try to create an issue related to https://github.com/rust-windowing/winit.

[risa2000]

@zbyna
Alright, I thought that the interaction between neovide and neovim was different (and was wrong :)). In that case, as @fredizzimo suggested, it looks like a problem with winit.

[risa2000]

I don't think that patch is relevant since it applies to gvim only.

Just one technical clarification as I am not clear on what you meant by "gvim only". The original problem affected both gvim (the GUI version) and vim the "console" version, and the patch solved it for both. Apart from that I guess you were right and it looks like a problem in the other library.

[zbyna]

Issue created: rust-windowing/winit#3621

[fredizzimo]

Thanks the issue looks good, I can help filling in more details if needed when we get a response from the Winit team.

[zbyna]

Since Winit team does not respond, I created a temporary naive fix in my neovide fork zbyna@c12b001 . I am not daring to fix Winit because I have no experience with Rust at all.

[zbyna]

I dared to create fix in library: rust-windowing/winit#3696 I have a feeling 🙂 I will need some sort of support. Thanks.