[Bug] webusb-serial issue with Chrome on Android

[hathach]

Describe the bug

On a Samsung S9+ phone, webusb-rgb works fine, webusb-serial will not write text in the receiver window when typed in the sender window.
All demos require connect which works, corresponding LED or NeoPixel light as expected on Connect
So connect code appears 100% ok
Just the echo for Android Chrome may be trouble.
I switched from Samsung keyboard to Google Keyboard, no change
Adafruit test with CPS board and code in GitHub

Set up (please complete the following information)

• Your Board and BSP version: Circuit Playground Express BSP 1.5.3
• TinyUSB library version: 0.6.0
• Your OS (mac/windows/linux) and its version: Samsung S9+ v
• Serial debug log if any: n/a

To Reproduce
Steps to reproduce the behavior: Normal testing

Expected behavior
Text should be echo to receiver on landing page

[mars-low]

The problem is not specific to USB implementation and is actually caused by keypress event being used in web application. Chrome on Android doesn't support keypress event for Gboard and it "won't be fixed". However some codes may be detected properly. There is a bug for this: https://code.google.com/p/chromium/issues/detail?id=118639. I've got some characters working with Hacker's Keyboard but not letters, specifically: 1234567890[]=\;',
Also I've been able to see output in receive window when I used Serial USB Terminal app for communicating with CDC interface.

Possible workaround is to replace keypress event with input event. But then it is necessary to parse content manually to get new characters as it returns full content of input field. The following is one possible way to see output in receiver window on Android.

    commandLine.addEventListener("keypress", function(event) {
      if (event.keyCode === 13) {
        if (commandLine.value.length > 0) {
          addLine('sender_lines', commandLine.value);
          commandLine.value = '';
          port.send(new TextEncoder('utf-8').encode('\r'));
        }
      }
    });

    commandLine.addEventListener("input", function(event) {
      let textContent = event.target.value;
      let lastChar = textContent.slice(textContent.length - 1);

      port.send(new TextEncoder('utf-8').encode(lastChar));
    });

Thanks for TinyUSB library!

[hathach]

thank you for explanation and the suggestion. #151 try to use keydown event instead of keypress, which seems to be supported on Android. would you mind trying it out to see if that works on your board.

[mars-low]

Unfortunately, it looks like keyboard events on Android Chrome are messy. Keydown event is detected, but doesn't return practical value under key property. It always returns 229 code (which is Undefined). You can test it yourself on your phone using this website: https://w3c.github.io/uievents/tools/key-event-viewer.html.

Result seen on WebUSB Serial page is as follows:

Frankly speaking I don't know any other cross-platform way than using input event and manually parsing for changes in input field. Other workaround would be to send message only after some button is clicked, but then it won't feel like data is propagated in real time, so it depends on your requirements.

For the sake of completeness, actually I'm not using Adafruit_TinyUSB_Arduino library or Arduino core.
I am testing WebUSB Serial example with Pi Pico board and https://github.com/raspberrypi/pico-sdk. Binary is built from repository: https://github.com/mars-low/pico-usb-gadget and USB part is copy-paste from official TinyUSB example.
I've posted it here, because issue is independent of used board. Some time ago I've also tested it with Arduino Nano 33 IoT used in conjuction with https://github.com/adafruit/ArduinoCore-samd (files to support Arduino Nano 33 IoT were copied from mainstream). I've tested it on web using Blazor app https://github.com/mars-low/HardwebBlazor and I saw for the first time that communication works correctly on Android when API is used programatically. Because it was working for my purpose I haven't digged further until recently I've come across Pi Pico and saw the same problem.

[hathach]

thanks @mars-low , the keyboard event viewer page is very handy. I confirm my android v9 keycode and key is both unidentified. Weirdly only Enter key is correctly reported. The issue with the walkaround is handling backspace and enter, in which we should send \r and backspace, instead it will send nothing (with enter) and the next-to-last char (with backspace).

For the sake of completeness, actually I'm not using Adafruit_TinyUSB_Arduino library or Arduino core. I am testing WebUSB Serial example with Pi Pico board and https://github.com/raspberrypi/pico-sdk.

Thanks for the info, it is pretty much the web application. So this would help other as well. Once it is confirmed to work, I will update code on the main repo (then get pull by pico-sdk etc ..). The webserial is actually not very helpful app, I am thinking to change/add another app that would allow to control the on-board LED and/or report the state of on-board button. Which is more apparent that the code run on the actual hardware. (the echo can be easily thought as web-only echoing).

Thank you for following this up, I am sure we could figure out way to handle enter/backspace with android. I am in the middle of other works, and will come back to this later on.