-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Full unicode support #306
Comments
(The challenge, of course, is that a lot of these features may interfere with the way the terminal is expected to work; and also it may be difficult to implement them in a performant way. If you have to call freetype to reraster things every frame you lose the benefits of GPU rendering. So it might be reasonable to put this off for a while.) |
You might want to check mlterm which supported bidirectional text for a long time, and it latest versions seems to have added more support for complex text using HarfBuzz as well. |
I just want to confirm that it's not a font issue - I'm running Alacritty on Arch Linux and all Hebrew text just shows up as boxes. I'm not talking being displayed right-to-left - as said before, many terminals just keep everything left to right - but no matter what font I choose I can't even see the letters. Is this normal or am I missing something? |
@bjesus Not sure why you say this isn't a font issue. That's definitely a font issue. The font alacritty is looking in for those glyphs doesn't have them. |
Hi! Thanks for Alacritty! I never said it wasn't font issue, I was only asking... and the reason I thought so is that I tried many different fonts that worked just fine on other terminals. Any suggestion for a font that I should try? I've tried monospace, "Droid Sans Mono", "Consolas" and even "Arial". None of them showed the Hebrew characters. |
@bjesus try Cousine (from ttf-croscore) |
@bjesus the reason those other fonts work is not because they have the glyphs, but because the application you're using supports fallback fonts--something we plan to add to Alacritty. |
Thanks @N-006 and @jwilm , it work just fine now. Sorry about the misunderstanding, I didn't think about fallback fonts. |
For people (like me) looking on how to enable unicode on Alacritty. There is now an RFC in #957 for font configuration - including fallback fonts, which would solve most unicode issues. Tooke me a few clicks to find out about it, figured linking back here would be useful! ✨ |
printf '\xF0\x9D\x9C\x99' is supposed to print 𝜙, but on Alacritty it prints an empty space. Is this due to the limitations that this thread refer to? Or is it a bug? |
@horta Most likely a font issue. |
I'm using macos, so it defaults to Menlo. However, using the same Menlo font on the Terminal.app it prints correctly. |
I'm also experiencing issues which I believe is due to unicode width. I use a "lambda" symbol in my prompt. Two issues noticed:
|
On macOS 10.14.4 using a german keyboard just typing an umlaut (like ä) produces |
Do you have the correct locale set? |
No, I didn’t know about that! iTerm seems to infer it from the system settings, as |
I missed this issue. related? |
I think I have a related problem: 2-byte long characters are not properly deleted using the backspace key, in fact only one byte is deleted, leading to invalid unicode characters. |
@jackroi Two byte characters, or two grapheme characters? It's most likely a problem with your shell. |
@chrisduerr I'm not sure. Here is an example of the characters in question: |
That should work just fine. It's likely a problem with your shell. |
But if it were a problem with my shell, the other terminals would also have the same problem, right? If someone wants to try to reproduce the problem, an easy way is as follows: cat > TEST
helloò<BACKSPACE>
helloò<BACKSPACE><BACKSPACE> # the terminal renders this as "hell", as it should be Then running
Note the |
can't repro on my system. |
there is certainly some weirdness happening if you use the U+0300 combining grave rather than the precomposed U+00F2: $ cat >TEST # typing "helloo", then inserting a U+0300
hello
hell
^C
$ hexdump -C TEST
00000000 68 65 6c 6c 6f 6f 0a 68 65 6c 6c 6f 0a |helloo.hello.|
0000000d
$ cat >TEST # typing "hello", then inserting a U+00F2
hello
hell
^C
$ hexdump -C TEST
00000000 68 65 6c 6c 6f 0a 68 65 6c 6c 0a |hello.hell.|
0000000b note that they render the same (I press backspace once on the first line, twice on the second line), but produce different files. |
what's the output of |
$ stty
speed 38400 baud; line = 0;
-brkint -imaxbel iutf8 |
I think this issue (306) is a sort of "meta" issue, so I think the problem is better to be tracked as a separate (new) issue. Anyway, I can't reproduce the issue (on zsh-5.8 on alacritty-0.5.0). |
just fyi, we don't perform unicode normalization, so it's expected? you should write unicode-normalize string to a file if you want it to be that way. From what I can see everything is working is expected with your file test. |
I agree, I'd rather not bump this issue. |
I have updated alacritty to version 0.6-dev (from 0.4.3) and the problem no longer occurs. Sorry, I should have updated before asking. Thank you all. |
I'm going to close this issue since it is rather unspecific. Many of the issues mentioned here already have dedicated issues like bidi (#663) or ligatures (#50). Though generally I don't see Alacritty implementing many of these features in the future, since most of them simply come with too big of a performance impact on all users to justify adding them for the few users that actually need them. |
TL;DR: Supporting unicode is hard, and it might be a good idea to use a library that knows how to do it well. Maybe look into using harfbuzz as well as freetype for font rendering. Unicode-width is good but doesn't do everything.
The problem of translating sequences of unicode codepoints to actual you-can-draw-this-on-screen glyphs, supporting things like character width (#265), ligatures (#50), bidirectional text (like in arabic), and text reordering (!), is called complex text layout. It is, appropriately, complex, and most terminals don't actually do it very well.
Windows and Mac both have systems that perform layout, integrated with their font rasterizers:
There's also ICU, which is cross-platform and supported by IBM (I think?)
Of the available options:
It seems to me like harfbuzz is the best option in terms of cross-platform support and level of control. You basically hand it a line of text and it tells you all the glyphs to draw in that line. Keep in mind I'm not actually a font rendering person, though, and it's possible I'm missing important details here.
This would probably have a performance cost, but I'm not sure how much of a performance cost. With a clever implementation it might not be too bad, and would be a huge boon to international users.
Other relevant links:
The text was updated successfully, but these errors were encountered: