Problems with encoding UTF-8 in Powershell

[antoningfors]

Per default Windows 10 and Windows 11 do not use UTF-8 unless you manually enable as a Beta feature under Region Settings in intl.cpl

Without that setting enabled both Sublime and Powershell are having problems reading and displaying special characters such as "åäö".

Here is an example below on a machine without UTF-8 enabled running files from the Powershell console.

File encoding: UTF-8

File encoding: UTF-8 with BOM

As you can see Powershell is unable read the string when the file encoding is "UTF-8" and everything works fine when the encoding is set to "UTF-8 with BOM".

Here is another example with the Build Panel is Sublime Text
File encoding: UTF-8

File encoding: UTF-8 with BOM

As you can see the output is still wrong in the Build Panel even when the encoding is set to "UTF-8 with BOM". But it can be fixed by running chcp.exe in cmd

And then changing the encoding settings in the build job.

As stated before this is only an issue when the UTF-8 Beta feature in Windows is disabled. If you enable UTF-8 support the Sublime Text build panel and powershell console works without any issues.

I am not sure how to handle the build panel problem but at least changing the default encoding format for Powershell files to "UTF-8 with BOM" would prevent scripts processing special characters from causing unwanted bugs. This could probably be done by updating PowershellSyntax.sublime-settings with "default_encoding": "UTF-8 with BOM". But I am not sure how that would affect Linux or Powershell Core users.