Map styling on Windows 11

[stevegdavis]

I want to change some of the road colors (not yet edited any files) and I have done:-

1. cloned the OM repository (builds OK)
2. Install Python 3.10 from Microsoft Store (installing the latest version of Python gives errors telling me to downgrade Python as a workaround when running generate_drules.sh)
3. pip install protobuf
4. run tools\unix\ bash ./generate_drules.sh but I run into this error:-

PS C:\src\mobile\organicmaps\tools\unix> bash ./generate_drules.sh
rm: cannot remove './../../data/classificator.txt': No such file or directory
rm: cannot remove './../../data/visibility.txt': No such file or directory
rm: cannot remove './../../data/colors.txt': No such file or directory
rm: cannot remove './../../data/patterns.txt': No such file or directory
Building drawing rules for style clear/clear
rm: cannot remove './../../data/drules_proto_clear.bin': No such file or directory
rm: cannot remove './../../data/drules_proto_clear.txt': No such file or directory
Traceback (most recent call last):
File "C:\src\mobile\organicmaps\tools\kothic\src\mapcss_init_.py", line 325, in parse
import_text = open(import_filename, "r").read()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2544.0_x64__qbz5n2kfra8p0\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 14995: character maps to < undefined >

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\src\mobile\organicmaps\tools\kothic\src\libkomwm.py", line 626, in
main()
File "C:\src\mobile\organicmaps\tools\kothic\src\libkomwm.py", line 616, in main
komap_mapswithme(options)
File "C:\src\mobile\organicmaps\tools\kothic\src\libkomwm.py", line 228, in komap_mapswithme
style.parse(filename=options.filename, static_tags=mapcss_static_tags,
File "C:\src\mobile\organicmaps\tools\kothic\src\mapcss_init_.py", line 364, in parse
raise Exception(msg)
Exception: 'charmap' codec can't decode byte 0x81 in position 14995: character maps to < undefined >
File: ./../../data/styles/clear/style-clear/style.mapcss
Line: 7

Same results on Win10 and 11. Thanks for any help.

[RedAuburn]

https://stackoverflow.com/a/49562606/13351126 this looks like the issue,
on Windows apparently the default encoding isn't utf-8 (you can check with [System.Text.Encoding]::Default in PowerShell)

try checking out organicmaps/kothic#6 for the kothic submodule, and report back :)

[biodranik]

Maybe we should force utf-8 in all our related python scripts? Can you please try it?

[stevegdavis]

Yes I did try utf-8 before but will try again maybe I missed something, thanks.

[stevegdavis]

There are many files that contain open(....) but I tried (that I had tried before) changed the following in \tools\kothic\src\mapcss_init_.py line 320 from:
elif IMPORT.match(css):
log.debug("import found")
import_filename = os.path.join(basepath, IMPORT.match(css).groups()[0])
try:
css = IMPORT.sub("", css, 1)
import_text = open(import_filename, "r").read()
stck[-1][1] = css # store remained part
stck.append([import_filename, import_text, import_text])
wasBroken = True
break
except IOError as e:
raise Exception("Cannot import file " + import_filename + "\n" + str(e))

to:-
elif IMPORT.match(css):
log.debug("import found")
import_filename = os.path.join(basepath, IMPORT.match(css).groups()[0])
try:
css = IMPORT.sub("", css, 1)
import_text = open(import_filename, "r", encoding='utf-8').read()
stck[-1][1] = css # store remained part
stck.append([import_filename, import_text, import_text])
wasBroken = True
break
except IOError as e:
raise Exception("Cannot import file " + import_filename + "\n" + str(e))

Now I get error:-

Traceback (most recent call last):
File "C:\src\mobile\organicmaps\tools\kothic\src\libkomwm.py", line 626, in
main()
File "C:\src\mobile\organicmaps\tools\kothic\src\libkomwm.py", line 616, in main
komap_mapswithme(options)
File "C:\src\mobile\organicmaps\tools\kothic\src\libkomwm.py", line 264, in komap_mapswithme
set_start_method('fork') # Use fork with multiprocessing to share global variables among Python instances
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2544.0_x64__qbz5n2kfra8p0\lib\multiprocessing\context.py", line 251, in set_start_method
self._actual_context = self.get_context(method)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2544.0_x64__qbz5n2kfra8p0\lib\multiprocessing\context.py", line 243, in get_context
return super().get_context(method)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2544.0_x64__qbz5n2kfra8p0\lib\multiprocessing\context.py", line 193, in get_context
raise ValueError('cannot find context for %r' % method) from None
ValueError: cannot find context for 'fork'

[stevegdavis]

Output from [System.Text.Encoding]::Default in PS

IsSingleByte : True
BodyName : iso-8859-1
EncodingName : Western European (Windows)
HeaderName : Windows-1252
WebName : Windows-1252
WindowsCodePage : 1252
IsBrowserDisplay : True
IsBrowserSave : True
IsMailNewsDisplay : True
IsMailNewsSave : True
EncoderFallback : System.Text.InternalEncoderBestFitFallback
DecoderFallback : System.Text.InternalDecoderBestFitFallback
IsReadOnly : True
CodePage : 1252

[stevegdavis]

1. Changed \tools\kothic\src\mapcss_init_.py (import_text = open(import_filename, "r", encoding='utf-8').read()
2. set MULTIPROCESSING=False in libkomwm.py. It all builds OK and works on Android phone.