WIP: Initial implementation of Maplibre/vector tile rendering

[yourealwaysbe]

Addresses #272

I made an initial attempt to render vector tiles using the Open Street Map demos Shortbread and MapTiler MVT.

It is possible to do something with Mapnik. In particular, MVT or PBF files can be loaded, but the style files openstreetmap.org uses are in Maplibre JSON format. As far as i know, this can't be converted to Mapnik easily.

Instead i have drafted a version that uses Maplibre.

There's a bit of a snag that Maplibre seems to support Android and iOS more than desktop platforms. Prebuilt binaries of the core library are provided for e.g. Linux, but the documentation does not recommend directly coding with this API unless you are a Maplibre developer.

They do provide an example mbgl-render, but it needs headers that are not in the headers provided by the pre-built core.

My current solution is to add maplibre-native as a submodule.

Building

I am not good with Automake, so the current build process is not ideal and only works for Linux with OpenGL so far (there's a commented out version that will work with Vulkan in the Makefiles).

1. git submodule init then git submodule update.
2. Enter maplibre-native and compile Maplibre Linux OpenGL core following the docs (you can build mbgl-core instead of mbgl-render).
3. Autogen and build Viking.

I have not put any flags in for controlling the build (e.g. selecting Windows or Linux or not using Maplibre at all). Instead i just bodged it in as best i could so that it compiles. This is something to work on if the general approach seems feasible.

Additions

I have added a new Maplibre Rendering layer based on the Mapnik layer. There are two main sets of files: vikmaplibrelayer and maplibre_interface. The interface follows the example Maplibre renderer.

Adding a Layer in Viking

The arguments are:

• The JSON Maplibre style file. Like Mapnik, this style file also includes the sources of the tiles.
• The cache file location. This is an sqlite3 file that the renderer will create and use to cache tiles. It is a backend feature of the renderer. Perhaps this should just be in the usual tile cache of Viking.
• An API Key. This may be needed if the style file somehow needs an API key (sometimes the renderer complains if you're using a file that fetches remote files but you haven't set this).
• The alpha.

The JSON file is the main thing that needs any effort. There are two examples being demoed on openstreetmap.org. I'm not sure what the legalities of using them are. Thunderforest also has one vector style file available.

• The Shortbread Colorful JSON. This is a demo style that works without an API key. You need to edit it first though by adding https://vector.openstreetmap.org before the URLs in glyphs, sprite, and tiles.
• The Shortbread Eclipse JSON. As above, but night mode.
• The MapTiler MVT JSON. You can either download the file or enter https://api.maptiler.com/maps/openstreetmap/style.json?key=<my_api_key> into the JSON field when configuring the layer (but it will not save if it's a URL with a ?, see limitations). It's free to sign up, but the free API key will only get you OSM data from 2020. For up-to-date data, the subscription is $2,500 per year.
• The Thunderforest Atlas JSON. You need at least a free plan API key from Thunderforest. You can enter the URL https://api.thunderforest.com/styles/atlas/style.json?apikey=<my_api_key> directly into the JSON text field when configuring the layer (after substituting your API key).

The JSON file requires sources of three types of data (these sources are defined inside the JSON itself):

• The tiles. This can be a local .mbtiles file with a URL mbtiles://<path to file>.
• The fonts (in the glyphs section).
• The sprites (at the sprite URL). It seems that things like @2x.png are added to this URL to get the style sheets.

To some extent you can try to mix and match different sources, but it doesn't always work out.

Limitations

The renderer cannot render multiple tiles on the same map at the same time. I have made a single-thread thread pool for it. Sometimes, with a badly configured JSON file, the renderer seems to hang, which stops all Maplibre Rendering layers from rendering.

Sometimes, i guess if text touches tile boundaries, it might get a bit clipped.

Only static libraries are compiled/provided by Maplibre. It adds about 140mb to the size of the Viking executable. An alternative approach might be to call out to mbgl-render, but this means an external process loading the cache (and possibly large .mbtiles file) for each tile.

The rendered tiles are not cached on disk. Perhaps we don't want to.

The style file URLs can be entered directly when configuring the layer, but they have ?s in before the API key argument. Viking does not seem to like saving/reloading URLs with ?s in them, so erases the value during save/load. Is there a fix for this?

TODO

Lots, it's very rough. But off the top of my head.

• Proper build system.
• Load/display copyright correctly.
• Maplibre licensing needed in docs.
• Docs for how to use (once settled).
• The "Maplibre features" dialog doesn't do anything.
• I'm not sure how much multiple Mapnik layers tread on each other's toes.

Video

maplibre.mp4

[rnorris]

So, I've finally had the time to look into this.

Looks and works great (I've only used the colourful.json).

Although it does complicate the build somewhat...
I've made fixes so that it can be included in the build (as per Point 1), defaulted to off; so interested users can enable it and try it.

Ultimately, I want to focus on trying to release v1.11 by the end of the year; so probably won't seek to include this at that point, but we shall see...

NB: May want the default tile cache file to be something like:

"/home/rob/.cache/viking/tile_cache_maplibre.sqlite"

So would want the cache_default() to use something like how vikmapslayer.c uses mpl_dir_default() + maps_layer_default_dir () with g_get_user_cache_dir() and also put maplibre in the name for clarity.

[yourealwaysbe]

Ultimately, I want to focus on trying to release v1.11 by the end of the year; so probably won't seek to include this at that point, but we shall see...

Yeah, i think it's too early for putting it in a release. It's just a proof of concept at this stage to see if it's a reasonable route. The vector tile provision seems to be a bit nascent at the moment too (or maybe i just miss the OpenCycleMap style). I'm also focused on another project for the near future.

[rnorris]

A few other thoughts:

• Ensure the 'Copyright' line in new files are yourself (not me!) and update the current year
• configure.ac: A few of the additional 'AC_CHECK_LIB' can be avoided since already tested
  • curl is already mandated
  • bz2, is already checked, so would only fail if explicitly turned off, hence can abort if so, e.g. (I've not tested this):

      if test "$ac_cv_enable_bzip2" = "no"; then
        AC_MSG_ERROR([libbz2 is needed for maplibre build])
      fi

• libjpeg + libpng are probably already dependencies of GTK, but I suppose no real harm in checking.
• configure.ac: typo in 'libwepb is needed but not found' --> libwebp
• Will need to update CI / devcontainer / docs for updated dependencies (libuv + libGL et al):
  • .github/workflow/build*yml
  • .devcontainer/devcontainer.json
  • README.md
• Ensure optional build works (I should have a patch for this...)
  • And then disable in Windows build (mingw64.spec.in - just like how mapnik is disabled).