[heatmap] Optionally use map as background

[laufhannes]

Depending on the heatmap's size, it may be nice to see at least some basic map in the background.

I have a first working solution with laufhannes@ebb0823 and laufhannes/staticmaplite@3758e11, but that's not perfect. The main part within this repo is to calculate lat/lng bounds of the visible part. Afterwards, a static map is requested, saved locally and inserted. (Without saving it locally, inkscape is not able to embed the image when converting to png)

The staticmaplite repo is written in php, but it should be possible to transform it into python. Using Google Maps Static may even be better (but requires an API key; in general free for non-commercial and low traffic apps), as it's possible to individually style the map. I created one first try for such a style at snazzy maps.

If using Google Maps, calculation of the required zoom level and cropping the image (or calculating the exact size) has to be done within this repo.

---

As this testing has been done in my local runalyze-environment, the example is decorated with our brandings ;)

And I guess this needs some additional padding within the draw area to not let tracks touch the border.

[lefty01]

wow ... nice!

[flopp]

Very nice.

I already have some experience with static maps: https://github.com/flopp/go-staticmaps ;)

[lowtower]

@flopp @laufhannes

Hello,
are You interested in getting this done or has it been dropped for a reason?
Cheers,
LT

[flopp]

@lowtower it would still be nice to have this feature. One could certainly use https://github.com/flopp/py-staticmaps to generate the background image.

[lowtower]

I have already had a look at py-staticmaps (which btw. is also mentioned at the osm staticmaps website).
To be honest, I would prefer not to take it because it has pycairo as requirement, which is quite hard to install. I think it is quite a big obstacle for a lot of users (You need dev packages under linux) .
The approach of laufhannes to take the staticmaps provider and url directly as argument seems to be straight forward.
I'll try and make a pr when ready.

[lowtower]

btw:
I find the carto dark no labels tiles the best as background.
As far as I have seen they are yet not supported by py-staticmaps, right?

[flopp]

@lowtower you're right, cairo is heavy - i already thought about making it an optional dependency of py-staticmaps (e.g. use it only if it's available).

Anyway, I'm glad you're taking the initiative on this issue!

[lowtower]

@flopp
I have seen that You already have made cairo optional in py-staticmaps - so, I will try my best to use it instead of other options!

[flopp]

@lowtower I'm about to add a PIL-based renderer - this is more lightweight than Cairo, but has no anti-aliasing

[lowtower]

@flopp I have a first working version of the background staticmap feature.
You can have a look at my fork.
I have also forked the py-staticmaps project to add object independent boundaries.
At the moment there is no margin to respect the line width of the tracks, means that the tracks are some pixels wider than the background image.
Hope You like it nevertheless.
Cheers,
LT.

[flopp]

@lowtower the results look nice!

For a final version the background image should be embedded into the SVG (to have a single output file) I guess.

[lowtower]

Reminder to myself:
the copyright notice is also missing because it's cropped away - must be added again.

[lowtower]

Somehow, I feel like the cropping should be done in the py-staticmaps.
What do You think?

[flopp]

Hm, py-staticmaps should create the desired image size - so that no cropping is necessary in GpxTrackPoster.

[lowtower]

Maybe, I understood something fundamentally wrong.
I have to look over it again.

[lowtower]

Hm, py-staticmaps should create the desired image size - so that no cropping is necessary in GpxTrackPoster.

The image size is as desired, but the boundaries get an additional border/margin.

I ran the example draw_gpx.py.
The resulting image has a left border of 60px and a right border of 59px to the gpx track (image border to object bounds - width=2 -> 2 extra pixels).
For the background image I need the staticmap without any borders to match the gpx heatmap exactly (or at least, I need to define the border myself).
That's why I cropped the borders of the staticmaps image away afterwards.
I haven't figured out yet where the extra 60/59px come from in py-staticmaps.
Can You help?

[lowtower]

A different approach would be to take the map from py-staticmaps as is and scale the tracks afterwards.

Pro:

• tracks have a padding/border
• copyright notice is not being cropped away
• map logic stays in py-staticmaps

Con:

• heatmap of tracks will be smaller than without background image

Shall I rather continue with this approach?
Suggestions?

[yihong0618]

A different approach would be to take the map from py-staticmaps as is and scale the tracks afterwards.

Pro:

• tracks have a padding/border
• copyright notice is not being cropped away
• map logic stays in py-staticmaps

Con:

• heatmap of tracks will be smaller than without background image

Shall I rather continue with this approach?
Suggestions?

I think it woud be nice, because user can choose with map or not.

[lowtower]

I think it woud be nice, because user can choose with map or not.

That's true for both variants.
In the first variant I keep the tracks as is and crop the background image accordingly. Tracks with or without bg image have the same scale.
In the second variant, I take the background image as is and scale the tacks accordingly - scale is different.

[lowtower]

WIP: First version of second variant (not pixel perfect yet):
https://github.com/lowtower/GpxTrackPoster/tree/heatmap-tiles-background-2

[lowtower]

Hello,
I have uploaded some showcase examples to illustrate the above:

• variant 1
  -- tracks are the same, background image is cropped to fit the tracks
  -- without background (former standard)
  -- with background (new)
• variant 2
  -- background image unmodified from py-staticmaps, tracks are scaled to fit the background image
  -- without background (former standard)
  -- with background (new)

Cheers,
LT

[yihong0618]

@lowtower that's cool

[lowtower]

@flopp @yihong0618
I think both versions are acceptable.
I tend to go further with version 2 (take map from py-staticmaps "as is" and scale tracks to match it)
Agree?

Cheers,
LT

[yihong0618]

@lowtower agree~

[laufhannes]

@lowtower Thanks for working on this! Just tried your current branch and it looks promising.

Is there anything missing or is it nearly production ready? I see that your required changes for py-staticmaps have been merged recently (only a new release version is missing - and I'd like to use a few other tile layers, but that needs to be done in py-staticmaps as well, right?). And it might be good to use cairo for rendering if available. (I'm not a python developer, but as far as I understand modules can be loaded within a try-block at runtime)

And is there a special reason why --heatmap-tile-max-size is limited to max 3600px? When transforming the resulting svg to png we're using a height of at least 4000px.

[lowtower]

Hello @laufhannes,

thanks for the reply and testing.
It's been a while since I worked on this.

Is there anything missing or is it nearly production ready?

As far as I remember, I was not fully satisfied with my solution.
It is not "pixel perfect", which I think has to do with the projection of geodata o 2D maps (like mentioned and explained well here pystaticmaps/Issue21).
Might not generally be a problem, but I haven't considered and tested "edge-cases".
The second point is that I did some "crazy" calculations to fit the tracks to the map, which has to do with how pystaticmaps generates the map (being discussed currently in pystaticmaps/Issue22).
Thirdly, the lack of feedback ;-)

And it might be good to use cairo for rendering if available

Cairo is available in pystaticmaps. I haven't and don't get cairo installed on my machine. Could possibly be made optional then.

And is there a special reason why --heatmap-tile-max-size is limited to max 3600px?

I don't remember - I have to look into this.

I am quite busy at the moment, but will look into it quite soon

Cheers,
LT

[lowtower]

Hello @laufhannes,

And is there a special reason why --heatmap-tile-max-size is limited to max 3600px? When transforming the resulting svg to png we're using a height of at least 4000px.

A very large pixel size results in large images and an increased time to fetch the tile.
I played around with some sizes and found a reasonable maximum size (for me at least).
Secondly, the copyright notice of the background image gets invisible small with larger number of pixels.
That should be handled in py-staticmaps.
I have increased the pixel size to 4800. With higher values the "warning" message is logged instead of printed
A pixel size of 4800 results in svg files of about 20MB.

And it might be good to use cairo for rendering if available

I have added an option to chose the renderer in py-staticmaps, available are "pillow" (default) and "cairo".
I cannot test "cairo".
Could You test it then?
https://github.com/lowtower/GpxTrackPoster/tree/heatmap-tiles-background-2

Cheers,
Lowtower

[lowtower]

and I'd like to use a few other tile layers, but that needs to be done in py-staticmaps as well, right?

That's right.
If You tell me which tile layers You need, I could implement it, test and file a pull request in py-staticmaps

[lowtower]

Hello @laufhannes,

And is there a special reason why --heatmap-tile-max-size is limited to max 3600px? When transforming the resulting svg to png we're using a height of at least 4000px.

A very large pixel size results in large images and an increased time to fetch the tile. I played around with some sizes and found a reasonable maximum size (for me at least). Secondly, the copyright notice of the background image gets invisible small with larger number of pixels. That should be handled in py-staticmaps. I have increased the pixel size to 4800. With higher values the "warning" message is logged instead of printed A pixel size of 4800 results in svg files of about 20MB.

And it might be good to use cairo for rendering if available

I have added an option to chose the renderer in py-staticmaps, available are "pillow" (default) and "cairo". I cannot test "cairo". Could You test it then? https://github.com/lowtower/GpxTrackPoster/tree/heatmap-tiles-background-2

Cheers, Lowtower

I have made a correction in cairo output - should be working now.

If You want to test it, You have to update the py-staticmaps library to latest source from github - as You said above, by now no new version has been pushed:
make setup then
pip uninstall -y py-staticmaps
pip install --upgrade git+https://github.com/lowtower/[email protected]#egg=py-staticmaps

Cheers,
LT

[laufhannes]

That's right. If You tell me which tile layers You need, I could implement it, test and file a pull request in py-staticmaps

I was planning on doing a pr myself, but I realized that those providers are not free for commercial applications (I was interested in Stadia.AlidadeSmooth, Stadia.AlidadeSmoothDark, Jawg.Dark, Jawg.Light, CartoDB.Positron, CartoDB.PositronNoLabels, CartoDB.DarkMatter, see https://leaflet-extras.github.io/leaflet-providers/preview/). For our usecase we probably need to use a custom provider, that means we'd need to set a provider dynamically withing GpxTrackPoster. Not sure if that's possible with py-staticmaps. But now I know how to use a custom py-staticmaps fork.

I have made a correction in cairo output - should be working now.

Yes, that looks fine. Though, the only difference I see is the rendering of copyright notice, which looks much smoother with cairo.

[lowtower]

Hello @laufhannes,

Stadia.AlidadeSmooth, Stadia.AlidadeSmoothDark, Jawg.Dark, Jawg.Light, CartoDB.Positron, CartoDB.PositronNoLabels, CartoDB.DarkMatter,

• CartoDB.PositronNoLabels is already supported (equals "carto-nolabels")
• CartoDB.DarkMatter NoLabels is already supported (equals "carto-darknolabels")
• Jawg requires an accesstoken
• Stadia requires an accesstoken

The Carto Maps with labels could be added.
Maybe something like a user defined tile_provider with registration key could be added to py-staticmaps? (@flopp: what do You think?)

Cheers,
LT

P.S.:

• my repo for testing
• supports api_keys on command line
• supports stadia-maps (requires key)
• supports jawg maps (requires key)