Caching not creating files

[dansku]

I am trying to generate cache for tiles, but some are not being generated, without an error message.

root@map-server ~/brain/tegola # ./tegola cache seed tiles 3/2/3
2022-08-29 10:48:58 [INFO] config.go:317: loading local config (config.toml)
2022-08-29 10:48:58 [WARN] postgis.go:332: Connecting to PostGIS with connection parameters is deprecated. Use 'uri' instead.
2022-08-29 10:48:58 [INFO] providers.go:82: registering provider(type): mappers (postgis)
2022-08-29 10:48:58 [INFO] seed_purge.go:191: zoom list: [0]
2022-08-29 10:48:58 [INFO] cache.go:204: waiting for workers to finish up
2022-08-29 10:48:58 [INFO] worker.go:72: cache seed set to not overwrite existing tiles. skipping map (mappers) tile (0/0/0)
2022-08-29 10:48:58 [INFO] cache.go:214: all workers are done
2022-08-29 10:48:58 [INFO] provider.go:256: cleaning up providers
2022-08-29 10:48:58 [INFO] postgis.go:1021: cleaning up postgis providers

root@map-server ~/brain/tegola/cache/mappers/tiles/3/2 # ll
total 44
drwxr-xr-x  2 root root  4096 Aug 29 00:53 ./
drwxr-xr-x 10 root root  4096 Aug 29 00:54 ../
-rw-r--r--  1 root root    59 Aug 29 00:52 1
-rw-r--r--  1 root root 28051 Aug 29 00:53 4
-rw-r--r--  1 root root   362 Aug 29 00:53 5

It's not generating the files, and to not return issues, anything I should be looking at?

Thanks

[ARolek]

@dansku looks like you closed this out, but was the issue related to the missing flag on the seed command? I believe it's --zxy

[dansku]

Hey @ARolek i did close because i didn't know anyone would answer it.

When I ran
./tegola cache seed --min-zoom 2 --max-zoom 3
there were some bigger tiles (1.4mb) on zoom 2 that didn't get cached, and tegola didn't return any error or anything.
the only way I got it to cache is by making the request via wget and waiting it to cache, as wget has a default timeout of 900s it, in the end worked out.

have you seen something like this?

[ARolek]

Can you post your config? If you're using the postgis provider, you should move to the mvt_postgis provider for better results.

https://tegola.io/documentation/configuration/#providers

[dansku]

here is the config, i will check the provider

hostname = "domain.com"
port = ":8080"

[webserver.headers]
Access-Control-Allow-Methods = "GET, OPTIONS"
Access-Control-Allow-Origin = "*"

[cache] # configure a tile cache
type = "redis"

# register data providers
[[providers]]
database = "maps" 
host = "xxx" 
name = "xxx"
password = "xxx" 
port = 5432 
srid = 4326 
type = "postgis" 
user = "tegola" 

#------------------------------------

[[providers.layers]]
geometry_fieldname = "geom"
id_fieldname = "h3_index_int"
name = "hiot_hotspots"
sql = "SELECT ST_AsBinary(geom) AS geom, h3_index_int, count, id FROM hiot_hotspots WHERE geom && !BBOX!"

[[providers.layers]]
geometry_fieldname = "geom"
id_fieldname = "h3_index_int"
name = "hiot_hotspots_denylist"
sql = "SELECT ST_AsBinary(geom) AS geom, h3_index_int, count, id, denylist FROM hiot_hotspots WHERE geom && !BBOX! AND denylist"

[[providers.layers]]
geometry_fieldname = "point"
id_fieldname = "h3_index_int"
name = "hiot_hotspots_points"
sql = "SELECT ST_AsBinary(point) AS point, h3_index_int, id FROM hiot_hotspots WHERE point && !BBOX!"

#------------------------------------

[[maps]]
# center = [22.4, -5.22, 3.0] # set the center of the map so the user is auto navigated to Bonn
name = "mappers"

#------------------------------------

[[maps.layers]]
max_zoom = 14
min_zoom = 7
provider_layer = "mappers.hiot_hotspots"

[[maps.layers]]
max_zoom = 14
min_zoom = 7
provider_layer = "mappers.hiot_hotspots_denylist"

[[maps.layers]]
max_zoom = 6
min_zoom = 2
provider_layer = "mappers.hiot_hotspots_points"

[ARolek]

Yeah you're using the postgis type for the provider. Look at the mvt_postgis provider for better results. Also make sure you have indexes on you're geometry columns.

[dansku]

Change provider type, give me a new error:

./tegola cache seed  --min-zoom 2 --max-zoom 3
2022-09-06 05:25:24 [INFO] config.go:317: loading local config (config.toml)
2022-09-06 05:25:24 [WARN] postgis.go:332: Connecting to PostGIS with connection parameters is deprecated. Use 'uri' instead.
2022-09-06 05:25:24 [INFO] providers.go:82: registering provider(type): mappers (mvt_postgis)
2022-09-06 05:25:24 [INFO] seed_purge.go:191: zoom list: [2 3]
2022-09-06 05:25:24 [INFO] cache.go:204: waiting for workers to finish up
2022-09-06 05:25:24 [INFO] worker.go:92: seeding map (mappers) tile (2/3/3) took: 105ms
2022-09-06 05:25:24 [INFO] worker.go:92: seeding map (mappers) tile (3/0/0) took: 105ms
2022-09-06 05:25:24 [INFO] worker.go:92: seeding map (mappers) tile (3/0/5) took: 140ms
2022-09-06 05:25:24 [INFO] worker.go:92: seeding map (mappers) tile (3/1/0) took: 154ms
2022-09-06 05:25:24 [INFO] worker.go:92: seeding map (mappers) tile (3/0/7) took: 165ms
2022-09-06 05:25:24 [INFO] worker.go:92: seeding map (mappers) tile (3/1/7) took: 165ms
2022-09-06 05:25:24 [INFO] worker.go:92: seeding map (mappers) tile (3/1/6) took: 168ms
2022-09-06 05:25:24 [INFO] cache.go:214: all workers are done
2022-09-06 05:25:24 [INFO] provider.go:256: cleaning up providers
2022-09-06 05:25:24 [INFO] postgis.go:1021: cleaning up postgis providers
Error: error seeding tile ({Z:2 X:2 Y:1}): ERROR: Unknown geometry type: -739748096 - Unknown (SQLSTATE XX000)

This could be an issue?!
Is there a way to know which is the row of this error, for example?

[ARolek]

Yea, you need to change you're query up a bit. Please refer to the link I posted earlier.

[dansku]

Yeah you're using the postgis type for the provider. Look at the mvt_postgis provider for better results. Also make sure you have indexes on you're geometry columns.

yeah just realized i don't, adding them now

[ARolek]

Here's the best documentation on the mvt_postgis provider: https://github.com/go-spatial/tegola/blob/master/mvtprovider/postgis/README.md

[dansku]

Yea, you need to change you're query up a bit. Please refer to the link I posted earlier.

ahh i see, working on it!

[dansku]

What a massive difference in performance!

Valid Geometries: Point, LineString, Polygon, MultiPoint, MultiLineString, MultiPolygon, GeometryCollection.

And this is the cell:

SRID=4326;POLYGON((-149.4079546040616 61.5682647740014,-149.416689067466 61.56599382773844,-149.4174479028639 61.56157822934728,-149.4094751212302 61.55943382297276,-149.4007425395111 61.56170418788818,-149.3999808578964 61.56611954041555,-149.4079546040616 61.5682647740014))

getting error:

could not register providers: error fetching geometry type for layer (hotspots): layer (hotspots) returned unsupported geometry type (<nil>)

[ARolek]

Glad that worked for you!

[dansku]

Glad that worked for you!

thank you for the help, so close from nailing this side of things ;)
any idea why it could be returning NIL?

[ARolek]

I would need to look at the data. You can turn on SQL debugging to see the query tegola is forming up and then inspect the results against the database directly.

https://tegola.io/documentation/debugging/

[dansku]

Will do, 5AM here, will go to sleep and check this in the morning.
Thank you very much 🚀

[Rub21]

I am facing a similar issue: I ran the command expecting to get tiles from zoom levels 0 to 7, but the result only includes a few of them, not all the tiles in the area zoom 0-7. Additionally, in my configuration, I am using mvt_postgis.

app# tegola cache seed \
    --config=/opt/tegola_config/config.toml \
    --map=osm \
    --min-zoom=0 \
    --max-zoom=7\
    --bounds=-180,-85.05112878,180,85.05112878 \
    --concurrency=128 \
    --overwrite=true

2024-11-27 18:42:22 [INFO] config.go:406: loading local config (/opt/tegola_config/config.toml)
2024-11-27 18:42:22 [INFO] providers.go:82: registering provider(type): osm (mvt_postgis)
2024-11-27 18:42:22 [INFO] providers.go:82: registering provider(type): ne (mvt_postgis)
2024-11-27 18:42:23 [INFO] seed_purge.go:194: zoom list: [0 1 2 3 4 5 6 7]
2024-11-27 18:42:23 [INFO] cache.go:213: waiting for workers to finish up
2024-11-27 18:42:35 [INFO] worker.go:94: seeding map (osm) tile (1/1/1) took: 12196ms
2024-11-27 18:42:36 [INFO] worker.go:94: seeding map (osm) tile (1/0/1) took: 13271ms
2024-11-27 18:42:45 [INFO] worker.go:94: seeding map (osm) tile (1/0/0) took: 22493ms
2024-11-27 18:42:58 [INFO] worker.go:94: seeding map (osm) tile (2/1/2) took: 35524ms
2024-11-27 18:42:59 [INFO] worker.go:94: seeding map (osm) tile (2/2/1) took: 36435ms
2024-11-27 18:43:09 [INFO] worker.go:94: seeding map (osm) tile (3/3/4) took: 46769ms
2024-11-27 18:43:11 [INFO] worker.go:94: seeding map (osm) tile (0/0/0) took: 48013ms
2024-11-27 18:43:11 [INFO] worker.go:94: seeding map (osm) tile (1/1/0) took: 48548ms
2024-11-27 18:43:18 [INFO] worker.go:94: seeding map (osm) tile (2/1/1) took: 55526ms
2024-11-27 18:43:20 [INFO] worker.go:94: seeding map (osm) tile (2/2/2) took: 57884ms
2024-11-27 18:43:23 [INFO] worker.go:94: seeding map (osm) tile (3/4/4) took: 60612ms
2024-11-27 18:43:24 [INFO] worker.go:94: seeding map (osm) tile (3/3/3) took: 61831ms
2024-11-27 18:43:32 [INFO] worker.go:94: seeding map (osm) tile (4/7/8) took: 69100ms
2024-11-27 18:43:33 [INFO] worker.go:94: seeding map (osm) tile (3/4/3) took: 70648ms
2024-11-27 18:43:36 [INFO] worker.go:94: seeding map (osm) tile (4/8/8) took: 73310ms
2024-11-27 18:43:36 [INFO] worker.go:94: seeding map (osm) tile (4/7/7) took: 73910ms
2024-11-27 18:43:45 [INFO] worker.go:94: seeding map (osm) tile (5/15/16) took: 82271ms
2024-11-27 18:43:45 [INFO] worker.go:94: seeding map (osm) tile (4/8/7) took: 82454ms
2024-11-27 18:43:47 [INFO] worker.go:94: seeding map (osm) tile (5/16/16) took: 84086ms
2024-11-27 18:43:48 [INFO] worker.go:94: seeding map (osm) tile (5/15/15) took: 85456ms
2024-11-27 18:43:53 [INFO] worker.go:94: seeding map (osm) tile (6/31/32) took: 90739ms
2024-11-27 18:43:56 [INFO] worker.go:94: seeding map (osm) tile (5/16/15) took: 92950ms
2024-11-27 18:43:56 [INFO] worker.go:94: seeding map (osm) tile (6/32/32) took: 93547ms
2024-11-27 18:43:57 [INFO] worker.go:94: seeding map (osm) tile (6/32/31) took: 94162ms
2024-11-27 18:44:05 [INFO] worker.go:94: seeding map (osm) tile (6/31/31) took: 102106ms
2024-11-27 18:44:07 [INFO] worker.go:94: seeding map (osm) tile (7/63/64) took: 104398ms
2024-11-27 18:44:07 [INFO] worker.go:94: seeding map (osm) tile (7/64/64) took: 104773ms
2024-11-27 18:44:08 [INFO] worker.go:94: seeding map (osm) tile (7/64/63) took: 105238ms
2024-11-27 18:44:15 [INFO] worker.go:94: seeding map (osm) tile (7/63/63) took: 112725ms
2024-11-27 18:44:15 [INFO] cache.go:223: all workers are done
2024-11-27 18:44:15 [INFO] provider.go:289: cleaning up providers
2024-11-27 18:44:15 [INFO] postgis.go:1039: cleaning up postgis providers

[Rub21]

Using the tiles-list option work well,
echo "0/0/0" > tile-list.tiles
tegola cache seed tile-list tile-list.tiles ...

[ARolek]

@Rub21 can you post the cache part of your config?

[Rub21]

Sure, here is my 👉 config . Maybe I missed something, but setting bounds=-180,-85.05112878,180,85.05112878 did not work. maybe that was removed. But using the tile-list to 0/0/0 an setting the zoom from 0-7 works fine.

$ echo "0/0/0" > tile-list.tiles
$ tegola cache seed tile-list tile-list.tiles \
    --config=/opt/tegola_config/config.toml \
    --map=osm \
    --min-zoom=0 \
    --max-zoom=7\
    --concurrency=128 \
    --overwrite=true