Honour "load layers as single WMS layer" and "get images directly from WMS server"

[maxencelaurent]

Problem description

In most of our QGIS projects, we usually define several WMS layers from third-party servers on top of the base map.
On top of that, we add several/many vector layers.

Since its introduction, the ‘Load as single WMS layer’ option has been extremely useful.
It has even become essential for ensuring legible labelling of the different layers.

Typical project looks like:

Project                                                                               
│                                                             
├── Group 1
│   ├── Postgis layer 1                                               
│   ├── Postgis layer 2
│   ├── Postgis layer ...  
│   └── Postgis layer n                                                                  
├── Group 2
│   ├── Postgis layer 1
│   ├── Postgis layer 2
│   ├── Postgis layer ...
│   └── Postgis layer n
├── Overlays
│   ├── WMS Overlay 1 (eg natural hazards, geological, cadastrial, water projection area...)
│   ├── WMS Overlay ...                     
│   └── WMS Overlay n  
└── baselayers
    ├── WMSLayer1 (Get images directly from WMS server)        
    ├── WMSLayer2 (Get images directly from WMS server) 
    ├── WMSLayer3 (Get images directly from WMS server)                                                              
    ├── WMSLayer4 (Get images directly from WMS server) 
    └── WMSLayer5 (Get images directly from WMS server) 

Unfortunately, third-party WMS layers we need are often provided by rather slow servers.
Such requests can take several seconds to complete.
Hence, including those layers in the "as single WMS query" leads to very slow responses.

It would be very helpful to have the ability to exclude those layers from the "single" query.

Current behaviour

If Load layers as single WMS layer is checked, the Get images directly from WMS server setting is ignored, except for baselayers :

Project                                                                   Request sent
│                                                             -------------------------------------------
├── Group1
│   ├── Layer1                                                ┐
│   ├── Layer2                                                │
│   ├── WMSLayer3  (Get images directly from WMS server)      │
│   ├── Layer4                                                │
│   └── Layer5                                                │ Get as single WMS request
├── Group2                                                    │
│   ├── WMSLayer1 (Get images directly from WMS server)       │
│   ├── WMSLayer2 (Get images directly from WMS server)       │
│   └── Layer3                                                ┘
└── baselayers
    ├── WMSLayer1 (Get images directly from WMS server)       ┐
    ├── WMSLayer2 (Get images directly from WMS server)       │
    ├── WMSLayer3 (Get images directly from WMS server)       │ => exclusive group, 1 request directly to its own WMS server
    ├── WMSLayer4 (Get images directly from WMS server)       │
    └── WMSLayer5 (Get images directly from WMS server)       ┘

Solution

Obviously, it's not possible to strictly honour both request directly from WMS server and Load layers as single WMS layer settings in non-trivial cases : either layers order or load as single setting couldn't be respected.

First variant: best-effort

With this variant, Load layers as single WMS layer means "Load layers using as few requests as possible".
LWC should group layers until reaching a non-groupable layer.

Effective number of queries depends on the layers orders. In trivial project (all external layers grouped front or back), only one request will be sent.

Project                                                                   Request sent
│                                                             -------------------------------------------
├── Group1
│   ├── Layer1                                                ┐ Get Layer 1 & 2 as single WMS request
│   ├── Layer2                                                ┘
│   ├── WMSLayer3 (Get images directly from WMS server)       - 1 request directly to its own WMS server
│   ├── Layer4                                                ┐ Get Layer 4 & 5 as single WMS request
│   └── Layer5                                                ┘
├── Group2                                                    
│   ├── WMSLayer1 (Get images directly from WMS server)       - 1 request directly to its own WMS server
│   ├── WMSLayer2 (Get images directly from WMS server)       - 1 request directly to its own WMS server
│   └── Layer3                                                - Get Layer 3 as single WMS request
└── baselayers
    ├── WMSLayer1 (Get images directly from WMS server)       ┐
    ├── WMSLayer2 (Get images directly from WMS server)       │
    ├── WMSLayer3 (Get images directly from WMS server)       │ => exclusive group, 1 request directly to its own WMS server
    ├── WMSLayer4 (Get images directly from WMS server)       │
    └── WMSLayer5 (Get images directly from WMS server)       ┘

2nd variant: reorder layers

Both settings are honoured, but the layers must be reordered.
In this case, one should decide whether to move non-groupable layers in front or behind.
In trivial projects layers order will be respected.

Here is an example moving external WMS layers back:

Project
│
├── Group1
│   ├── Layer1                                                ┐
│   ├── Layer2                                                │
│   ├── Layer4                                                │ Layers 1, 2, 4, 5 and 3 as single WMS request 
│   └── Layer5                                                │ 
├── Group2                                                    │ 
│   └── Layer3                                                ┘
├── Group1
│   ├── WMSLayer3 (Get images directly from WMS server)       - 1 request directly to its own WMS server
├── Group2                                                    
│   ├── WMSLayer1 (Get images directly from WMS server)       - 1 request directly to its own WMS server
│   ├── WMSLayer2 (Get images directly from WMS server)       - 1 request directly to its own WMS server
└── baselayers
    ├── WMSLayer1 (Get images directly from WMS server)       ┐
    ├── WMSLayer2 (Get images directly from WMS server)       │
    ├── WMSLayer3 (Get images directly from WMS server)       │ => exclusive group, 1 request directly to its own WMS server
    ├── WMSLayer4 (Get images directly from WMS server)       │
    └── WMSLayer5 (Get images directly from WMS server)       ┘

Feature Request summary

Valid values for the "Load layers as single WMS layer" setting should be

value	Single WMS Query	Legible Labelling	directly from WMS server	Layer order respected
false, undefined	❌	❌	✅	✅
true (current behaviour)	✅	✅	❌	✅
at_best	partially	partially	✅	✅
stack_front, stack_back	✅	✅	✅	❌

Related PR

• Original PR [Feature] Load layers as single WMS layer #3947 contains some discussion about cached WMTS layers
• PR Exclude basemaps from single WMS layer option #6351

[meyerlor]

I get what you are pointing at, and I have though also about it a bit - the solution which came to my mind was to simply add a per-layer toggle "exclude from single wms stack" (which should only be visibile/selectable when the option is used) - so leave it to the admin to be smart rather then some lizmap-magic which you can not fine-tune to your liking.
@mind84 do you have any thoughts about it?

[mind84]

@maxencelaurent, @meyerlor,

We've faced a very similar problem; the most effective approach is probably the one suggested by @meyerlor and giving control to the administrator. This obviously increases the complexity of the plugin configuration, but in this specific case, I think it's a good choice.

Cheers!