Make consolidated metadata fetch opt-in

[dcherian]

Zarr version

main

Numcodecs version

...

Python Version

...

Operating System

...

Installation

...

Description

Icechunk for example, does not implement this and these requests are wasted gets/sets. Stores should opt-in to this behaviour

Steps to reproduce

...

Additional output

No response

[TomAugspurger]

Stores should opt-in to this behaviour

Probably opt-out, right? The current behavior, of use_consolidated=None, matches the behavior from V2, which is to use it if present. With something like Store.supports_consolidated_metadata, we could set use_consolidated=store.supports_consolidated_metadata and skip it when we know it can't be present.

And just a note: I think this only affects Zarr v2, since in Zarr v3 the consolidated metadata is in the same zarr.json as the other metadata.

[dcherian]

Yes, now I think that the issue that motivated this comment is on the Xarray side: pydata/xarray#9848