VRTWarpedDataset::IRasterIO(): optimize I/O when requesting whole image at a resolution that doesn't match an overview#10952
Conversation
|
@rouault thanks for the fix, In your example, we can see the other issue I mentioned cogeotiff/rio-tiler#697 about the do you want me to open an issue about this? |
| bool GDALTransformIsAffineNoRotation(GDALTransformerFunc pfnTransformer, | ||
| void *pTransformerArg); | ||
|
|
||
| bool GDALTransformHasFastClone(void *pTransformerArg); |
There was a problem hiding this comment.
is there a chance we do not use void * anymore?
They make the code almost impossible to read.
There was a problem hiding this comment.
Without impacting too much the API, we could probably have a "typedef void* GDALTransformerArg:". Doing better would involve ditching those C based APIs and just use C++ type inheritance, but that would impact external code (there are users having custom transformer implementations)
There was a problem hiding this comment.
I wonder if it's feasible to move to C++ type inheritance internally but have a compatibility layer that's able to pass the various function and argument struct pointers to a C++ type that can hold them. (I haven't dug into the details, it's just a general thought from the time I've spent in this code.)
…ge at a resolution that doesn't match an overview Fixes rasterio/rasterio#3203 With that optimization: ``` $ gdalwarp /vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif in.vrt $ CPL_DEBUG=VSICURL GDAL_INGESTED_BYTES_AT_OPEN=32768 GDAL_HTTP_MULTIRANGE=YES GDAL_HTTP_MERGE_CONSECUTIVE_RANGES=YES GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR python -c "from osgeo import gdal; ds = gdal.Open('in.vrt'); ds.ReadRaster(0, 0, ds.RasterXSize, ds.RasterYSize, ds.RasterXSize // 5, ds.RasterYSize // 5)" /home/even/gdal/gdal/build_cmake/swig/python/osgeo/gdal.py:311: FutureWarning: Neither gdal.UseExceptions() nor gdal.DontUseExceptions() has been explicitly called. In GDAL 4.0, exceptions will be enabled by default. warnings.warn( VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)=200959614 response_code=200 VSICURL: Downloading 0-32767 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)... VSICURL: Got response_code=206 VSICURL: GetFileList(/vsicurl/https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A) VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.xml)=0 response_code=404 VSICURL: GetFileSize(https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.XML)=0 response_code=404 VSICURL: Downloading 3096576-13139967 (https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/15/T/VK/2023/10/S2B_15TVK_20231008_0_L2A/TCI.tif)... VSICURL: Got response_code=206 ```
4 participants
Fixes rasterio/rasterio#3203
With that optimization: