Performance improvement in mapping function#15
Performance improvement in mapping function#15delphinedobler wants to merge 6 commits intoArgoDMQC:masterfrom
Conversation
DD (2024/08/01) creation
If la_wmo_boxes_file is removed, automatically recreate it from climatology repository (useful when changing climatology version).
auto-creation of output directories if they do not exist already.
Includes: 3.1 - Performance: save boxes_id from get_region_ow and recompute the index to provide as input to retr_region_ow 3.2 - Performance: Inside retr_region_ow, save loaded data, and at each following profile, reload only new boxes and delete unused. 3.3 - Performance: Inside map_data_grid, comment calculation of vdataerror as this is never used afterwards.
However, the difference between mapped salinities is not zero on some cycles. I have changed line 270 in update_salinity_mapping.m: With this change, the mapped salinities are now the same for the initial and 3.3 runs
This float is in the South Atlantic and is crossing the 0-360 degree mark. |
correct la_bhist_Z after CC validation tests
|
Thanks Cécile, I've changed the PR15 and PR16 to include the necessary correction. |
|
changes are included in PR #16. Closing without merge. |

This pull request contains three steps of performance improvements:
3.1 - Performance: save boxes_id from get_region_ow and recompute the index to provide as input to retr_region_ow
3.2 - Performance: Inside retr_region_ow, save loaded data, and at each following profile, reload only new boxes and delete unused.
3.3 - Performance: Inside map_data_grid, comment calculation of vdataerror as this is never used afterwards.
Improvement and integrity has been tested using wmo 3902007 until cycle 89, with the default ow_config.txt from the master branch: (300 max casts, MAP_USE_PV=0, MAP_USE_SAF=0, MAPSCALE_LONGITUDE_LARGE=6, MAPSCALE_LONGITUDE_SMALL=3, MAPSCALE_LATITUDE_LARGE=3, MAPSCALE_LATITUDE_SMALL=1, MAPSCALE_PHI_LARGE=0.5, MAPSCALE_PHI_SMALL=0.1, MAPSCALE_AGE_LARGE=20, MAPSCALE_AGE_SMALL=10, MAP_P_EXCLUDE=200, MAP_P_DELTA=250) and latest climatology to date (CTD (2024v01) and Argo (2023v03)).
In the testing case, it was observed an average 9.7 wmo boxes used for each profile (min 8, max 12).
The following execution times were observed:
initial execution : 18.63 minutes
3.1 change : 12.65 minutes
3.2 change : 8.77 minutes
3.3 change : 7.92 minutes
The gain in performance depends on several factors (max casts, number of loaded boxes, etc). For instance, with smaller ellipse (i.e. less wmo boxes to probe), the 2.3 gain will be much smaller. This recorded times gives a rough estimate.
Output plots were compared to ensure integrity.