Skip to content

Performance improvement in mapping function#15

Closed
delphinedobler wants to merge 6 commits intoArgoDMQC:masterfrom
delphinedobler:Performance-improvement-in-mapping-function
Closed

Performance improvement in mapping function#15
delphinedobler wants to merge 6 commits intoArgoDMQC:masterfrom
delphinedobler:Performance-improvement-in-mapping-function

Conversation

@delphinedobler
Copy link
Collaborator

@delphinedobler delphinedobler commented Aug 2, 2024

This pull request contains three steps of performance improvements:
3.1 - Performance: save boxes_id from get_region_ow and recompute the index to provide as input to retr_region_ow
3.2 - Performance: Inside retr_region_ow, save loaded data, and at each following profile, reload only new boxes and delete unused.
3.3 - Performance: Inside map_data_grid, comment calculation of vdataerror as this is never used afterwards.

Improvement and integrity has been tested using wmo 3902007 until cycle 89, with the default ow_config.txt from the master branch: (300 max casts, MAP_USE_PV=0, MAP_USE_SAF=0, MAPSCALE_LONGITUDE_LARGE=6, MAPSCALE_LONGITUDE_SMALL=3, MAPSCALE_LATITUDE_LARGE=3, MAPSCALE_LATITUDE_SMALL=1, MAPSCALE_PHI_LARGE=0.5, MAPSCALE_PHI_SMALL=0.1, MAPSCALE_AGE_LARGE=20, MAPSCALE_AGE_SMALL=10, MAP_P_EXCLUDE=200, MAP_P_DELTA=250) and latest climatology to date (CTD (2024v01) and Argo (2023v03)).

In the testing case, it was observed an average 9.7 wmo boxes used for each profile (min 8, max 12).

The following execution times were observed:
initial execution : 18.63 minutes
3.1 change : 12.65 minutes
3.2 change : 8.77 minutes
3.3 change : 7.92 minutes

The gain in performance depends on several factors (max casts, number of loaded boxes, etc). For instance, with smaller ellipse (i.e. less wmo boxes to probe), the 2.3 gain will be much smaller. This recorded times gives a rough estimate.

Output plots were compared to ensure integrity.

DD (2024/08/01) creation
If la_wmo_boxes_file is removed, automatically recreate it from climatology repository (useful when changing climatology version).
auto-creation of output directories if they do not exist already.
Includes:
3.1 - Performance: save boxes_id from get_region_ow and recompute the index to provide as input to retr_region_ow
3.2 - Performance: Inside retr_region_ow, save loaded data, and at each following profile, reload only new boxes and delete unused.
3.3 - Performance: Inside map_data_grid, comment calculation of vdataerror as this is never used afterwards.
@cabanesc
Copy link
Collaborator

  • I have tested the code using wmo 6902865 (North Atlanic) until cycle 112, with the default ow_config.txt from the master branch and the latest climatology to date (CTD (2024v01) and Argo (2023v03)).
    The gain in performance is in line with the one observed for float 3902007
    initial execution : 28.17 minutes
    3.3 change : 13.12 minutes

  • I have tested the integrity by comparing mapped salinities from the initial execution and the 3.3 execution. Mapped salinity should be the same as the rng() function in find_best_hist(l.49) ensures that the function ran() produces the same sequence of random numbers on each run.

However, the difference between mapped salinities is not zero on some cycles.

image

I have changed line 270 in update_salinity_mapping.m:

% la_bhist_Z = la_grid_Z(index);
la_bhist_Z = la_grid_Z(index_ini);

With this change, the mapped salinities are now the same for the initial and 3.3 runs

  • I have then tested updating the mapping of float 6902865 up to cycle 120 (using the previous mapped files up to cycle 112). The mapped salinities are the same for the initial and 3.3 runs.

  • I have also tested the code using wmo 3901901 until cycle 187, with the default ow_config.txt from the master branch and the latest climatology to date (CTD (2024v01) and Argo (2023v03)).

This float is in the South Atlantic and is crossing the 0-360 degree mark.
initial execution : 32.72 minutes
3.3 change : 14.02 minutes
Again, the mapped salinities are the same for the initial and 3.3 runs.

correct la_bhist_Z  after CC validation tests
@delphinedobler
Copy link
Collaborator Author

Thanks Cécile, I've changed the PR15 and PR16 to include the necessary correction.

@cabanesc
Copy link
Collaborator

cabanesc commented Sep 4, 2025

changes are included in PR #16. Closing without merge.

@cabanesc cabanesc closed this Sep 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants