Investigate memory usage of data refresh HLA processing step

[zabeen]

The above image shows CPU and memory usage during data refresh job run on UAT-WMDA.

The first half of the chart (before the line at 7:12AM) was the donor import step - and the second half of the chart is the HLA processing step.

It's not clear to me why the memory usage during HLA processing gradually increases to high levels, and then suddenly drops off at the end of the stage.

It suggests either some data is either being cached to memory on purpose, or it is not being disposed of correctly while batches of donors are being processed.

Either way, addressing this issue would mean data refresh could be safely run during business hours and days.