You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We currently use a hard-coded formula for determining how many processes to use in DISP-S1 SCIFLO processing. We should make this formula a bit more flexible by getting some of the parameters from settings.yaml so that change to this formula doesn't require a new PCM release and deployment. It's currently here: https://github.com/nasa/opera-sds-pcm/blob/develop/opera_chimera/precondition_functions.py#L385 n_parallel_bursts = max(int(round(available_cores / 2)) + 1, 1)
Make the following changes to that function:
Parameterize the division factor and the constant factor so that they are retrieved from settings.yaml. Have reasonable values if they are not found.
Make the default value of the denominator be 4 instead of 2. The constant factor can still be 1.
I'd also make change that formula so that it's a multiplication instead of division. This allows for maximum flexibility and the most straightforward. So the currently formula would look: n_parallel_bursts = max(int(round(available_cores * 0.5 )) + 1, 1)
Checked for duplicates
Yes - I've already checked
Alternatives considered
Yes - and alternatives don't suffice
Related problems
No response
Describe the feature request
We currently use a hard-coded formula for determining how many processes to use in DISP-S1 SCIFLO processing. We should make this formula a bit more flexible by getting some of the parameters from
settings.yaml
so that change to this formula doesn't require a new PCM release and deployment. It's currently here:https://github.com/nasa/opera-sds-pcm/blob/develop/opera_chimera/precondition_functions.py#L385
n_parallel_bursts = max(int(round(available_cores / 2)) + 1, 1)
Make the following changes to that function:
settings.yaml
. Have reasonable values if they are not found.n_parallel_bursts = max(int(round(available_cores * 0.5 )) + 1, 1)
theads_per_worker
toohttps://github.com/nasa/opera-sds-pcm/blob/develop/opera_chimera/precondition_functions.py#L380
threads_per_worker = available_cores
The text was updated successfully, but these errors were encountered: