-
Notifications
You must be signed in to change notification settings - Fork 186
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Are WindowedTimeAverage
s working properly when picking up simulations?
#3485
Comments
I don't think this is done correctly right now. Somehow the |
Ah, I see. Sounds like it wouldn't be trivial to add that support. I guess a workaround to avoid partially-averaged results when picking up would be to set the |
There are two things. One is to fix the flow of information... that's probably pretty easy because we can either 1) make Oceananigans.jl/src/Simulations/run.jl Line 147 in 643b484
then with a fallback The other task is to figure out how to save down the "state" of the time-averaging apparatus so that it can be restored correctly. That's maybe the harder part but of course unavoidable to make checkpointing work with it. |
This might be also relevant for when output files are split? See #3506. |
Hmm yes, perhaps the output writers need to be re-initialized when picking up as well? That would require extending what we do when we pick up here: Oceananigans.jl/src/Simulations/run.jl Lines 87 to 90 in 3bb62a6
|
I've been using
WindowedTimeAverage
s for my simulations (by settingschedule = AveragedTimeInterval(...)
in aNetCDFOutputWriter
). I noticed that whenever I run out of walltime and have to checkpoint my simulations, when I pick them up again I get the following warning for each of the time-averaged outputs:(which comes from this call.)
Does this mean that the time averages aren't being correctly calculated after picking up? I tried following the trail to figure it out but couldn't determine the answer...
The text was updated successfully, but these errors were encountered: