Hi all, Thanks, <a class="user-mention notranslate" data-hovercard-t

I think this is essentially a duplicate of <a class="issue-link js-issue-link" data-er

It looks like the time step calculated here <div class="Box Box-

Error in AveragedTimeInterval Output,about clima/oceananigans.jl

Comments (32)

glwagner commented on August 20, 2024 3

But Δt = clock.time - wta.previous_collection_time is not the simulation time-step. It's the time increment between collections. We can change the symbol to T = clock.time - wta.previous_collection_time to help clarify this.

Moreover, do you think that a small error in T would cause the kind of issue that @liuchihl is seeing? It looks to me like the average is not calculated at all --- rather than having a small round-off error.

PS I updated your comment to show more of the code so that we could see explicitly the part of the code you were referring to.

EDIT: and just to clarify the time increment between collections can be more than one time-step because we support strided averages (eg computing the averaged quantity less frequently than every time step).

from oceananigans.jl.

simone-silvestri commented on August 20, 2024 2

The zeros are written down at iteration 38, 50, 74, 98, 119, 140, 146, 167, 188, 194, and 201 from what I can see. If we find a pattern regarding the parameters of the WindowedTimeAverage we can probably find what the problem is

from oceananigans.jl.

hdrake commented on August 20, 2024 1

I think this is essentially a duplicate of #3485, although @liuchihl found a MWE where similar issues pop up even in the initial period (without picking up from a checkpoint).

from oceananigans.jl.

liuchihl commented on August 20, 2024 1

I tested the simulation without picking up a checkpoint (running only the first simulation from 0-6s), but the error still unexpectedly persists. It seems the issue might be related to a bug in AveragedTimeInterval and maybe not necessarily just related to the checkpoint. (Hence I change the name of this issue.)

from oceananigans.jl.

glwagner commented on August 20, 2024 1

Let me see if I understand this. We want to run a simulation in which the output interval and the averaging window is between 2-3 time-steps, right? Maybe we need to write out the sequence of steps that we expect to occur and then we can look at the source code and see why that is not happening. Does the problem also happen when δt = 0.02 (eg output exactly every two time-steps)?

from oceananigans.jl.

simone-silvestri commented on August 20, 2024 1

It looks like the time step calculated here

Oceananigans.jl/src/OutputWriters/windowed_time_average.jl

Lines 211 to 213 in 406eb9c

 function accumulate_result!(wta, clock::Clock, integrand=wta.operand) 

 # Time increment: 

 Δt = clock.time - wta.previous_collection_time

does not agree with the time step used in the simulation because of finite precision. I don't think we need to recompute it, we can use the simulation time step size. Probably this will not solve this issue but it will help to make the averaging more consistent

from oceananigans.jl.

glwagner commented on August 20, 2024 1

Yes! Make sure to work from main because remember we changed how TimeInterval works recently. TimeInterval actuates on this criteria:

Oceananigans.jl/src/Utils/schedules.jl

Lines 65 to 79 in dc5dc28

 function (schedule::TimeInterval)(model) 

 t = model.clock.time 

 t★ = next_actuation_time(schedule) 

 if t >= t★ 

 if schedule.actuations < typemax(Int) 

 schedule.actuations += 1 

 else # re-initialize the schedule to t★ 

 initialize!(schedule, t★) 

 end 

 return true 

 else 

 return false 

 end 

 end

when t exceeds or is equal to next_actuation_time:

Oceananigans.jl/src/Utils/schedules.jl

Lines 58 to 63 in dc5dc28

 function next_actuation_time(schedule::TimeInterval) 

 t₀ = schedule.first_actuation_time 

 N = schedule.actuations 

 T = schedule.interval 

 return t₀ + (N + 1) * T 

 end

The first actuation time and number of actuations have to be initialized:

Oceananigans.jl/src/Utils/schedules.jl

Lines 50 to 56 in dc5dc28

 function initialize!(schedule::TimeInterval, first_actuation_time::Number) 

 schedule.first_actuation_time = first_actuation_time 

 schedule.actuations = 0 

 return true 

 end 

 initialize!(schedule::TimeInterval, model) = initialize!(schedule, model.clock.time)

I think WindowedTimeAverage works differently and looks at previous_stop_time, plus the interval of the time-averaging

Oceananigans.jl/src/OutputWriters/windowed_time_average.jl

Line 90 in dc5dc28

 (sch::AveragedTimeInterval)(model) = sch.collecting || model.clock.time >= sch.previous_interval_stop_time + sch.interval - sch.window 

and previous_stop_time is recorded here:

Oceananigans.jl/src/OutputWriters/windowed_time_average.jl

Line 91 in dc5dc28

 initialize_schedule!(sch::AveragedTimeInterval, clock) = sch.previous_interval_stop_time = clock.time - rem(clock.time, sch.interval) 

which in turn is called here:

Oceananigans.jl/src/OutputWriters/windowed_time_average.jl

Line 280 in dc5dc28

initialize_schedule!(wta.schedule, model.clock)

So there's something about using the previous stop time, rather than the number of actuations, which incurs a floating point error issue.

I think AveragedTimeInterval should somehow use a number of actuations as a criteria. Not sure actually how that would work, need to study the flow of information in detail. It will probably help to sketch out a diagram / make a cartoon on a piece of paper so that you understand precisely the flow of information.

from oceananigans.jl.

glwagner commented on August 20, 2024 1

Thanks a lot for your hint, I wanted to make sure what you mean by number of actuations as a criteria. Do you mean
first_actuation_time::Number
and do you think this will potentially prevent the floating point issue?

Not quite.

TimeInterval keeps a count of how many times it has been actuated. This is stored in the variable TimeInterval.actuations. It's use is illustrated in the computation of the next_actuation_time:

Oceananigans.jl/src/Utils/schedules.jl

Lines 58 to 63 in dc5dc28

 function next_actuation_time(schedule::TimeInterval) 

 t₀ = schedule.first_actuation_time 

 N = schedule.actuations 

 T = schedule.interval 

 return t₀ + (N + 1) * T 

 end

To understand this consider an example. TimeInterval has been actuated 2 times, with an interval of 1.1. This means that the next actuation time --- the third actuation_ -- will occur at the time

t = 3 * 1.1 = 3.3

Thus rather than storing the "previous actuation time" (a floating point number), we instead store the number of actuations (an integer, and exact), and compute the next actuation time with the formula above.

Does this make sense?

from oceananigans.jl.

liuchihl commented on August 20, 2024

The issue not only happens after picking up checkpoint, it also occurs before that. For example, with the same MWE, if I set:

Δt = .01    # timestep (s)
T1 = 6      # first simulation stop time (s)
T2 = 2T1    # second simulation stop time (s)
δt = .03   # progress message interval and output saving interval

The issue also occurs in the first run (e.g., 0-6 s):

from oceananigans.jl.

glwagner commented on August 20, 2024

Hmm ok. Can you identify the difference between the cases where it works and where it fails?

I'd like some other devs to help out with this @navidcy @simone-silvestri

from oceananigans.jl.

liuchihl commented on August 20, 2024

Sure, I ran a simple test using the MWE above without picking up a checkpoint and set:

Δt = .01    # timestep (s)
T1 = 5      # first simulation stop time (s)
δt = .03   # progress message interval and output saving interval
# Run a simulation
simulation = test_simulation(T1, Δt, δt, true)
run!(simulation)

I've noticed that this strange behavior occurs when δt is smaller than or equal to 0.03, but it disappears when δt is greater than 0.03 (even at 0.031). It seems there is a cutoff value of δt below which the error emerges.

from oceananigans.jl.

hdrake commented on August 20, 2024

By the way, we originally thought we might be able to solve this issue by setting the checkpointing interval to be an integer multiple of the averaging interval (and window) but that is evidently not the case since the bug shows up in the MWEs above that satisfy this condition (where the checkpoint time is T1 = 6 and the averaging interval is δt = 0.03).

from oceananigans.jl.

liuchihl commented on August 20, 2024

We want to run a simulation in which the output interval and the averaging window is between 2-3 time-steps, right?

Yes, exactly. The error occurs in this range.

Maybe we need to write out the sequence of steps that we expect to occur and then we can look at the source code and see why that is not happening. Does the problem also happen when δt = 0.02 (eg output exactly every two time-steps)?

Yes, I agree, the problem still happens when δt = 0.02.

from oceananigans.jl.

simone-silvestri commented on August 20, 2024

It seems like the Δt that is calculated in accumulate_result! does not agree with the simulation Δt because of finite precision arithmetic. Wouldn't it be better to pass the simulation Δt to the schedule so everything becomes more consistent and we avoid discrepancies between schedule time and simulation time?

from oceananigans.jl.

glwagner commented on August 20, 2024

@simone-silvestri can you spell out what you're saying more explicitly by referencing / linking into the source code? I can't figure out what you're saying (it seems I would have to dive into the source code to figure it out).

from oceananigans.jl.

hdrake commented on August 20, 2024

I'm also doubtful small round off errors in the timestep would be related, unless there a place in the code where the scheduled times for outputting / checkpointing need to exactly match some value.

In the MWE, the problematic values look like they're all zeros. In our more complicated example, where the averaging interval is a large (decimal) multiple of the timestep, it's not clear if the values are underestimated because the velocities are of both signs so a biased average could go either way.

from oceananigans.jl.

simone-silvestri commented on August 20, 2024

I am not sure if this helps, but this might shed some light.
This is the same MWE with the progress function set as:

    function progress_message(sim) 
        wta = sim.output_writers[:timeavg2].outputs[1]
        wd = wta.window_start_time
        ws = wta.window_start_iteration
        pc = wta.previous_collection_time
        fo = wta.fetch_operand
        res = sum(wta.result)
        
        interval = wta.schedule.interval
        pis = wta.schedule.previous_interval_stop_time
        cll = wta.schedule.collecting
        @info string("Iter: ", iteration(sim), ", time: ", prettytime(sim), " , u-avg: ", res, ", window_start_time: ", wd, ", window_start_iteration: ", ws, ", previous_collection_time: ", pc, ", fetch_operand: ", fo, ", interval: ", interval, ", previous_interval_stop_time: ", pis, ", collecting: ", cll)
    end

julia> include("test.jl")
[ Info: Initializing simulation...
[ Info: Iter: 0, time: 0 seconds , u-avg: 0.0, window_start_time: 0.0, window_start_iteration: 0, previous_collection_time: 0.0, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.0, collecting: true
[ Info:     ... simulation initialization complete (257.729 ms)
[ Info: Executing initial time step...
[ Info:     ... initial time step complete (1.212 seconds).
[ Info: Iter: 1, time: 10 ms , u-avg: 0.0031413629825035438, window_start_time: 0.0, window_start_iteration: 0, previous_collection_time: 0.01, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.0, collecting: true
[ Info: Iter: 2, time: 20 ms , u-avg: 0.007851828677677537, window_start_time: 0.0, window_start_iteration: 0, previous_collection_time: 0.02, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.0, collecting: true
[ Info: Iter: 3, time: 30 ms , u-avg: 0.014652384734839867, window_start_time: 0.0, window_start_iteration: 0, previous_collection_time: 0.03, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.03, collecting: false
[ Info: Iter: 4, time: 40 ms , u-avg: 0.0, window_start_time: 0.04, window_start_iteration: 4, previous_collection_time: 0.04, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.03, collecting: true
[ Info: Iter: 5, time: 50 ms , u-avg: 0.07837860105159718, window_start_time: 0.04, window_start_iteration: 4, previous_collection_time: 0.05, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.03, collecting: true
[ Info: Iter: 6, time: 60 ms , u-avg: 0.09557081820157169, window_start_time: 0.04, window_start_iteration: 4, previous_collection_time: 0.06, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.06, collecting: false
[ Info: Iter: 7, time: 70.000 ms , u-avg: 0.0, window_start_time: 0.06999999999999999, window_start_iteration: 7, previous_collection_time: 0.06999999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.06, collecting: true
[ Info: Iter: 8, time: 80.000 ms , u-avg: 0.20000603541011058, window_start_time: 0.06999999999999999, window_start_iteration: 7, previous_collection_time: 0.07999999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.06, collecting: true
[ Info: Iter: 9, time: 90.000 ms , u-avg: 0.22639226913537686, window_start_time: 0.06999999999999999, window_start_iteration: 7, previous_collection_time: 0.08999999999999998, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.06, collecting: true
[ Info: Iter: 10, time: 90 ms , u-avg: 0.22639226913537688, window_start_time: 0.06999999999999999, window_start_iteration: 7, previous_collection_time: 0.09, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.09, collecting: false
[ Info: Iter: 11, time: 100.000 ms , u-avg: 0.0, window_start_time: 0.09999999999999999, window_start_iteration: 11, previous_collection_time: 0.09999999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.09, collecting: true
[ Info: Iter: 12, time: 110.000 ms , u-avg: 0.3763650237305524, window_start_time: 0.09999999999999999, window_start_iteration: 11, previous_collection_time: 0.10999999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.09, collecting: true
[ Info: Iter: 13, time: 120.000 ms , u-avg: 0.41171106795258217, window_start_time: 0.09999999999999999, window_start_iteration: 11, previous_collection_time: 0.11999999999999998, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.09, collecting: true
[ Info: Iter: 14, time: 120 ms , u-avg: 0.4117110679525822, window_start_time: 0.09999999999999999, window_start_iteration: 11, previous_collection_time: 0.12, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.12, collecting: false
[ Info: Iter: 15, time: 130 ms , u-avg: 0.0, window_start_time: 0.13, window_start_iteration: 15, previous_collection_time: 0.13, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.12, collecting: true
[ Info: Iter: 16, time: 140 ms , u-avg: 0.6058901911474607, window_start_time: 0.13, window_start_iteration: 15, previous_collection_time: 0.14, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.12, collecting: true
[ Info: Iter: 17, time: 150 ms , u-avg: 0.649882311875457, window_start_time: 0.13, window_start_iteration: 15, previous_collection_time: 0.15, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.15, collecting: false
[ Info: Iter: 18, time: 160 ms , u-avg: 0.0, window_start_time: 0.16, window_start_iteration: 18, previous_collection_time: 0.16, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.15, collecting: true
[ Info: Iter: 19, time: 170 ms , u-avg: 0.8865442560258744, window_start_time: 0.16, window_start_iteration: 18, previous_collection_time: 0.17, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.15, collecting: true
[ Info: Iter: 20, time: 180 ms , u-avg: 0.9387919760818845, window_start_time: 0.16, window_start_iteration: 18, previous_collection_time: 0.18, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.18, collecting: false
[ Info: Iter: 21, time: 190 ms , u-avg: 0.0, window_start_time: 0.19, window_start_iteration: 21, previous_collection_time: 0.19, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.18, collecting: true
[ Info: Iter: 22, time: 200 ms , u-avg: 1.2158361130173538, window_start_time: 0.19, window_start_iteration: 21, previous_collection_time: 0.2, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.18, collecting: true
[ Info: Iter: 23, time: 210 ms , u-avg: 1.2758756779393434, window_start_time: 0.19, window_start_iteration: 21, previous_collection_time: 0.21, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.21, collecting: false
[ Info: Iter: 24, time: 220 ms , u-avg: 0.0, window_start_time: 0.22, window_start_iteration: 24, previous_collection_time: 0.22, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.21, collecting: true
[ Info: Iter: 25, time: 230 ms , u-avg: 1.5908429442874084, window_start_time: 0.22, window_start_iteration: 24, previous_collection_time: 0.23, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.21, collecting: true
[ Info: Iter: 26, time: 240 ms , u-avg: 1.6581414386466988, window_start_time: 0.22, window_start_iteration: 24, previous_collection_time: 0.24, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.24, collecting: false
[ Info: Iter: 27, time: 250 ms , u-avg: 0.0, window_start_time: 0.25, window_start_iteration: 27, previous_collection_time: 0.25, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.24, collecting: true
[ Info: Iter: 28, time: 260 ms , u-avg: 2.008236162653511, window_start_time: 0.25, window_start_iteration: 27, previous_collection_time: 0.26, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.24, collecting: true
[ Info: Iter: 29, time: 270 ms , u-avg: 2.082196240249851, window_start_time: 0.25, window_start_iteration: 27, previous_collection_time: 0.27, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.27, collecting: false
[ Info: Iter: 30, time: 280 ms , u-avg: 0.0, window_start_time: 0.28, window_start_iteration: 30, previous_collection_time: 0.28, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.27, collecting: true
[ Info: Iter: 31, time: 290.000 ms , u-avg: 2.464310956360575, window_start_time: 0.28, window_start_iteration: 30, previous_collection_time: 0.29000000000000004, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.27, collecting: true
[ Info: Iter: 32, time: 300 ms , u-avg: 2.5442761423093008, window_start_time: 0.28, window_start_iteration: 30, previous_collection_time: 0.3, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.27, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 33, time: 310 ms , u-avg: 2.6254722813218105, window_start_time: 0.28, window_start_iteration: 30, previous_collection_time: 0.31, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.3, collecting: false
[ Info: Iter: 34, time: 320 ms , u-avg: 0.0, window_start_time: 0.32, window_start_iteration: 34, previous_collection_time: 0.32, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.3, collecting: true
[ Info: Iter: 35, time: 330.000 ms , u-avg: 3.1255402085515462, window_start_time: 0.32, window_start_iteration: 34, previous_collection_time: 0.32999999999999996, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.3, collecting: false
[ Info: Iter: 36, time: 340.000 ms , u-avg: 0.0, window_start_time: 0.33999999999999997, window_start_iteration: 36, previous_collection_time: 0.33999999999999997, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.3, collecting: true
[ Info: Iter: 37, time: 350 ms , u-avg: 3.476005252456648, window_start_time: 0.33999999999999997, window_start_iteration: 36, previous_collection_time: 0.35, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.32999999999999996, collecting: false
[ Info: Iter: 38, time: 360 ms , u-avg: 0.0, window_start_time: 0.36, window_start_iteration: 38, previous_collection_time: 0.36, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.32999999999999996, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 39, time: 370 ms , u-avg: 3.837876565646086, window_start_time: 0.36, window_start_iteration: 38, previous_collection_time: 0.37, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.36, collecting: false
[ Info: Iter: 40, time: 380 ms , u-avg: 0.0, window_start_time: 0.38, window_start_iteration: 40, previous_collection_time: 0.38, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.36, collecting: true
[ Info: Iter: 41, time: 390 ms , u-avg: 4.209726007369981, window_start_time: 0.38, window_start_iteration: 40, previous_collection_time: 0.39, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.39, collecting: false
[ Info: Iter: 42, time: 400 ms , u-avg: 0.0, window_start_time: 0.4, window_start_iteration: 42, previous_collection_time: 0.4, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.39, collecting: true
[ Info: Iter: 43, time: 410.000 ms , u-avg: 4.590086057763697, window_start_time: 0.4, window_start_iteration: 42, previous_collection_time: 0.41000000000000003, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.39, collecting: true
[ Info: Iter: 44, time: 420 ms , u-avg: 4.6865378368773, window_start_time: 0.4, window_start_iteration: 42, previous_collection_time: 0.42, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.39, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 45, time: 430 ms , u-avg: 4.783510427744303, window_start_time: 0.4, window_start_iteration: 42, previous_collection_time: 0.43, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.42, collecting: false
[ Info: Iter: 46, time: 440 ms , u-avg: 0.0, window_start_time: 0.44, window_start_iteration: 46, previous_collection_time: 0.44, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.42, collecting: true
[ Info: Iter: 47, time: 450.000 ms , u-avg: 5.370305891865513, window_start_time: 0.44, window_start_iteration: 46, previous_collection_time: 0.44999999999999996, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.42, collecting: false
[ Info: Iter: 48, time: 460.000 ms , u-avg: 0.0, window_start_time: 0.45999999999999996, window_start_iteration: 48, previous_collection_time: 0.45999999999999996, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.42, collecting: true
[ Info: Iter: 49, time: 470 ms , u-avg: 5.76708650433695, window_start_time: 0.45999999999999996, window_start_iteration: 48, previous_collection_time: 0.47, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.44999999999999996, collecting: false
[ Info: Iter: 50, time: 480 ms , u-avg: 0.0, window_start_time: 0.48, window_start_iteration: 50, previous_collection_time: 0.48, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.44999999999999996, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 51, time: 490 ms , u-avg: 6.166231535087016, window_start_time: 0.48, window_start_iteration: 50, previous_collection_time: 0.49, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.48, collecting: false
[ Info: Iter: 52, time: 500 ms , u-avg: 0.0, window_start_time: 0.5, window_start_iteration: 52, previous_collection_time: 0.5, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.48, collecting: true
[ Info: Iter: 53, time: 510 ms , u-avg: 6.566165741031355, window_start_time: 0.5, window_start_iteration: 52, previous_collection_time: 0.51, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.51, collecting: false
[ Info: Iter: 54, time: 520 ms , u-avg: 0.0, window_start_time: 0.52, window_start_iteration: 54, previous_collection_time: 0.52, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.51, collecting: true
[ Info: Iter: 55, time: 530 ms , u-avg: 6.965310764571667, window_start_time: 0.52, window_start_iteration: 54, previous_collection_time: 0.53, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.51, collecting: true
[ Info: Iter: 56, time: 540 ms , u-avg: 7.064702771106775, window_start_time: 0.52, window_start_iteration: 54, previous_collection_time: 0.54, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.54, collecting: false
[ Info: Iter: 57, time: 550 ms , u-avg: 0.0, window_start_time: 0.55, window_start_iteration: 57, previous_collection_time: 0.55, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.54, collecting: true
[ Info: Iter: 58, time: 560 ms , u-avg: 7.559105120877197, window_start_time: 0.55, window_start_iteration: 57, previous_collection_time: 0.56, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.54, collecting: true
[ Info: Iter: 59, time: 570 ms , u-avg: 7.657023372200447, window_start_time: 0.55, window_start_iteration: 57, previous_collection_time: 0.57, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.54, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 60, time: 580 ms , u-avg: 7.7544847826693255, window_start_time: 0.55, window_start_iteration: 57, previous_collection_time: 0.58, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.57, collecting: false
[ Info: Iter: 61, time: 590 ms , u-avg: 0.0, window_start_time: 0.59, window_start_iteration: 61, previous_collection_time: 0.59, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.57, collecting: true
[ Info: Iter: 62, time: 600 ms , u-avg: 8.333461880244451, window_start_time: 0.59, window_start_iteration: 61, previous_collection_time: 0.6, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.6, collecting: false
[ Info: Iter: 63, time: 610 ms , u-avg: 0.0, window_start_time: 0.61, window_start_iteration: 63, previous_collection_time: 0.61, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.6, collecting: true
[ Info: Iter: 64, time: 620 ms , u-avg: 8.709752264017125, window_start_time: 0.61, window_start_iteration: 63, previous_collection_time: 0.62, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.6, collecting: true
[ Info: Iter: 65, time: 630 ms , u-avg: 8.802136412533468, window_start_time: 0.61, window_start_iteration: 63, previous_collection_time: 0.63, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.63, collecting: false
[ Info: Iter: 66, time: 640 ms , u-avg: 0.0, window_start_time: 0.64, window_start_iteration: 66, previous_collection_time: 0.64, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.63, collecting: true
[ Info: Iter: 67, time: 650 ms , u-avg: 9.256391825350283, window_start_time: 0.64, window_start_iteration: 66, previous_collection_time: 0.65, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.63, collecting: true
[ Info: Iter: 68, time: 660.000 ms , u-avg: 9.344764747359864, window_start_time: 0.64, window_start_iteration: 66, previous_collection_time: 0.6599999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.63, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 69, time: 670.000 ms , u-avg: 9.432128769553014, window_start_time: 0.64, window_start_iteration: 66, previous_collection_time: 0.6699999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.6599999999999999, collecting: false
[ Info: Iter: 70, time: 680.000 ms , u-avg: 0.0, window_start_time: 0.6799999999999999, window_start_iteration: 70, previous_collection_time: 0.6799999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.6599999999999999, collecting: true
[ Info: Iter: 71, time: 690 ms , u-avg: 9.944532401619153, window_start_time: 0.6799999999999999, window_start_iteration: 70, previous_collection_time: 0.69, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.6599999999999999, collecting: false
[ Info: Iter: 72, time: 700 ms , u-avg: 0.0, window_start_time: 0.7, window_start_iteration: 72, previous_collection_time: 0.7, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.6599999999999999, collecting: true
[ Info: Iter: 73, time: 710 ms , u-avg: 10.268085937114538, window_start_time: 0.7, window_start_iteration: 72, previous_collection_time: 0.71, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.69, collecting: false
[ Info: Iter: 74, time: 720 ms , u-avg: 0.0, window_start_time: 0.72, window_start_iteration: 74, previous_collection_time: 0.72, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.69, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 75, time: 730 ms , u-avg: 10.576240502438422, window_start_time: 0.72, window_start_iteration: 74, previous_collection_time: 0.73, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.72, collecting: false
[ Info: Iter: 76, time: 740 ms , u-avg: 0.0, window_start_time: 0.74, window_start_iteration: 76, previous_collection_time: 0.74, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.72, collecting: true
[ Info: Iter: 77, time: 750 ms , u-avg: 10.867779952303898, window_start_time: 0.74, window_start_iteration: 76, previous_collection_time: 0.75, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.75, collecting: false
[ Info: Iter: 78, time: 760 ms , u-avg: 0.0, window_start_time: 0.76, window_start_iteration: 78, previous_collection_time: 0.76, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.75, collecting: true
[ Info: Iter: 79, time: 770 ms , u-avg: 11.141553713694051, window_start_time: 0.76, window_start_iteration: 78, previous_collection_time: 0.77, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.75, collecting: true
[ Info: Iter: 80, time: 780 ms , u-avg: 11.206495836906438, window_start_time: 0.76, window_start_iteration: 78, previous_collection_time: 0.78, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.78, collecting: false
[ Info: Iter: 81, time: 790 ms , u-avg: 0.0, window_start_time: 0.79, window_start_iteration: 81, previous_collection_time: 0.79, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.78, collecting: true
[ Info: Iter: 82, time: 800 ms , u-avg: 11.516560410575497, window_start_time: 0.79, window_start_iteration: 81, previous_collection_time: 0.8, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.78, collecting: true
[ Info: Iter: 83, time: 810.000 ms , u-avg: 11.574058559459141, window_start_time: 0.79, window_start_iteration: 81, previous_collection_time: 0.8099999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.78, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 84, time: 820 ms , u-avg: 11.62982461713531, window_start_time: 0.79, window_start_iteration: 81, previous_collection_time: 0.82, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.8099999999999999, collecting: false
[ Info: Iter: 85, time: 830 ms , u-avg: 0.0, window_start_time: 0.83, window_start_iteration: 85, previous_collection_time: 0.83, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.8099999999999999, collecting: true
[ Info: Iter: 86, time: 840 ms , u-avg: 11.944939757472355, window_start_time: 0.83, window_start_iteration: 85, previous_collection_time: 0.84, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.84, collecting: false
[ Info: Iter: 87, time: 850 ms , u-avg: 0.0, window_start_time: 0.85, window_start_iteration: 87, previous_collection_time: 0.85, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.84, collecting: true
[ Info: Iter: 88, time: 860 ms , u-avg: 12.126506036338247, window_start_time: 0.85, window_start_iteration: 87, previous_collection_time: 0.86, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.84, collecting: true
[ Info: Iter: 89, time: 870 ms , u-avg: 12.167655766858058, window_start_time: 0.85, window_start_iteration: 87, previous_collection_time: 0.87, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.87, collecting: false
[ Info: Iter: 90, time: 880 ms , u-avg: 0.0, window_start_time: 0.88, window_start_iteration: 90, previous_collection_time: 0.88, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.87, collecting: true
[ Info: Iter: 91, time: 890 ms , u-avg: 12.356031044664883, window_start_time: 0.88, window_start_iteration: 90, previous_collection_time: 0.89, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.87, collecting: true
[ Info: Iter: 92, time: 900.000 ms , u-avg: 12.388421440868372, window_start_time: 0.88, window_start_iteration: 90, previous_collection_time: 0.8999999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.87, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 93, time: 910.000 ms , u-avg: 12.418820112689675, window_start_time: 0.88, window_start_iteration: 90, previous_collection_time: 0.9099999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.8999999999999999, collecting: false
[ Info: Iter: 94, time: 920.000 ms , u-avg: 0.0, window_start_time: 0.9199999999999999, window_start_iteration: 94, previous_collection_time: 0.9199999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.8999999999999999, collecting: true
[ Info: Iter: 95, time: 930.000 ms , u-avg: 12.579076993274457, window_start_time: 0.9199999999999999, window_start_iteration: 94, previous_collection_time: 0.9299999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.8999999999999999, collecting: false
[ Info: Iter: 96, time: 940 ms , u-avg: 0.0, window_start_time: 0.94, window_start_iteration: 96, previous_collection_time: 0.94, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.8999999999999999, collecting: true
[ Info: Iter: 97, time: 950 ms , u-avg: 12.654017134137858, window_start_time: 0.94, window_start_iteration: 96, previous_collection_time: 0.95, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.9299999999999999, collecting: false
[ Info: Iter: 98, time: 960 ms , u-avg: 0.0, window_start_time: 0.96, window_start_iteration: 98, previous_collection_time: 0.96, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.9299999999999999, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 99, time: 970 ms , u-avg: 12.704142124423381, window_start_time: 0.96, window_start_iteration: 98, previous_collection_time: 0.97, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.96, collecting: false
[ Info: Iter: 100, time: 980 ms , u-avg: 0.0, window_start_time: 0.98, window_start_iteration: 100, previous_collection_time: 0.98, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.96, collecting: true
[ Info: Iter: 101, time: 990 ms , u-avg: 12.729254143694298, window_start_time: 0.98, window_start_iteration: 100, previous_collection_time: 0.99, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.99, collecting: false
[ Info: Iter: 102, time: 1 second , u-avg: 0.0, window_start_time: 1.0, window_start_iteration: 102, previous_collection_time: 1.0, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.99, collecting: true
[ Info: Iter: 103, time: 1.010 seconds , u-avg: 12.729254086283136, window_start_time: 1.0, window_start_iteration: 102, previous_collection_time: 1.01, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.99, collecting: true
[ Info: Iter: 104, time: 1.020 seconds , u-avg: 12.724543620587964, window_start_time: 1.0, window_start_iteration: 102, previous_collection_time: 1.02, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.02, collecting: false
[ Info: Iter: 105, time: 1.030 seconds , u-avg: 0.0, window_start_time: 1.03, window_start_iteration: 105, previous_collection_time: 1.03, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.02, collecting: true
[ Info: Iter: 106, time: 1.040 seconds , u-avg: 12.682195964176623, window_start_time: 1.03, window_start_iteration: 105, previous_collection_time: 1.04, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.02, collecting: true
[ Info: Iter: 107, time: 1.050 seconds , u-avg: 12.668106406195333, window_start_time: 1.03, window_start_iteration: 105, previous_collection_time: 1.05, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.05, collecting: false
[ Info: Iter: 108, time: 1.060 seconds , u-avg: 0.0, window_start_time: 1.06, window_start_iteration: 108, previous_collection_time: 1.06, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.05, collecting: true
[ Info: Iter: 109, time: 1.070 seconds , u-avg: 12.579076594562144, window_start_time: 1.06, window_start_iteration: 108, previous_collection_time: 1.07, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.05, collecting: true
[ Info: Iter: 110, time: 1.080 seconds , u-avg: 12.555733004208836, window_start_time: 1.06, window_start_iteration: 108, previous_collection_time: 1.08, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.08, collecting: false
[ Info: Iter: 111, time: 1.090 seconds , u-avg: 0.0, window_start_time: 1.09, window_start_iteration: 111, previous_collection_time: 1.09, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.08, collecting: true
[ Info: Iter: 112, time: 1.100 seconds , u-avg: 12.420811272264622, window_start_time: 1.09, window_start_iteration: 111, previous_collection_time: 1.1, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.08, collecting: true
[ Info: Iter: 113, time: 1.110 seconds , u-avg: 12.388420848899857, window_start_time: 1.09, window_start_iteration: 111, previous_collection_time: 1.1099999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.08, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 114, time: 1.120 seconds , u-avg: 12.354060011630247, window_start_time: 1.09, window_start_iteration: 111, previous_collection_time: 1.1199999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.1099999999999999, collecting: false
[ Info: Iter: 115, time: 1.130 seconds , u-avg: 0.0, window_start_time: 1.13, window_start_iteration: 115, previous_collection_time: 1.13, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.1099999999999999, collecting: true
[ Info: Iter: 116, time: 1.140 seconds , u-avg: 12.126505258118181, window_start_time: 1.13, window_start_iteration: 115, previous_collection_time: 1.14, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.1099999999999999, collecting: false
[ Info: Iter: 117, time: 1.150 seconds , u-avg: 0.0, window_start_time: 1.15, window_start_iteration: 117, previous_collection_time: 1.15, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.1099999999999999, collecting: true
[ Info: Iter: 118, time: 1.160 seconds , u-avg: 11.944938876944853, window_start_time: 1.15, window_start_iteration: 117, previous_collection_time: 1.16, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.14, collecting: false
[ Info: Iter: 119, time: 1.170 seconds , u-avg: 0.0, window_start_time: 1.17, window_start_iteration: 119, previous_collection_time: 1.17, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.14, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 120, time: 1.180 seconds , u-avg: 11.741355753127747, window_start_time: 1.17, window_start_iteration: 119, previous_collection_time: 1.18, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.17, collecting: false
[ Info: Iter: 121, time: 1.190 seconds , u-avg: 0.0, window_start_time: 1.19, window_start_iteration: 121, previous_collection_time: 1.19, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.17, collecting: true
[ Info: Iter: 122, time: 1.200 seconds , u-avg: 11.51655933624829, window_start_time: 1.19, window_start_iteration: 121, previous_collection_time: 1.2, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.2, collecting: false
[ Info: Iter: 123, time: 1.210 seconds , u-avg: 0.0, window_start_time: 1.21, window_start_iteration: 123, previous_collection_time: 1.21, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.2, collecting: true
[ Info: Iter: 124, time: 1.220 seconds , u-avg: 11.271436795064185, window_start_time: 1.21, window_start_iteration: 123, previous_collection_time: 1.22, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.2, collecting: true
[ Info: Iter: 125, time: 1.230 seconds , u-avg: 11.206494650021211, window_start_time: 1.21, window_start_iteration: 123, previous_collection_time: 1.23, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.23, collecting: false
[ Info: Iter: 126, time: 1.240 seconds , u-avg: 0.0, window_start_time: 1.24, window_start_iteration: 126, previous_collection_time: 1.24, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.23, collecting: true
[ Info: Iter: 127, time: 1.250 seconds , u-avg: 10.867778659886214, window_start_time: 1.24, window_start_iteration: 126, previous_collection_time: 1.25, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.23, collecting: true
[ Info: Iter: 128, time: 1.260 seconds , u-avg: 10.795968973249174, window_start_time: 1.24, window_start_iteration: 126, previous_collection_time: 1.26, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.26, collecting: false
[ Info: Iter: 129, time: 1.270 seconds , u-avg: 0.0, window_start_time: 1.27, window_start_iteration: 129, previous_collection_time: 1.27, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.26, collecting: true
[ Info: Iter: 130, time: 1.280 seconds , u-avg: 10.424164173642733, window_start_time: 1.27, window_start_iteration: 129, previous_collection_time: 1.28, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.26, collecting: true
[ Info: Iter: 131, time: 1.290 seconds , u-avg: 10.3461243332739, window_start_time: 1.27, window_start_iteration: 129, previous_collection_time: 1.29, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.29, collecting: false
[ Info: Iter: 132, time: 1.300 seconds , u-avg: 0.0, window_start_time: 1.3, window_start_iteration: 132, previous_collection_time: 1.3, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.29, collecting: true
[ Info: Iter: 133, time: 1.310 seconds , u-avg: 9.944530889918811, window_start_time: 1.3, window_start_iteration: 132, previous_collection_time: 1.31, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.29, collecting: true
[ Info: Iter: 134, time: 1.320 seconds , u-avg: 9.860953582966044, window_start_time: 1.3, window_start_iteration: 132, previous_collection_time: 1.3199999999999998, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.29, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 135, time: 1.330 seconds , u-avg: 9.776254135548728, window_start_time: 1.3, window_start_iteration: 132, previous_collection_time: 1.3299999999999998, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.3199999999999998, collecting: false
[ Info: Iter: 136, time: 1.340 seconds , u-avg: 0.0, window_start_time: 1.3399999999999999, window_start_iteration: 136, previous_collection_time: 1.3399999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.3199999999999998, collecting: true
[ Info: Iter: 137, time: 1.350 seconds , u-avg: 9.256390196808994, window_start_time: 1.3399999999999999, window_start_iteration: 136, previous_collection_time: 1.3499999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.3199999999999998, collecting: false
[ Info: Iter: 138, time: 1.360 seconds , u-avg: 0.0, window_start_time: 1.3599999999999999, window_start_iteration: 138, previous_collection_time: 1.3599999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.3199999999999998, collecting: true
[ Info: Iter: 139, time: 1.370 seconds , u-avg: 8.894518883619556, window_start_time: 1.3599999999999999, window_start_iteration: 138, previous_collection_time: 1.3699999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.3499999999999999, collecting: false
[ Info: Iter: 140, time: 1.380 seconds , u-avg: 0.0, window_start_time: 1.38, window_start_iteration: 140, previous_collection_time: 1.38, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.3499999999999999, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 141, time: 1.390 seconds , u-avg: 8.522669441895662, window_start_time: 1.38, window_start_iteration: 140, previous_collection_time: 1.39, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.38, collecting: false
[ Info: Iter: 142, time: 1.400 seconds , u-avg: 0.0, window_start_time: 1.4, window_start_iteration: 142, previous_collection_time: 1.4, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.38, collecting: true
[ Info: Iter: 143, time: 1.410 seconds , u-avg: 8.142309391501945, window_start_time: 1.4, window_start_iteration: 142, previous_collection_time: 1.41, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.38, collecting: false
[ Info: Iter: 144, time: 1.420 seconds , u-avg: 0.0, window_start_time: 1.42, window_start_iteration: 144, previous_collection_time: 1.42, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.38, collecting: true
[ Info: Iter: 145, time: 1.430 seconds , u-avg: 7.754939839787331, window_start_time: 1.42, window_start_iteration: 144, previous_collection_time: 1.43, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.41, collecting: false
[ Info: Iter: 146, time: 1.440 seconds , u-avg: 0.0, window_start_time: 1.44, window_start_iteration: 146, previous_collection_time: 1.44, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.41, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 147, time: 1.450 seconds , u-avg: 7.362089557400126, window_start_time: 1.44, window_start_iteration: 146, previous_collection_time: 1.45, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.44, collecting: false
[ Info: Iter: 148, time: 1.460 seconds , u-avg: 0.0, window_start_time: 1.46, window_start_iteration: 148, previous_collection_time: 1.46, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.44, collecting: true
[ Info: Iter: 149, time: 1.470 seconds , u-avg: 6.965308944928689, window_start_time: 1.46, window_start_iteration: 148, previous_collection_time: 1.47, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.47, collecting: false
[ Info: Iter: 150, time: 1.480 seconds , u-avg: 0.0, window_start_time: 1.48, window_start_iteration: 150, previous_collection_time: 1.48, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.47, collecting: true
[ Info: Iter: 151, time: 1.490 seconds , u-avg: 6.566163914178623, window_start_time: 1.48, window_start_iteration: 150, previous_collection_time: 1.49, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.47, collecting: true
[ Info: Iter: 152, time: 1.500 seconds , u-avg: 6.466180362467067, window_start_time: 1.48, window_start_iteration: 150, previous_collection_time: 1.5, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.5, collecting: false
[ Info: Iter: 153, time: 1.510 seconds , u-avg: 0.0, window_start_time: 1.51, window_start_iteration: 153, previous_collection_time: 1.51, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.5, collecting: true
[ Info: Iter: 154, time: 1.520 seconds , u-avg: 5.966459950002811, window_start_time: 1.51, window_start_iteration: 153, previous_collection_time: 1.52, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.5, collecting: true
[ Info: Iter: 155, time: 1.530 seconds , u-avg: 5.866772317348391, window_start_time: 1.51, window_start_iteration: 153, previous_collection_time: 1.53, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.53, collecting: false
[ Info: Iter: 156, time: 1.540 seconds , u-avg: 0.0, window_start_time: 1.54, window_start_iteration: 156, previous_collection_time: 1.54, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.53, collecting: true
[ Info: Iter: 157, time: 1.550 seconds , u-avg: 5.370304086613586, window_start_time: 1.54, window_start_iteration: 156, previous_collection_time: 1.55, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.53, collecting: true
[ Info: Iter: 158, time: 1.560 seconds , u-avg: 5.271797207501014, window_start_time: 1.54, window_start_iteration: 156, previous_collection_time: 1.56, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.56, collecting: false
[ Info: Iter: 159, time: 1.570 seconds , u-avg: 0.0, window_start_time: 1.57, window_start_iteration: 159, previous_collection_time: 1.57, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.56, collecting: true
[ Info: Iter: 160, time: 1.580 seconds , u-avg: 4.782987845658554, window_start_time: 1.57, window_start_iteration: 159, previous_collection_time: 1.58, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.56, collecting: true
[ Info: Iter: 161, time: 1.590 seconds , u-avg: 4.686536074120517, window_start_time: 1.57, window_start_iteration: 159, previous_collection_time: 1.5899999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.56, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 162, time: 1.600 seconds , u-avg: 4.59066857242074, window_start_time: 1.57, window_start_iteration: 159, previous_collection_time: 1.5999999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.5899999999999999, collecting: false
[ Info: Iter: 163, time: 1.610 seconds , u-avg: 0.0, window_start_time: 1.6099999999999999, window_start_iteration: 163, previous_collection_time: 1.6099999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.5899999999999999, collecting: true
[ Info: Iter: 164, time: 1.620 seconds , u-avg: 4.022643185248518, window_start_time: 1.6099999999999999, window_start_iteration: 163, previous_collection_time: 1.6199999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.5899999999999999, collecting: false
[ Info: Iter: 165, time: 1.630 seconds , u-avg: 0.0, window_start_time: 1.63, window_start_iteration: 165, previous_collection_time: 1.63, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.5899999999999999, collecting: true
[ Info: Iter: 166, time: 1.640 seconds , u-avg: 3.6556017405745704, window_start_time: 1.63, window_start_iteration: 165, previous_collection_time: 1.64, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.6199999999999999, collecting: false
[ Info: Iter: 167, time: 1.650 seconds , u-avg: 0.0, window_start_time: 1.65, window_start_iteration: 167, previous_collection_time: 1.65, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.6199999999999999, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 168, time: 1.660 seconds , u-avg: 3.2992577798961906, window_start_time: 1.65, window_start_iteration: 167, previous_collection_time: 1.66, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.65, collecting: false
[ Info: Iter: 169, time: 1.670 seconds , u-avg: 0.0, window_start_time: 1.67, window_start_iteration: 169, previous_collection_time: 1.67, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.65, collecting: true
[ Info: Iter: 170, time: 1.680 seconds , u-avg: 2.9550176300281055, window_start_time: 1.67, window_start_iteration: 169, previous_collection_time: 1.68, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.68, collecting: false
[ Info: Iter: 171, time: 1.690 seconds , u-avg: 0.0, window_start_time: 1.69, window_start_iteration: 171, previous_collection_time: 1.69, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.68, collecting: true
[ Info: Iter: 172, time: 1.700 seconds , u-avg: 2.6242398495734784, window_start_time: 1.69, window_start_iteration: 171, previous_collection_time: 1.7, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.68, collecting: true
[ Info: Iter: 173, time: 1.710 seconds , u-avg: 2.54427468086229, window_start_time: 1.69, window_start_iteration: 171, previous_collection_time: 1.71, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.71, collecting: false
[ Info: Iter: 174, time: 1.720 seconds , u-avg: 0.0, window_start_time: 1.72, window_start_iteration: 174, previous_collection_time: 1.72, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.71, collecting: true
[ Info: Iter: 175, time: 1.730 seconds , u-avg: 2.1561549468272188, window_start_time: 1.72, window_start_iteration: 174, previous_collection_time: 1.73, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.71, collecting: true
[ Info: Iter: 176, time: 1.740 seconds , u-avg: 2.0821948885524773, window_start_time: 1.72, window_start_iteration: 174, previous_collection_time: 1.74, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.74, collecting: false
[ Info: Iter: 177, time: 1.750 seconds , u-avg: 0.0, window_start_time: 1.75, window_start_iteration: 177, previous_collection_time: 1.75, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.74, collecting: true
[ Info: Iter: 178, time: 1.760 seconds , u-avg: 1.7254386818218563, window_start_time: 1.75, window_start_iteration: 177, previous_collection_time: 1.76, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.74, collecting: true
[ Info: Iter: 179, time: 1.770 seconds , u-avg: 1.6581402086967247, window_start_time: 1.75, window_start_iteration: 177, previous_collection_time: 1.77, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.77, collecting: false
[ Info: Iter: 180, time: 1.780 seconds , u-avg: 0.0, window_start_time: 1.78, window_start_iteration: 180, previous_collection_time: 1.78, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.77, collecting: true
[ Info: Iter: 181, time: 1.790 seconds , u-avg: 1.3359141226176374, window_start_time: 1.78, window_start_iteration: 180, previous_collection_time: 1.79, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.77, collecting: true
[ Info: Iter: 182, time: 1.800 seconds , u-avg: 1.2758745806538938, window_start_time: 1.78, window_start_iteration: 180, previous_collection_time: 1.7999999999999998, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.77, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 183, time: 1.810 seconds , u-avg: 1.217529300743549, window_start_time: 1.78, window_start_iteration: 180, previous_collection_time: 1.8099999999999998, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.7999999999999998, collecting: false
[ Info: Iter: 184, time: 1.820 seconds , u-avg: 0.0, window_start_time: 1.8199999999999998, window_start_iteration: 184, previous_collection_time: 1.8199999999999998, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.7999999999999998, collecting: true
[ Info: Iter: 185, time: 1.830 seconds , u-avg: 0.8865433256230806, window_start_time: 1.8199999999999998, window_start_iteration: 184, previous_collection_time: 1.8299999999999998, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.7999999999999998, collecting: false
[ Info: Iter: 186, time: 1.840 seconds , u-avg: 0.0, window_start_time: 1.8399999999999999, window_start_iteration: 186, previous_collection_time: 1.8399999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.7999999999999998, collecting: true
[ Info: Iter: 187, time: 1.850 seconds , u-avg: 0.6938736028202257, window_start_time: 1.8399999999999999, window_start_iteration: 186, previous_collection_time: 1.8499999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.8299999999999998, collecting: false
[ Info: Iter: 188, time: 1.860 seconds , u-avg: 0.0, window_start_time: 1.8599999999999999, window_start_iteration: 188, previous_collection_time: 1.8599999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.8299999999999998, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 189, time: 1.870 seconds , u-avg: 0.5235899518877757, window_start_time: 1.8599999999999999, window_start_iteration: 188, previous_collection_time: 1.8699999999999999, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.8599999999999999, collecting: false
[ Info: Iter: 190, time: 1.880 seconds , u-avg: 0.0, window_start_time: 1.88, window_start_iteration: 190, previous_collection_time: 1.88, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.8599999999999999, collecting: true
[ Info: Iter: 191, time: 1.890 seconds , u-avg: 0.3763644046007603, window_start_time: 1.88, window_start_iteration: 190, previous_collection_time: 1.89, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.8599999999999999, collecting: false
[ Info: Iter: 192, time: 1.900 seconds , u-avg: 0.0, window_start_time: 1.9, window_start_iteration: 192, previous_collection_time: 1.9, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.8599999999999999, collecting: true
[ Info: Iter: 193, time: 1.910 seconds , u-avg: 0.252777992933367, window_start_time: 1.9, window_start_iteration: 192, previous_collection_time: 1.91, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.89, collecting: false
[ Info: Iter: 194, time: 1.920 seconds , u-avg: 0.0, window_start_time: 1.92, window_start_iteration: 194, previous_collection_time: 1.92, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.89, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 195, time: 1.930 seconds , u-avg: 0.15331845599118626, window_start_time: 1.92, window_start_iteration: 194, previous_collection_time: 1.93, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.92, collecting: false
[ Info: Iter: 196, time: 1.940 seconds , u-avg: 0.0, window_start_time: 1.94, window_start_iteration: 196, previous_collection_time: 1.94, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.92, collecting: true
[ Info: Iter: 197, time: 1.950 seconds , u-avg: 0.07837831512778862, window_start_time: 1.94, window_start_iteration: 196, previous_collection_time: 1.95, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.95, collecting: false
[ Info: Iter: 198, time: 1.960 seconds , u-avg: 0.0, window_start_time: 1.96, window_start_iteration: 198, previous_collection_time: 1.96, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.95, collecting: true
[ Info: Iter: 199, time: 1.970 seconds , u-avg: 0.0282533248422682, window_start_time: 1.96, window_start_iteration: 198, previous_collection_time: 1.97, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.95, collecting: true
[ Info: Iter: 200, time: 1.980 seconds , u-avg: 0.020407752224732935, window_start_time: 1.96, window_start_iteration: 198, previous_collection_time: 1.98, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.98, collecting: false
[ Info: Iter: 201, time: 1.990 seconds , u-avg: 0.0, window_start_time: 1.99, window_start_iteration: 201, previous_collection_time: 1.99, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.98, collecting: true
[ Info: Simulation is stopping after running for 2.041 seconds.
[ Info: Simulation time 2 seconds equals or exceeds stop time 2 seconds.
[ Info: Iter: 202, time: 2 seconds , u-avg: 8.929976983568233e-15, window_start_time: 1.99, window_start_iteration: 201, previous_collection_time: 2.0, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 1.98, collecting: true

from oceananigans.jl.

hdrake commented on August 20, 2024

What is going on here? I don't understand it but it looks wrong...

[ Info: Iter: 9, time: 90.000 ms , u-avg: 0.22639226913537686, window_start_time: 0.06999999999999999, window_start_iteration: 7, previous_collection_time: 0.08999999999999998, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.06, collecting: true
[ Info: Iter: 10, time: 90 ms , u-avg: 0.22639226913537688, window_start_time: 0.06999999999999999, window_start_iteration: 7, previous_collection_time: 0.09, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.09, collecting: false

Maybe the actuations are being counted incorrectly?

from oceananigans.jl.

hdrake commented on August 20, 2024

@simone-silvestri, I think the problems arise earlier than the iterations you highlight, where zeros are written.

At first the averages are correctly being collected over three iteration, which makes sense because the interval is equal to three times the timestep.

[ Info: Iter: 1, time: 10 ms , u-avg: 0.0031413629825035438, window_start_time: 0.0, window_start_iteration: 0, previous_collection_time: 0.01, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.0, collecting: true
[ Info: Iter: 2, time: 20 ms , u-avg: 0.007851828677677537, window_start_time: 0.0, window_start_iteration: 0, previous_collection_time: 0.02, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.0, collecting: true
[ Info: Iter: 3, time: 30 ms , u-avg: 0.014652384734839867, window_start_time: 0.0, window_start_iteration: 0, previous_collection_time: 0.03, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.03, collecting: false

But after the warnings it goes out of whack:

[ Info: Iter: 29, time: 270 ms , ... collecting: false
[ Info: Iter: 30, time: 280 ms , ... collecting: true
[ Info: Iter: 31, time: 290.000 ms , ... collecting: true
[ Info: Iter: 32, time: 300 ms , ... collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/development/Oceananigans.jl/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 33, time: 310 ms , ... collecting: false
[ Info: Iter: 34, time: 320 ms , ... collecting: true
[ Info: Iter: 35, time: 330.000 ms , ... collecting: false
[ Info: Iter: 36, time: 340.000 ms , ... collecting: true
[ Info: Iter: 37, time: 350 ms , ... collecting: false

It looks like between Iter: 33 and Iter: 35 it only collects the time average for 20 ms (whereas the interval is supposed to be 30 ms) and then writes out this incomplete average. Maybe this missing 10 ms is where the zeros get written in? I don't know.

from oceananigans.jl.

glwagner commented on August 20, 2024

Not sure if it helps but the result is zero'd out here:

Oceananigans.jl/src/OutputWriters/windowed_time_average.jl

Lines 255 to 268 in 406eb9c

 elseif !(wta.schedule.collecting) 

 # run_diagnostic! has been called on schedule but we are not currently collecting data. 

 # Initialize data collection: 

 # Start averaging period 

 wta.schedule.collecting = true 

 # Zero out result 

 wta.result .= 0 

 # Save averaging start time and the initial data collection time 

 wta.window_start_time = model.clock.time 

 wta.window_start_iteration = model.clock.iteration 

 wta.previous_collection_time = model.clock.time

so when collecting flips from false to true, the result should be 0 after

from oceananigans.jl.

glwagner commented on August 20, 2024

@tomchor uses this or did at some point. Not sure who else has used it but would nice to tag them here to gather a good consensus on tests to write up.

from oceananigans.jl.

hdrake commented on August 20, 2024

so when collecting flips from false to true, the result should be 0 after

Yeah, I think the problem is that collecting is being flipped at the wrong time.

from oceananigans.jl.

hdrake commented on August 20, 2024

@liuchihl, you tested that this bug is reproduced in which versions of Oceananigans again?

from oceananigans.jl.

liuchihl commented on August 20, 2024

@hdrake I am using v0.91.5, but v0.90.11 already has this issue, so it doesn't seem like a new problem.

from oceananigans.jl.

glwagner commented on August 20, 2024

The problem is this iteration:

[ Info: Iter: 32, time: 300 ms , u-avg: 2.5442761423093008, window_start_time: 0.28, window_start_iteration: 30, previous_collection_time: 0.3, fetch_operand: true, interval: 0.03, previous_interval_stop_time: 0.27, collecting: true

We should have collecting: false.

That means end_of_window

Oceananigans.jl/src/OutputWriters/windowed_time_average.jl

Line 270 in 406eb9c

elseif end_of_window(wta.schedule, model.clock)

spuriously returns false.

Oceananigans.jl/src/OutputWriters/windowed_time_average.jl

Line 93 in 406eb9c

 end_of_window(sch::AveragedTimeInterval, clock) = clock.time >= sch.previous_interval_stop_time + sch.interval 

So we want to look at model.clock.time and sch.previous_interval_stop_time + sch.interval. It does seem probable the issue is roundoff error.

Probably AveragedTimeInterval needs to be updated to align more with the (new) criteria currently being used for TimeInterval eg we need a similar criteria as used for next_actuation_time:

Oceananigans.jl/src/Utils/schedules.jl

Lines 56 to 61 in 406eb9c

 function next_actuation_time(schedule::TimeInterval) 

 t₀ = schedule.first_actuation_time 

 N = schedule.actuations 

 T = schedule.interval 

 return t₀ + N * T 

 end

Would be helpful to have a test too.

from oceananigans.jl.

hdrake commented on August 20, 2024

Probably AveragedTimeInterval needs to be updated to align more with the (new) criteria currently being used for TimeInterval

@glwagner, are you working on this or would you like me and @liuchihl to take it on?

from oceananigans.jl.

glwagner commented on August 20, 2024

It would be great if you are able to take this up! I can help of course. It will also be useful to have fresh eyes on this algorithm.

from oceananigans.jl.

liuchihl commented on August 20, 2024

I will try to figure it out.

from oceananigans.jl.

liuchihl commented on August 20, 2024

So we want to look at model.clock.time and sch.previous_interval_stop_time + sch.interval. It does seem probable the issue is roundoff error.

It does seem like a roundoff error, below is the progress message. At iter:64, for example, model.clock.time is slightly larger than sch.previous_interval_stop_time + sch.interval （which I think they should've been equal):

[ Info: Initializing simulation...
[ Info: Iter: 0, time: 0 seconds, model clock time:0.0, previous_interval_stop_time + interval:0.1, u-avg: 0.0, window_start_time: 0.0, window_start_iteration: 0, previous_collection_time:0.0, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 0.0, collecting: true
[ Info:     ... simulation initialization complete (231.038 ms)
[ Info: Executing initial time step...
[ Info:     ... initial time step complete (546.837 ms).
[ Info: Iter: 11, time: 100 ms, model clock time:0.1, previous_interval_stop_time + interval:0.2, u-avg: 0.1202985845671951, window_start_time: 0.0, window_start_iteration: 0, previous_collection_time:0.1, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 0.1, collecting: false
[ Info: Iter: 22, time: 200 ms, model clock time:0.2, previous_interval_stop_time + interval:0.30000000000000004, u-avg: 0.8057920910530502, window_start_time: 0.11, window_start_iteration: 12, previous_collection_time:0.2, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 0.2, collecting: false
[ Info: Iter: 32, time: 300.000 ms, model clock time:0.30000000000000004, previous_interval_stop_time + interval:0.4, u-avg: 2.0225593735128355, window_start_time: 0.21, window_start_iteration: 23, previous_collection_time:0.30000000000000004, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 0.30000000000000004, collecting: false
[ Info: Iter: 42, time: 400 ms, model clock time:0.4, previous_interval_stop_time + interval:0.5, u-avg: 3.664512241688186, window_start_time: 0.3100000000000001, window_start_iteration: 33, previous_collection_time:0.4, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 0.4, collecting: false
[ Info: Iter: 52, time: 500 ms, model clock time:0.5, previous_interval_stop_time + interval:0.5, u-avg: 5.570924908683772, window_start_time: 0.4100000000000001, window_start_iteration: 43, previous_collection_time:0.5, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 0.4, collecting: false
[ Info: Iter: 64, time: 600.000 ms, model clock time:0.6000000000000001, previous_interval_stop_time + interval:0.6, u-avg: 0.0, window_start_time: 0.6000000000000001, window_start_iteration: 64, previous_collection_time:0.6000000000000001, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 0.5, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/.julia/packages/Oceananigans/OMBY0/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 75, time: 700.000 ms, model clock time:0.7000000000000001, previous_interval_stop_time + interval:0.8, u-avg: 9.512220799868613, window_start_time: 0.6199999999999999, window_start_iteration: 66, previous_collection_time:0.7000000000000001, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 0.7000000000000001, collecting: false
[ Info: Iter: 86, time: 800 ms, model clock time:0.8, previous_interval_stop_time + interval:0.9, u-avg: 10.99170409268473, window_start_time: 0.71, window_start_iteration: 76, previous_collection_time:0.8, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 0.8, collecting: false
[ Info: Iter: 97, time: 900 ms, model clock time:0.9, previous_interval_stop_time + interval:0.9, u-avg: 12.107573765048185, window_start_time: 0.8099999999999999, window_start_iteration: 87, previous_collection_time:0.9, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 0.8, collecting: false
[ Info: Iter: 108, time: 1 second, model clock time:1.0, previous_interval_stop_time + interval:1.0, u-avg: 12.691638724405168, window_start_time: 0.9299999999999997, window_start_iteration: 100, previous_collection_time:1.0, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 0.9, collecting: false
[ Info: Iter: 118, time: 1.100 seconds, model clock time:1.1, previous_interval_stop_time + interval:1.2000000000000002, u-avg: 12.56681992191321, window_start_time: 1.0300000000000007, window_start_iteration: 111, previous_collection_time:1.1, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 1.1, collecting: false
[ Info: Iter: 128, time: 1.200 seconds, model clock time:1.2000000000000002, previous_interval_stop_time + interval:1.3000000000000003, u-avg: 11.926603358212661, window_start_time: 1.1100000000000003, window_start_iteration: 119, previous_collection_time:1.2000000000000002, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 1.2000000000000002, collecting: false
[ Info: Iter: 139, time: 1.300 seconds, model clock time:1.3, previous_interval_stop_time + interval:1.3000000000000003, u-avg: 10.709836075752904, window_start_time: 1.2100000000000004, window_start_iteration: 129, previous_collection_time:1.3, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 1.2000000000000002, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/.julia/packages/Oceananigans/OMBY0/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 149, time: 1.400 seconds, model clock time:1.4000000000000001, previous_interval_stop_time + interval:1.5000000000000002, u-avg: 8.9791965740231, window_start_time: 1.3200000000000005, window_start_iteration: 141, previous_collection_time:1.4000000000000001, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 1.4000000000000001, collecting: false
[ Info: Iter: 160, time: 1.500 seconds, model clock time:1.5, previous_interval_stop_time + interval:1.5000000000000002, u-avg: 7.161470540582019, window_start_time: 1.4100000000000004, window_start_iteration: 150, previous_collection_time:1.5, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 1.4000000000000001, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/.julia/packages/Oceananigans/OMBY0/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 170, time: 1.600 seconds, model clock time:1.6, previous_interval_stop_time + interval:1.7000000000000002, u-avg: 5.078554914290692, window_start_time: 1.5200000000000005, window_start_iteration: 162, previous_collection_time:1.6, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 1.6, collecting: false
[ Info: Iter: 180, time: 1.700 seconds, model clock time:1.7000000000000002, previous_interval_stop_time + interval:1.8000000000000003, u-avg: 3.309337820047445, window_start_time: 1.6100000000000003, window_start_iteration: 171, previous_collection_time:1.7000000000000002, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 1.7000000000000002, collecting: false
[ Info: Iter: 191, time: 1.800 seconds, model clock time:1.8, previous_interval_stop_time + interval:1.8000000000000003, u-avg: 1.7406913565812152, window_start_time: 1.7100000000000004, window_start_iteration: 181, previous_collection_time:1.8, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 1.7000000000000002, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/.julia/packages/Oceananigans/OMBY0/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 201, time: 1.900 seconds, model clock time:1.9000000000000001, previous_interval_stop_time + interval:2.0, u-avg: 0.5790445551477348, window_start_time: 1.8200000000000005, window_start_iteration: 193, previous_collection_time:1.9000000000000001, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 1.9000000000000001, collecting: false
[ Info: Iter: 212, time: 2 seconds, model clock time:2.0, previous_interval_stop_time + interval:2.0, u-avg: 0.07095790120929331, window_start_time: 1.9100000000000004, window_start_iteration: 202, previous_collection_time:2.0, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 1.9000000000000001, collecting: false
[ Info: Iter: 223, time: 2.100 seconds, model clock time:2.1000000000000005, previous_interval_stop_time + interval:2.2, u-avg: 0.16557552735280148, window_start_time: 2.0299999999999994, window_start_iteration: 215, previous_collection_time:2.1000000000000005, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 2.1, collecting: false
[ Info: Iter: 234, time: 2.200 seconds, model clock time:2.2, previous_interval_stop_time + interval:2.3000000000000003, u-avg: 0.8057920910533722, window_start_time: 2.1100000000000003, window_start_iteration: 224, previous_collection_time:2.2, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 2.2, collecting: false
[ Info: Iter: 245, time: 2.300 seconds, model clock time:2.3000000000000007, previous_interval_stop_time + interval:2.4000000000000004, u-avg: 2.0225593735131833, window_start_time: 2.21, window_start_iteration: 235, previous_collection_time:2.3000000000000007, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 2.3000000000000003, collecting: false
[ Info: Iter: 256, time: 2.400 seconds, model clock time:2.4000000000000004, previous_interval_stop_time + interval:2.5000000000000004, u-avg: 3.6645122416885725, window_start_time: 2.3100000000000005, window_start_iteration: 246, previous_collection_time:2.4000000000000004, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 2.4000000000000004, collecting: false
[ Info: Iter: 267, time: 2.500 seconds, model clock time:2.5, previous_interval_stop_time + interval:2.5000000000000004, u-avg: 5.570924908684207, window_start_time: 2.41, window_start_iteration: 257, previous_collection_time:2.5, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 2.4000000000000004, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/.julia/packages/Oceananigans/OMBY0/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 278, time: 2.600 seconds, model clock time:2.6000000000000005, previous_interval_stop_time + interval:2.7, u-avg: 7.6538405349756085, window_start_time: 2.5199999999999996, window_start_iteration: 269, previous_collection_time:2.6000000000000005, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 2.6, collecting: false
[ Info: Iter: 289, time: 2.700 seconds, model clock time:2.7, previous_interval_stop_time + interval:2.8000000000000003, u-avg: 9.42305762921893, window_start_time: 2.6100000000000003, window_start_iteration: 279, previous_collection_time:2.7, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 2.7, collecting: false
[ Info: Iter: 300, time: 2.800 seconds, model clock time:2.8000000000000007, previous_interval_stop_time + interval:2.9000000000000004, u-avg: 10.991704092685225, window_start_time: 2.71, window_start_iteration: 290, previous_collection_time:2.8000000000000007, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 2.8000000000000003, collecting: false
[ Info: Iter: 311, time: 2.900 seconds, model clock time:2.9000000000000004, previous_interval_stop_time + interval:3.0000000000000004, u-avg: 12.107573765048683, window_start_time: 2.8100000000000005, window_start_iteration: 301, previous_collection_time:2.9000000000000004, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 2.9000000000000004, collecting: false
[ Info: Iter: 322, time: 3 seconds, model clock time:3.0, previous_interval_stop_time + interval:3.0000000000000004, u-avg: 12.66143754805721, window_start_time: 2.91, window_start_iteration: 312, previous_collection_time:3.0, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 2.9000000000000004, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/.julia/packages/Oceananigans/OMBY0/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 333, time: 3.100 seconds, model clock time:3.1000000000000005, previous_interval_stop_time + interval:3.2, u-avg: 12.583985175726626, window_start_time: 3.0199999999999996, window_start_iteration: 324, previous_collection_time:3.1000000000000005, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 3.1, collecting: false
[ Info: Iter: 344, time: 3.200 seconds, model clock time:3.2, previous_interval_stop_time + interval:3.3000000000000003, u-avg: 11.926603358213136, window_start_time: 3.1100000000000003, window_start_iteration: 334, previous_collection_time:3.2, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 3.2, collecting: false
[ Info: Iter: 355, time: 3.300 seconds, model clock time:3.3000000000000007, previous_interval_stop_time + interval:3.4000000000000004, u-avg: 10.709836075753326, window_start_time: 3.21, window_start_iteration: 345, previous_collection_time:3.3000000000000007, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 3.3000000000000003, collecting: false
[ Info: Iter: 366, time: 3.400 seconds, model clock time:3.4000000000000004, previous_interval_stop_time + interval:3.5000000000000004, u-avg: 9.067883207577935, window_start_time: 3.3100000000000005, window_start_iteration: 356, previous_collection_time:3.4000000000000004, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 3.4000000000000004, collecting: false
[ Info: Iter: 377, time: 3.500 seconds, model clock time:3.5, previous_interval_stop_time + interval:3.5000000000000004, u-avg: 7.161470540582303, window_start_time: 3.41, window_start_iteration: 367, previous_collection_time:3.5, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 3.4000000000000004, collecting: true
┌ Warning: Returning a WindowedTimeAverage before the collection period is complete.
└ @ Oceananigans.OutputWriters ~/.julia/packages/Oceananigans/OMBY0/src/OutputWriters/windowed_time_average.jl:201
[ Info: Iter: 388, time: 3.600 seconds, model clock time:3.6000000000000005, previous_interval_stop_time + interval:3.7, u-avg: 5.078554914290898, window_start_time: 3.5199999999999996, window_start_iteration: 379, previous_collection_time:3.6000000000000005, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 3.6, collecting: false
[ Info: Iter: 399, time: 3.700 seconds, model clock time:3.7, previous_interval_stop_time + interval:3.7, u-avg: 3.309337820047581, window_start_time: 3.6100000000000003, window_start_iteration: 389, previous_collection_time:3.7, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 3.6, collecting: false
[ Info: Iter: 410, time: 3.800 seconds, model clock time:3.8000000000000007, previous_interval_stop_time + interval:3.9000000000000004, u-avg: 1.6002624850128424, window_start_time: 3.7299999999999995, window_start_iteration: 402, previous_collection_time:3.8000000000000007, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 3.8000000000000003, collecting: false
[ Info: Iter: 421, time: 3.900 seconds, model clock time:3.9000000000000004, previous_interval_stop_time + interval:4.0, u-avg: 0.6248216842178228, window_start_time: 3.8100000000000005, window_start_iteration: 411, previous_collection_time:3.9000000000000004, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 3.9000000000000004, collecting: false
[ Info: Iter: 432, time: 4 seconds, model clock time:4.0, previous_interval_stop_time + interval:4.0, u-avg: 0.07095790120929706, window_start_time: 3.91, window_start_iteration: 422, previous_collection_time:4.0, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 3.9000000000000004, collecting: false
[ Info: Iter: 443, time: 4.100 seconds, model clock time:4.1000000000000005, previous_interval_stop_time + interval:4.2, u-avg: 0.16557552735280162, window_start_time: 4.029999999999999, window_start_iteration: 435, previous_collection_time:4.1000000000000005, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 4.1000000000000005, collecting: false
[ Info: Iter: 454, time: 4.200 seconds, model clock time:4.2, previous_interval_stop_time + interval:4.2, u-avg: 0.8057920910533768, window_start_time: 4.11, window_start_iteration: 444, previous_collection_time:4.2, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 4.1000000000000005, collecting: false
[ Info: Iter: 465, time: 4.300 seconds, model clock time:4.3, previous_interval_stop_time + interval:4.3, u-avg: 2.164461823304153, window_start_time: 4.2299999999999995, window_start_iteration: 457, previous_collection_time:4.3, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 4.2, collecting: false
[ Info: Iter: 476, time: 4.400 seconds, model clock time:4.4, previous_interval_stop_time + interval:4.5, u-avg: 3.8428643990560714, window_start_time: 4.329999999999999, window_start_iteration: 468, previous_collection_time:4.4, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 4.4, collecting: false
[ Info: Iter: 487, time: 4.500 seconds, model clock time:4.5, previous_interval_stop_time + interval:4.5, u-avg: 5.57092490868422, window_start_time: 4.41, window_start_iteration: 477, previous_collection_time:4.5, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 4.4, collecting: false
[ Info: Iter: 498, time: 4.600 seconds, model clock time:4.6000000000000005, previous_interval_stop_time + interval:4.7, u-avg: 7.7522019307475505, window_start_time: 4.529999999999999, window_start_iteration: 490, previous_collection_time:4.6000000000000005, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 4.6000000000000005, collecting: false
[ Info: Iter: 509, time: 4.700 seconds, model clock time:4.7, previous_interval_stop_time + interval:4.7, u-avg: 9.42305762921895, window_start_time: 4.61, window_start_iteration: 499, previous_collection_time:4.7, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 4.6000000000000005, collecting: false
[ Info: Iter: 520, time: 4.800 seconds, model clock time:4.800000000000001, previous_interval_stop_time + interval:4.9, u-avg: 11.132132964253687, window_start_time: 4.7299999999999995, window_start_iteration: 512, previous_collection_time:4.800000000000001, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 4.800000000000001, collecting: false
[ Info: Iter: 531, time: 4.900 seconds, model clock time:4.9, previous_interval_stop_time + interval:5.0, u-avg: 12.10757376504871, window_start_time: 4.8100000000000005, window_start_iteration: 521, previous_collection_time:4.9, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 4.9, collecting: false
[ Info: Iter: 542, time: 5 seconds, model clock time:5.0, previous_interval_stop_time + interval:5.0, u-avg: 12.661437548057236, window_start_time: 4.91, window_start_iteration: 532, previous_collection_time:5.0, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 4.9, collecting: false
[ Info: Iter: 553, time: 5.100 seconds, model clock time:5.1000000000000005, previous_interval_stop_time + interval:5.2, u-avg: 12.566819921913737, window_start_time: 5.029999999999999, window_start_iteration: 545, previous_collection_time:5.1000000000000005, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 5.1000000000000005, collecting: false
[ Info: Iter: 564, time: 5.200 seconds, model clock time:5.2, previous_interval_stop_time + interval:5.2, u-avg: 11.926603358213159, window_start_time: 5.11, window_start_iteration: 554, previous_collection_time:5.2, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 5.1000000000000005, collecting: false
[ Info: Iter: 575, time: 5.300 seconds, model clock time:5.300000000000001, previous_interval_stop_time + interval:5.4, u-avg: 10.567933625962374, window_start_time: 5.2299999999999995, window_start_iteration: 567, previous_collection_time:5.300000000000001, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 5.300000000000001, collecting: false
[ Info: Iter: 586, time: 5.400 seconds, model clock time:5.4, previous_interval_stop_time + interval:5.5, u-avg: 9.067883207577948, window_start_time: 5.3100000000000005, window_start_iteration: 576, previous_collection_time:5.4, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 5.4, collecting: false
[ Info: Iter: 597, time: 5.500 seconds, model clock time:5.5, previous_interval_stop_time + interval:5.5, u-avg: 7.1614705405823145, window_start_time: 5.41, window_start_iteration: 587, previous_collection_time:5.5, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 5.4, collecting: false
[ Info: Iter: 608, time: 5.600 seconds, model clock time:5.6000000000000005, previous_interval_stop_time + interval:5.7, u-avg: 4.980193518518983, window_start_time: 5.529999999999999, window_start_iteration: 600, previous_collection_time:5.6000000000000005, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 5.6000000000000005, collecting: false
[ Info: Iter: 619, time: 5.700 seconds, model clock time:5.7, previous_interval_stop_time + interval:5.7, u-avg: 3.309337820047582, window_start_time: 5.61, window_start_iteration: 609, previous_collection_time:5.7, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 5.6000000000000005, collecting: false
[ Info: Iter: 630, time: 5.800 seconds, model clock time:5.800000000000001, previous_interval_stop_time + interval:5.9, u-avg: 1.6002624850128442, window_start_time: 5.7299999999999995, window_start_iteration: 622, previous_collection_time:5.800000000000001, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 5.800000000000001, collecting: false
[ Info: Iter: 641, time: 5.900 seconds, model clock time:5.9, previous_interval_stop_time + interval:6.0, u-avg: 0.6248216842178208, window_start_time: 5.8100000000000005, window_start_iteration: 631, previous_collection_time:5.9, fetch_operand: true, interval: 0.1, previous_interval_stop_time: 5.9, collecting: false
[ Info: Simulation is stopping after running for 1.413 seconds.

from oceananigans.jl.

liuchihl commented on August 20, 2024

Probably AveragedTimeInterval needs to be updated to align more with the (new) criteria currently being used for TimeInterval eg we need a similar criteria as used for next_actuation_time:

@glwagner Could you maybe provide some hints on where the source code in AveragedTimeInterval should be modified to align with the implementation in TimeInterval? So far I haven't been able to fix the bug. My guess is something along these lines, but I could be wrong:

Oceananigans.jl/src/OutputWriters/windowed_time_average.jl

Lines 130 to 142 in 406eb9c

 function outside_window(schedule::AveragedSpecifiedTimes, clock) 

 next = schedule.specified_times.previous_actuation + 1 

 next > length(schedule.specified_times.times) && return true 

 next_time = schedule.specified_times.times[next] 

 return clock.time < next_time - schedule.window 

 end 

 function end_of_window(schedule::AveragedSpecifiedTimes, clock) 

 next = schedule.specified_times.previous_actuation + 1 

 next > length(schedule.specified_times.times) && return true 

 next_time = schedule.specified_times.times[next] 

 return clock.time >= next_time 

 end

from oceananigans.jl.

liuchihl commented on August 20, 2024

Thanks a lot for your hint, I wanted to make sure what you mean by number of actuations as a criteria. Do you mean

first_actuation_time::Number

and do you think this will potentially prevent the floating point issue?

from oceananigans.jl.

liuchihl commented on August 20, 2024

yes, it makes sense to me now! Thanks for the clear explanation.

from oceananigans.jl.

Error in AveragedTimeInterval Output about oceananigans.jl HOT 32 OPEN

Comments (32)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	function accumulate_result!(wta, clock::Clock, integrand=wta.operand)
	# Time increment:
	Δt = clock.time - wta.previous_collection_time

	function (schedule::TimeInterval)(model)
	t = model.clock.time
	t★ = next_actuation_time(schedule)

	if t >= t★
	if schedule.actuations < typemax(Int)
	schedule.actuations += 1
	else # re-initialize the schedule to t★
	initialize!(schedule, t★)
	end
	return true
	else
	return false
	end
	end

	function next_actuation_time(schedule::TimeInterval)
	t₀ = schedule.first_actuation_time
	N = schedule.actuations
	T = schedule.interval
	return t₀ + (N + 1) * T
	end

	function initialize!(schedule::TimeInterval, first_actuation_time::Number)
	schedule.first_actuation_time = first_actuation_time
	schedule.actuations = 0
	return true
	end

	initialize!(schedule::TimeInterval, model) = initialize!(schedule, model.clock.time)

	elseif !(wta.schedule.collecting)
	# run_diagnostic! has been called on schedule but we are not currently collecting data.
	# Initialize data collection:

	# Start averaging period
	wta.schedule.collecting = true

	# Zero out result
	wta.result .= 0

	# Save averaging start time and the initial data collection time
	wta.window_start_time = model.clock.time
	wta.window_start_iteration = model.clock.iteration
	wta.previous_collection_time = model.clock.time

	function outside_window(schedule::AveragedSpecifiedTimes, clock)
	next = schedule.specified_times.previous_actuation + 1
	next > length(schedule.specified_times.times) && return true
	next_time = schedule.specified_times.times[next]
	return clock.time < next_time - schedule.window
	end

	function end_of_window(schedule::AveragedSpecifiedTimes, clock)
	next = schedule.specified_times.previous_actuation + 1
	next > length(schedule.specified_times.times) && return true
	next_time = schedule.specified_times.times[next]
	return clock.time >= next_time
	end