`JLD2OutputWriter` and `Checkpointer` don't work when `max_filesize` and `part` are specified. about oceananigans.jl HOT 2 OPEN

The intended user experience is that only one line should need to be changed: pickup=false to pickup=true in run!.

Therefore, users should not have to manually specify the "part" that they want to pick up from. I don't like option 2 above.

I think that fixing this problem may become much easier if we can "delay" the creation of the output file. Right now, the output file is created when we build the output writer. But at that point, we have no way of knowing whether we are going to pick up or not.

I've long wanted to implement this "delay" but more pressing matters have intervened...

The basic thing we need to do is to add an initialize!(output_writer, sim) utility, which will create the output file. That function then will know whether the simulation is starting fresh (because iteration(sim) == 0, or whether it is "continuing"). One huge feature this will enable is the ability to avoid overwriting an existing file when it represents the output from the current continuing run. That's a huge problem with the current interface, is that you have to be really careful about overwrite_existing if you are trying to pickup from a checkpoint. And I think that's a big problem.

With that feature I think we can also figure out how to handle output that is split into multiple files --- because we know if a simulation is continuing that we will have to figure out which part to use (if any).

continues writing into the most recent output file once it catches up to the latest unsaved iteration.

This is a separate feature from what I was talking about, but I think it's also a great idea! There also may be a clue how to solve a roundoff error issue, where two outputs are written one iteration separate from one another, but at virtually identical times (eg distinguished only by machine epsilon).

PS: I simplified the example a bit to help me understand it

from oceananigans.jl.

Why do we even have the "part" kw for JLD2OutputWriter? I feel this is a weird detail and users should not have to set that.

from oceananigans.jl.