Atomic file saves? about rustbreak HOT 20 CLOSED

This sounds like a sensible solution. I’ll add it.

from rustbreak.

tempfile crate implements it nicely:

use tempfile::NamedTempFile;
…
let tmp_path = NamedTempFile::new_in(self.path.parent().unwrap())?;
fs::write(&tmp_path, serialize()?)?;
tmp_path.persist(&self.path)?;

Ensures the temp file is deleted on error, etc.

from rustbreak.

I think it would be extremely counter-intuitive to have the save behaviour depend on the creation method (file or path). I think it would be a lot clearer to split it out to two back-ends (say FileBackend without atomic saves and PathBackend with atomic saves).

Is backwards compatibility really required? From version 1 to 2 you can make breaking changes (and 2 hasn't been released yet).

from rustbreak.

I'm looking forward to this! I'm using panic="abort" and saving app state periodically to disk using rustbreak. If a panic happens in the middle of the saving the db, it corrupts the db file..

@kornelski With the current version of rustbreak (and v1), it wouldn't corrupt the db file with panic="unwind" if a panic happens in the middle of the saving the db, right?

from rustbreak.

The process can get SIGTERM/SIGKILL at any time. I'm most worried about unfortunate press of Ctrl-C corrupting the db.

from rustbreak.

Ah yes, that happens more often than panics :)

from rustbreak.

Any update on this?
In my experience, corrupting the db happens a lot when pressing Ctrl-C.
My application saves its state to the db every few seconds, and writing takes some time even though I use the binary encoding. So the chance of Ctrl-C interrupting the write are very high.
I'm really looking forward to atomic file saves.

from rustbreak.

I've used this instead: https://gitlab.com/crates.rs/simple_cache/blob/master/src/kv.rs

from rustbreak.

@Boscop

In my experience, corrupting the db happens a lot when pressing Ctrl-C.
My application saves its state to the db every few seconds, and writing takes some time even though I use the binary encoding. So the chance of Ctrl-C interrupting the write are very high.

You can use some signal handling to migrate this risk (except SIGKILL of course, which isn't really a signal...). The cli-wg has a short introduction to signal handling in rust: https://rust-lang-nursery.github.io/cli-wg/in-depth/signals.html. Atomic saves would be very nice though.

from rustbreak.

@niluxv But using the ctrlc crate to intercept Ctrl-C doesn't work when running my exe through cargo watch -x run which I'm always using during development!
rust-lang/cargo#4575

from rustbreak.

Ah, oke. cargo-watch should probably forward signals. I saw you already created an issue for that.

from rustbreak.

@TheNeikos Any update on this? Btw: https://crates.io/crates/atomicwrites

from rustbreak.

I thought lets give it a try, but there is a catch. You need to close the file before you can replace it by the temporary file. I can think of 3 options:

1. Change the Backend (trait) API to give ownership of self to put_data (and return a new instance in the Result).
2. Do not keep an open file handle in FileBackend, but rather a path to the file (This is the trick used by atomicwrites).
3. Use a crate like take_mut (or the more recent replace_with). This will pull in a dependancy with unsafe code though.

from rustbreak.

If you need fast reads (that is, Backend::get_data) I'd advice against option 2, since it requires to open the file on every read. But most of the time you are probably reading the in memory representation (not loading from the backend).

from rustbreak.

Yes, in the current state, adding this is somewhat difficult generally. Since the FileBackend only takes a File, it has no notion of a path and thus can't re-open it. I'd accept a PR adding a backwards compatible option if the FileBackend was created with a path where it would re-open it.

from rustbreak.

@niluxv That's actually a good idea. And you're correct, it's only in RC, I forgot that it is technically a pre-release and thus could introduce breaking changes from 1 to 2.

Adding a PathBackend is also feasible and the correct path forward I feel. It side-steps the constructor specific problem.

from rustbreak.

I will create a PR.

from rustbreak.

I think this can be closed now. Atomic saves are now available through the PathBackend backend (and a database with this backend is aliased PathDatabase).

from rustbreak.

Any idea when this version with atomic saving will be available on crates.io? :)
Or can you at least please re-generate the docs based on master?

from rustbreak.

@TheNeikos Maybe release a 2.0.0-rc4?

Before the final stable 2.0.0 I'd like to solve issue #67 (if you agree to the proposed solution). That would be a breaking change, so it has to be done before 2.0.0.

Updating the dependencies as in pull request #73 is also a breaking change. Maybe this could be merged even before an rc4 release, since it updates a vulnarable dependency?

from rustbreak.