yoshuawuyts / fd-lock Goto Github PK

Advisory cross-platform file locks using file descriptors

License: Apache License 2.0

Rust 100.00%

fd-lock's Introduction

fd-lock

Advisory cross-platform file locks using file descriptors. Adapted from mafintosh/fd-lock.

Note that advisory lock compliance is opt-in, and can freely be ignored by other parties. This means this crate should never be used for security purposes, but solely to coordinate file access.

Examples

Basic usage

use std::io::prelude::*;
use std::fs::File;
use fd_lock::RwLock;

// Lock a file and write to it.
let mut f = RwLock::new(File::open("foo.txt")?);
write!(f.write()?, "chashu cat")?;

// A lock can also be held across multiple operations.
let mut f = f.write()?;
write!(f, "nori cat")?;
write!(f, "bird!")?;

Installation

$ cargo add fd-lock

Safety

This crate uses unsafe on Windows to interface with windows-sys. All invariants have been carefully checked, and are manually enforced.

Contributing

Want to join us? Check out our "Contributing" guide and take a look at some of these issues:

References

License

MIT OR Apache-2.0

fd-lock's People

Contributors

Stargazers

Watchers

Forkers

mexus veetaha tecywiz121 songlinshu puzzlewolf sunfishcode icodein khonsulabs kennykerr sinhpham jgoerzen mati865 nathaniel-daniel lit-protocol imberflur psumbera embarkstudios limeth

fd-lock's Issues

create a .lock function

https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-lockfileex has a way of enabling "block-on-fail" semantics that would allow us to define lock method, similar to how a regular mutex works.

For Unix we could omit the NB_LOCK flag to achieve the same result.

read_owned and write_owned

My use case: keeping the lock guard tied to an object

struct MyStruct {
    // .... data
    lock: OwnedRwLockWriteGuard
}

It's similar to tokio's RwLock API: https://docs.rs/tokio/latest/tokio/sync/struct.RwLock.html#method.write_owned

I noticed the documentation for the locks states that the drop impl may panic if there is a problem with unlocking. Looking at unix and windows, only windows seems to panic on drop due to errors while unlocking, while the unix implementation seems to ignore errors. Is it even possible for unlocks on unix to create an error?

Regardless, I think making all platforms avoid panics and ignore errors would be better, as that mirrors Rust file behavior more closely.
At the very least, I think adding a method to allow manually unlocking and returning a result on errors would be helpful so that users who wish to do so can avoid panicking if they fail to unlock a file.

If this is a desired change, I am willing to submit a PR.

If the panic on drop behavior is intended, I think that unix should also panic on drop so that all platforms behave the same.

Investigate `fd-lock` failure for `spidermonkey-wasm`

bytecodealliance/spidermonkey-wasm-rs#40 (comment) reported a failure of the fd-lock crate, requiring a dependency to be pinned to an earlier version.

Quote:

From my digging it looks like https://github.com/yoshuawuyts/fd-lock relies on some features that were removed in one of the more recent Rust releases.

Does not build on Windows

   Compiling fd-lock v1.1.0 (C:\Users\Peter\Code\wincheck\fd-lock)
error[E0432]: unresolved import `winapi::um::fileapi::LOCKFILE_EXCLUSIVE_LOCK`
 --> src\windows.rs:3:61
  |
3 | use winapi::um::fileapi::{LockFile, LockFileEx, UnlockFile, LOCKFILE_EXCLUSIVE_LOCK};
  |                                                             ^^^^^^^^^^^^^^^^^^^^^^^ no `LOCKFILE_EXCLUSIVE_LOCK` in `um::fileapi`

error[E0433]: failed to resolve: use of undeclared type or module `ErrorKind`
  --> src\windows.rs:11:13
   |
11 |         Err(ErrorKind::Locked.into())
   |             ^^^^^^^^^ use of undeclared type or module `ErrorKind`

error[E0433]: failed to resolve: use of undeclared type or module `ErrorKind`
  --> src\windows.rs:21:13
   |
21 |         Err(ErrorKind::Locked.into())
   |             ^^^^^^^^^ use of undeclared type or module `ErrorKind`

error[E0433]: failed to resolve: use of undeclared type or module `ErrorKind`
  --> src\windows.rs:81:17
   |
81 |             Err(ErrorKind::Other.into())
   |                 ^^^^^^^^^ use of undeclared type or module `ErrorKind`

error[E0433]: failed to resolve: use of undeclared type or module `ErrorKind`
  --> src\windows.rs:97:17
   |
97 |             Err(ErrorKind::Other.into())
   |                 ^^^^^^^^^ use of undeclared type or module `ErrorKind`

error[E0412]: cannot find type `LockGuard` in this scope
 --> src\windows.rs:7:42
  |
7 | pub fn lock(handle: RawHandle) -> Result<LockGuard, Error> {
  |                                          ^^^^^^^^^ help: a struct with a similar name exists: `FdLockGuard`

error[E0412]: cannot find type `Error` in this scope
 --> src\windows.rs:7:53
  |
7 | pub fn lock(handle: RawHandle) -> Result<LockGuard, Error> {
  |                                                     ^^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
  |
1 | use core::fmt::Error;
  |
1 | use crate::error::Error;
  |
1 | use failure::Error;
  |
1 | use std::error::Error;
  |
and 2 other candidates

error[E0422]: cannot find struct, variant or union type `LockGuard` in this scope
 --> src\windows.rs:9:12
  |
9 |         Ok(LockGuard { handle })
  |            ^^^^^^^^^ help: a struct with a similar name exists: `FdLockGuard`

error[E0412]: cannot find type `HANDLE` in this scope
  --> src\windows.rs:17:19
   |
17 | fn unlock(handle: HANDLE) -> Result<(), Error> {
   |                   ^^^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
   |
1  | use std::os::windows::raw::HANDLE;
   |
1  | use winapi::shared::ntdef::HANDLE;
   |
1  | use winapi::um::winnt::HANDLE;
   |

error[E0412]: cannot find type `Error` in this scope
  --> src\windows.rs:17:41
   |
17 | fn unlock(handle: HANDLE) -> Result<(), Error> {
   |                                         ^^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
   |
1  | use core::fmt::Error;
   |
1  | use crate::error::Error;
   |
1  | use failure::Error;
   |
1  | use std::error::Error;
   |
and 2 other candidates

error[E0425]: cannot find value `handle` in this scope
  --> src\windows.rs:51:33
   |
51 |         if unsafe { !UnlockFile(handle, 0, 0, 1, 0) } {
   |                                 ^^^^^^ not found in this scope

error[E0412]: cannot find type `Error` in this scope
  --> src\windows.rs:76:58
   |
76 |     pub fn lock(&mut self) -> Result<FdLockGuard<'_, T>, Error> {
   |                                                          ^^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
   |
1  | use core::fmt::Error;
   |
1  | use crate::error::Error;
   |
1  | use failure::Error;
   |
1  | use std::error::Error;
   |
and 2 other candidates

error[E0425]: cannot find value `handle` in this scope
  --> src\windows.rs:78:32
   |
78 |         if unsafe { LockFileEx(handle, LOCKFILE_EXCLUSIVE_LOCK, 0, 1, 0) } {
   |                                ^^^^^^ not found in this scope

error[E0412]: cannot find type `Error` in this scope
  --> src\windows.rs:92:62
   |
92 |     pub fn try_lock(&mut self) -> Result<FdLockGuard<'_, T>, Error> {
   |                                                              ^^^^^ not found in this scope
help: possible candidates are found in other modules, you can import them into scope
   |
1  | use core::fmt::Error;
   |
1  | use crate::error::Error;
   |
1  | use failure::Error;
   |
1  | use std::error::Error;
   |
and 2 other candidates

error[E0425]: cannot find value `handle` in this scope
  --> src\windows.rs:94:30
   |
94 |         if unsafe { LockFile(handle, 0, 0, 1, 0) } {
   |                              ^^^^^^ not found in this scope

error[E0308]: mismatched types
 --> src\windows.rs:8:17
  |
8 |     if unsafe { LockFile(handle, 0, 0, 1, 0) } {
  |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected bool, found i32

error[E0308]: mismatched types
  --> src\windows.rs:18:17
   |
18 |     if unsafe { UnlockFile(handle, 0, 0, 1, 0) } {
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected bool, found i32

error[E0599]: no method named `as_raw_fd` found for type `T` in the current scope
  --> src\windows.rs:50:30
   |
50 |         let fd = self.lock.t.as_raw_fd();
   |                              ^^^^^^^^^

error[E0308]: mismatched types
  --> src\windows.rs:51:21
   |
51 |         if unsafe { !UnlockFile(handle, 0, 0, 1, 0) } {
   |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected bool, found i32

error[E0599]: no method named `as_raw_fd` found for type `T` in the current scope
  --> src\windows.rs:77:25
   |
77 |         let fd = self.t.as_raw_fd();
   |                         ^^^^^^^^^

error[E0061]: this function takes 6 parameters but 5 parameters were supplied
  --> src\windows.rs:78:21
   |
78 |         if unsafe { LockFileEx(handle, LOCKFILE_EXCLUSIVE_LOCK, 0, 1, 0) } {
   |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected 6 parameters

error[E0308]: mismatched types
  --> src\windows.rs:78:21
   |
78 |         if unsafe { LockFileEx(handle, LOCKFILE_EXCLUSIVE_LOCK, 0, 1, 0) } {
   |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected bool, found i32

error[E0599]: no method named `as_raw_fd` found for type `T` in the current scope
  --> src\windows.rs:93:25
   |
93 |         let fd = self.t.as_raw_fd();
   |                         ^^^^^^^^^

error[E0308]: mismatched types
  --> src\windows.rs:94:21
   |
94 |         if unsafe { LockFile(handle, 0, 0, 1, 0) } {
   |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected bool, found i32

error: aborting due to 24 previous errors

Some errors have detailed explanations: E0061, E0308, E0412, E0422, E0425, E0432, E0433, E0599.
For more information about an error, try `rustc --explain E0061`.
error: Could not compile `fd-lock`.

mutable read guard

Hi,

Thanks for this crate! I have a question - I'm finding it a bit impractical that the read guard doesn't have mutable access as e.g. reading from a file requires it (all the read methods in std::io::Read use &mut self). Because there's no DerefMut, it's not possible to call these directly on the guard without e.g. wrapping it in BufReader first, e.g.:

        let lock = RwLock::new(File::open(path).unwrap());
        let mut guard = lock.try_read().unwrap();
        let mut read = vec![];
        // doesn't compile
        guard.read_to_end(&mut read).unwrap();

Is this intended? I see that you want to allow shared read access which makes sense, but perhaps a lock/guard does not need to be re-used for that so allowing a mutable access for reading might have a more ergonomic api. Wdyt?

Return proper Error for `try_*` on Windows

On Unix, using the try_* family of functions compares the error result to check if the error is a WouldBlock error, as seen here. On Windows however, any error that occurs while using the try_* is incorrectly reported as a WouldBlock error, as the actual error is ignored.

This can be seen on Windows with the following example:

use fd_lock::RwLock;
use std::fs::File;
use std::os::windows::fs::OpenOptionsExt;

fn main() {
    let file = File::options()
        .read(true)
        .write(false)
        .access_mode(0)
        .open("test.lock")
        .expect("failed to create file");
    
    let mut file = RwLock::new(file);

    let _lock = file.write().expect("failed to lock"); 
}

This code will panic with failed to lock: Os { code: 5, kind: PermissionDenied, message: "Access is denied." }, due to the access_mode(0) flag. However, if the code is modified to instead use try_write, the panic becomes failed to lock: Kind(WouldBlock), hiding the actual error.

The error returned by Windows to denote that a call to try_* failed seems to be error 33, ERROR_LOCK_VIOLATION. Unfortunately, Rust's std::io::Error seems to classify this as std::io::ErrorKind::Uncategorized, so a match would need to be done on the error code.

I think that the Windows implementation should perform a match on the returned error for the try_* functions, as that is what Unix does. Additionally, I think its important for callers to be able to differentiate between a lock already being held on a file and a lock that failed due to some I/O error.

Repo and release not in sync?

Hi I noticed the latest crates.io version for this repo is 3.0.3 but I don't see that change reflected in this repo, the latest commit to Cargo.toml on the master branch shows version 3.0.2. Am I missing something or are you using a different repo to publish to crates.io?

How to write `try_write` followed by `write`?

This program does a try_write(), and if that fails due to another process holding the lock, prints a message, and then does a write() to wait until the lock is available:

use std::io::Write;
use fd_lock::RwLock;

fn main() -> std::io::Result<()> {
    let file = std::fs::File::create("test.file")?;
    let mut lock = RwLock::new(file);

    let mut guard = match lock.try_write() {
        Ok(guard) => guard,
        Err(err) if err.kind() == std::io::ErrorKind::WouldBlock => {
            eprintln!("blocking on other processes...");
            lock.write()?
        }
        Err(err) => return Err(err),
    };

    eprintln!("writing to the file");

    writeln!(&mut guard, "hello")?;

    Ok(())
}

However, this doesn't compile:

error[E0499]: cannot borrow `lock` as mutable more than once at a time
  --> examples/foo.rs:12:13
   |
8  |     let mut guard = match lock.try_write() {
   |                           ----------------
   |                           |
   |                           first mutable borrow occurs here
   |                           a temporary with access to the first borrow is created here ...
...
12 |             lock.write()?
   |             ^^^^^^^^^^^^ second mutable borrow occurs here
...
15 |     };
   |      - ... and the first borrow might be used here, when that temporary is dropped and runs the destructor for type `Result<fd_lock::RwLockWriteGuard<'_, File>, std::io::Error>`

Is there a way to do what this code is doing that avoids this problem?

Alternatively, would it makes sense for try_write and write to take a &self instead of a &mut self? They don't need the mut for any of their own state.

Shared locks?

Hi, as far as I can see this is not yet possible. It would be great with shared locks, essentially LOCK_SH in flock. fs3 has shared_lock, but not in a LockGuarded out-of-scopey-unlockey way.

Reduce dependency on failure

Feature Request

Summary

Remove failure dependency

Motivation

Unnecessary dependency which may not be used on dependent packages.

Guide-level explanation

Explain the proposal as if it was already included in the project and you
were teaching it to another programmer. That generally means:

Handle error manually.

Reference-level explanation

This is the technical portion of the feature request. Explain the design in
sufficient detail that:

Remove fail and implement Error.

Drawbacks

Unnecessary dependency on failure as a library.

Rationale and alternatives

Switch to other failure alternative such as thiserror and anyhow, but library should not dictates which error handling library to be used and let the application handles it.

Unresolved Questions

None.

Provide a `try_new` API for unsupported platforms

Right now, if running this code on an unsupported platform it just panics. It would be nice if there was an API to check if locks were supported and handle it gracefully.

With v3.0.1, getting error "an unknown tool name found in scoped lint"

I am using rust 1.56.1 and after doing a cargo update on my project the fd-lock version changed from v03.0.0 to v3.0.1 (as a transitive dependency). I now get the below error:

...
Compiling fd-lock v3.0.1
error[E0710]: an unknown tool name found in scoped lint: `rustdoc::missing_doc_code_examples`
  --> /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.1/src/lib.rs:33:23
   |
33 | #![warn(missing_docs, rustdoc::missing_doc_code_examples)]
   |                       ^^^^^^^

I believe this is because rustdoc::missing_doc_code_example is only available in nightly. I worked around the issue by pinning the version of fd-lock to 3.0.0 in my project, but I believe your MSRV is stable so I shouldn't get this error.

Rewrite using io_lifetimes crate / IO safety RFC

https://io-experiment.sunfishcode.online/x86_64-unknown-linux-gnu/io_lifetimes/index.html now exists, which is an experimental implementation of the IO safety RFC. We should create a branch of this code which uses that instead of what we're currently doing. Not in the least to validate that the RFC enables what it promises to.

flock returns -1, and sets errno

Bug Report

Your Environment

Software	Version(s)
fd-lock	1.1.1
Rustc	rustc 1.47.0-nightly (bf4342114 2020-08-25)
Operating System	Linux XPS-15-9570 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Expected Behavior

FdLock::try_lock on an already locked file should return a Locked error kind.

Current Behavior

Returns Other error kind instead.

Code Sample

use fd_lock::FdLock;
use std::fs::File;
use tempfile::tempdir;

fn main() {
    let dir = tempdir().unwrap();
    let path = dir.path().join("lockfile");

    let mut l0 = FdLock::new(File::create(&path).unwrap());
    let mut l1 = FdLock::new(File::open(path).unwrap());

    let g0 = l0.lock();

    let err = l1.try_lock().unwrap_err();
    println!("{:?}", err.kind());

    drop(g0);
}