openssh-rust / openssh-sftp-client Goto Github PK

sftp v3 client implemented using pure rust

License: MIT License

Rust 99.36% Shell 0.64%

openssh-sftp-client's Introduction

openssh-sftp-client

openssh-sftp-client, implements sftp v3 accodring to openssh-portable/sftp-client.c in rust using tokio and serde.

It exposes highlevel async APIs that models closely after std::fs that are easy to use.

Extensions

This crate support the following extensions:

limits
expand path
fsync
hardlink
posix rename
copy-data

How to run tests

For macOS, please install latest rsync from homebrew.

./run_tests.sh

openssh-sftp-client's People

Contributors

Stargazers

Watchers

Forkers

kylegodbold yah01 xis19

openssh-sftp-client's Issues

openssh-sftp-clien 0.14 seems broken

apache/opendal#3466 (comment)

   Compiling concurrent_arena v0.1.8
   Compiling openssh-sftp-protocol v0.24.0
   Compiling openssh-sftp-error v0.3.1
   Compiling openssh-sftp-client-lowlevel v0.5.1
   Compiling openssh-sftp-client v0.14.0
error[E0277]: the trait bound `openssh_sftp_client_lowlevel::Error: From<openssh::Error>` is not satisfied
  --> /home/xuanwo/.cargo/registry/src/index.crates.io-6f17d22bba15001f/openssh-sftp-client-0.14.0/src/sftp/openssh_session.rs:65:30
   |
65 |         Err(err) => Some(err.into()),
   |                              ^^^^ the trait `From<openssh::Error>` is not implemented for `openssh_sftp_client_lowlevel::Error`
   |
   = help: the following other types implement trait `From<T>`:
             <openssh_sftp_client_lowlevel::Error as From<openssh_sftp_error::AwaitableError>>
             <openssh_sftp_client_lowlevel::Error as From<SshFormatError>>
             <openssh_sftp_client_lowlevel::Error as From<openssh::error::Error>>
             <openssh_sftp_client_lowlevel::Error as From<std::io::Error>>
             <openssh_sftp_client_lowlevel::Error as From<JoinError>>
             <openssh_sftp_client_lowlevel::Error as From<TryFromIntError>>
   = note: required for `openssh::Error` to implement `Into<openssh_sftp_client_lowlevel::Error>`

error[E0277]: the trait bound `openssh_sftp_client_lowlevel::Error: From<openssh::Error>` is not satisfied
  --> /home/xuanwo/.cargo/registry/src/index.crates.io-6f17d22bba15001f/openssh-sftp-client-0.14.0/src/sftp/openssh_session.rs:76:26
   |
76 |     let occuring_error = session.close().await.err().map(Error::from);
   |                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `From<openssh::Error>` is not implemented for `openssh_sftp_client_lowlevel::Error`
   |
   = help: the following other types implement trait `From<T>`:
             <openssh_sftp_client_lowlevel::Error as From<openssh_sftp_error::AwaitableError>>
             <openssh_sftp_client_lowlevel::Error as From<SshFormatError>>
             <openssh_sftp_client_lowlevel::Error as From<openssh::error::Error>>
             <openssh_sftp_client_lowlevel::Error as From<std::io::Error>>
             <openssh_sftp_client_lowlevel::Error as From<JoinError>>
             <openssh_sftp_client_lowlevel::Error as From<TryFromIntError>>

error[E0277]: the trait bound `openssh_sftp_client_lowlevel::Error: From<openssh::Error>` is not satisfied
  --> /home/xuanwo/.cargo/registry/src/index.crates.io-6f17d22bba15001f/openssh-sftp-client-0.14.0/src/sftp/openssh_session.rs:76:58
   |
76 |     let occuring_error = session.close().await.err().map(Error::from);
   |                                                          ^^^^^^^^^^^ the trait `From<openssh::Error>` is not implemented for `openssh_sftp_client_lowlevel::Error`
   |
   = help: the following other types implement trait `From<T>`:
             <openssh_sftp_client_lowlevel::Error as From<openssh_sftp_error::AwaitableError>>
             <openssh_sftp_client_lowlevel::Error as From<SshFormatError>>
             <openssh_sftp_client_lowlevel::Error as From<openssh::error::Error>>
             <openssh_sftp_client_lowlevel::Error as From<std::io::Error>>
             <openssh_sftp_client_lowlevel::Error as From<JoinError>>
             <openssh_sftp_client_lowlevel::Error as From<TryFromIntError>>

error: cannot check whether the hidden type of opaque type satisfies auto traits
   --> /home/xuanwo/.cargo/registry/src/index.crates.io-6f17d22bba15001f/openssh-sftp-client-0.14.0/src/sftp/openssh_session.rs:105:35
    |
105 |         let handle = tokio::spawn(create_session_task(session, tx));
    |                      ------------ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |                      |
    |                      required by a bound introduced by this call
    |
note: opaque type is declared here
   --> /home/xuanwo/.cargo/registry/src/index.crates.io-6f17d22bba15001f/openssh-sftp-client-0.14.0/src/sftp/openssh_session.rs:25:6
    |
25  | ) -> Option<Error> {
    |      ^^^^^^^^^^^^^
note: this item depends on auto traits of the hidden type, but may also be registering the hidden type. This is not supported right now. You can try moving the opaque type and the item that actually registers a hidden type into a new submodule
   --> /home/xuanwo/.cargo/registry/src/index.crates.io-6f17d22bba15001f/openssh-sftp-client-0.14.0/src/sftp/openssh_session.rs:99:18
    |
99  |     pub async fn from_session(
    |                  ^^^^^^^^^^^^
note: required by a bound in `tokio::spawn`
   --> /home/xuanwo/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.33.0/src/task/spawn.rs:166:21
    |
164 |     pub fn spawn<F>(future: F) -> JoinHandle<F::Output>
    |            ----- required by a bound in this function
165 |     where
166 |         F: Future + Send + 'static,
    |                     ^^^^ required by this bound in `spawn`

error[E0277]: `?` couldn't convert the error to `openssh_sftp_client_lowlevel::Error`
   --> /home/xuanwo/.cargo/registry/src/index.crates.io-6f17d22bba15001f/openssh-sftp-client-0.14.0/src/sftp/openssh_session.rs:110:27
    |
110 |             Ok(res) => res?,
    |                           ^ the trait `From<openssh::Error>` is not implemented for `openssh_sftp_client_lowlevel::Error`
    |
    = note: the question mark operation (`?`) implicitly performs a conversion on the error value using the `From` trait
    = help: the following other types implement trait `From<T>`:
              <openssh_sftp_client_lowlevel::Error as From<openssh_sftp_error::AwaitableError>>
              <openssh_sftp_client_lowlevel::Error as From<SshFormatError>>
              <openssh_sftp_client_lowlevel::Error as From<openssh::error::Error>>
              <openssh_sftp_client_lowlevel::Error as From<std::io::Error>>
              <openssh_sftp_client_lowlevel::Error as From<JoinError>>
              <openssh_sftp_client_lowlevel::Error as From<TryFromIntError>>
    = note: required for `Result<Sftp, openssh_sftp_client_lowlevel::Error>` to implement `FromResidual<Result<Infallible, openssh::Error>>`

For more information about this error, try `rustc --explain E0277`.
error: could not compile `openssh-sftp-client` (lib) due to 5 previous errors
cargo build  22.54s user 3.50s system 251% cpu 10.364 total

Support extension copy-data

The latest openssh-9.0 has released, adding support for the "copy-data" extension to allow server-side copying of files/data, following the design in section 7 in draft-ietf-secsh-filexfer-extensions-00.

TODO:

Update submodule openssh-portable to tag V_9_0_P1.
Support extension copy-data.

FR: Allow create a metadata::Permissions struct using octets

e.g.

let permissions = Permissions::from(0o755u16);

would be helpful

`check.sh` would fail on Mac systems

check.sh, which depends on rsync, will not work on Mac since rsync on mac has version 2.6.9 which was built over 20 years ago.

Refactor `WriteEnd` internals

Extracting the Bytes buffer out and refactor it to looks like a mpsc, that might make the internals look easier.

Is it possible to remove lifetime from `File`?

Hi, is it possbile to remove the lifetime from File so that users don't need to hack them?

Current

To allow using File like std::fs::File/tokio::fs::File, we have to do something like the following:

https://github.com/apache/incubator-opendal/blob/main/core/src/services/sftp/utils.rs#L39-L66

pub struct SftpReader {
    // similar situation to connection struct
    // We can make sure the file can live as long as the connection.
    file: OwningHandle<
        Box<PooledConnection<'static, Manager>>,
        Box<FdReader<Compat<TokioCompatFile<'static>>>>,
    >,
}

impl SftpReader {
    pub async fn new(
        conn: PooledConnection<'static, Manager>,
        path: PathBuf,
        start: u64,
        end: u64,
    ) -> Result<Self> {
        let mut file = OwningHandle::new_with_fn(Box::new(conn), |conn| unsafe {
            let file = block_on((*conn).sftp.open(path)).unwrap();
            let f = Compat::new(TokioCompatFile::from(file));
            Box::new(oio::into_reader::from_fd(f, start, end))
        });

        file.seek(SeekFrom::Start(0)).await?;

        Ok(SftpReader { file })
    }
}

Expected

pub struct SftpReader {
    file: sftp::File,
}

impl SftpReader {
    pub async fn new(
        conn: PooledConnection<'static, Manager>,
        path: PathBuf,
        start: u64,
        end: u64,
    ) -> Result<Self> {
        let mut file = conn.sftp.open(path).await?;

        file.seek(SeekFrom::Start(0)).await?;

        Ok(SftpReader { file })
    }
}

Invalid response id after closing file in blocking context

With #98, the problem fixed by #77 will exist. It occurs in action https://github.com/apache/incubator-opendal/actions/runs/5798363627/job/15715936917?pr=2815.

Maybe we need to find a better solution.

Examples?

I looked on tests but it seems to require binaries like openssh-server or openssh-client? Is it possible to use this in windows?

My use case is uploading files from windows machine to Linux in Rust Program. Especially I had concerns if the following will even work in windows?

pub async fn launch_sftp() -> (process::Child, PipeWrite, PipeRead) {
    let mut child = process::Command::new(get_sftp_path())
        .args(&["-e", "-l", "DEBUG"])
        .stdin(Stdio::piped())
        .stdout(Stdio::piped())
        .stderr(Stdio::inherit())
        .kill_on_drop(true)
        .spawn()
        .unwrap();

    let stdin = child_stdin_to_pipewrite(child.stdin.take().unwrap()).unwrap();
    let stdout = child_stdout_to_pipewrite(child.stdout.take().unwrap()).unwrap();

    (child, stdin, stdout)
}

pub fn get_sftp_path() -> &'static path::Path {
    static SFTP_PATH: OnceCell<path::PathBuf> = OnceCell::new();

    SFTP_PATH.get_or_init(|| {
        let mut sftp_path: path::PathBuf = env::var("OUT_DIR").unwrap().into();
        sftp_path.push("openssh-portable");
        sftp_path.push("sftp-server");

        eprintln!("sftp_path = {:#?}", sftp_path);

        sftp_path
    })
}

Where does openssh-portable in sftp_path.push("openssh-portable") comes? Can we link it statically like sqlite3 etc.?

Missing Sftp::from_session

Hey there,

I wanted to follow the example from examples folder but I get:

no function or associated item named `from_session` found for struct `Sftp` in the current scope
function or associated item not found in `Sftp`

When using

openssh = "0.10.1"
openssh-sftp-client = "0.14.1"

exaxmples/ folder demonstrating how to use this crate.

There are tests here but reading them is just leaving me more confused: https://github.com/openssh-rust/openssh-sftp-client/blob/main/tests/highlevel.rs
It seems like we are spawning a server but I thought this was a client library?

I think a simple example demonstrating how to use this library in the most common use case would go a long way.

Examples of how to use it?

Hi, nice to see this library.

It looks like it is meant to be used over the top of an SSH transport implementation that does authentication and encryption? Could you give some example code of how to connect it to OpenSSH or whatever transport is supported?

Possible bug in lifetime checker of rustc 1.62

I found that rustc rejects when I added this commit that changes nothing related to the error.

The code where error resides worked just fine without this commit or on 1.61 or earlier.

@jonhoo Could this be a bug in the rustc?

P.S. It failed on my MacOS M1 but succeeded in the CI... Really strange.
I also experienced clippy reports the lifetime error but cargo does not even without this commit.

I really have no idea what is going on.

FR: Add `path()` method to `DirEntry`

I noticed that openssh_sftp_client::fs::DirEntry is missing a path() method which is provided in the std-equivalent.
Is there a technical reason this isn't implemented?

chore: Cleanup the integration tests for highlevel API

The tests for highlevel API currently relies on run_tests.sh to work.

To fix this, we just need to use tempfile to create temporary dir.

How can I upload big File

Is there a way instead of loading up a huge file in RAM to upload, how can I split the file in parts to upload?

How to check if the sftp session is disconnected

Hi,

Is there a way to check if an sftp session got disconnected? I am planning to maintain long running sftp connections to a remote server and would ideally like to reconnect in case of intermittent disconnections

openssh-sftp-client = { version = "0.13.6", features = ["openssh"]

the underlying ssh session has a .check() method. but is there a way to access it from the sftp session?

Thanks

Misleading naming of `file::write` and `file::write_all`

Hi there,
I've stumbled upon a bug in our code where we used file::write instead of file::write_all and our file got truncated without us noticing because we didn't read the result success case but only the error.

Is there any way you would consider a naming change as the documentation claims to be closely modelled to the standard library? The standard library uses std::fs::write to write all the contents of a slice.

This might be confusing to other users as well. What do you and other people think?

chore: prepare for v0.11.0 release

With #9 and #11 done, now we are almost ready for v0.11.0 release.

Since this will be a new major release, I would like to clean up the API and remove anything that will be confusing.

@jonhoo Can you help me with this, by looking at the public APIs exposed by openssh-sftp-client and see if there is anything you would like to remove or add?

`openssh-sftp-error` needs to bump major version

openssh-sftp-error has a break change about bumping openssh version in 0.3.2, but it only updates a minor version from 0.3.1.

This will result in a compilation failure, which occurs in https://github.com/apache/incubator-opendal/actions/runs/6885228025/job/18729057412

Incorrect behavior about `AsyncSeek` of `TokioCompatFile`

Sometimes AsyncSeek to TokioCompatFile will set the offset to the wrong place. This error occurs in apache/opendal#2263 https://github.com/apache/incubator-opendal/actions/runs/4976970726/jobs/8905529556

I guess the problem is about

openssh-sftp-client/src/file/tokio_compat_file.rs

Line 354 in ec57895

self.consume(offset);

openssh-sftp-client/src/file/tokio_compat_file.rs

Lines 383 to 391 in ec57895

 fn consume(self: Pin<&mut Self>, amt: usize) { 

 let this = self.project(); 

 let buffer = this.buffer; 

 let amt = min(buffer.len(), amt); 

 buffer.advance(amt); 

 this.inner.offset += amt as u64; 

 }

When the offset amount is larger than buffer.len(), it will only offset to the end of buffer.

I haven't found the code which can reproduce stably. I will try to fix it these days.

`read_dir` miss some files in multiple file directories

I use fs.open_dir("$path").await.unwrap().read_dir().await.unwarp() to list all files in a directory. It will always miss some files in multiple file directories. I tries reconnecting to server, rebooting the servers, rewriting the files, using other network environment. It will trigger this issue steadily.

Reproduction code

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    let url = "SECRET";

    let session = SessionBuilder::default()
        .user("ymz".to_owned())
        .keyfile("SECRET")
        .connect(url)
        .await?;

    let mut child = session
        .subsystem("sftp")
        .stdin(Stdio::piped())
        .stdout(Stdio::piped())
        .spawn()
        .await?;

    let sftp = Sftp::new(
        child.stdin().take().unwrap(),
        child.stdout().take().unwrap(),
        Default::default(),
    )
    .await?;

    let mut fs = sftp.fs();
    fs.set_cwd("/home/ymz/sftp_test/");
    let dir = fs.open_dir("./test_list_rich_dir/").await.unwrap().read_dir().await?;
    let mut actual = vec![];
    for entry in dir {
        actual.push(entry.filename().to_owned());
    }
    actual.sort_unstable();
    println!("{:?}", actual);

    Ok(())
}

shell output

[".", "..", "file-0", "file-1", "file-10", "file-100", "file-11", "file-12", "file-13", "file-14",
 "file-15", "file-16", "file-17", "file-18", "file-19", "file-2", "file-20", "file-21", "file-22",
 "file-23", "file-24", "file-25", "file-26", "file-27", "file-28", "file-29", "file-3", "file-30", 
 "file-31", "file-32", "file-33", "file-34", "file-35", "file-36", "file-37", "file-38", "file-39",
 "file-4", "file-40", "file-41", "file-42", "file-43", "file-44", "file-45", "file-46", "file-47",
 "file-48", "file-49", "file-50", "file-51", "file-52", "file-53", "file-54", "file-55", "file-56", 
 "file-57", "file-58", "file-59", "file-6", "file-60", "file-61", "file-62", "file-63", "file-64", 
 "file-65", "file-66", "file-67", "file-68", "file-69", "file-7", "file-70", "file-72", "file-73", 
 "file-74", "file-75", "file-76", "file-77", "file-78", "file-79", "file-8", "file-80", "file-82", 
 "file-83", "file-84", "file-85", "file-86", "file-87", "file-88", "file-89", "file-9", "file-90", 
 "file-91", "file-92", "file-93", "file-94", "file-95", "file-96", "file-97", "file-98", "file-99"]

ssh terminal screenshot

We can see that it misses file-5, file-71, file-81. In multiple tests conducted, these three files were always lost.

According to my tests, as long as the number of files in the directory is greater than 100, this error will be generated. If there are only 90 files, it will run correctly.

unix only ?

i have this error on window at compilation time : error[E0433]: failed to resolve: could not find unix in os

TokioCompatFile has an infinite write buffer size, which is unexpected

From what I can tell, TokioCompatFile will sneakily buffer up an unlimited number of bytes into memory and doesn't apply any backpressure when writing.

The poll_write implementation will spawn new write requests and from what I can see always returns Poll::Ready — never Poll::Pending.

Other than leading to an unexpectedly high write throughput to start with, then an unexpectedly slow final flush(), this also results in very high / unbounded memory usage. :)

The best workaround for library users I can find today is to flush() every so often, but this worsens performance because it waits for the whole write request queue to be drained, when it is actually quite healthy to have some write requests in-flight.

I think a better approach would be for poll_write to return Poll::Pending if there are too many write requests in the queue (counted either by number of requests or number of bytes), then to wake the context once that number decreases beneath a threshold.

Move upstream dependency into this org

This crate has two dependencies that worth putting into this org:

awaitable an awaitable type to convey input to the ReadEnd and then get the response from it.
concurrent_arena concurrent arena that uses a u32 as key. P.S. ~~I'm now working on avoiding this dependency on openssh_sftp_client_lowlevel, but openssh_sftp_client would still depend on it. It requires GAT to parameterize it, so it will stay as-is for now.~~ It took too much effort and doesn't have much benefit, so I gave it up.

These two crates are owned by me and created specifically for openssh-sftp-client and there is no other dependent of it right now.

I'd like to move these two repos into this org and I'd like your review on concurrent_arena since it contains some unsafe code and invovles concurrency @jonhoo ,

Something in `Sftp::new` does not implement `Send`

Description

I am trying to use Sftp in an app that I am developing with tauri. tauri is an Electron alternative which haș the concept of commands, which allow the frontend to invoke some functionality from the backend. My use case of this crate involves uploading some file to a remote server upon some user's invocation. However, tauri::commands must implement Send. I am getting an error that state's that Send is not implemented for the Id newtype in openssh-sftp-client.

I was wondering if you could help me figure out a) where exactly this error was coming from, b) if this was fixable, and c) what we/I can do or contribute to fix it.

Full error message from `rustc`/`tauri`

error: future cannot be sent between threads safely
   --> src/ssh/commands.rs:55:1
    |
55  |   #[tauri::command]
    |   ^^^^^^^^^^^^^^^^^
    |   |
    |   future returned by `send_backup_command` is not `Send`
    |   in this expansion of `ssh::commands::__cmd__send_backup_command!` (#2)
    |
   ::: /Users/msamdars/.cargo/registry/src/github.com-1ecc6299db9ec823/tauri-macros-1.1.1/src/lib.rs:49:1
    |
49  |   pub fn generate_handler(item: TokenStream) -> TokenStream {
    |   --------------------------------------------------------- in this expansion of `tauri::generate_handler!` (#1)
    |
   ::: src/main.rs:491:25
    |
491 |           .invoke_handler(tauri::generate_handler![
    |  _________________________-
492 | |             persist_state_to_disk,
493 | |             load_state_from_disk,
494 | |             load_privacy_policy_accepted,
...   |
546 | |             upload::gcp::upload_file_gcs,
547 | |         ])
    | |         -
    | |         |
    | |_________in this macro invocation (#1)
    |           in this macro invocation (#2)
    |
    = help: the trait `std::marker::Send` is not implemented for `dyn futures::Future<Output = std::result::Result<(openssh_sftp_client_lowlevel::awaitable_responses::Id<bytes::bytes_mut::BytesMut>, openssh_sftp_protocol::response::Limits), openssh_sftp_error::Error>>`
note: required by a bound in `tauri::command::private::ResultFutureTag::future`
   --> /Users/msamdars/.cargo/registry/src/github.com-1ecc6299db9ec823/tauri-1.1.1/src/command.rs:289:42
    |
289 |       F: Future<Output = Result<T, E>> + Send,
    |                                          ^^^^ required by this bound in `tauri::command::private::ResultFutureTag::future`

Reproduction

Make sure you have the necessary prerequisites
Run yarn create tauri-app and choose your preferred front-end stack (since this error occurs in the backend, any frontend will do)
Replace your src-tauri/src/main.rs file with the following:

#![cfg_attr(
    all(not(debug_assertions), target_os = "windows"),
    windows_subsystem = "windows"
)]

use openssh::Stdio;
use openssh_sftp_client::{Sftp, SftpOptions};

const PRIVATE_KEY_PATH: &str = "";
const USERNAME: &str = "";
const IP: &str = "";
const PORT: u16 = 22;

async fn sftp_open() -> anyhow::Result<Sftp> {
    let sess = openssh::SessionBuilder::default()
    .user(USERNAME.to_string())
        .port(PORT)
        .keyfile(PRIVATE_KEY_PATH)
        .control_directory(std::env::temp_dir())
        .connect(IP)
        .await?;

    // Request a SFTP subsystem, and wait until it completes.
    let mut child = sess
        .subsystem("sftp")
        .stdin(Stdio::piped())
        .stdout(Stdio::piped())
        .spawn()
        .await?;

    let sftp = Sftp::new(
        child.stdin().take().unwrap(),
        child.stdout().take().unwrap(),
        SftpOptions::default(),
    )
    .await?;

    Ok(sftp)
}

#[tauri::command]
async fn upload_new_file() -> Result<(), String> {
    let sftp = sftp_open().await.map_err(|e| e.to_string())?;
    let mut file = sftp.create("test.txt").await.map_err(|e| e.to_string())?;

    file.write_all(b"hello world").await.map_err(|e| e.to_string())?;
    file.close().await.map_err(|e| e.to_string())?;
    sftp.close().await.map_err(|e| e.to_string())?;
    Ok(())
}

fn main() {
    tauri::Builder::default()
    .invoke_handler(tauri::generate_handler![upload_new_file])
        .run(tauri::generate_context!())
        .expect("error while running tauri application");
}

Run yarn build, and then yarn tauri dev to see the error

TODO: split the lowlevel and highlevel as separate crate

It seems that the API of lowlevel and highlevel aren't that tightly coupled.

Changes to the lowlevel can often be wrapped up by the highlevel, thus not posing a breaking change in the highlevel API, yet with the current model, users of the highlevel API are also forced to bump their major version number.

Splitting them into two different crates would make it possible to release these crates separately at different pace, though it would make the maintenance of it a bit harder.

If we are going with this new release model, shall we use the workspace features so that all scripts and ci setups can be shared between them, or shall we create a separate repository for the lowlevel part?

Performance when downloading files

I'm currently trying to use this crate in a small project for the purpose of downloading a file via sftp.
I don't know if I'm missing something, but with my naive implementation, downloading the file is several times slower than with the standard sftp client. What I'm doing is this:

use bytes::{BytesMut};
use openssh::{KnownHosts, Session, Stdio};
use openssh_sftp_client::Sftp;
use std::error::Error;
use std::fs::File;
use std::io::Write;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    let session = Session::connect_mux(
        "ssh://user@host",
        KnownHosts::Accept,
    )
    .await?;
    let mut child = session
        .subsystem("sftp")
        .stdin(Stdio::piped())
        .stdout(Stdio::piped())
        .spawn()
        .await?;

    let sftp_client = Sftp::new(
        child.stdin().take().unwrap(),
        child.stdout().take().unwrap(),
        Default::default(),
    )
    .await?;

    let mut zip_file = sftp_client
        .options()
        .read(true)
        .open("myfile")
        .await?;

    let mut tmp_file = File::create("myfile").unwrap();
    let buf_size = 100 * 1024 * 1024;

    while let Ok(read_result) = zip_file
        .read(
            buf_size,
            BytesMut::with_capacity(buf_size.try_into().unwrap()),
        )
        .await
    {
        if let Some(data) = read_result {
            tmp_file.write_all(&data)?;
        } else {
            break;
        }
    }

    Ok(())
}

Any hints are appreciated.

Create remote sftp subsystem error

The key logs are as follows

...
  2023-08-01T14:55:52.537  INFO openssh_sftp_client::sftp::openssh_session: Connecting to sftp subsystem, session = Session(ProcessImpl(Session { tempdir: Some(TempDir { path: "/tmp/.ssh-connectionHl54xJ" }), ctl: "/tmp/.ssh-connectionHl54xJ/master", master_log: Some("/tmp/.ssh-connectionHl54xJ/log") }))
    at /usr/local/cargo/git/checkouts/openssh-sftp-client/8aadf10/src/sftp/openssh_session.rs:27
    in openssh_sftp_client::sftp::openssh_session::session_task with session: Session(ProcessImpl(Session { tempdir: Some(TempDir { path: "/tmp/.ssh-connectionHl54xJ" }), ctl: "/tmp/.ssh-connectionHl54xJ/master", master_log: Some("/tmp/.ssh-connectionHl54xJ/log") }))

  2023-08-01T14:55:52.537  INFO openssh_sftp_client::sftp::openssh_session: Connection to sftp subsystem established, session = Session(ProcessImpl(Session { tempdir: Some(TempDir { path: "/tmp/.ssh-connectionHl54xJ" }), ctl: "/tmp/.ssh-connectionHl54xJ/master", master_log: Some("/tmp/.ssh-connectionHl54xJ/log") }))
    at /usr/local/cargo/git/checkouts/openssh-sftp-client/8aadf10/src/sftp/openssh_session.rs:51
    in openssh_sftp_client::sftp::openssh_session::session_task with session: Session(ProcessImpl(Session { tempdir: Some(TempDir { path: "/tmp/.ssh-connectionHl54xJ" }), ctl: "/tmp/.ssh-connectionHl54xJ/master", master_log: Some("/tmp/.ssh-connectionHl54xJ/log") }))

  2023-08-01T14:55:56.545  INFO openssh_sftp_client::tasks: Requesting graceful shutdown of flush_task from read_task, shared_data = 0xaaae209a7320
    at /usr/local/cargo/git/checkouts/openssh-sftp-client/8aadf10/src/tasks.rs:185
    in openssh_sftp_client::tasks::read_task with read_end_buffer_size: 1024

  2023-08-01T14:55:56.545 ERROR openssh_sftp_client::tasks: error: IO Error (Excluding `io::ErrorKind::WouldBlock`): unexpected end of file.
    at /usr/local/cargo/git/checkouts/openssh-sftp-client/8aadf10/src/tasks.rs:165
    in openssh_sftp_client::tasks::read_task with read_end_buffer_size: 1024

  2023-08-01T14:55:56.545 ERROR openssh_sftp_client::sftp::openssh_session: Waiting on remote sftp subsystem to exit failed: Failed to create sftp from session: the remote process has terminated, session = Session(ProcessImpl(Session { tempdir: Some(TempDir { path: "/tmp/.ssh-connectionHl54xJ" }), ctl: "/tmp/.ssh-connectionHl54xJ/master", master_log: Some("/tmp/.ssh-connectionHl54xJ/log") }))
    at /usr/local/cargo/git/checkouts/openssh-sftp-client/8aadf10/src/sftp/openssh_session.rs:70
    in openssh_sftp_client::sftp::openssh_session::session_task with session: Session(ProcessImpl(Session { tempdir: Some(TempDir { path: "/tmp/.ssh-connectionHl54xJ" }), ctl: "/tmp/.ssh-connectionHl54xJ/master", master_log: Some("/tmp/.ssh-connectionHl54xJ/log") }))

  2023-08-01T14:55:56.551 ERROR openssh_sftp_client::sftp::openssh_session: Closing session failed: Failed to create sftp from session: the local ssh command could not be executed, session = Session(ProcessImpl(Session { tempdir: Some(TempDir { path: "/tmp/.ssh-connectionHl54xJ" }), ctl: "/tmp/.ssh-connectionHl54xJ/master", master_log: Some("/tmp/.ssh-connectionHl54xJ/log") }))
    at /usr/local/cargo/git/checkouts/openssh-sftp-client/8aadf10/src/sftp/openssh_session.rs:80
    in openssh_sftp_client::sftp::openssh_session::session_task with session: Session(ProcessImpl(Session { tempdir: Some(TempDir { path: "/tmp/.ssh-connectionHl54xJ" }), ctl: "/tmp/.ssh-connectionHl54xJ/master", master_log: Some("/tmp/.ssh-connectionHl54xJ/log") }))
...

Can we implement `AsyncWrite` for `openssh_sftp_client::File`?

Hi, File provides most it's API through async fn like:

impl File
    pub async fn write(&mut self, buf: &[u8]) -> Result<usize, Error> {
        ...
    }
}

This make it's hard to implement AsyncWrite upon it: users will need to build their own state machine to make implement poll_write correctly.

Is it possible to add native AsyncWrite and AsyncRead support? (we already have AsyncSeek now).

	fn consume(self: Pin<&mut Self>, amt: usize) {
	let this = self.project();

	let buffer = this.buffer;
	let amt = min(buffer.len(), amt);

	buffer.advance(amt);
	this.inner.offset += amt as u64;
	}