Giter Club home page Giter Club logo

minio-rs's Introduction

MinIO Rust SDK for Amazon S3 Compatible Cloud Storage Slack Sourcegraph Apache V2 License

MinIO Rust SDK is Simple Storage Service (aka S3) client to perform bucket and object operations to any Amazon S3 compatible object storage service.

For a complete list of APIs and examples, please take a look at the MinIO Rust Client API Reference

Example:: file-uploader.rs

Upload a file to MinIO

License

This SDK is distributed under the Apache License, Version 2.0, see LICENSE for more information.

minio-rs's People

Contributors

anjalshireesh avatar balamurugana avatar donatello avatar dvaldivia avatar grumbach avatar harshavardhana avatar hjlebbink avatar krisis avatar kyldvs avatar olalalalao avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

minio-rs's Issues

Please drop dependency on dead chrono-locale

chrono-locale crate is dead and does not compile with the latest chrono. It is used only in util.rs - to_http_header_value().
I suggest dropping the dependency and replacing

time.formatl("%a, %d %b %Y %H:%M:%S GMT", "C").to_string()

with

time.format("%a, %d %b %Y %H:%M:%S GMT").to_string()

Generate Pre-Signed URLs to Objects

Hi all,

First of all thanks for your very useful bindings.
I wonder if there is any plan to support the generation of Pre-Signed URLs, like the ones that can be generated by mc share.

Maybe it is already supported, but I was unable to find it in the docs.

Thank you.

dyn minio::s3::sse::Sse needs to implement Send trait for thread safety

Hello,

I'm currently using your library in a multithreaded context and I've encountered an issue related to Rust's concurrency safety guarantees.

The error message I'm getting is:

dyn minio::s3::sse::Sse` cannot be shared between threads safely
the trait `Sync` is not implemented for `dyn minio::s3::sse::Sse`
required for `&dyn minio::s3::sse::Sse` to implement `Send`

This suggests that the dyn minio::s3::sse::Sse type does not implement the Sync trait, which means it can't be shared between threads safely.

I believe the solution to this issue would be to ensure that dyn minio::s3::sse::Sse implements the Send trait, allowing it to be safely sent between threads.

(or maybe it's not thread safe with no workaround?)

create_multipart_upload,上传包含中文的文件名报错

S3Error(ErrorResponse { code: "SignatureDoesNotMatch", message: "The request signature we calculated does not match the signature you provided. Check your key and signing method.", resource: "/wccloud/1716536982708_屏幕截102754.png", request_id: "17D25D4A8A2345AA", host_id: "dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8", bucket_nam"", object_name: "1716536982708_屏幕截102754.png" }

raise "SignatureDoesNotMatch" error if object_name contains Chinese charactor

I tried to use "use minio::s3::utils::urlencode" to encode Chinese charactors, but this does not help;

s3 operation failed; code: SignatureDoesNotMatch, message: The request signature we calculated does not match the signature you provided. Check your key and signing method., resource: /historical/ElementaryFactor/复权因子, request_id: 17ACEB23C21FD868, host_id: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8, bucket_name: , object_name: ElementaryFactor/复权因子

Release roadmap

Feel free to modify/suggest modifications for the release. Strictly bounded with #8

Will it be v2.0.0 or v1.1.0?

  • repository cleanup #31
  • quotas (admin api)

Use GIT

If you are having problems with Send + Sync, crates.io version 0.1.0 does not implement them, yet.

So import using:

minio = {git="https://github.com/minio/minio-rs.git"}

bucket_exists only works if the user is the owner

It’s nice that the API offers a simple method for checking if a bucket exists. However, the implementation of client.bucket_exists() isn’t great. I have an IAM user that has access to AWS S3 and I can list buckets just fine but when calling client.bucket_exists() I get access denied. The reason for this is that under the hood client.bucket_exists() calls self.get_region().

match self.get_region(args.bucket, args.region).await {

According to the AWS documentation, you have to provide the ID for the bucket owner in order to be allowed to perform this operation. If the provided ID doesn’t match the bucket owner, you get access denied.
https://stackoverflow.com/questions/55195343/aws-s3-access-denied-when-getting-bucket-location
https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketLocation.html#API_GetBucketLocation_RequestSyntax
This makes getting the region an impractical way of checking if a bucket exists.

Another reason why calling self.get_region() isn’t a great way of checking if a bucket exists is that the client caches this information locally (which makes sense for other use cases). This means that if the bucket gets deleted while the client object is still alive, client.bucket_exists() will still return true. Of course this will be very rare in practice, you don’t delete buckets that often, but it’s still not the best design.

The object name contains a Chinese message indicating a signature error

The object_name and path contain Chinese characters, so a signature error will be displayed if the object_name or path is uploaded or downloaded

2023-11-14T03:02:44Z INFO  minio_manager::minio_util] 上传文件名称:"test/绿色指针说明手册.pdf"
[2023-11-14T03:02:46Z INFO  minio_manager::minio_util] Err(S3Error(ErrorResponse { code: "SignatureDoesNotMatch", message: "The request signature we calculated does not match the sig
nature you provided. Check your key and signing method.", resource: "/test/test/绿色指针说明手册.pdf", request_id: "17975E351E27D69A", host_id: "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", bucket_name: "", object_name: "test/绿色指针说明手册.pdf" }))

time estimate for stable release

i saw that the rust sdk is not listed on the website, so i'm assuming it is not stable yet to use, can anyone give me an estimate of when it might be stable enough to use?

UploadObjectArgs does not implement send

pub async fn upload_object(bucket_name: String, object_name: PathBuf, filename: PathBuf) {
    let upload_object_args = UploadObjectArgs::new(&bucket_name, object_name.to_str().expect("路径不支持"), filename.to_str().expect("路径不支持")).unwrap();
    let _ = MINIO_CLIENT.upload_object(&upload_object_args).await;
}

I'm prompted when I submit my upload method to tokio,The upload method does this. Other methods, such as remove_object and list_objects, do not have this problem。Maybe I'm using it the wrong way. I'd like some Pointers

error: future cannot be sent between threads safely
   --> src\main.rs:160:45
    |
160 |                     return Command::perform(minio_util::upload_object(self.curr_bucket.clone(), object_name, file), |_| Message::Refresh);
    |                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ future returned by `upload_object` is not `Send`
    |
    = help: the trait `Sync` is not implemented for `dyn Sse`
note: future is not `Send` as this value is used across an await
   --> src\minio_util.rs:92:61
    |
90  |     let upload_object_args = UploadObjectArgs::new(&bucket_name, object_name.to_str().expect("路径不支持"), filename.to_str().expect("路径不     ...
    |         ------------------ has type `UploadObjectArgs<'_>` which is not `Send`
91  |     // let upload_object_args = UploadObjectArgs::new(&bucket_name, object_name.to_str().expect(""), &filename).unwrap();
92  |     let _ = MINIO_CLIENT.upload_object(&upload_object_args).await;
    |                                                             ^^^^^ await occurs here, with `upload_object_args` maybe used later
...
95  | }
    | - `upload_object_args` is later dropped here
note: required by a bound in `iced::Command::<T>::perform`
   --> C:\Users\17274\.cargo\git\checkouts\iced-3815be7e76d03e0b\8d77971\runtime\src\command.rs:43:53
    |
42  |     pub fn perform<A>(
    |            ------- required by a bound in this associated function
43  |         future: impl Future<Output = T> + 'static + MaybeSend,
    |                                                     ^^^^^^^^^ required by this bound in `Command::<T>::perform`


cannot get file by presigned-url

I was using presigned url to get file, but it raises The request signature we calculated does not match the signature you provided. Check your key and signing method. exception,

image

here is my rust code.

 #[tokio::test]
    async fn test4() -> anyhow::Result<()> {
        let mut b = BaseUrl::from_string("http://192.168.50.144:9000".to_string())?;
        b.https = false;
        b.region = "cn-shanghai".to_string();

        let static_provider =
            StaticProvider::new("Wqjy5OmuKrjKmJoKC6XX", "dd0DxXPgWhyRjCDhGlP7UnHolF7EJuxKvroroI7p", None);

        let client = Client::new(b, Some(&static_provider), None, None)?;

        let args = minio::s3::args::GetPresignedObjectUrlArgs::new(
            "xiaoshuyuilocaltest",
            "demo.pdf",
            http::Method::GET,
        )?;

        let r = client.get_presigned_object_url(&args).await?;
        println!("[rust]  ====> {:?}", r.url);

        anyhow::Ok(())
    }

But when i use python sdk to generate presigned url, I can download files successfully.

Here is my python code.

from minio import Minio
from minio.error import S3Error


def main():
    # Create a client with the MinIO server playground, its access key
    # and secret key.
    client = Minio(
        "192.168.50.144:9000",
        access_key="Wqjy5OmuKrjKmJoKC6XX",
        secure=False,
        secret_key="dd0DxXPgWhyRjCDhGlP7UnHolF7EJuxKvroroI7p",
        session_token=None
    )

    # Upload '/home/user/Photos/asiaphotos.zip' as object name
    # 'asiaphotos-2015.zip' to bucket 'asiatrip'.
    f = client.get_presigned_url(
        method="GET",
        bucket_name="xiaoshuyuilocaltest",
        object_name="demo.pdf",)
    print(f)


if __name__ == "__main__":
    try:
        main()
    except S3Error as exc:
        print("error occurred.", exc)

access_key, secret_key and endpoint are same

Upload Bytes/Vec<u8> rather than just files

Currently the only way, to my understanding, to upload an object to Minio is pointing to a file stored somewhere in the OS. What about uploading just an array of bytes? I have a special case where the objects to be uploaded to Minio are generated on the fly by my app therefore they don't exist on the disk. I could workaround this in my app, writing the files to the disk and only then upload them to Minio but since my files are very small it would be easier to avoid the workaround.

Any ideas about how to do it? If it is not implemented yet I would be happy to contribute (I would just need some hints about where to put the code)

Thanks in advance!

client
        .upload_object(
            &mut UploadObjectArgs::new(
                &bucket_name,
                "asiaphotos-2015.zip",
                "/home/user/Photos/asiaphotos.zip", // HOW CAN I POINT TO AN ARRAY OF BYTES?
            )
            .unwrap(),
        )
        .await
        .unwrap();

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.