Unable to read files about s3mock HOT 7 OPEN

@marad Can you provide more concrete & full code example for this failure? As I cannot reproduce this issue.

from s3mock.

@shuttie Ok, I've found the cause of my problems. In my code I need to create connection with:

val clientConfig = new ClientConfiguration()
clientConfig.setSignerOverride("S3SignerType")
val client = new AmazonS3Client(credentials, clientConfig)

And this signer override is causing the files to be empty. Is there a way to support other signing algorithms in s3mock?

from s3mock.

Sure, I'll create small example code and send it here.

from s3mock.

@shuttie The problem is, really that the files are stored empty in the temporary directory when putting objects. Setting signer type is one way to achieve this. I'm suspecting other ways but will investigate later.

from s3mock.

@marad, I had a similar problem using S3Mock for Presigned URLs. I originally used AnonymousAWSCredentials in my tests but the code path for presigned URLs requires an access key. I found your earlier comment and switched to BasicAWSCredentials and overrode the signer per your comment and repro'd the same "zero bytes" problem.

It turns out that "S3SignerType" is incorrect. Both "AWS3SignerType" and "NoOpSignerType" work for me. I also can verify that the files are no longer zero bytes in size.

from s3mock.

We experience the same issue, caused by using S3SignerType. I suggest to change the name of this issue accordingly, and I'd appreciate a fix.

from s3mock.

The code expects the PUT payload after a chunk signature (example below). According to https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html this seems to be specific to V4. With V2 the client only sends the raw data, causing s3mock to fail because of the missing chunk signature.

V4:

org.apache.http.wire - http-outgoing-0 >> "3;chunk-signature=0d1d8b31c94d5e4effc1acb4a97632ae10355cf018fb454eb5400faa5be61bec[\r][\n]"
org.apache.http.wire - http-outgoing-0 >> "FOO[\r][\n]"
org.apache.http.wire - http-outgoing-0 >> "0;chunk-signature=4b2cc53ba80357f42d34d77d596cc47552d586bfa71fb506c0ecf94d9854723e[\r][\n]"

V2 is just org.apache.http.wire - http-outgoing-0 >> "FOO".

The failing logic is in S3ChunkedProtocolStage. In PutObject the S3ChunkedProtocolStage seems to be hardcoded in completeSigned.

from s3mock.