SSE options for AWS s3 about funnel HOT 10 CLOSED

I haven't had time to look into this yet, I'll try to get to it early next week.

from funnel.

Hi,
is there any update on this?
Thanks again for your help

from funnel.

Sorry, I've been stuck on other projects. I am working on this now. I hope to have something for you to try out by EOD tomorrow.

from funnel.

I've prototyped out a solution in #623. Let me know if this works for you.

For SSE-KMS:

AmazonS3:
SSE:
    KMSKey: "1a03ce70-5f03-484e-8396-0e97de661b79"

For SSE-C:

Generate a key file:

openssl rand -out sse-c.key 32

Then configure funnel to use it:

AmazonS3:
  SSE:
    CustomerKeyFile: "./sse-c.key"

from funnel.

Note for SSE-KMS:

As long as your credentials can access the KMS key used for the given bucket, everything seems to work with no special configuration.

from funnel.

Great thanks, I will give it a try tomorrow.
Regarding the sse-c I assume the file ./sse-c.key is expected to be available to all workers, correct? how about the server? Are there any special permissions required for this file?
Thanks again for your help

from funnel.

The server doesn't require any storage configuration in this case. And yes the sse-c.key file is assumed to be accessible by all of the workers.

Upon further testing, I found that in the sse-c case my solution assumes that all files tasks will reference were encrypted using that key. So, for example, if you were to reference a file in an unencrypted bucket the task would fail upon trying to download it.

from funnel.

Thanks for the clarifications.
I will work with the assumption that all files are encrypted.
Is there any way and are you planning to change this behavior and allow encrypted (with sse-c) or unencrypted files to be used at the same time?
Thanks

from funnel.

Yes, I plan to add support for using sse-c encrypted and unencrypted files at the same time.

from funnel.

Yes, I plan to add support for using sse-c encrypted and unencrypted files at the same time.

Done.

from funnel.