sharperad / cake.aws.s3 Goto Github PK

I have my deployment vm set up with Instance Credentials using IAM roles, and access key/secret is not (shouldn't be) needed. However, it looks like this library has a hard requirement on the AccessKey existing in a UploadSettings object.

The expected behavrior would be that it falls back to the default AWS authentication methods if accesskey/secrets isn't explicitly set.

When using SyncUpload method to sync a local directory, no matter what KeyPrefix I specified. The local folder will always be put in the bucket root folder on S3.

The GetKey method in S3Manager.cs seems not using the keyPrefix, it just use the local folder path as the S3 key.

string key = this.GetKey(file, fullPath, settings.LowerPaths);

The NuGet Gallery will soon include a "Cake" tab with instructions on how to use NuGet packages in Cake build scripts. I'd like to suggest that you include the tag cake-addin in the NuGet package of this addin, so that the NuGet Gallery can display the correct instructions to install this addin.

The UI will look similar to this:

Documentation on best practices on tags for Cake addins: https://cakebuild.net/docs/extending/addins/best-practices#tags

The comments for uploading mention utilization of the multi-part upload capability, but no usage of the defined AWS SDK API (https://docs.aws.amazon.com/AmazonS3/latest/dev/HLuploadFileDotNet.html?shortFooter=true) is seen in the code that I can find.

Hi @SharpeRAD,

I am using Cake.AWS.S3 version 1.0.0 which references version 1.0.6 of MimeTypesMap package. But it throws a missing package error for the above version of MimeTypesMap.
I have checked in NuGet for MimeTypesMap but couldn't find 1.0.6 version in the version history for this package.

Do we know if it's an existing issue and if we have any fix for the same?

Thanks

We've recently released new icons for Cake extensions, to help more easily differentiate Cake Addins, Modules, Recipes, etc.

More information: https://github.com/cake-contrib/graphics

Thanks again for creating this Cake Addin, we really appreciate the effort that you have put in to creating it.

We, the Cake Team, recently announced a new Cake Contrib Icon, details of which can be found here:

http://cakebuild.net/blog/2017/06/new-cake-contrib-icon

Would you consider changing the nuspec file for your NuGet Package to use this new Cake Contrib Icon? If so, the recommended URL to use is:

https://cdn.rawgit.com/cake-contrib/graphics/a5cf0f881c390650144b2243ae551d5b9f836196/png/cake-contrib-medium.png

Details of the above URL can be found in the repository here:

https://github.com/cake-contrib/graphics

Please let me know if you have any questions.

Cake 2.0.0 has been released 🎉

We performed an automated audit of your Cake addin and found that it does not follow all the best practices.

We encourage you to make the following modifications:

• You are currently referencing Cake.Core 0.26.0. Please upgrade to 0.28.0
• The Cake.Core reference should be private. Specifically, your addin's .csproj should have a line similar to this: <PackageReference Include="Cake.Core" Version="0.28.0" PrivateAssets="All" />
• Your addin should target netstandard2.0. Please note that there is no need to multi-target, netstandard2.0 is sufficient.

Apologies if this is already being worked on, or if there are existing open issues, this issue was created based on what is currently published for this package on NuGet.org and in the project on github.

Add support for transfer acceleration, which would just involve adding a property to the UploadSettings and applying it when creating the S3 client.

https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration-examples.html#transfer-acceleration-examples-dotnet

In 0.6.6, along with Cake.AWS.S3.dll the following dependencies were included in the lib/ folder:

AWSSDK.S3.dll
AWSSDK.Core.dll
MimeTypesMap.dll

In 0.6.7 the only library present is Cake.AWS.S3.dll and cake script now immediately fails with the following errors:

Could not load [repo]\tools\Addins\Cake.AWS.S3.0.6.7\lib\net46\Cake.AWS.S3.dll (missing AWSSDK.S3, Version=3.3.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)
error CS1061: 'ICakeContext' does not contain a definition for 'CreateDownloadSettings' and no extension method 'CreateDownloadSettings' accepting a first argument of type 'ICakeContext' could be found (are you missing a using directive or an assembly reference?)
error CS0103: The name 'S3Download' does not exist in the current context

We've started using GitHub Discussions as the preferred communication channel moving forward (instead of Gitter) because it makes it easier to keep track of discussions in a structured way, especially if multiple discussions are happening at the same time. It also allows to search for previous questions/answers, which can be a helpful resource.

As such, we're recommending addin maintainers to update any links to Gitter to point to the GH discussions.

Template:

## Discussion

For questions and to discuss ideas & feature requests, use the [GitHub discussions on the Cake GitHub repository](https://github.com/cake-build/cake/discussions), under the [Extension Q&A](https://github.com/cake-build/cake/discussions/categories/extension-q-a) category.

[![Join in the discussion on the Cake repository](https://img.shields.io/badge/GitHub-Discussions-green?logo=github)](https://github.com/cake-build/cake/discussions)

I have a cake script that looks like this:

#addin nuget:?package=AWSSDK.Core&version=3.3.101.9
#addin nuget:?package=AWSSDK.S3&version=3.3.101.9
#addin "Cake.AWS.S3&version=0.6.8"

// ... and eventually
var uploadSettings = CreateUploadSettings();

As you can see, I am trying to call CreateUploadSettings.

Unfortunately, it fails with this message:

error CS0103: The name 'CreateUploadSettings' does not exist in the current context

Do you know what I am doing wrong?

My goal of using CreateUploadSettings is to use my aws credentials that are stored in my aws credentials file. Could you please also let me know if this will take these credentials?

I am trying to setup this so that I don't have to set the AccessKey and SecretKey in my UploadSettings. I would expect Cake.AWS.S3 to pick these credentials automatically just like the aws cli does.

Task("Download-File-Fallback")
    .Description("Download a file from S3 using AWS Fallback credentials")
    .Does(() =>
{
    S3Download("C:/Files/test.zip", "test.zip", Context.CreateDownloadSettings()
    {
        BucketName = "cake-s3"
    });
});

gets error CS1026: ) expected
pointing at the end of the line:
S3Download("C:/Files/test.zip", "test.zip", Context.CreateDownloadSettings()

If I code like this it works:

Task("Download-File-Fallback")
    .Description("Download a file from S3 using AWS Fallback credentials")
    .Does(() =>
{
    var settings = Context.CreateDownloadSettings(); 
    settings.BucketName = "cake-s3";
     S3Download("C:/Files/test.zip", "test.zip",   settings);
});

issuing a dotnet cake

Could not load D:\source\psnbooster\build\tools\Addins\Cake.AWS.S3.0.6.6\lib\netstandard2.0\Cake.AWS.S3.dll (missing AWSSDK.S3, Version=3.3.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)
D:/source/psnbooster/build/build.cake(212,9): error CS0103: The name 'S3Upload' does not exist in the current context
D:/source/psnbooster/build/build.cake(213,8): error CS0246: The type or namespace name 'UploadSettings' could not be found (are you missing a using directive or an assembly reference?)
D:/source/psnbooster/build/build.cake(218,19): error CS0103: The name 'S3CannedACL' does not exist in the current context
D:/source/psnbooster/build/build.cake(222,9): error CS0103: The name 'S3Upload' does not exist in the current context
D:/source/psnbooster/build/build.cake(223,8): error CS0246: The type or namespace name 'UploadSettings' could not be found (are you missing a using directive or an assembly reference?)
D:/source/psnbooster/build/build.cake(228,19): error CS0103: The name 'S3CannedACL' does not exist in the current context
Error: Error(s) occurred when compiling build script:
D:/source/psnbooster/build/build.cake(212,9): error CS0103: The name 'S3Upload' does not exist in the current context
D:/source/psnbooster/build/build.cake(213,8): error CS0246: The type or namespace name 'UploadSettings' could not be found (are you missing a using directive or an assembly reference?)
D:/source/psnbooster/build/build.cake(218,19): error CS0103: The name 'S3CannedACL' does not exist in the current context
D:/source/psnbooster/build/build.cake(222,9): error CS0103: The name 'S3Upload' does not exist in the current context
D:/source/psnbooster/build/build.cake(223,8): error CS0246: The type or namespace name 'UploadSettings' could not be found (are you missing a using directive or an assembly reference?)
D:/source/psnbooster/build/build.cake(228,19): error CS0103: The name 'S3CannedACL' does not exist in the current context

any ideas?

I am trying the set CacheControl to "no-cache" but no matter what I do CacheControl always reverts to "public, max-age=864000"

var settings = CreateSyncSettings(awsAccessKey, awsSecretKey)
            .SetBucketName(s3Bucket)
            .SetRegion(regionName)
            .SetLowerPaths(false);
        
settings.CacheControl = "no-cache";

var invalidationList = await S3SyncUpload((DirectoryPath)"site", settings);

We performed an automated audit of your Cake addin and found that it does not follow all the best practices.

We encourage you to make the following modifications:

• You are currently referencing Cake.Core 0.29.0. Please upgrade to 0.33.0
• The Cake.Core reference should be private. Specifically, your addin's .csproj should have a line similar to this: <PackageReference Include="Cake.Core" Version="0.33.0" PrivateAssets="All" />
• The nuget package for your addin should use the cake-contrib icon. Specifically, your addin's .csproj should have a line like this: <PackageIconUrl>https://cdn.jsdelivr.net/gh/cake-contrib/graphics/png/cake-contrib-medium.png</PackageIconUrl>.

Apologies if this is already being worked on, or if there are existing open issues, this issue was created based on what is currently published for this package on NuGet.

This issue was created by a tool: Cake.AddinDiscoverer version 3.12.1

In order to use Cake.AWS.S3 in a Cake 0.26+ environment without having to suppress warnings, etc., it would be useful to update references.

C:\code> .\build.ps1                                 
Preparing to run build script...                                                         
Running build script...                                                                  
Error: The assembly 'Cake.AWS.S3, Version=0.6.0.0, Culture=neutral, PublicKeyToken=null' 
is referencing an older version of Cake.Core (0.22.0).                                   
This assembly need to reference at least Cake.Core version 0.26.0.                       
Another option is to downgrade Cake to an earlier version.                               
It's not recommended, but you can explicitly opt-out of assembly verification            
by configuring the Skip Verification setting to true                                     
(i.e. command line parameter "--settings_skipverification=true",                         
environment variable "CAKE_SETTINGS_SKIPVERIFICATION=true",                              
read more about configuration at https://cakebuild.net/docs/fundamentals/configuration)  

Will have a PR shortly

First of all, I wanted to thank you for adding to the Cake community by adding this addin.

I was just wondering if you had seen this blog post:

http://cakebuild.net/blog/2016/08/cake-contribution-organization

We are currently going through a process of increasing the visibility of addins, and also trying to ensure their long term maintainability.

To that end, we are asking addin creators to add the cake-contrib user on NuGet as a co-owner (this can be done through the NuGet website by clicking on Manage Owners on the package page).

Would you be interested in doing this for Cake.AWS.S3? If you have any questions about this, please let me know. There was some initial concern that the Cake Team were trying to "take over" packages, and that couldn't be further from the truth, and if you have this concern, or others, I would like to address them.

Thanks!

This is a very useful addin for Cake, thanks! 😉

One problem I hit was that the AWS environment variables used by CakeEnvironmentExtensions don't match what's used by the AWS SDK:

• CakeEnvironmentExtensions.cs#L31
• AWSCredentials.cs#L649

Was this a deliberate choice to differentiate them from the fallback credentials used by the SDK?

Normally I use the ctors which leverage the FallbackCredentialsFactory:

• AmazonS3Client.cs#L60
• AmazonS3Client.cs#L79

... which gives you a few ways to automatically inject the credentials - via an instance profile when running inside AWS or via the standard environment variables when running outside AWS.

Not a major problem obviously, but it does require more setup than expected for the common case, so this addin should ideally use the same standard conventions already established by the AWS SDK.

I would love to be able to do the following and use S3's AES256 server-side encryption with AWS-managed (or customer managed) KMS keys:

    var settings = Context.CreateUploadSettings(); 
    settings.BucketName = myBucket;
    settings.ServerSideEncryption = ServerSideEncryptionMethod.AES256;
    await S3Upload(myFile, myS3Key, settings);

or

    var settings = Context.CreateUploadSettings(); 
    settings.BucketName = myBucket;
    settings.ServerSideEncryption = ServerSideEncryptionMethod.AWSKMS;
    settings.ServerSideEncryptionKeyManagementServiceKeyId = "MY_KMS_KEY_ARN";
    await S3Upload(myFile, myS3Key, settings);

Since S3 now supports these types of server-side encryption (as opposed to customer-managed encryption), adding support for stuff like this would be very helpful. If I find time, I will see if I can put together a pull request. The reference guide for these encryption methods lives here: https://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html

versions Cake 0.32.1
versions Cake.AWS.S3 0.6.8
Operating system I am on Windows 10.

The file uploads fine, but the headers appear to not be getting set.

It may be that I am missing something, however I am not seeing it would appreciate any guidance.

The file is just a index.html.

Here is the relevant part of my cake script.

    var sync = new SyncSettings()
    {
        AccessKey = accessKey,
        SecretKey = secretKey,
        BucketName = bucketName,
        CannedACL = S3CannedACL.PublicRead,
        CompressContent = true,
        DefaultContentType = "text/html",
        GenerateContentType = true,
        Region = RegionEndpoint.APSoutheast2,
    };
    sync.Headers.CacheControl = "private, no-cache, no-store, proxy-revalidate, no-transform";
    sync.Headers["Pragma"] = "no-cache";
    await S3SyncUpload(file, sync);

I get the following error:
Error: The assembly 'Cake.AWS.S3, Version=0.4.6.0, Culture=neutral, PublicKeyToken=null' is referencing an older version of Cake.Core (0.17.0). This assembly need to reference at least Cake.Core version 0.22.0. Another option is to downgrade Cake to an earlier version. It's not recommended, but you can explicitly opt-out of assembly verification by configuring the Skip Verification setting to true (i.e. command line parameter "--settings_skipverification=true", envrionment variable "CAKE_SETTINGS_SKIPVERIFICATION=true", read more about configuration at https://cakebuild.net/docs/fundamentals/configuration)