hypnoglow / helm-s3 Goto Github PK

Here are integration tests, but reindex action is not tested. I think we can also split tests into separate files for each action.

This may not be a bug, maybe I'm missing some setting but I can't find how to make helm use a repo in a different region from the default set in my AWS profile. Here is my .aws/config file:

[profile-us-east-1]
region = us-east-1

[profile-us-west-2]
region = us-west-2

The helm s3 repo s3://mybucket-in-us-east-1/helm-charts-repo is in us-east-1. The following works:

$ export AWS_PROFILE=profile-us-east-1
$ helm repo add myrepo s3://mybucket-in-us-east-1/helm-charts-repo
"myrepo" has been added to your repositories

Now change the profile default region:

$ helm repo remove myrepo
"myrepo" has been removed from your repositories
$ export AWS_PROFILE=profile-us-west-2
$ helm repo add myrepo s3://mybucket-in-us-east-1/helm-charts-repo
fetch from s3: fetch object from s3: BucketRegionError: incorrect region, the bucket is not in 'us-west-2' region
	status code: 301, request id: , host id: 
Error: Looks like "s3://mybucket-in-us-east-1/helm-charts-repo" is not a valid chart repository or cannot be reached: plugin "bin/helms3" exited with error

What am I doing wrong?

I get this error whenever attempting to use an s3 repo inside requirements.yaml

-> % helm dep update
Error: no repository definition for mys3repo. Please add them via 'helm repo add'
Note that repositories must be URLs or aliases. For example, to refer to the stable
repository, use "https://kubernetes-charts.storage.googleapis.com/" or "@stable" instead of
"stable". Don't forget to add the repo, too ('helm repo add').

It does work with full s3 path ie. s3://mys3repo

  - name: somechart
    version: 0.1.0
    repository: "s3://mys3repo/charts"
    condition: somechart.enabled

Currently, operations like push, reindex, delete affect only the remote index, and an additional execution of helm repo update is required after each command. This can easily be improved and the plugin can implicitly update the local index after each command.

Running this command
helm repo add my-helm-repo "s3://"

getting the following error,
fetch from s3: fetch object from s3: MissingRegion: could not find region configuration

How do I specify the S3 region? I am running this in a docker container

In addition to the --force flag, an --ignore-if-exists flag for the push command could be useful

My CICD test and package my whole helm repo after each commit and push the charts with helm s3.

I don't want/need to use --force for existing charts but I would like the push not to fail on them.

Most of the time only 1 chart is updated per commit so most of the pushes fail. A clean exit could be very useful and avoid some error management in the CICD.

It would be nice to process the redirect to the correct region (or allow configuring it per repo) rather than having to swap the profile's region around.

Hey there, thanks for the awesome plugin. Has there been any discussion about pre-packaging this with Helm in a Docker image? If not, would that be useful to this community?

I am noticing when I am trying to install or inspect my charts it is unable to find unless I specify the verison number.

Is there no way to set the latest one as the default?

Examples:
root@SURFACE:/mnt/c/Temp# helm install myrepo/myapp --debug
[debug] Created tunnel using local port: '49901'

[debug] SERVER: "127.0.0.1:49901"

[debug] Original chart version: ""
Error: chart "myapp" matching not found in myrepo index. (try 'helm repo update'). No chart version found for myapp-

and..

root@SURFACE:/mnt/c/Temp# helm inspect myrepo/myapp --debug
Error: chart "myapp" matching not found in myrepo index. (try 'helm repo update'). No chart version found for myapp-

I'm trying to update dependencies on a chart in a private repo, but the download of the chart dependency is failing with NoSuchBucket from S3.

I've checked that I have permission to view the bucket and I've successfully pulled the file directly with the AWS cli, so I'm kind of stumped as to why this is failing.

Current set up:

• Project has a chart dependency stored at private-repo/private-chart
• private-repo was added using helm add repo private-repo s3://private-repo

➜  helm dep update
Hang tight while we grab the latest from your chart repositories...
...Unable to get an update from the "local" chart repository (http://127.0.0.1:8879/charts):
        Get http://127.0.0.1:8879/charts/index.yaml: dial tcp 127.0.0.1:8879: getsockopt: connection refused
...Successfully got an update from the "private-repo" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 13 charts
Downloading private-chart from repo s3://private-repo
2017/10/10 13:37:16 failed to fetch from s3: failed to download object from s3: NoSuchBucket: The specified bucket does not exist
        status code: 404, request id: SOMEREQUESTID, host id: SOMEHOSTID
Save error occurred:  could not download http://private-repo.s3-us-west-2.amazonaws.com/private-chart-0.0.1.tgz: plugin "bin/helms3" exited with error
Deleting newly downloaded charts, restoring pre-update state
Error: could not download http://private-repo.s3-us-west-2.amazonaws.com/private-chart-0.0.1.tgz: plugin "bin/helms3" exited with error

To be consistent with helm, we should use the same CLI library instead of kingpin.

https://github.com/spf13/cobra

While the Helm S3 plugin is very helpful it seems to be available for online installation only (i.e., requires an Internet connection). We are going to use Helm on Internet connected systems as well as offline (air-gap) systems with private, non AWS-provided S3 buckets. For this reason it would be helpful to have an offline installation package for every release. This would also help in pinning a tested version.

In our current setup we have two Amazon accounts, AccountA and AccountB. Charts are pushed using the s3 plugin to a bucket repo by AccountA, but AccountB needs to install charts from the repo. Unfortunately, as AccountA owns the objects, this isn't possible.

We effectively have this scenario. The only way around it is to use the AccountA to explicitly grant permission to AccountB on the files after pushing them, or by using an acl parameter, e.g.

aws s3api put-object --bucket examplebucket --key chart.tgz --grant-full-control id="canonicalUserId-ofAccountA" --body ...

when pushing the chart up initially.

What do you think about the ability to customise the permissions when pushing charts (and the index file)?

Ref: #15

it would be great to be able to detect if the version of the chart as not been changed as part of PR validation instead of waiting until release.

this option would indicate if a chart upload would fail due to the version not being changed.

Any easy way to use minio instead of the real s3 with helm-s3?

https://www.minio.io/

Chart tarballs are uploaded using the "binary/octet-stream" content-type. It would be nice to be able to specify a content-type like "application/x-tar" or "application/x-gzip"

Example HeadObject() response

{                                                                                                                     
    "AcceptRanges": "bytes",                                                                                          
    "ContentType": "binary/octet-stream",                                                                             
    "LastModified": "Mon, 15 Oct 2018 18:51:03 GMT",                                                                  
    "ContentLength": 25232,                                                                                           
    "VersionId": "null",                                                                                              
    "ETag": "\"11f69dd7d7dd80f72e5a6b5c6f7119bf\"",                                                                   
    "Metadata": {                                                                                                     
        "chart-metadata": "..................",          
        "chart-digest": "............................"                            
    }                                                                                                                 
} 

Ref: #5

helm s3 delete <chartName> --version <version> <repo>

Its not very easy atall to use the new changes until a new release has been built and published. Can you cut a new release please?

It would be nice to have the list of the IAM permissions required on the bucket in order to be able to fetch, push and reindex

S3 plugin was throwing semver validation error even if my version was valid, got fixed by re-indexing, error handling should be improved.

Helm fetch does not work unless you first issue s3 reindex. This is unexpected since fetch is able to interact with regular helm repos without any special caveats.

helm s3 push CHART REPO has a race condition when multiple charts are pushed around the same time. It occurs in the following situation:

1. UserA runs: helm s3 push CHART_A s3://REPO
2. UserB runs: helm s3 push CHART_B s3://REPO
3. UserA's process fetches s3://REPO/index.yaml
4. UserB's process fetches s3://REPO/index.yaml
5. UserA's process updates his fetched index.yaml with CHART_A's new version and replaces the remote s3://REPO/index.yaml with his updated version.
6. UserB's process updates his fetched (and now out-of-date) index.yaml with CHART-B's new version and replaces the remote s3://REPO/index.yaml with his updated version which does not contain CHART_A.

At the end of this process, both CHART_A and CHART_B are present in the repository, but CHART_A is missing from the index so any downstream charts that require it will fail when running helm dep update CHART_THAT_DEPENDS_ON_CHART_A.

A simple solution would be for the plugin to create a mutex, e.g. index.yaml.lock before fetching index.yaml which it would delete after replacing index.yaml with the updated version.
If the lockfile is already present, the plugin should wait until it has been deleted and a new one can be created before fetching index.yaml and proceeding. In the worst case, this could cause cascading delays if many charts are frequently updated, but slow is better than broken.

Ref: #5 (comment) #5

This cannot be fully implemented via --profile flag, because you cannot pass it to native helm commands, in case of s3 protocol usage (example: helm repo add myrepo s3://mybucket/charts).

AWS CLI also supports named profiles via AWS_PROFILE environment variable. So, adding AWS_PROFILE support for helm-s3 will be trivial and will work for all cases.

When using the standard AWS CLI profile defined as:

[helm-chart-uploader]
role_arn = arn:aws:iam::#########:role/MyUploaderRole
credential_source = Ec2InstanceMetadata

I am able to upload artifacts as the cross-account user via the AWS CLI (AWS_PROFILE=helm-chart-uploader), but when running the helm s3 push command it seems to ignore the AWS_PROFILE value and not upload as the expected assumed role. If the AWS_PROFILE functionality is being delegated to the Go SDK that capability was added via aws/aws-sdk-go#2201 (which is in v1.15.54+ -- latest patch release being v1.15.89)

When using an s3 bucket without index.yaml, the error message states:

The index file does not exist by the path s3://bucketname/index.yaml. If you haven't initialized the repository yet, try running "helm s3 init s3:/bucketname"

That is missing a slash after "s3:/". If you run the command as is you'll get:

upload index to s3: uri s3:/bucketname/index.yaml protocol is not s3

The error message needs to have a double slash after s3:

Hi,

Just a small issue. The docs state:

To use the plugin, you do not need any special dependencies.

The semantics of "special" may be debated here, but we encountered errors installing this plugin in an Ubuntu image because it did not make. It may help to list out in the docs that you do actually need the make utility.

Thanks!

When I installed helm-s3 and tried to push to S3, it fails with the below error

$ helm s3 push --force "./abc-$(./version)-${CI_COMMIT_REF_NAME:0:11}-${CI_COMMIT_SHA:0:8}.tgz" "<repo_name>"
Error: unknown command "s3" for "helm"
Run 'helm --help' for usage.

There is a library: https://godoc.org/k8s.io/helm/pkg/chartutil

On push command, the target file should be loaded as a chart just to check if it is a valid chart.

In #10 you added support for profiles would it be possible to expand support for this to include assume roles and MFA (similar to https://docs.aws.amazon.com/cli/latest/userguide/cli-roles.html) ?

I don't use Amazon S3 bucket, but s3 api compatible bucket.
So how to configure your helm-s3 tool to use another s3 compatible bucket ?

Hi,

We use the following version scheme for our charts "1.0.0+24.abc" : 1.0.0 is the version and the rest is build metadata (# of commits since last tag, and git commit abbrev)
The helm docs says it is supported: https://docs.helm.sh/developing_charts/#charts-and-versioning

The first time a chart with version 1.0.0+24.abc is pushed it works fine.
If we make changes in our code and create a new chart with version 1.0.0+25.def is generated.
When trying to push that chart, it gets rejected, saying the version already exists:

The chart already exists in the repository and cannot be overwritten without an explicit intent. If you want to replace existing chart, use --force flag:

    helm s3 push --force server-0.0.1+128.62d899d.tgz charts

Error: plugin "s3" exited with error

After some digging, the problem seems to be related to the way caches are built in the plugin. They rely on major, minor, patch only. In my case the cache would already contain a 1.0.0 entry so it would refuse to push a new 1.0.0 version.

Would it be wrong to use the full version, including the metadata?

Thanks,
Guenael

https://circleci.com/gh/hypnoglow/helm-s3/391

Latest commit e6d13b3 on Nov 8, 2018

Downloading and installing helm-s3 v0.8.0 ...
Invalid checksum
helm-s3 install hook failed. Please remove the plugin using 'helm plugin remove s3' and install again.
Error: plugin install hook for "s3" exited with error
Exited with code 1

Helm version: 2.14.1
plugin version: 0.8.0
I have installed helm and helm-s3 plugin on our Jenkins server. I want to build Jenkins pipeline that will package and push our helm charts to our helm repo in AWS. The Jenkins server has an IAM role with a policy allowing all s3 actions on the s3 bucket holding our helm repo. On the commandline I can list and put-object to the s3 bucket.
$ aws s3 ls 123456789-helm-bucket
PRE my-charts/
However, when I try to add this repo to the helm, I get access denied error.
helm repo add my-charts s3://123456789-helm-bucket/my-charts fetch from s3: fetch object from s3: AccessDenied: Access Denied status code: 403, request id: C641C8B8F9131614, host id: Tnu5Cu+iI+WqOoiTlousoefgn5Dfr8rKOzqY+mpebDpHPfDwKJn9JQ5xoTNBvKPfc5uHpouRchU= Error: Looks like "s3://123456789-helm-bucket/my-charts" is not a valid chart repository or cannot be reached: plugin "bin/helms3" exited with error
I have tried to run aws configure command and added credentials and also tried by setting AWS_ACCESS_KEY, AW_SECRET_KEY and AWS_REGION variables as well, but each time I get the same error.

When I'm trying to install the plugin into our CI runner which runs inside AWS / EC2 / kubernetes and which gets assigned/assumes an IAM role I can't add a repository because there is no ~/aws/credentials file. Even though s3 API / cli access is working.

Repo steps:

$ aws s3 ls my-helm-bucket
> 2017-12-12 15:31:01        890 index.yaml

$ helm repo add private s3://my-helm-bucket
> 2017/12/12 15:41:25 get aws config: failed to parse aws credentials file: failed to open aws credentials file: open /root/.aws/credentials: no such file or directory
Error: Looks like "s3://my-helm-bucket" is not a valid chart repository or cannot be reached: plugin "bin/helms3" exited with error

helm env
HELM_BIN="helm" HELM_DEBUG="false" HELM_PLUGINS="/Users/user/Library/helm/plugins" HELM_REGISTRY_CONFIG="/Users/user/Library/Preferences/helm/registry.json" HELM_REPOSITORY_CACHE="/Users/user/Library/Caches/helm/repository" HELM_REPOSITORY_CONFIG="/Users/user/Library/Preferences/helm/repositories.yaml"

helm s3 reindex my repo
load repo file: open /Users/user/.helm/repository/repositories.yaml: no such file or directory

The bug seems to be here:
https://github.com/hypnoglow/helm-s3/blob/master/internal/helmutil/repo_entry.go

Now my env var HELM HOME is empty

`echo $HELM_HOME

`

but if I populate in the session and force it to fail

export HELM_HOME=/Users/user/Library/Caches/helm/repository/

helm s3 reindex boingodev load repo file: open /Users/user/Library/Caches/helm/repository/repository/repositories.yaml: no such file or directory

So the code is assuming the /repository/repositories.yaml subfolder always exists. Which is not the case for me.

Q:Why are you using the OS environment variables instead of the helm's internal configuration?
A: the env parameter is not supported on 2.x but only on 3.x which just became stable recently.

Hi!

Having a weird issue. See below

helm version
Client: &version.Version{SemVer:"v2.14.1", GitCommit:"5270352a09c7e8b6e8c9593002a73535276507c0", GitTreeState:"clean"}

plugin verison 0.8.0

16:38:41  + AWS_DEFAULT_REGION=us-east-1
16:38:41  + helm repo add local s3://artifacts-xxx-us-east-1/charts

16:38:42  fetch from s3: fetch object from s3: BucketRegionError: incorrect region, the bucket is not in 'us-east-2' region
16:38:42  	status code: 301, request id: , host id: 
16:38:42  Error: Looks like "s3://artifacts-xxx-us-east-1/charts" is not a valid chart repository or cannot be reached: plugin "bin/helms3" exited with error

This script works just fine when there is no AWS instance profile available. Any ideas?

I think that the push should be rejected if the version is already there.
Maybe a strict flag if this not what you want by default?

In addition if i push the same version the index file doesn't overrides the old one.

I tried setting up the plugin to use cross account s3 buckets and it doesn't seem to work. Using the aws cli works just fine for fetching charts, but the plugin returns an access denied error.

Currently, the init command is destructive and will overwrite an index.yaml file if one already exists. It should handle this scenario and fail if that file already exists. There could be an explicit flag to force overwriting of existing index.yaml file.

There's many providers who offer an S3-compatible storage service, e.g. DreamObjects. It would be great to allow a custom endpoint URL to be set via environment variable. Thanks!

Would be great to get this to work in Terraform.
Helm repo for terraform exists: https://www.terraform.io/docs/providers/helm/repository.html

Helm-s3 is really useful for uploading and managing helm charts via in S3.
However, I want my repo to be public, so ideally the provider chain would fall back to anonymous access to s3, or a baseurl could be specified to be used in the index.yaml instead?

Due to many requests from users, it is necessary to implement this feature.

Refs:

• #5 manual actions inside S3 bucket where charts are held may lead to broken index
• #18 race conditions may lead to broken index

Any way to support passing in the region for the bucket as sometimes we use helm with different environments than the S3 bucket for the chart repo and we get the following error:

fetch from s3: fetch object from s3: BucketRegionError: incorrect region, the bucket is not in 'us-west-2' region
    status code: 301, request id: , host id: 
Error: failed to download "REPO/PACKAGE" (hint: running `helm repo update` may help)

Hi,

I was wondering if there were any plans for windows support?
When installing the plugin under windows (git bash, in admin mode) I am getting:

Downloading and installing helm-s3 v0.8.0 ...
Unsupported OS type
helm-s3 install hook failed. Please remove the plugin using 'helm plugin remove s3' and install again.
Error: plugin install hook for "s3" exited with error

I also see that there are no windows releases here:
https://github.com/hypnoglow/helm-s3/releases/

I use helm 2.11.0.

Thanks!
Guenael

First of all, great work with this plugin, it works like a charm for us, and alleviated a lot of friction we had using a private s3 bucket for our chart repo. Here a few feature suggestions that we would love to see (if I had the time, I would love to jump in and help out):

AWS Profiles

In my use case, my AWS credentials file has multiple profiles (one personal and one for work). It would be nice to see that there is an option to specify which credentials profile to use to access the bucket.

Possible usage: helm s3 ... [--profile (default: default)]

Reindexing the repository

If for some reason we manually go into the s3 bucket, and modify the existing charts, I think we should have a method to easily reindex the index.yaml directly from the helm command through this plugin

Possible usage: helm s3 reindex [repo]

Deleting charts from the repository

I think that there should be a method to delete charts from the repo, and regenerate the index.yaml to reflect that.

Possible usage: helm s3 delete [chartName] [repo] [--version (default: latest)].

Helm s3 version: 0.7.0

When trying to index to index my repo I get the following message:

traverse the chart repository: load archive from s3 object: EOF

I am logged in and have all permissions for s3