cmattoon / aws-ssm Goto Github PK
View Code? Open in Web Editor NEWPopulates Kubernetes Secrets from AWS Parameter Store
Home Page: https://hub.docker.com/r/cmattoon/aws-ssm/
License: Apache License 2.0
Populates Kubernetes Secrets from AWS Parameter Store
Home Page: https://hub.docker.com/r/cmattoon/aws-ssm/
License: Apache License 2.0
Any chance to move the chart to upstream, maybe to incubator for now?
We are using the app in our k8 clusters and we are hosting the chart ourselves, which makes it harder to get the updates. And if it will be moved to a popular chart museum, it will also become more popular.
Hey @cmattoon -
The current version of the code uses an outdated aws-sdk which doesn't support the AssumeRoleWithWebIdentity IAM call. The architecture also depends on the node running the container having access to the secrets.
I'm using your container on EKS with the OIDC identity stuff enabled which allows pods to assume roles in AWS, thus I can limit the permissions i need to set on a node.
I updated that with some other packages as well as the dependency system you were using before hand ๐
I think i'm also using a newer version of golang but that's not as important.
I'll write up the PR for this shortly.
I just found that this package is not available on helm hub, would be nice to have it there for visibility.
I could not get your helm install process to work witch looked like this:
make install
helm upgrade --install aws-ssm \
--namespace aws-ssm \
--set image.tag=latest \
--set aws.region=us-east-2 \
--set aws.access_key=<therealkey \
--set aws.secret_key=<therealsecretaccesskey> \
aws-ssm
Error: UPGRADE FAILED: configmaps is forbidden: User "system:serviceaccount:kube-system:default" cannot list configmaps in the namespace "kube-system"
make: *** [install] Error 1
So I created a deployment yaml file instead that places the aws-ssm pod in a namespace called aws-ssm
. Here is the log output of that pod:
time="2018-09-04T22:48:02Z" level=info msg="Using config: &{us-east-2 %!s(int=30) 0.0.0.0:9999 aws}"
time="2018-09-04T22:48:02Z" level=info msg=Running...
time="2018-09-04T22:48:02Z" level=info msg="Connected to cluster at https://10.43.0.1:443"
time="2018-09-04T22:48:02Z" level=fatal msg="Error retrieving secrets: secrets is forbidden: User \"system:serviceaccount:aws-ssm:default\" cannot list secrets at the cluster scope"
Our Kubernetes setup isolates teams by namespaces. Our serviceaccounts can only possible access own-namespace objects.
Similar to this:
external-secrets/kubernetes-external-secrets#106
Hello,
Does the helm chart deployment support the passing of an IAM role?.. I don't see the option within the deployment.yml file.
Thanks..
Hi there,
Thank you for putting this tool together - it's incredibly helpful and works great!
I'm syncing multiple parameters from SSM, some are 'secret' and some are just plain configuration data. To keep things semantically correct, I'd ideally like to sync the plain config data to a ConfigMap instead of a secret.
I'm fairly new to Kubernetes but my understanding at the moment is that ConfigMaps and Secrets are almost the same (Secrets are just base64 encoded for now and may have encryption features added in future). I'm wondering therefore whether it might be possible for this tool to also sync ConfigMap resources, using pretty much the same annotations?
I am trying to run the aws-ssm with docker:
sudo docker run --env KUBE_CONFIG=~/.kube/config --env MASTER_URL=<API server endpoint from EKS> cmattoon/aws-ssm
and I keep getting
level=fatal msg="Error with kubernetes client: stat /home/ec2-user/.kube/config: no such file or directory
even when I am sure ~/.kube/config
exist.
am i missing something?
From conversation in PR#12
In pkg/provider/aws.go
, GetParameterDataByPath
and GetParameterValue
fail with log.Fatalf
when the AWS client returns an error. If IAM credentials are not configured, the Pod will enter a crashloop until the pod is authorized. This is probably correct - or at least consistent - behavior when no IAM role/policy exists at all.
The bug exists when different permissions exist on different SSM keys, resulting in some values being resolved and some providing errors.
However, it prevented other secrets (for which no permission problem existed) to be updated, and since there is one single aws-ssm deployment on the cluster, serving potentially more than one application, the side effect of one broken secret in one namespace affecting operations on an unrelated application can be a bit problematic.
As a user, I'd like to provide values directly to Pods via environment variables, given a StringList
in the format key1=val1,key2=val2,key3=val3
Looks like installing the chart fails with the following error:
"Missing required field "apiGroup" and "kind" in io.k8s.api.rbac.v1.RoleRef"
This suggests a missing apiGroup: rbac.authorization.k8s.io on the RoleRef defined in aws-ssm/templates/rbac.yaml:
Chart fails with Missing required field "apiGroup" and "kind" in io.k8s.api.rbac.v1.RoleRefhttps://github.com/cmattoon/aws-ssm/blob/c5570006473e359dfe20b84f601e278550729d96/aws-ssm/templates/rbac.yaml#L43
Make container fails on step 5 with this error:
Step 5/27 : RUN go install -v ./...
---> Running in 4dee84188e3d
go: downloading k8s.io/apimachinery v0.17.4
go: downloading github.com/aws/aws-sdk-go v1.29.23
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading k8s.io/client-go v0.15.11
go: downloading k8s.io/api v0.17.4
go: downloading golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527
go: downloading k8s.io/klog v1.0.0
go: downloading golang.org/x/net v0.0.0-20200301022130-244492dfa37a
go: downloading github.com/gogo/protobuf v1.3.1
go: downloading github.com/google/gofuzz v1.1.0
go: downloading gopkg.in/inf.v0 v0.9.1
go: downloading golang.org/x/text v0.3.2
go: downloading github.com/jmespath/go-jmespath v0.3.0
/go/pkg/mod/golang.org/x/[email protected]/idna/idna10.0.0.go:25:2: golang.org/x/[email protected]: unexpected EOF
/go/pkg/mod/golang.org/x/[email protected]/idna/idna10.0.0.go:26:2: golang.org/x/[email protected]: unexpected EOF
/go/pkg/mod/golang.org/x/[email protected]/idna/idna10.0.0.go:27:2: golang.org/x/[email protected]: unexpected EOF
pkg/secret/secret.go:27:2: k8s.io/[email protected]: unexpected EOF
pkg/secret/secret.go:28:2: k8s.io/[email protected]: unexpected EOF
pkg/controller/client.go:21:2: k8s.io/[email protected]: unexpected EOF
The command '/bin/sh -c go install -v ./...' returned a non-zero code: 1
Makefile:65: recipe for target 'container' failed
make: *** [container] Error 1
There should be an option to configure CloudWatch Events and respond to configuration changes rather than polling SSM. Events can be Create
, Update
, or Delete
, and I expect to make Delete
-ing an opt-in feature.
It seems aws-ssm
get only 10 parameters using directory when it syncs, and I can't find any configuration to make it get more than 10 parameters for a single secret with directory configuration.
I want to request one of
MaxItems
and this may not be needed
Given the following SSM Params, and aws-ssm/aws-param-name: /path/to/db
/path/to/db/passwd
/path/to/db/etc/foo
/path/to/db/etc/passwd
Since the current implementation uses basename
, the keys passwd
and foo
will be set in the Secret, but which passwd
seems to be undefined. Also, I feel like the keys should probably be passwd
, etc_foo
, and etc_passwd
, and/or a limit placed on the recursion.
There are multiple projects that try to do the same thing conceptually, with CRD-s very similar to this project:
Consider unifying efforts to provide a Common API (eg. CRD) for sourcing secrets into kubernetes.
Ongoing discussion:
Looks like currently it's hardcoded to 30sec
Seems like we need volume mount CA certs or provide some in the docker image:
level=info msg="Couldn't get value for aws-ssm/test-secret: RequestError: send request failed\ncaused by: Post https://ssm.us-east-2.amazonaws.com/: x509: failed to load system roots and no roots provided"
Hello guys,
I noticed the last commit was ~3 years ago. Is this project alive or deprecated?
Thank you.
I'm a bit confused regarding the mapping of values to keys in the created k8s secret. The README.md states that an AWS securestring value of "foo = bar" would set a k8s value of "foo:bar"
However I always get
data:
SecureString: bar
Where bar is base64 encoded as expected.
I've tried setting the parameter value as follows
bar # K8s secret get 'SecureString: bar'
foo=bar # K8s secret gets 'SecureString: foo=bar'
Again the values are base64 encoded as expected.
If this is expected behavior then I think it would be nice to provide an annotation that overrides the default data.SecureString name so that I can adjust the name to match what applications are expecting to find in their secret.
Build failing due to something with aws-iam-authenticator:
# github.com/kubernetes-sigs/aws-iam-authenticator/cmd/aws-iam-authenticator
../../kubernetes-sigs/aws-iam-authenticator/cmd/aws-iam-authenticator/root.go:68:18: cannot use config.DefaultFeatureGates (type map["sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".Feature]"sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".FeatureSpec) as type map["github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".Feature]"github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".FeatureSpec in argument to featureGates.Add
../../kubernetes-sigs/aws-iam-authenticator/cmd/aws-iam-authenticator/root.go:96:15: cannot use featureGates (type *"github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".featureGate) as type "sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".MutableFeatureGate in field value:
*"github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".featureGate does not implement "sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".MutableFeatureGate (wrong type for Add method)
have Add(map["github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".Feature]"github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".FeatureSpec) error
want Add(map["sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".Feature]"sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".FeatureSpec) error
../../kubernetes-sigs/aws-iam-authenticator/cmd/aws-iam-authenticator/server.go:82:43: cannot use k8sconfig (type *"github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/client-go/rest".Config) as type *"sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/client-go/rest".Config in argument to versioned.NewForConfig
../../kubernetes-sigs/aws-iam-authenticator/cmd/aws-iam-authenticator/server.go:89:26: cannot use kubeClient (type "github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/client-go/kubernetes".Interface) as type "sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/client-go/kubernetes".Interface in argument to controller.New:
"github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/client-go/kubernetes".Interface does not implement "sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/client-go/kubernetes".Interface (wrong type for AdmissionregistrationV1beta1 method)
have AdmissionregistrationV1beta1() "github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/client-go/kubernetes/typed/admissionregistration/v1beta1".AdmissionregistrationV1beta1Interface
want AdmissionregistrationV1beta1() "sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/client-go/kubernetes/typed/admissionregistration/v1beta1".AdmissionregistrationV1beta1Interface
../../kubernetes-sigs/aws-iam-authenticator/cmd/aws-iam-authenticator/server.go:102:56: cannot use []"github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/apimachinery/pkg/runtime".Object literal (type []"github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/apimachinery/pkg/runtime".Object) as type []"sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/apimachinery/pkg/runtime".Object in argument to "sigs.k8s.io/aws-iam-authenticator/pkg/generated/clientset/versioned/fake".NewSimpleClientset
The command '/bin/sh -c go get -u -v github.com/kubernetes-sigs/aws-iam-authenticator/cmd/aws-iam-authenticator' returned a non-zero code: 2
Hi cmattoon!
I've made a few PRs that were merged since 1.7. I've been using my own dirty build for the past few months. I'm hoping you'd be able to build and push to docker hub so I can use the official one.
Thanks,
-Ryan!
If I understand this correctly, currently it's only possible to map 1-1 parameters and secrets key (one SSM parameter per secret).
It would be nice if multiple parameters could be added as multiple key-value pairs in the same secret
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.