Giter Club home page Giter Club logo

vault-secret's Introduction

CircleCI Go Report Card

Kubernetes Secrets from Hashicorp Vault

Problem: My secret are stored in Vault, how can I inject them into Kubernetes secret ?

Solution: Use vault-secret custom resource to specify Vault server, path and keys and the operator will retrieve all the needed information from vault and push them into a Kubernetes secret resource ready to be used in the cluster.

Note on upgrading to 1.0.1 onward

From version 1.0.1, k8s auth method switches from using the local service account configured on the operator side to using the one from the client's namespace defined in the custom resource. This is improving security but as a result, you will probably have to check your vault configuration is in adequation with this change.

Note for Kubernetes 1.24+

From Kubernetes 1.24, secrets are not created along a service account anymore. A secret needs to be manually created to make the controller happy. See #40 for more info.

Installation

Kubernetes version requirements

This operator is supported from Kubernetes 1.10.

If using Kubernetes 1.10 version, the feature gate CustomResourceSubresources must be enabled for the Custom Resource status field to get updated! This feature is enabled by default starting from Kubernetes 1.11.

Operator

Get the latest release from https://github.com/nmaupu/vault-secret/releases

Deploy the Custom Resource Definition and the operator:

$ kubectl apply -f config/crd/bases/maupu.org_vaultsecrets.yaml
$ kubectl apply -f config/doc-samples/operator.yaml
$ kubectl apply -f config/doc-samples/role.yaml
$ kubectl apply -f config/doc-samples/role_binding.yaml
$ kubectl apply -f config/doc-samples/service_account.yaml

Configuration

Env vars

The vault-secret operator can be configured to watch a unique namespace, a set of namespaces or can also be cluster wide. In that case, modify RBAC role and role binding to be cluster scoped. The following environment variables are available to configure the operator:

  • WATCH_NAMESPACE: namespace to watch for new CR. If not defined, use WATCH_MULTINAMESPACES or configure a cluster wide operator.
  • WATCH_MULTINAMESPACES: comma separated list of namespaces to watch for new CR, if not defined, the operator will be cluster scoped except if WATCH_NAMESPACE is set.
  • OPERATOR_NAME: name of the operator.

Label filtering

One can use the command line flag --filter-label to filter which vaultsecret custom resource to process by the operator. This flag can be used multiple times.

Example usage:

--filter-label=mylabel=myvalue

Custom resource

Here is an example (config/doc-samples/maupu.org_v1beta1_vaultsecrets_cr.yaml) :

apiVersion: maupu.org/v1beta1
kind: VaultSecret
metadata:
  name: example-vaultsecret
  namespace: nma
spec:
  secretName: vault-secret-test
  secretLabels:
    foo: bar
  secretAnnotations:
    foo: bar
  secrets:
    - secretKey: username
      kvPath: secrets/kv
      path: test
      field: username
    - secretKey: password
      kvPath: secrets/kv
      path: test
      field: password
  syncPeriod: 1h
  config:
    addr: https://vault.example.com
    auth:
      kubernetes:
        role: myrole
        cluster: kubernetes

A corresponding secret would be created in the same namespace as the VaultSecret custom resource. This secret would contain two keys filled with vault content:

  • username
  • password

It's possible to add annotations and labels to the generated secret with secretAnnotations and secretLabels.

Here is another example for "dockerconfig" secrets:

apiVersion: maupu.org/v1beta1
kind: VaultSecret
metadata:
  name: dockerconfig-example
  namespace: nma
spec:
  secretName: dockerconfig-test
  secretType: kubernetes.io/dockerconfigjson
  secrets:
    - secretKey: .dockerconfigjson
      kvPath: secrets/dockerconfig
      field: dockerconfigjson
      path: /
  config:
    addr: https://vault.example.com
    auth:
      kubernetes:
        role: myrole
        cluster: kubernetes

It's possible to set the secret type in the spec with secretType, if it isn't specified the default value is Opaque.


Secret are resynced periodically (after a maximum of 10h) but it's possible to reduce this delay with the syncPeriod option (syncPeriod: 1h).


If your Vault is using TLS but if its certificates are not signed by a known authority, one can use the config option insecure to skip tls verification.

Do not use TLS_SKIP_VERIFY env variable when starting the operator, it's not being taken into account.

Here is an example:

apiVersion: maupu.org/v1beta1
kind: VaultSecret
metadata:
  name: example-vaultsecret-insecure
spec:
  secretName: vault-secret-test
  secrets:
    - secretKey: foo
      kvPath: secret
      path: foo/bar
      field: value
  config:
    insecure: true
    addr: https://localhost
    auth:
      ...

Vault configuration

To authenticate, the operator uses the config section of the Custom Resource Definition. The following options are supported:

The prefered way is to use Vault Kubernetes Auth Method because the other authentication methods require to push a secret into the custom resource (e.g. token or role_id/secret_id).

Kubernetes Auth Method usage

  config:
    addr: https://vault.example.com
    auth:
      kubernetes:
        role: myrole
        cluster: kubernetes

The section kubernetes takes two arguments:

  • role: role associated with the service account configured.
  • cluster: name used in the url when configuring auth on vault side.

Token

  config:
    addr: https://vault.example.com
    auth:
      token: <mytoken>

AppRole

  config:
    addr: https://vault.example.com
    auth:
      approle:
        roleId: <myroleid>
        secretId: <mysecretid>

If several configuration options are specified, there are used in the following order:

  • Token
  • AppRole
  • Kubernetes Auth Method

Development

Prerequisites

Building

To build, simply use make:

make docker-build
IMG=local/vault-secret:test make docker-build

This task will:

  • build the binary (using docker)
  • create a docker image

You can then push it to any docker repository or use it locally.

vault-secret's People

Contributors

afritzler avatar eugenebnd avatar gdlx avatar mool avatar nmaupu avatar primeroz avatar rokrokss avatar vizv avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

vault-secret's Issues

Migrate CI to the new operator SDK

When building openapi, this error occurs:

[Deprecation Notice] Operator SDK has a new CLI and project layout that is aligned with Kubebuilder. 
See `operator-sdk init -h` and the following doc on how to scaffold a new project:
https://sdk.operatorframework.io/docs/golang/quickstart/
To migrate existing projects to the new layout see:
https://sdk.operatorframework.io/docs/golang/migration/project_migration_guide/
 
The 'operator-sdk generate' command invokes a specific generator to generate
code or manifests.

Usage: 
  operator-sdk generate [command]
 
Available Commands:
  bundle           Generates bundle data for the operator
  crds             Generates CRDs for API's
  csv              Generates a ClusterServiceVersion YAML file for the operator
  k8s              Generates Kubernetes code for custom resource
  packagemanifests Generates a package manifests forma
 
Flags:
  -h, --help   help for generat
 
Global Flags:
      --verbose   Enable verbose loggin
 
Use "operator-sdk generate [command] --help" for more information about a command.

How to proceed with deprecated apiextensions.k8s.io/v1beta1 for crd ?

Hi,

I was wondering what the plan is going forward with the deprecated apiextensions.k8s.io/v1beta1. This was supposed to be removed in 1.19 but has been postponed to 1.22 which give us more time to address the issue.

Also given the fact that 1.16 is EOL maybe is time to think about it ?

I have testied the vault-secret operator on 1.20 and the operator is working as expected. Secrets are created with no issue

$ kubectl apply -f https://raw.githubusercontent.com/nmaupu/vault-secret/master/config/crd/bases/maupu.org_vaultsecrets.yaml --dry-run=client

Warning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
customresourcedefinition.apiextensions.k8s.io/vaultsecrets.maupu.org created (dry run)

the version is returned as v1 from the apiserver both on 1.20 and 1.17 so maybe we can just move the CRD spec and code to v1 rather than v1beta1 and deprecate older version of the code ?

$ kubectl get crd vaultsecrets.maupu.org -o yaml | grep apiVersion | egrep -v "\   {"
apiVersion: apiextensions.k8s.io/v1
  - apiVersion: apiextensions.k8s.io/v1
  - apiVersion: apiextensions.k8s.io/v1beta1
              f:apiVersion:
          apiVersion:

I can totally try getting this as a PR but wanted to check what your thoughts are first about moving

Thanks for the tool though!

Repetitive update of the reconciled VaultSecret status

While preparing to test #37, another problem was discovered. It can be reproduced with both 1.1.2 and 1.2.1.

The scenario is:

  • the Vault operator runs in argocd namespace and watches on its own namespace. It runs using default SA
  • the git-github VaultSecret is being created in the namespace. It should be synced with the git-github secret

Everything works smoothly, but the following gets logged by the Vault operator

2022-03-11T15:00:56.732Z	ERROR	controller	Reconciler error	{"reconcilerGroup": "maupu.org", "reconcilerKind": "VaultSecret", "controller": "vaultsecret", "name": "git-github", "namespace": "argocd", "error": "Operation cannot be fulfilled on vaultsecrets.maupu.org \"git-github\": the object has been modified; please apply your changes to the latest version and try again"}
github.com/go-logr/zapr.(*zapLogger).Error
	/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:237
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:209
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:188
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.Until
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90

Seems the operator attempts to reconcile the secret three times. The first two attempts start at nearly the same time and one of the threads gets into the conflict trying to update the status of the VaultSecret.

This is confirmed by the logs of the API server. it gets two requests to update /apis/maupu.org/v1beta1/namespaces/argocd/vaultsecrets/git-github/status from system:serviceaccount:argocd:default. The second update is rejected with "Conflict" 409.

❯ kubectl -n argocd logs vault-secret-7978575d49-88b77 | grep "Reconciling VaultSecret"
2022-03-11T15:00:41.050Z	INFO	vaultsecret-operator	Reconciling VaultSecret	{"Request.Namespace": "argocd", "Request.Name": "git"}
2022-03-11T15:00:41.244Z	INFO	vaultsecret-operator	Reconciling VaultSecret	{"Request.Namespace": "argocd", "Request.Name": "git-github-internal"}
2022-03-11T15:00:41.322Z	INFO	vaultsecret-operator	Reconciling VaultSecret	{"Request.Namespace": "argocd", "Request.Name": "argocd-secret"}
2022-03-11T15:00:41.387Z	INFO	vaultsecret-operator	Reconciling VaultSecret	{"Request.Namespace": "argocd", "Request.Name": "git"}
2022-03-11T15:00:56.574Z	INFO	vaultsecret-operator	Reconciling VaultSecret	{"Request.Namespace": "argocd", "Request.Name": "git-github"}
2022-03-11T15:00:56.668Z	INFO	vaultsecret-operator	Reconciling VaultSecret	{"Request.Namespace": "argocd", "Request.Name": "git-github"}
2022-03-11T15:00:57.732Z	INFO	vaultsecret-operator	Reconciling VaultSecret	{"Request.Namespace": "argocd", "Request.Name": "git-github"}

Kubernetest 1.21 Compatibility issue

Hi Team,

we have recently rebuild our environment from the Kubernetes 1.19 to 1.21 ( EKS ) and using Vault - 1.4.6. In that we facing following errors for the Java services, There is now a requirement to validate issuer of JWTs issues by k8s in subsequent token review calls, this change worked for Java applications which are directly making a call to vault, but it appears to not work for calls made by vault-secret. Pl let us know if you can help this.

vault-secret:

2022-02-22T12:06:07.337Z INFO vaultsecret-operator Reconciling VaultSecret {"Request.Namespace": "testing", "Request.Name": "testing-service"}
2022-02-22T12:06:07.337Z INFO vault-auth-provider Authenticating using Kubernetes auth method {"func": "KubernetesProvider.Login"}
2022-02-22T12:06:11.627Z ERROR controller Reconciler error {"reconcilerGroup": "maupu.org", "reconcilerKind": "VaultSecret", "controller": "vaultsecret", "name": "testing-secrets-service", "namespace": "testing", "error": "Error making API request.\n\nURL: PUT https://vaultURL/v1/auth/eks/login\nCode: 500. Errors:\n\n* claim "iss" is invalid"}

Secret type support

It would be great to be able to specify the secret type, for example to be able to set it to kubernetes.io/dockerconfigjson for storing the credentials for private docker registries.

label controller is not right

When creating a secret, a label controller is added containing vaultsecret-controller. It would be better if it's filled with the OPERATOR_NAME directly.

Skip Tls Verify with vault Option

When i put key-value VAULT_SKIP_VERIFY=true in environment variable, it seems to be ignored
I tested this option in container with vault-vli successfully.

Add service account namespace in kubernetes auth

Hi,
We have the requirement to use one service account to auth with Vault and use it for all the namespaces in one cluster. I updated the KubernetesAuthType with adding one argument: serviceAccountNamespace.
With this update, user could choose using centralized service account or specified service account in the same namespace with VaultSecret.
I have test this update in my local and confirmed it works well. I have no permission to push new branch to this repo. It would be much helpful for both of you and others to evaluate this update if you could give me permission to push new branch. Could you kindly help to consider this request? Thank you.

Permanent secret reconciliation

I'm trying to set up vault-secret and have come to a problem where it permanently keeps reconciling my secret without any delays in between.

$ kubectl logs vault-secret-698c49fdc6-qcws9 -n vault

{"level":"info","ts":1593812965.213965,"logger":"vault-auth-provider","msg":"Authenticating using Kubernetes auth method"}
{"level":"info","ts":1593812965.4210057,"logger":"vaultsecret-operator","msg":"Reconciling existing Secret keycloak/keycloak","Request.Namespace":"keycloak","Request.Name":"keycloak"}
{"level":"info","ts":1593812965.513911,"logger":"vaultsecret-operator","msg":"Updating CR status information","Request.Namespace":"keycloak","Request.Name":"keycloak"}
{"level":"info","ts":1593812965.5969684,"logger":"vaultsecret-operator","msg":"Reconciling VaultSecret","Request.Namespace":"keycloak","Request.Name":"keycloak"}
{"level":"info","ts":1593812965.5970542,"logger":"vault-auth-provider","msg":"Authenticating using Kubernetes auth method"}
{"level":"info","ts":1593812965.8533182,"logger":"vaultsecret-operator","msg":"Reconciling existing Secret keycloak/keycloak","Request.Namespace":"keycloak","Request.Name":"keycloak"}
{"level":"info","ts":1593812965.8989706,"logger":"vaultsecret-operator","msg":"Updating CR status information","Request.Namespace":"keycloak","Request.Name":"keycloak"}
{"level":"info","ts":1593812965.9886577,"logger":"vaultsecret-operator","msg":"Reconciling VaultSecret","Request.Namespace":"keycloak","Request.Name":"keycloak"}
{"level":"info","ts":1593812965.9887373,"logger":"vault-auth-provider","msg":"Authenticating using Kubernetes auth method"}
{"level":"info","ts":1593812966.2226007,"logger":"vaultsecret-operator","msg":"Reconciling existing Secret keycloak/keycloak","Request.Namespace":"keycloak","Request.Name":"keycloak"}
{"level":"info","ts":1593812966.2807412,"logger":"vaultsecret-operator","msg":"Updating CR status information","Request.Namespace":"keycloak","Request.Name":"keycloak"}
{"level":"info","ts":1593812966.3277357,"logger":"vaultsecret-operator","msg":"Reconciling VaultSecret","Request.Namespace":"keycloak","Request.Name":"keycloak"}
{"level":"info","ts":1593812966.3278165,"logger":"vault-auth-provider","msg":"Authenticating using Kubernetes auth method"}
{"level":"info","ts":1593812966.6002426,"logger":"vaultsecret-operator","msg":"Reconciling existing Secret keycloak/keycloak","Request.Namespace":"keycloak","Request.Name":"keycloak"}
{"level":"info","ts":1593812966.6574135,"logger":"vaultsecret-operator","msg":"Updating CR status information","Request.Namespace":"keycloak","Request.Name":"keycloak"}
{"level":"error","ts":1593812966.7721353,"logger":"vaultsecret-operator","msg":"Error occurred when updating CR status","Request.Namespace":"keycloak","Request.Name":"keycloak","error":"Operation cannot be fulfilled on vaultsecrets.maupu.org \"keycloak\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tvault-secret/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/nmaupu/vault-secret/pkg/controller/vaultsecret.(*ReconcileVaultSecret).Reconcile\n\tvault-secret/pkg/controller/vaultsecret/vaultsecret_controller.go:173\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tvault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:216\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tvault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:192\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\tvault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:171\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\tvault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\tvault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\tvault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}
{"level":"info","ts":1593812966.7725334,"logger":"vaultsecret-operator","msg":"Reconciling VaultSecret","Request.Namespace":"keycloak","Request.Name":"keycloak"}
{"level":"info","ts":1593812966.772643,"logger":"vault-auth-provider","msg":"Authenticating using Kubernetes auth method"}
{"level":"info","ts":1593812966.977385,"logger":"vaultsecret-operator","msg":"Reconciling existing Secret keycloak/keycloak","Request.Namespace":"keycloak","Request.Name":"keycloak"}
{"level":"info","ts":1593812967.0389547,"logger":"vaultsecret-operator","msg":"Updating CR status information","Request.Namespace":"keycloak","Request.Name":"keycloak"}
{"level":"info","ts":1593812967.0978956,"logger":"vaultsecret-operator","msg":"Reconciling VaultSecret","Request.Namespace":"keycloak","Request.Name":"keycloak"}

This is just a snippet from 2 seconds of log output. As you can see it tries to update the secret so often that sometimes it fails since the previous update wasn't fully finished, yet.

$ cat vaultsecret-keycloak.yaml

apiVersion: maupu.org/v1beta1
kind: VaultSecret
metadata:
  name: keycloak
  namespace: keycloak
spec:
  config:
    addr: https://myvault.instance/
    auth:
      kubernetes:
        cluster: kubernetes
        role: keycloak_vault-secret
    insecure: true
  secretName: keycloak
  secrets:
    - field: username
      kvPath: mariadb
      path: creds/keycloak
      secretKey: username
    - field: password
      kvPath: mariadb
      path: creds/keycloak
      secretKey: password
  syncPeriod: 1h

I've tried different values for syncPeriod but it seems to have no effect whatsoever. By now my Vault instance has over 221 pages of active Keycloak MariaDB credentials.

If there is any further information needed for debugging, let me know.

No linux/arm64 support

I'm running my bare-metal Kubernetes cluster on linux/arm64 platform and would love to use vault-secret CRD. Unfortunately the Docker image targets linux/amd64. Would it be possible to implement multi-arch support and include arm64?

I made small modifications in the Docker file and implemented my build using GitHub actions but CircleCI is used in this project and I guess multi-arch should be implemented using CricleCI's capabilities.

Feat Req: Ability to use k8s secret for VaultApp Role / token auth

Add the ability to specify a k8s secret to use to set the vault app role or token auth instead of defining directly in CR. With having the ability to store k8s secrets encrypted it would be more secure to pull vault creds from a secret instead of defining directly in the cr.

Secret cannot be created, see CR Status field for details

Hi ,

I deployed vault-secret project on AWS EKS 1.11 and 1.13, when I try to deploy my application tries to get the secret from the vault, it shows up following error :

{"level":"error","ts":1561627361.0858045,"logger":"vaultsecret-controller","msg":"Some errors have been issued in the CR status information, please check","Request.Namespace":"kube-system","Request.Name":"mychart","error":"Secret mychart cannot be created, see CR Status field for details","stacktrace":"github.com/nmaupu/vault-secret/vendor/github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/github.com/nmaupu/vault-secret/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/nmaupu/vault-secret/pkg/controller/vaultsecret.(*ReconcileVaultSecret).Reconcile\n\t/go/src/github.com/nmaupu/vault-secret/pkg/controller/vaultsecret/vaultsecret_controller.go:136\ngithub.com/nmaupu/vault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/nmaupu/vault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:215\ngithub.com/nmaupu/vault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/github.com/nmaupu/vault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\ngithub.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/github.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\ngithub.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/github.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\ngithub.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/github.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}
{"level":"error","ts":1561627361.0988958,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"vaultsecret-controller","request":"kube-system/mychart","error":"Secret mychart cannot be created, see CR Status field for details","stacktrace":"github.com/nmaupu/vault-secret/vendor/github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/github.com/nmaupu/vault-secret/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/nmaupu/vault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/nmaupu/vault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:217\ngithub.com/nmaupu/vault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/github.com/nmaupu/vault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\ngithub.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/github.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\ngithub.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/github.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\ngithub.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/github.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}

Seems like it can connect to the vault and get the secret back,
but it can't create Secret file on k8s,
not sure what to do.

Thank for the help

Support for secrets stored in to vault in base64 encoded format(eg Truststore and Keystore) to Kubernetes secrets

Hey
Thanks for this tool.

We have our keystore and truststore stored in base64 encoded in vault(as this is vault limitation, we can not store a binary file without encoding it to base64) and when we fetch them into kubernetes secret, the value get encoded again so when we mount these kubernetes secret into pod , we get base64 encoded value. while we want it to be actual keystore.

is it possible to add a flag(like below valyetype) in definition of "VaultSecret" which can tell that value is already base64 encoded and not to encode it when creating kubernetes secret.

- secretKey: trust.jks 
  kvPath: secret
  path: path 
  field: truststore
  **_valuetype: base64encoded_**

permission denied

Hi ,

today when I try to deploy our project on k8s, it's failed by can't found any secret,

after check CR statue, it's showing:

URL: GET https://vault.example.com/v1/data/secret/dev/api
Code: 403. Errors:

* 1 error occurred:
  * permission denied


    Secret:
      Field:       config
      Kv Path:
      Path:        secret/dev/api
      Secret Key:  config
    Status:        false
    Message:       Problem occured getting secret
    Root Error:    Error making API request.

and after some checking, I found out it will only happen on the newest version 0.0.8,

even I deploy newest CustomResourceDefinition , CR status not showing any error,

but log still showing:

{"level":"error","ts":1567508261.160536,"logger":"vaultsecret-controller","msg":"Error occured when updating CR status","Request.Namespace":"backend","Request.Name":"dev-secrets","error":"the server could not find the requested resource (put vaultsecrets.maupu.org dev-secrets)","stacktrace":"github.com/nmaupu/vault-secret/vendor/github.com/go-logr/zapr.(*zapLogger).Error\n\tvault-secret/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/nmaupu/vault-secret/pkg/controller/vaultsecret.(*ReconcileVaultSecret).Reconcile\n\tvault-secret/pkg/controller/vaultsecret/vaultsecret_controller.go:138\ngithub.com/nmaupu/vault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tvault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:215\ngithub.com/nmaupu/vault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\tvault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\ngithub.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\tvault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\ngithub.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil\n\tvault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\ngithub.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait.Until\n\tvault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}

if I change controller image back to 0.0.7, it's work perfectly

Is there any new config that I have to set up on 0.0.8 ?

Thanks

operator-sdk build broken

$ make build
operator-sdk generate k8s
INFO[0000] Running deepcopy code-generation for Custom Resource group versions: [maupu:[v1beta1], ] 
INFO[0001] Code-generation complete.                    
operator-sdk build nmaupu/vault-secret:latest
# vault-secret/vendor/github.com/nmaupu/vault-secret/pkg/apis/maupu/v1beta1
vendor/github.com/nmaupu/vault-secret/pkg/apis/maupu/v1beta1/vaultsecret_types.go:82:25: cannot use &VaultSecret literal (type *VaultSecret) as type runtime.Object in argument to SchemeBuilder.Register:
	*VaultSecret does not implement runtime.Object (missing DeepCopyObject method)
vendor/github.com/nmaupu/vault-secret/pkg/apis/maupu/v1beta1/vaultsecret_types.go:82:41: cannot use &VaultSecretList literal (type *VaultSecretList) as type runtime.Object in argument to SchemeBuilder.Register:
	*VaultSecretList does not implement runtime.Object (missing DeepCopyObject method)

Is this connected to deps that aren't all pinned?

Error occured when updating CR status

I'm using 0.0.8, Not sure why, the is secret updated successfully, but CR status remains empty.

Here is log:

{"level":"info","ts":1571024733.3686614,"logger":"vaultsecret-controller","msg":"Reconciling VaultSecret","Request.Namespace":"security","Request.Name":"vault.eden.viz.software"}
{"level":"info","ts":1571024733.3686988,"logger":"vault-auth-provider","msg":"Authenticating using Kubernetes auth method"}
2019/10/14 03:45:33 [DEBUG] PUT https://vault.eden.viz.software/v1/auth/kubernetes/login
2019/10/14 03:45:33 [DEBUG] GET https://vault.eden.viz.software/v1/kubernetes/eden.viz.software/kv/gateway/traefik.eden.viz.software
2019/10/14 03:45:34 [DEBUG] GET https://vault.eden.viz.software/v1/kubernetes/eden.viz.software/kv/data/gateway/traefik.eden.viz.software
2019/10/14 03:45:34 [DEBUG] GET https://vault.eden.viz.software/v1/kubernetes/eden.viz.software/kv/gateway/traefik.eden.viz.software
2019/10/14 03:45:34 [DEBUG] GET https://vault.eden.viz.software/v1/kubernetes/eden.viz.software/kv/data/gateway/traefik.eden.viz.software
{"level":"info","ts":1571024734.5569997,"logger":"vaultsecret-controller","msg":"Reconciling existing Secret security/vault.eden.viz.software","Request.Namespace":"security","Request.Name":"vault.eden.viz.software"}
{"level":"info","ts":1571024734.8736734,"logger":"vaultsecret-controller","msg":"Updating CR status information","Request.Namespace":"security","Request.Name":"vault.eden.viz.software"}
{"level":"error","ts":1571024735.011213,"logger":"vaultsecret-controller","msg":"Error occured when updating CR status","Request.Namespace":"security","Request.Name":"vault.eden.viz.software","error":"vaultsecrets.maupu.org \"vault.eden.viz.software\" not found","stacktrace":"github.com/nmaupu/vault-secret/vendor/github.com/go-logr/zapr.(*zapLogger).Error\n\tvault-secret/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/nmaupu/vault-secret/pkg/controller/vaultsecret.(*ReconcileVaultSecret).Reconcile\n\tvault-secret/pkg/controller/vaultsecret/vaultsecret_controller.go:167\ngithub.com/nmaupu/vault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tvault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:215\ngithub.com/nmaupu/vault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\tvault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\ngithub.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\tvault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\ngithub.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil\n\tvault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\ngithub.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait.Until\n\tvault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}

Here is pretty-printed error shown on the last line,

{
  "level": "error",
  "ts": 1571024735.011213,
  "logger": "vaultsecret-controller",
  "msg": "Error occured when updating CR status",
  "Request.Namespace": "security",
  "Request.Name": "vault.eden.viz.software",
  "error": "vaultsecrets.maupu.org \"vault.eden.viz.software\" not found",
  "stacktrace": "github.com/nmaupu/vault-secret/vendor/github.com/go-logr/zapr.(*zapLogger).Error\n\tvault-secret/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/nmaupu/vault-secret/pkg/controller/vaultsecret.(*ReconcileVaultSecret).Reconcile\n\tvault-secret/pkg/controller/vaultsecret/vaultsecret_controller.go:167\ngithub.com/nmaupu/vault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tvault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:215\ngithub.com/nmaupu/vault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\tvault-secret/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\ngithub.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\tvault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\ngithub.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil\n\tvault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\ngithub.com/nmaupu/vault-secret/vendor/k8s.io/apimachinery/pkg/util/wait.Until\n\tvault-secret/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"
}

Maybe it's not a good idea use . in name for CRD?

Release manifests contain CRD only

the README.md states:

Get the latest release from https://github.com/nmaupu/vault-secret/releases

Deploy the Custom Resource Definition and the operator:

$ kubectl apply -f deploy/crds/maupu_v1beta1_vaultsecret_crd.yaml
$ kubectl apply -f deploy/service_account.yaml
$ kubectl apply -f deploy/role.yaml
$ kubectl apply -f deploy/role_binding.yaml
$ kubectl apply -f deploy/operator.yaml

But the latest two releases contains maupu.org_vaultsecrets.yaml only

Kubernetest 1.24 Compatibility

Hello,

The operator does not work with kubernetes 1.24

The error is:
2022-08-30T20:02:58.968Z ERROR controller Reconciler error {"reconcilerGroup": "maupu.org", "reconcilerKind": "VaultSecret", "controller": "vaultsecret", "name": "external-dns", "namespace": "external-dns-system", "error": "No secret associated with the service account external-dns-system/default"}

It does look like it's happening since CHANGELOG
Secret API objects containing service account tokens are no longer auto-generated for every ServiceAccount

Affected code is here: resources.go

Release with updated CRDs

Thank you very much for upgrading the CRD definitions with #32. Although the new version of the Operator is not released yet, we tried to use the changed CRD from https://github.com/nmaupu/vault-secret/blob/master/config/crd/bases/maupu.org_vaultsecrets.yaml.

The operator manages to update the secrets but then logs errors, which seem to occur at attempts to update the VaultSecret status. Example is below

2022-02-22T12:46:47.686Z     ERROR  controller     Reconciler error      {"reconcilerGroup": "maupu.org", "reconcilerKind": "VaultSecret", "controller": "vaultsecret", "name": "git-github", "namespace": "argocd", "error": "vaultsecrets.maupu.org \"git-github\" not found"}
github.com/go-logr/zapr.(*zapLogger).Error
       /go/pkg/mod/github.com/go-logr/[[email protected]](mailto:[email protected])/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
       /go/pkg/mod/sigs.k8s.io/[[email protected]](mailto:[email protected])/pkg/internal/controller/controller.go:237
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
       /go/pkg/mod/sigs.k8s.io/[[email protected]](mailto:[email protected])/pkg/internal/controller/controller.go:209

Would it be possible to release the new version of the Operator?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.