Giter Club home page Giter Club logo

sidecar-injector's Introduction

CyberArk Sidecar Injector

Sidecars are used in Kubernetes to introduce additional features to application pods. Manual sidecar injection can be cumbersome and repetitive. CyberArk Sidecar Injector enables automatic sidecar injection through it being a mutating admission webhook controller. This means when CyberArk Sidecar Injector is deployed and enabled in a namespace, any pod created in that namespace with the appropriate annotations will result in automated sidecar injection.

CyberArk Sidecar Injector provides support for a selection of available sidecars that are configurable through annotations.

Note that unlike manual injection, automatic injection occurs at the pod-level. You will not see any change to the deployment itself. Instead you will want to check individual pods (via kubectl describe) to see the injected sidecar.


##Status: Beta

The CyberArk Sidecar Injector is currently in beta.

Naming and functionality are still subject to breaking changes.


Certification Level

The Secrets Provider push to File feature is a Certified level project. Certified projects have been reviewed and are supported by CyberArk. For more detailed information on our certification levels, see our community guidelines.


This document shows how to deploy and use the CyberArk Sidecar Injector mutating admission webhook controller which injects sidecar container(s) into a pod prior to persistence of the underlying object.

Prerequisites

Kubernetes 1.16.0 or above with the admissionregistration.k8s.io/v1 API enabled. Verify that by the following command:

~$ kubectl api-versions | grep admissionregistration.k8s.io/v1

The result should be:

admissionregistration.k8s.io/v1

In addition, the MutatingAdmissionWebhook and ValidatingAdmissionWebhook admission controllers should be added and listed in the correct order in the admission-control flag of kube-apiserver. Please see the Kubernetes documentation. It is likely that this is set by default if your cluster is running on GKE.

If using minikube, start your cluster as follows:

~$ minikube start --kubernetes-version=v1.10.0

Available Sidecars

This section enumerates the available sidecars. The choice of sidecar is made via annotations - see the Configuration section for details.

  1. Secretless
  2. Authenticator
  3. Secrets Provider

Secretless

See the README

Injects:

  • a newly created Volume named secretless-config sourced from a ConfigMap
  • a configurable Secretless container with:
    • a Volume Mount at /etc/secretless of the secretless-config Volume

Authenticator

See the README

Injects:

  • a newly created shareable in-memory Volume named conjur-access-token used to store the access token
  • a configurable Authenticator container with:
    • a Volume Mount at /run/conjur of the conjur-access-token Volume
  • a read-only Volume Mount at /run/conjur of the conjur-access-token Volume for each of the containers whose names appears in the comma-separated list of container names that you may configure.

Secrets Provider

See the README

Injects:

  • Newly created shareable in-memory Volumes named podInfo and conjur-secrets.
  • a configurable Secrets Provider container
  • Volume Mounts at /conjur/secrets and /conjur/podinfo

Requires:

  • a configMap in the namespace that the sidecar resides in. The config map can be the Golden config or the Conjur connect config map.
  • The sidecar deployment manifest must include the conjur-connect configmap or the Golden configmap

For example to add the Golden config to the sidecar manifest:

    envFrom:
      - configMapRef:
          name: conjur-configmap

Mandatory TLS

Supporting TLS for external webhook server is required because admission is a high security operation. As part of the installation process, we need to create a TLS certificate signed by a trusted CA (shown below is the Kubernetes CA but you can use your own) to secure the communication between the webhook server and kube-apiserver. For the complete steps of creating and approving Certificate Signing Requests(CSR), please refer to Managing TLS in a cluster.

Docker Image

The docker image for the mutating admission webhook server is publicly available on Dockerhub as cyberark/sidecar-injector.

The docker image entrypoint is the server binary. The binary supports the following flags:

Usage of cyberark-sidecar-injector:
  -authenticator-image string
        Container image for the Kubernetes Authenticator sidecar (default "cyberark/conjur-kubernetes-authenticator:latest")
  -noHTTPS
        Run Webhook server as HTTP (not HTTPS).
  -port int
        Webhook server port. (default 443)
  -secretless-image string
        Container image for the Secretless sidecar (default "cyberark/secretless-broker:latest")
  -tlsCertFile string
        Path to file containing the x509 Certificate for HTTPS. (default "/etc/webhook/certs/cert.pem")
  -tlsKeyFile string
        Path to file containing the x509 Private Key for HTTPS. (default "/etc/webhook/certs/key.pem")
  -version
        Show current version

Installation

Installation is possible either

Installing the Sidecar Injector (Manually)

Dedicated Namespace

Create a namespace injectors, where you will deploy the CyberArk Sidecar Injector Webhook components.

  1. Create namespace
    ~$ kubectl create namespace injectors

Deploy Sidecar Injector

  1. Create a signed cert/key pair and store it in a Kubernetes secret that will be consumed by sidecar injector deployment

    ~$ ./deployment/webhook-create-signed-cert.sh \
        --service cyberark-sidecar-injector \
        --secret cyberark-sidecar-injector \
        --namespace injectors
  2. Patch the MutatingWebhookConfiguration by setting caBundle with correct value from Kubernetes cluster

    ~$ cat deployment/mutatingwebhook.yaml | \
        deployment/webhook-patch-ca-bundle.sh \
          --namespace-selector-label cyberark-sidecar-injector \
          --service cyberark-sidecar-injector \
          --namespace injectors > \
        deployment/mutatingwebhook-ca-bundle.yaml
  3. Generate sidecar injector deployment manifest. Note: add --secrets-provider if using secrets provider.

    ~$ ./deployment/deployment.yaml.sh \
         --deployment-api-version apps/v1 \
         --sidecar-injector-image cyberark/sidecar-injector:latest \
         --secretless-image cyberark/secretless-broker:latest \
         --authenticator-image cyberark/conjur-kubernetes-authenticator:latest \
         --secrets-provider-image cyberark/secrets-provider-for-k8s:latest
         > ./deployment/deployment.yaml
  4. Deploy resources

    ~$ kubectl -n injectors apply -f deployment/deployment.yaml
    ~$ kubectl -n injectors apply -f deployment/service.yaml
    ~$ kubectl -n injectors apply -f deployment/mutatingwebhook-ca-bundle.yaml
    ~$ kubectl -n injectors apply -f deployment/crd.yaml

Verify Sidecar Injector Installation

  1. The sidecar injector webhook should be running
    ~$ kubectl -n injectors get pods
    NAME                                                  READY     STATUS    RESTARTS   AGE
    cyberark-sidecar-injector-bbb689d69-882dd   1/1       Running   0          5m
    
    ~$ kubectl -n injectors get deployment
    NAME                                  DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
    cyberark-sidecar-injector             1         1         1            1           5m
    

Installing the Sidecar Injector (Helm)

To install the sidecar injector in the injectors namespace run the following:

helm --namespace injectors \
 install \
 --set "deploymentApiVersion=apps/v1" \
 --set "caBundle=$(kubectl -n kube-system \
   get configmap \
   extension-apiserver-authentication \
   -o=jsonpath='{.data.client-ca-file}' \
 )" \
 ./helm/cyberark-sidecar-injector/  --generate-name

Optionally, if you want to specify the Secretless, Conjur Authenticator and/or Secrets Provider Docker image references, you can specify this in the helm install command:

helm --namespace injectors \
 install \
 --set "deploymentApiVersion=apps/v1" \
 --set "caBundle=$(kubectl -n kube-system \
    get configmap \
    extension-apiserver-authentication \
    -o=jsonpath='{.data.client-ca-file}' \
  )" \
 --set secretlessImage=path/to/secretless/container/image/repo/and/tag" \
 --set authenticatorImage=path/to/authenticator/container/image/repo/and/tag" \
 --set secretsProviderImage=path/to/secretsProvider/container/image/repo/and/tag" \
 ./helm/cyberark-sidecar-injector/  --generate-name

Below is example output from the helm install command:

...

NOTES:
## Instructions
Before you can proceed to use the sidecar-injector, there's one last step.
You will need to approve the CSR (Certificate Signing Request) made by the
sidecar-injector.
This allows the sidecar-injector to communicate securely with the Kubernetes API.

### Watch initContainer logs for when CSR is created
kubectl -n injectors logs deployment/cyberark-sidecar-injector -c
init-webhook -f

### You can check and inspect the CSR
kubectl describe csr "cyberark-sidecar-injector.injectors"

### Approve the CSR
kubectl certificate approve "cyberark-sidecar-injector.injectors"

Now that everything is setup you can enjoy the Cyberark Sidecar Injector.
This is the general workflow:

1. Annotate your application to enable the injector and to configure the sidecar (see
README.md)
2. Webhook intercepts and injects containers as needed

Enjoy.

Make sure to read the NOTES section once the chart is installed; instructions are provided on how to accept the CSR request. The CSR request must be approved.

Using the Sidecar Injector

Configuration

The configurable parameters should be set as annotations on the Pod template spec and NOT on the Deployment, Job or otherwise*. The sidecar injector will not inject the sidecar into pods by default. Add the conjur.org/inject annotation with value true to the Pod template spec to enable injection. Injection will not go ahead if the annotations are not on the Pod template spec.

Note: Configurable parameters of the Sidecar Injector have changed!

For Sidecar injector versions 0.1.1 and below the parameters used sidecar-injector.cyberark.com/xxx. This has been updated to conjur.org/xxx in version 0.2.0 and above.

The following table lists the configurable parameters of the Sidecar Injector and their default values.

Parameter Description Default
conjur.org/inject Enable the Sidecar Injector by setting to true nil (required)
conjur.org/secretless-config ConfigMap holding Secretless configuration nil (required for secretless)
conjur.org/conjurAuthConfig ConfigMap holding Conjur authentication configuration nil (required for authenticator
conjur.org/conjurConnConfig ConfigMap holding Conjur connection configuration nil (required for authenticator
conjur.org/inject-type Injected Sidecar type (secretless, authenticator or secrets-provider) nil (required)
conjur.org/conjur-inject-volumes Comma-separated list of the names of containers, in the pod, that will be injected with conjur-access-token or conjur-secrets and conjur-status VolumeMounts. (e.g. app-container-1,app-container-2) nil (applies to authenticator and secrets provider)
conjur.org/container-mode Sidecar Container mode (init or sidecar) (secretless only supports sidecar) defaults to sidecar
conjur.org/container-name Sidecar Container name nil (only applies to authenticator and secrets-provider)
conjur.org/container-image Sidecar Container image defaults to the value configured for the sidecar-injector at startup, using the -secretless-image or -authenticator-image or -secrets-provider CLI arguments.

conjur.org/secretless-config

There are three options for the value of secretless-config:

  1. configmapName 1. configfile#configmapname 1. k8s/crd#crdName

Option one is legacy, providing backwards compatibility by only specifying a configmap name. The general format is provider#identifier.

If a config map is referenced, it should contain the following path:

  • secretless.yml - Secretless Configuration File

For help using a CRD to configure secretless, Refer to the secretless CRD readme.

conjur.org/conjurConnConfig

Expected to contain the following paths:

  • CONJUR_VERSION - the version of your Conjur instance (4 or 5)
  • CONJUR_APPLIANCE_URL - the URL of the Conjur appliance instance you are connecting to
  • CONJUR_AUTHN_URL - the URL of the authenticator service endpoint
  • CONJUR_ACCOUNT - the account name for the Conjur instance you are connecting to
  • CONJUR_SSL_CERTIFICATE - the x509 certificate that was created when Conjur was initiated

conjur.org/conjurAuthConfig

Expected to contain the following path:

  • CONJUR_AUTHN_LOGIN - Host login for pod e.g. namespace/service_account/some_service_account

Secretless Sidecar Injection Example

For this section, you'll work from a test namespace $TEST_APP_NAMESPACE_NAME (see below). Later you will label this namespace with cyberark-sidecar-injector=enabled so as to allow the cyberark-sidecar-injector to operate on pods created in this namespace.

  1. Set test namespace environment variable

    export TEST_APP_NAMESPACE_NAME=secretless-sidecar-test 
  2. Create test namespace

    ~$ kubectl create namespace ${TEST_APP_NAMESPACE_NAME}
  3. Label the default namespace with cyberark-sidecar-injector=enabled

    ~$ kubectl label \
      namespace ${TEST_APP_NAMESPACE_NAME} \
      cyberark-sidecar-injector=enabled
    
    ~$ kubectl get namespace \
      -L cyberark-sidecar-injector
    NAME                            STATUS    AGE       CYBERARK-SIDECAR-INJECTOR
    default                         Active    18h
    kube-public                     Active    18h
    kube-system                     Active    18h
    cyberark-sidecar-injector       Active    18h
    secretless-sidecar-test         Active    18h       enabled
    
  4. Create Secretless ConfigMap

    This configuration sets up an http service authenticator listening on 0.0.0.0:3000 using the basic_auth authentication strategy. The service authenticator is passed the actual values for the user and password using the literal secret provider.

    As shown below, the username and password are the literal values my-username and secretpassword, respectively.

    ~$ cat << EOL | kubectl -n ${TEST_APP_NAMESPACE_NAME} create configmap secretless --from-file=secretless.yml=/dev/stdin
    version: "2"
    services:
      my-service-proxy:
        protocol: http
        listenOn: tcp://0.0.0.0:3000
        credentials:
          username: my-username
          password: secretpassword
        config:
          authenticationStrategy: basic_auth
          authenticateURLsMatching:
            - ^http.*
    EOL
  5. Deploy an echo server app with the Secretless Sidecar:

    The app is an echo server listening on port 8080, which echoes the request header of any requests sent to it.

    The Secretless Sidecar is injected into the application pod on pod creation via the sidecar injector. The injection is configured via annotations.

    • The secretless ConfigMap is used as a source for Secretless configuration.
    ~$ kubectl -n ${TEST_APP_NAMESPACE_NAME} \
      delete pod \
      test-app --ignore-not-found
    
    ~$ cat << EOF | kubectl -n ${TEST_APP_NAMESPACE_NAME} create -f -
    apiVersion: v1
    kind: Pod
    metadata:
      name: test-app
      annotations:
        conjur.org/inject: "yes"
        conjur.org/secretless-config: "secretless"
        conjur.org/inject-type: "secretless"
      labels:
        app: test-app
    spec:
      containers:
        - name: app
          env:
            - name: http_proxy
              value: "http://0.0.0.0:3000"
          image: googlecontainer/echoserver:1.1
    EOF
  6. Verify Secretless sidecar container injected

    ~$ kubectl -n ${TEST_APP_NAMESPACE_NAME} get pods
    NAME                     READY     STATUS        RESTARTS   AGE
    test-app                 2/2       Running       0          1m
    
  7. Test Secretless

    In this step, you test Secretless by execing into the application pod's main container and issuing an HTTP request against the echo server proxied by Secretless.

    The pod spec for the echo server sets the environment variable http_proxy within the application to the Secretless http service authenticator's address http://0.0.0.0:3000. This allows Secretless to inject HTTP Authorization headers when proxying the request, as per the Secretless configuration above.

    The HTTP Authorization headers are extracted and base64 decoded from the response to retrieve the username and password.

    ~$ kubectl -n ${TEST_APP_NAMESPACE_NAME} \
      exec test-app \
      -c app \
      -i \
      -- \
      curl --silent localhost:8080 \
        | grep authorization \
        | sed -e s/^authorization=Basic\ // \
        | base64 --decode; echo
    
    my-username:secretpassword
    

Conjur Authenticator/Secretless Sidecar/Secrets Provider Injection Example

For this section, you'll work from a test namespace $TEST_APP_NAMESPACE_NAME (see below). Later you will label this namespace with cyberark-sidecar-injector=enabled so as to allow the cyberark-sidecar-injector to operate on pods created in this namespace.

  1. Setup a Conjur appliance running with the Kubernetes authenticator installed and enabled. e.g. run ./start in kubernetes-conjur-deploy

  2. Load Conjur policy to create a host for the service account $TEST_APP_SERVICE_ACCOUNT. e.g. test-app-secretless is made available by walking through kubernetes-conjur-demo up to and including ./3_init_conjur_cert_authority.sh

  3. Set up environment variables, if not already set from kubernetes-conjur-demo

    # REQUIRED values, identical to those used for
    # kubernetes-conjur-deploy and kubernetes-conjur-demo
    export CONJUR_VERSION=...
    export CONJUR_NAMESPACE_NAME=...
    export CONJUR_ACCOUNT=...
    export AUTHENTICATOR_ID=...
    export TEST_APP_NAMESPACE_NAME=...
  4. Set up pod-specific environment variables - modify to match your environment

    # REQUIRED values
    export TEST_APP_SERVICE_ACCOUNT=test-app-secretless
    export containerMode=sidecar
  5. Generate derived Conjur connection environment variables

    # derived values
    ## CONJUR_APPLIANCE_URL
    CONJUR_APPLIANCE_URL="https://conjur-follower.${CONJUR_NAMESPACE_NAME}.svc.cluster.local/api"
    ## CONJUR_AUTHN_URL
    CONJUR_AUTHN_URL="https://conjur-follower.${CONJUR_NAMESPACE_NAME}.svc.cluster.local/api/authn-k8s/${AUTHENTICATOR_ID}"
    ## CONJUR_AUTHN_LOGIN
    if [ ${CONJUR_VERSION} == '4' ]; then
      CONJUR_AUTHN_LOGIN=${TEST_APP_NAMESPACE_NAME}/service_account/${TEST_APP_SERVICE_ACCOUNT}
    else
      CONJUR_AUTHN_LOGIN=host/conjur/authn-k8s/${AUTHENTICATOR_ID}/apps/${TEST_APP_NAMESPACE_NAME}/service_account/${TEST_APP_SERVICE_ACCOUNT}
    fi
    ## CONJUR_SSL_CERTIFICATE
    ### get one of the follower pod names
    follower_pod_name=$(kubectl -n ${CONJUR_NAMESPACE_NAME} \
      get pods \
      -l role=follower --no-headers \
      | awk '{ print $1 }' | head -1);
    
    CONJUR_SSL_CERTIFICATE=$(kubectl -n ${CONJUR_NAMESPACE_NAME} \
      exec \
      $follower_pod_name \
      -- cat /opt/conjur/etc/ssl/conjur.pem)
  6. Create test namespace

    ~$ kubectl create namespace ${TEST_APP_NAMESPACE_NAME}
  7. Label the default namespace with cyberark-sidecar-injector=enabled

    ~$ kubectl label \
      namespace ${TEST_APP_NAMESPACE_NAME} \
      cyberark-sidecar-injector=enabled
    
    ~$ kubectl get namespace -L cyberark-sidecar-injector
    NAME                            STATUS    AGE       CYBERARK-SIDECAR-INJECTOR
    default                         Active    18h
    kube-public                     Active    18h
    kube-system                     Active    18h
    cyberark-sidecar-injector       Active    18h
    conjur-sidecar-test             Active    18h       enabled
    
  8. Create service account (might already exist from kubernetes-conjur-demo) to be used by the application pod

    This service account maps to the Conjur identity for the pod

    ~$ kubectl -n ${TEST_APP_NAMESPACE_NAME} \
    create serviceaccount ${TEST_APP_SERVICE_ACCOUNT}
  9. Create Conjur ConfigMap

    This ConfigMap named conjur stores the connection details to the Conjur appliance. These details are necessary for both the Authenticator and Secretless sidecars to communicate with the Conjur appliance.

    ~$ cat << EOL | kubectl -n ${TEST_APP_NAMESPACE_NAME} apply -f -
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: conjur
    data:
      CONJUR_ACCOUNT: "${CONJUR_ACCOUNT}"
      CONJUR_VERSION: "${CONJUR_VERSION}"
      CONJUR_APPLIANCE_URL: "${CONJUR_APPLIANCE_URL}"
      CONJUR_AUTHN_URL: "${CONJUR_AUTHN_URL}"
      CONJUR_SSL_CERTIFICATE: |
    $(echo "${CONJUR_SSL_CERTIFICATE}" | awk '{ print "    " $0 }')
      CONJUR_AUTHN_LOGIN: "${CONJUR_AUTHN_LOGIN}"
    EOL
  10. You can now leverage Conjur by either

    1. Deploying the Authenticator Sidecar or
    2. Deploying the Secretless Sidecar or
    3. Deploying the Secrets Provider Sidecar

Deploy Authenticator Sidecar

  1. Deploy an app with the Authenticator Sidecar:

    The Authenticator Sidecar is injected into the application pod on pod creation via the sidecar injector. The injection is configured via annotations.

    • The conjur ConfigMap is used for both Conjur Authentication and Connection configuration
    • The conjur.org/container-name is set to "secretless" because the corresponding Conjur identity to the service account used expects the sidecar container to be named secretless.
    ~$ kubectl -n ${TEST_APP_NAMESPACE_NAME} \
      delete pod \
      test-app --ignore-not-found
    
    ~$ cat << EOF | kubectl -n ${TEST_APP_NAMESPACE_NAME} apply -f -
    apiVersion: v1
    kind: Pod
    metadata:
      annotations:
        conjur.org/conjurAuthConfig: conjur
        conjur.org/conjurConnConfig: conjur
        conjur.org/container-mode: ${containerMode}
        conjur.org/conjur-token-receivers: "app"
        conjur.org/inject: "yes"
        conjur.org/inject-type: authenticator
        conjur.org/container-name: secretless
      labels:
        app: test-app
      name: test-app
    spec:
      containers:
      - image: googlecontainer/echoserver:1.1
        name: app
      serviceAccountName: ${TEST_APP_SERVICE_ACCOUNT}
    EOF
  2. Verify Authenticator sidecar container injected

    ~$ kubectl -n ${TEST_APP_NAMESPACE_NAME} get pods
    NAME                     READY     STATUS        RESTARTS   AGE
    test-app                 2/2       Running       0          1m
    
  3. Test Authenticator

    In this step, you test the Authenticator by execing into the application pod's main container and read the contents of /run/conjur/access-token.

    The /run/conjur/access-token file contains the access token which is injected by the Authenticator sidecar upon successful authentication against the Conjur appliance. Note that this file is volume mounted into the application pod's main container as a result of the annotation conjur.org/conjur-token-receivers being set to that container's name.

    ~$ kubectl -n ${TEST_APP_NAMESPACE_NAME} \
      exec test-app \
      -c app \
      -i \
      -- \
      cat /run/conjur/access-token | jq .
    {
      "data": "host/conjur/authn-k8s/sidecar-test/apps/sidecar-example-app/service_account/test-app-secretless",
      "timestamp": "2018-09-20 16:54:04 UTC",
      "signature": "aICQDREA2S-ulOxu8yMWqT9o8h_JDKuuDKIJOFBbQsL_uKZuovManGn-q2Yr4wdT9f_kJdgCNsxh9q54w2ciptn5sAFB3YzDAmqfUzjWv9pIwel2o7N2nuzIw-h7Ho6hA2PQ8V1Iz3NSILCT2JAnWDTi_--bplxqa6g72-j0xprkuFMkDvj2cd084WtMMWXii4W_5WG6BWA9jtnd72-tzhoaU4LFSRfSK7LON8aDdzyFexkM1IbjIuiF1sASBIsvnuY2GeghNDO8VciKh6dXe-sBqNlISlYOTOaQoEMIxA8Nm2t9jeYxmDHJ0IFkTmneeC2dgaJWWoF7MtfJnyPvwn_Z-bF49hkcYDL37-xJxUHPDA4QoU_4p82oqgC3NPnI",
      "key": "11cd239ab55175a3c0f93a7376abe663"
    }
    

Deploy Secretless Sidecar

  1. Create Secretless ConfigMap:

    This configuration sets up an http service authenticator on 0.0.0.0:3000 using the basic_auth authentication strategy that retrieves user and password using the conjur secret provider.

    The username and password are set and stored within the Conjur appliance.

    ~$ cat << EOL | kubectl -n ${TEST_APP_NAMESPACE_NAME} create configmap secretless --from-file=secretless.yml=/dev/stdin
    services:
      my-service-proxy:
        protocol: http
        listenOn: tcp://0.0.0.0:3000
        credentials:
          username:
            from: conjur
            get: test-secretless-app-db/username
          password:
            from: conjur
            get: test-secretless-app-db/password
        config:
          authenticationStrategy: basic_auth
          authenticateURLsMatching:
            - ^http.*
    EOL
  2. Deploy an echo server app with the Secretless Sidecar:

    The app is an echo server listening on port 8080, which echoes the request header of any requests sent to it.

    The Secretless Sidecar is injected into the application pod on pod creation via the sidecar injector. The injection is configured via annotations.

    • The conjur ConfigMap is used for both Conjur Authentication and Connection configuration
    • The secretless ConfigMap is used as a source for Secretless configuration.
    ~$ kubectl -n ${TEST_APP_NAMESPACE_NAME} \
    delete pod \
    test-app --ignore-not-found
    
    ~$ cat << EOF | kubectl -n ${TEST_APP_NAMESPACE_NAME} apply -f -
    apiVersion: v1
    kind: Pod
    metadata:
      annotations:
        conjur.org/conjurAuthConfig: conjur
        conjur.org/conjurConnConfig: conjur
        conjur.org/inject: "yes"
        conjur.org/inject-type: secretless
        conjur.org/secretless-config: secretless
      labels:
        app: test-app
      name: test-app
    spec:
      containers:
      - env:
          - name: http_proxy
            value: "http://0.0.0.0:3000"
        image: googlecontainer/echoserver:1.1
        name: app
      serviceAccountName: ${TEST_APP_SERVICE_ACCOUNT}
    EOF
  3. Verify Secretless sidecar container injected

    ~$ kubectl -n ${TEST_APP_NAMESPACE_NAME} get pods
    NAME                     READY     STATUS        RESTARTS   AGE
    test-app                 2/2       Running       0          1m
    
  4. Test Secretless with Conjur

    In this step, you test Secretless by execing into the application pod's main container and issuing an HTTP request against the echo server proxied by Secretless.

    The pod spec for the echo server sets the environment variable http_proxy within the application to the Secretless http service authenticator's address http://0.0.0.0:3000. This allows Secretless to inject HTTP Authorization headers when proxying the request, as per the Secretless configuration above.

    The HTTP Authorization headers are extracted and base64 decoded from the response to retrieve the username and password.

    ~$ kubectl -n ${TEST_APP_NAMESPACE_NAME} \
      exec test-app \
      -c app \
      -i \
      -- \
      curl --silent localhost:8080 \
        | grep authorization \
        | sed -e s/^authorization=Basic\ // \
        | base64 --decode; echo
    "test_app:84674b2874a5d7c952e7fec8"
    

Deploy Secrets Provider Sidecar

  1. Deploy an app with the Secrets Provider Sidecar:

    The Secrets Provider Sidecar is injected into the application pod on pod creation via the sidecar injector. The injection is configured via annotations.

  • The conjur.org/container-name is set to "cyberark-secrets-provider-for-k8s".

    ~$ kubectl -n ${TEST_APP_NAMESPACE_NAME} \
      delete pod \
      test-app --ignore-not-found
    
    ~$ cat << EOF | kubectl -n ${TEST_APP_NAMESPACE_NAME} apply -f -
    apiVersion: v1
    kind: Pod
    metadata:
      annotations:
        conjur.org/inject: "true"
        conjur.org/inject-type: "secrets-provider"
        conjur.org/container-name: "cyberark-secrets-provider-for-k8s"
        conjur.org/container-image: "docker.io/cyberark/secrets-provider-for-k8s:edge"
        conjur.org/conjur-token-receivers: "test-app"
        conjur.org/authn-identity: host/conjur/authn-k8s/my-authenticator-id/apps/test-app-secrets-provider-p2f
        conjur.org/container-mode: init
        conjur.org/secrets-destination: file
        conjur.org/conjur-secrets.test-app: |
          - admin-username: username
          - admin-password: password
        conjur.org/conjur-secrets-policy-path.test-app: test-secrets-provider-p2f-app-db/
        conjur.org/secret-file-path.test-app: "./application.yaml"
        conjur.org/secret-file-format.test-app: "yaml"
      labels:
        app: test-app
      name: test-app
    spec:
      containers:
      - image: googlecontainer/echoserver:1.1
        name: app
      serviceAccountName: ${TEST_APP_SERVICE_ACCOUNT}
    EOF
  1. Verify Authenticator sidecar container injected

    ~$ kubectl -n ${TEST_APP_NAMESPACE_NAME} get pods
    NAME                     READY     STATUS        RESTARTS   AGE
    test-app                 2/2       Running       0          1m
    
  2. Test Secrets Provider

    In this step, you test the Secrets Provider by execing into the application pod's main container and read the file /conjur/secrets/application.yaml.

    The /conjur/secrets/application.yaml file contains the secrets that are injected by the Secrets Provider sidecar upon successful authentication against the Conjur appliance. Note that this file is volume mounted into the application pod's main container as a result of the annotation conjur.org/conjur-inject-volumes being set to that container's name.

    ~$ kubectl -n ${TEST_APP_NAMESPACE_NAME} \
      exec test-app \
      -c app \
      -i \
      -- \
      cat /conjur/secrets/application.yaml
    "admin-username": "test_app"
    "admin-password": "9622c67ebd1f5fa94ef114da"
    

Contributing

We welcome contributions of all kinds to this repository. For instructions on how to get started and descriptions of our development workflows, please see our contributing guide.

License

The Sidecar Injector is licensed under Apache License 2.0 - see LICENSE for more details.

sidecar-injector's People

Contributors

andytinkham avatar bradleyboutcher avatar captainfluffytoes avatar diverdane avatar doodlesbykumbi avatar garymoon avatar hughsaunders avatar izgeri avatar jakequilty avatar jtuttle avatar juniortaeza avatar phisco avatar rpothier avatar sgnn7 avatar szh avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sidecar-injector's Issues

Add ability to inject conjur-access-token volume mounts to a selection of containers

Update the sidecar injector to enable injecting a volume mount to the conjur-access-token into the container definition of specified containers.

Context

The sidecar injector when used to inject the authenticator will create the conjur-access-token volume and inject the authenticator sidecar. If you want your application container to create a mount to the conjur-access-token volume:

volumeMounts:
  - mountPath: /run/conjur
    name: conjur-access-token

then you have to create a volume mount on that container to a volume that doesn't exist at the time you invoke kubectl apply -f ... This can have mixed results depending on when validation of the pod spec takes place.

A better approach is to introduce an annotation option to the sidecar-injector that allows the sidecar-injector to automatically add these volume mounts to containers whose names you specify. Perhaps sidecar-injector.cyberark.com/conjurTokenReceivers, which would take a comma separated list of container names.

Plz be consistent w/ access token file name

The authenticator client names its token "access-token", but the injected authenticator client names its token "conjur-access-token". It means we can't create one config map to use across injected and non-injected deployments unless we create multiple config map entries for the token file name. Which seems unnecessary.

Sidecar Injector supports configuring secretlessbroker via CRD

As a K8S user, I want to to configure secretless broker using a specialised configuration object, so that I can use K8S' native type system to organise my configuration data.

When I create a configuration object and refer to it using an annotation on a deployment object, sidecar injector will correctly pass the configuration source to secretless broker.

Dev Notes:

  • @hughsaunders worked on this during his interview process and hit a problem where containers injected via the sidecar injector do not have service account token volume mounts. This prevents secretless broker from authenticating with the k8s api. This issue occurs because the mutating plugin that adds the service account token volume mount to containers runs before the plugin that adds the sidecar container.
  • KEP-36 proposes re-running mutating plugins when a mutation has occurred to ensure the other plugins get to see the result of the mutation. The implication of that is that any hacks around the ordering problem need a deprecation note, and object modifications must be idempotent.

There is a trusted release of the sidecar injector

To ease the deployment of the sidecars, we need to create injectors (webhook) for all our sidecars:

In the following order of priorities

  • Authn K8s Client
  • Secretless
  • K8s Secret Provider

MVP1: Trusted
MVP2 : Because this interacts with your production cluster it will need to be certified

Guidance for order of work:

  • Authn K8s client: MVP1, MVP2
  • Secretless: MVP1, MVP2

If there is an opportunity to parallelize it should can be considered

TLS handshake error when following README in local cluster

Summary

When executing the readme (manual deployment) in a rancher desktop local environment to get to know the product, I am receiving Internal error occurred: failed calling webhook "sidecar-injector.conjur.org": failed to call webhook: Post "https://cyberark-sidecar-injector.injectors.svc:443/mutate?timeout=10s": x509: certificate signed by unknown authority:Deployment does not have minimum availability. in my deployment of the testing app.

In the logs of the mutating webhook pod I am seeing multiple http: TLS handshake error from 10.42.0.1:42142: remote error: tls: bad certificate error lines.
This is being tested by bypassing the conjur setup locally as I expect the mutation to happen ( authentication sidecar gets injected) and only then the conjur connection not being available would cause errors.

We decided to give it a go and test this in our openshift test environment using the helm chart, but are getting exactly same issue over there. In that environment conjur is set up and working. Would love to get it running locally first so that we can test the product in a "playground" environment.

Reproducible

  • Always
  • Sometimes
  • Non-Reproducible

Version/Tag number

latest/master

Environment setup

Rancher desktop on MAC + Openshift

Upgrade instructions exist

Clear instructions of how a user might upgrade a pre-running sidecar-injector to the latest version, and what the implications of upgrading are for the different deployment modes.

Roughly this would

  • helm upgrade
  • kubectl apply on the newly generated deployment manifest for the sidecar injector

Configurable sidecar container images

At present the container images for the Authenticator, and Secretless sidecars are hardcoded to the latest versions available from Dockerhub. Users of the injector will likely want to customize this.

This could be achieved in a variety of ways. This issue is to discuss them.

deprecation warning

Summary

Running the manual steps for deployment. Generating the certificates results in this deprecation warnings:

  • Warning: certificates.k8s.io/v1beta1 CertificateSigningRequest is deprecated in v1.19+, unavailable in v1.22+; use certificates.k8s.io/v1 CertificateSigningRequest
  • W0222 23:19:50.734919 15383 helpers.go:553] --dry-run is deprecated and can be replaced with --dry-run=client.

Steps to Reproduce

Steps to reproduce the behavior:

  1. Execute certificate deployment script: ./deployment/webhook-create-signed-cert.sh --service cyberark-sidecar-injector --secret cyberark-sidecar-injector --namespace injectors

Expected Results

No warnings

Actual Results (including error logs, if applicable)

Warnings for deprecation of the API for certificates and for using dry run.

Reproducible

  • [x ] Always

Version/Tag number

Latest Master branch.

Environment setup

Openshift 4.16
Azure

There exists tagged releases for the sidecar-injector

This is an important step towards towards having trusted releases. It also makes for a better UX for consumers of this. Using latest is always a risk.

  • Docker image exists on Dockerhub with tagged image
  • There's a release in the releases section of the repo
  • Tag images using commit

Add troubleshooting section for missing annotations

Update the README of the sidecar injector with a short troubleshooting section that just has what the error looks like if you put the annotations in the deployment (for example) instead of the pod template spec, and how to fix it.

This likely needs updating the logging and logic around handling missing annotations.

A "What is this section" exists in the README

There's a broad audience of readers that will consult the README for insight into what this project is about. The range of k8s knowledge among them will range from “almost none” to “expert”. It's worthwhile to provide some description that is comprehensible to all and clearly states the high-level purpose of this project.

Manifests use older deprecated API versions

Summary

Some of our manifests still use apps/v1beta1 or apps/v1beta2, but Kubernetes officially deprecated and removed support for the v1beta1 API for several resources in v1.16. We should update our manifests to use v1 instead of v1beta1 - we already updated our deployment scripts and our demo.

Steps to Reproduce

Deploy the sidecar injector in K8s 1.16+

Expected Results

A successful deployment

Actual Results (including error logs, if applicable)

You might see errors like

Error: validation failed: unable to recognize "": no matches for kind "Deployment" in version "apps/v1beta2"

Additional Information

Add any other context about the problem here.

Helm Chart should have no network dependecies

Trying to install the helm chart in a strictly segregated environment where no access to the internet can be assumed, we noticed the helm chart is trying to create a deployment with an initContainer downloading kubectl at run time.

It would be really nice if you could use an image that already has such a dependency and make the helm chart less dependent on network connectivity as possible.

I would open a PR, but I guess it would be better for you to own the image and therefore this repository could not be the right place for that, let me know if that's not the case and I'll be happy to contribute.

Sidecar containers images configurable at runtime

Is your feature request related to a problem? Please describe.

At present the sidecar injector relies on configuration of the sidecar container images at deploy time. This means if you want to upgrade/change sidecar container images you have to redeploy. This would incur downtime for such a critical component.

Describe the solution you would like

Runtime configuration avoids the problem altogether. It allows the application to seamlessly change the container images at runtime.

Describe alternatives you have considered

N/A

Additional context

N/A

Tests are unreliable

Currently the master build is alternating between failure and success, the reliability of the test must be improved.

Update API version in deployment.yaml

Summary

The deploylment.yaml uses apiVersion: extensions/v1beta1 for the deployment declaration. This is deprecated on newer clusters and should be apps/v1. Also, you need to add in a selector to the deployment once you do change to apps/v1.

Steps to Reproduce

Steps to reproduce the behavior:

  1. kubectl apply -f deployment.yaml
  2. See error that

Expected Results

Deployment should successfully load.

Actual Results (including error logs, if applicable)

kubectl complains that there is no Type deployment in extensions/v1beta1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.