cert-manager / aws-privateca-issuer Goto Github PK

View Code? Open in Web Editor NEW

180.0 15.0 73.0 16.55 MB

Addon for cert-manager that issues certificates using AWS ACM PCA.

License: Apache License 2.0

Dockerfile 1.02% Makefile 8.53% Go 82.23% Smarty 1.30% Shell 6.92%

aws kubernetes cert-manager

aws-privateca-issuer's Issues

[Feature Request]: Create `Kind: AWSPCAClusterIssuers` resource

Describe why this change is needed

Save me from creating this with a local helm chart

Describe solutions and alternatives considered (optional)

An alternative is to install the current helm chart, and then kubectl apply -f this file, or helm install a local chart with this.

apiVersion: awspca.cert-manager.io/v1beta1
kind: AWSPCAClusterIssuer
metadata:
  name: demo-test-root-ca
spec:
  arn: arn:aws:acm-pca:us-east-2:snip:certificate-authority/snip
  region: us-east-2

Is there anything else you would like to add?

Original source of the above yaml https://aws.amazon.com/blogs/security/tls-enabled-kubernetes-clusters-with-acm-private-ca-and-amazon-eks-2/

Docs of what AWS permissions it requires

It is not clear from the Readme what kind of minimal AWS policy we need to have in order to make this operator fully functional

Testing Issue

Just a test.

[Known Issue]: Incorrect Certificate Validity for certificates with specified validity duration. (v1.0.0/v1.1.0)

During the release of v1.0.0 in Nov 2021, the plugin would add 30 days as the validity period for every certificate as a default for certificates with no validity duration specified, with this a bug was introduced which would add 30 days to the validity period on top of the specified validity duration; this was carried forward to v1.1.0 as well. This issue has been rectified in v1.2.0 (For more information please look in to PR. If the incorrect validity is a breaking issue then revoke the old certificates, move to version v1.2.0, and issue new certs.

Can we limit secretRef scope

it seems thats, AWSPCAIssuer can use any secrets in any namespaces. it may cause security problems.

[Bug]: Bash end of file errors with new testing

Describe the expected outcome

make cluster runs with no errors locally on MacOS running Bash 5.x

Describe the actual outcome

make cluster

RUNNING=$(docker inspect -f '{{.State.Running}}' "kind-registry" 2>/dev/null || true)
if [ "$RUNNING" != 'true' ]; then
bash: -c: line 2: syntax error: unexpected end of file

Steps to reproduce

MacOS 11.6

$ bash --version
bash --version
GNU bash, version 5.1.8(1)-release (x86_64-apple-darwin20.3.0)

Relevant log output

No response

Version

0.3.1 (Latest)

Have you tried the following?

Check the Troubleshooting section
Search open issues

Category

Broken Testing Infrastructure

Severity

Severity 4

[Feature Request]: Ability to set PathLength for Sub CA certificates

Describe why this change is needed

With this PR #55 we are introducing the ability to issue subCA certificate when the isCA flag is toggled. This is a feature request to add the ability to modify the subCA template ARN to use something besides a path length of zero.

Describe solutions and alternatives considered (optional)

This could possibly be done as a flag set on the issuer, that when set, is evaluated and used to set the appropriate template arn when issuing the subCA certificate.

Is there anything else you would like to add?

No response

[Feature Request]: Unable to Issue a CA certificate with the desired pathlen constraint value

Describe the expected outcome

I created a Root CA in AWS Private CA with the template: RootCACertificate/V1
Created a Subordinate CA with the help of the same Root CA. Template used for Subordinate: SubordinateCACertificate_PathLen3/V1
Verified that the certificate of Subordinate CA has the pathlen:3 constraint i.e.

            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:3

Next, I created a CA certificate (using Certificate of cert-manager.io/v1). The generated CA certificate has the following constraints-

            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:0

Expected a CA certificate with pathlen:2

Describe the actual outcome

The generated CA certificate had a pathlen:0 constraint, instead of the expected pathlen:2 constraint.

Is this happening due to this section in pca.go?

	if spec.IsCA {
		return prefix + "acm-pca:::template/SubordinateCACertificate_PathLen0/V1"
	}

Steps to reproduce

Create an EKS Cluster.
Install Cert Manager v1.10.0

# https://cert-manager.io/docs/installation/helm/
CERT_MANAGER_VERSION=v1.10.0
helm repo add jetstack https://charts.jetstack.io
helm repo update

helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version ${CERT_MANAGER_VERSION} \
  --set installCRDs=true \
  --wait

Create a Root CA and a Subordinate CA in AWS Private CA
You could run this script to create a Root CA and a Subordinate CA- https://raw.githubusercontent.com/find-arka/k8s-misc/main/create-ca-hierarchy-aws-pca.sh

With the output from the script, save the Intermediate CA ARN in an environment variable-

export CA_ARN=arn:aws:acm-pca:REDACTED:REDACTED-AC:certificate-authority/REDACTED

Setup an IAM Policy to access the CA

cat <<EOF > AWSPCAIssuerPolicyTest.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "awspcaissuer",
      "Action": [
        "acm-pca:DescribeCertificateAuthority",
        "acm-pca:GetCertificate",
        "acm-pca:IssueCertificate"
      ],
      "Effect": "Allow",
      "Resource": "${CA_ARN}"
    }
  ]
}
EOF

POLICY_ARN=$(aws iam create-policy \
    --policy-name AWSPCAIssuerPolicyTest \
    --policy-document file://AWSPCAIssuerPolicyTest.json \
    --output json | jq -r '.Policy.Arn')

echo "POLICY_ARN = ${POLICY_ARN}"

Setup a k8s Service Account and an associated IAM role to access the subordinate CA.

CURRENT_CLUSTER="Please put your cluster name here"
echo "${POLICY_ARN}"
echo "${REGION}"
echo "${CURRENT_CLUSTER}"

# Currently, we are installing the plugin in the same namespace as cert-manager
export PCA_NAMESPACE=cert-manager
# latest version https://github.com/cert-manager/aws-privateca-issuer/releases
export AWSPCA_ISSUER_TAG=v1.2.2

# Enable the IAM OIDC Provider for the cluster
eksctl utils associate-iam-oidc-provider \
    --cluster=${CURRENT_CLUSTER} \
    --region=${REGION} \
    --approve;

# Create IAM role bound to a service account
eksctl create iamserviceaccount --cluster=${CURRENT_CLUSTER} \
    --region=${REGION} \
    --namespace=${PCA_NAMESPACE} \
    --attach-policy-arn=${POLICY_ARN} \
    --override-existing-serviceaccounts \
    --tags "created-by=${USER},team=${TEAM},purpose=customer-support" \
    --name=aws-pca-issuer \
    --role-name "ServiceAccountRolePrivateCA-${CURRENT_CLUSTER}" \
    --approve;

# Install AWS Private CA Issuer Plugin 
# https://github.com/cert-manager/aws-privateca-issuer/#setup
helm repo add awspca https://cert-manager.github.io/aws-privateca-issuer
helm repo update
helm install aws-pca-issuer awspca/aws-privateca-issuer \
    --namespace ${PCA_NAMESPACE} \
    --set image.tag=${AWSPCA_ISSUER_TAG} \
    --set serviceAccount.create=false \
    --set serviceAccount.name=aws-pca-issuer \
    --kube-context ${CURRENT_CLUSTER} \
    --wait;

# Verify deployment status
kubectl --context ${CURRENT_CLUSTER} -n ${PCA_NAMESPACE} \
    rollout status deploy/aws-pca-issuer-aws-privateca-issuer;

Create the Issuer-

# edit the var if your CA is in a different region
export CA_REGION="YOUR CA REGION"
export CA_ARN="arn:aws:acm-pca:redacted:redacted:certificate-authority/redacted"

cat << EOF | kubectl apply -f -
apiVersion: awspca.cert-manager.io/v1beta1
kind: AWSPCAClusterIssuer
metadata:
  name: my-cluster-issuer
spec:
  arn: ${CA_ARN}
  region: ${CA_REGION}
EOF

Create a Certificate object

cat << EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: my-internal-ca
spec:
  commonName: my-internal-ca
  dnsNames:
    - "*.internal.my-org.ca"
  isCA: true
  duration: 2160h #90d
  secretName: my-internal-ca-cert-manager
  subject:
    organizations:
    - cluster.local
    - cert-manager
  issuerRef:
    group: awspca.cert-manager.io
    kind: AWSPCAClusterIssuer
    name: my-cluster-issuer
EOF

Verify:

kubectl get Certificate my-internal-ca

Expected output:

NAME             READY   SECRET                        AGE
my-internal-ca   True    my-internal-ca-cert-manager   4s

extract the secret to read the value-

kubectl get secret my-internal-ca-cert-manager -o yaml | yq -r '.data."tls.crt"' | base64 -d > my-internal-ca-cert-manager-tls-crt.pem
kubectl get secret my-internal-ca-cert-manager -o yaml | yq -r '.data."ca.crt"' | base64 -d > my-internal-ca-cert-manager-ca-crt.pem

my-internal-ca-cert-manager-tls-crt.pem has the CA cert chained with the Issuer. Extract the top section from the pem file and copy it to a different file. I named it generated-ca-cert.pem

openssl x509 -in generated-ca-cert.pem -noout -text | grep -A3 Constraint

            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:0
            X509v3 Authority Key Identifier:
                5C:F9:5F:9D:CF:86:DD:56:94:64:36:C4:REDACTED

Get the Subordinate cert and verify that the pathlen 3 constraint is present-

ROOT_CAARN="arn:aws:acm-pca:REDACTED:REDACTED:certificate-authority/REDACTED"
SUBORDINATE_CERTARN="arn:aws:acm-pca:REDACTED:REDACTED:certificate-authority/REDACTED/certificate/REDACTED"

aws acm-pca get-certificate \
    --certificate-authority-arn "${ROOT_CAARN}" \
    --certificate-arn "${SUBORDINATE_CERTARN}" \
    --output json | jq -r '.Certificate' > "intermediate-cert-common-purpose.pem"

openssl x509 -in intermediate-cert-common-purpose.pem -noout -text | grep -A3 Constraint

            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:3
            X509v3 Authority Key Identifier:
                0B:97:66:22:D3:3A:FF:7D:51:10:2F:46:D1:F8:E8:E9:1D:4E:64:CA

Subordinate CA cert has pathlen:3 but the generated CA cert from that CA cert doesn't have pathlen:2, instead it has pathlen:0

Relevant log output

N/A.
Have already attached the expected output along with the commands in the above section.

Version

Cert Manager -> v1.10.0
aws-privateca-issuer-> v1.2.2
Kubernetes -> 1.22
Amazon EKS platform version -> eks.6

Have you tried the following?

Check the Troubleshooting section
Search open issues

Category

Supported Workflow Broken

Severity

Severity 3

Consider bumping to v2 of the go SDK

In private EKS clusters, VPC endpoints are typically created for regional endpoints.

The AWS sdk in v1 defaults to using the global sts endpoint so the issuer doesn't work without setting an additional env var: AWS_STS_REGIONAL_ENDPOINTS: 'regional'. If using the v2 sdk it uses regional by default as documented here.

[Feature Request]: Integration with cert-manager 1.10

Describe why this change is needed

It appears that the AWS PCA Issuer integration with cert-manager version 1.10 does not quite work. After upgrading to cert-manager 1.10 and attempting to use the latest version of AWS PCA Cluster Issuer resources, these are the messages I see (this works with cert-manager 1.9.1):

E1121 15:03:27.073943       1 checks.go:55] cert-manager/controller/certificaterequests-issuer-selfsigned/handleSecretReference "msg"="failed to determine affected certificate requests" "error"="invalid value \"AWSPCAClusterIssuer\" for issuerRef.kind. Must be empty, \"Issuer\" or \"ClusterIssuer\"" "resource_kind"="Secret" "resource_name"="external-secrets-webhook" "resource_namespace"="my-platform" "resource_version"="v1"

Key thing to point out is "error"="invalid value \"AWSPCAClusterIssuer\" for issuerRef.kind. Must be empty, \"Issuer\" or \"ClusterIssuer\"".

Here is my certificate definition:

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: external-secrets-cert
  namespace: my-platform
spec:
  commonName: external-secrets-webhook.my-platform.svc
  dnsNames:
    - external-secrets-webhook.my-platform.svc
    - external-secrets-webhook
    - external-secrets-webhook.svc.cluster.local
    - external-secrets-webhook.cluster.local
    - localhost
  duration: 2160h0m0s
  ipAddresses:
    - 127.0.0.1
  issuerRef:
    kind: ClusterIssuer
    name: intermediate-ca
  renewBefore: 360h0m0s
  secretName: external-secrets-webhook

And my ClusterIssuer/AWS PCA ClusterIssuer:

---
apiVersion: awspca.cert-manager.io/v1beta1
kind: AWSPCAClusterIssuer
metadata:
  name: root-ca
spec:
  arn: REDACTED
  region: us-gov-west-1
---
kind: Certificate
apiVersion: cert-manager.io/v1
metadata:
  name: intermediate-ca
  namespace: my-platform
spec:
  commonName: intermediate-ca
  duration: 2160h0m0s
  isCA: true
  privateKey:
    algorithm: RSA
    size: 4096
  issuerRef:
    group: awspca.cert-manager.io
    kind: AWSPCAClusterIssuer
    name: root-ca
  renewBefore: 360h0m0s
  secretName: intermediate-ca
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: intermediate-ca
spec:
  ca:
    secretName: intermediate-ca

Describe solutions and alternatives considered (optional)

No response

Is there anything else you would like to add?

Looking for some guidance to see if this has been tested with 1.10.

CA Certificates are not currently supported via the PCA external issuer

As the code says at:
https://github.com/jniebuhr/aws-pca-issuer/blob/e5e5d80ede895ecc079ffa3bb3f2e0821b8d6c3b/pkg/controllers/certificaterequest_controller.go#L136

Is it an intentional limitation? Or is it still in progress? If yes than do you have any timeline for it?

We would like to issue a sub-ordinate CA signed by ACMPCA. We can do that using aws or the web console but it would be nice if aws-pca-issuer supports this too.

[Bug]: Using the default version causes image pull failures

Describe the expected outcome

When using the chart without specifying a version number, the latest version should be deployed.

Describe the actual outcome

The chart attempts to pull a malformed image version and the deployment fails.

The image it's trying to pull is: cert-manager-aws-privateca-issuer:vv1.2.3

There's code here to specify the default version should be prefixed with a v
The helm chart downloads with a v in the version number by default:

Chart.yaml:

appVersion: v1.2.3

Hence 2 v's.

The workaround is to specify the chart version explicitly

Steps to reproduce

Attempt to use the chart but do not specifying a version to deploy the latest version of the aws-privateca-issuer.
Deployment should fail pulling the image due to the malformed version

Relevant log output

No response

Version

1.2.3

Have you tried the following?

Check the Troubleshooting section
Search open issues

Category

Supported Workflow Broken

Severity

Severity 2

[Bug]: Helm chart use latest image instead of appVersion

Describe the expected outcome

Installing the Helm chart should use the image matching the appVersion.

This will allow a reproducible deployment.

Describe the actual outcome

The latest image get always used, which will lead to unexpected upgrade in case the pod gets restarted.

Steps to reproduce

helm template pca-issuer awspca/aws-privateca-issuer --version 1.2.0

Output:

...
     containers:
        - name: aws-privateca-issuer
          image: "public.ecr.aws/k1n1h4h4/cert-manager-aws-privateca-issuer:latest"
...

Note:
I'm aware of the possibility to set image.tag to "" in the values file to fallback to the Chart.AppVersion.

Relevant log output

No response

Version

= 1.0.0

Have you tried the following?

Check the Troubleshooting section
Search open issues

Category

Other

Severity

Severity 4

Known Issue: STS GetCallerIdentity failing because of a region not specified bug

There is currently a known issue with the plugin that is preventing certificate issuance due to STS GetCallerIdentity failing because of a region not specified bug, regardless of whether a region was specified or not (#54). There is an existing pull request to fix this (#53), but we are holding off on accepting any pull requests until our testing is redesigned. To fix this issue until then, please checkout the cleanup branch by running

git fetch -a
git checkout cleanup

Also, please be sure you are using the plugin with an IAM user, as that is the most reliable workflow https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html#Using_CreateAccessKey
This user must have minimum permissions listed here: https://github.com/cert-manager/aws-privateca-issuer#configuration

export AWS_SECRET_ACCESS_KEY=<Secret Access Key you generated>
export AWS_ACCESS_KEY_ID=<Access Key you generated>

pod keeps crashing

Followed the below guide:
https://aws.amazon.com/blogs/containers/setting-up-end-to-end-tls-encryption-on-amazon-eks-with-the-new-aws-load-balancer-controller/

Logs:

I0923 17:41:51.050435       1 request.go:655] Throttling request took 1.019788438s, request: GET:https://172.20.0.1:443/apis/policy/v1beta1?timeout=32s
{"level":"info","ts":1632418911.1048965,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":":8080"}
{"level":"info","ts":1632418911.1053565,"logger":"setup","msg":"starting manager"}
I0923 17:41:51.105568       1 leaderelection.go:243] attempting to acquire leader lease aws-pca-issuer/b858308c.awspca.cert-manager.io...
{"level":"info","ts":1632418911.105594,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}
I0923 17:42:08.605338       1 leaderelection.go:253] successfully acquired lease aws-pca-issuer/b858308c.awspca.cert-manager.io
{"level":"info","ts":1632418928.6055086,"logger":"controller-runtime.manager.controller.certificaterequest","msg":"Starting EventSource","reconciler group":"cert-manager.io","reconciler kind":"CertificateRequest","source":"kind source: /, Kind="}
{"level":"info","ts":1632418928.6055412,"logger":"controller-runtime.manager.controller.awspcaclusterissuer","msg":"Starting EventSource","reconciler group":"awspca.cert-manager.io","reconciler kind":"AWSPCAClusterIssuer","source":"kind source: /, Kind="}
{"level":"info","ts":1632418928.6055186,"logger":"controller-runtime.manager.controller.awspcaissuer","msg":"Starting EventSource","reconciler group":"awspca.cert-manager.io","reconciler kind":"AWSPCAIssuer","source":"kind source: /, Kind="}
{"level":"info","ts":1632418928.7065413,"logger":"controller-runtime.manager.controller.certificaterequest","msg":"Starting Controller","reconciler group":"cert-manager.io","reconciler kind":"CertificateRequest"}
I0923 17:42:09.656073       1 request.go:655] Throttling request took 1.045965413s, request: GET:https://172.20.0.1:443/apis/discovery.k8s.io/v1beta1?timeout=32s
{"level":"error","ts":1632418929.7086253,"logger":"controller-runtime.source","msg":"if kind is a CRD, it should be installed before calling Start","kind":"AWSPCAClusterIssuer.awspca.cert-manager.io","error":"no matches for kind \"AWSPCAClusterIssuer\" in version \"awspca.cert-manager.io/v1beta1\"","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/log.(*DelegatingLogger).Error\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/log/deleg.go:144\nsigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/source/source.go:117\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:167\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:223\nsigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).startRunnable.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/manager/internal.go:681"}
{"level":"info","ts":1632418929.708644,"logger":"controller-runtime.manager.controller.certificaterequest","msg":"Starting workers","reconciler group":"cert-manager.io","reconciler kind":"CertificateRequest","worker count":1}
{"level":"info","ts":1632418929.7087255,"logger":"controller-runtime.manager.controller.certificaterequest","msg":"Stopping workers","reconciler group":"cert-manager.io","reconciler kind":"CertificateRequest"}
{"level":"error","ts":1632418932.3092573,"logger":"controller-runtime.source","msg":"if kind is a CRD, it should be installed before calling Start","kind":"AWSPCAIssuer.awspca.cert-manager.io","error":"no matches for kind \"AWSPCAIssuer\" in version \"awspca.cert-manager.io/v1beta1\"","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/log.(*DelegatingLogger).Error\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/log/deleg.go:144\nsigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/source/source.go:117\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:167\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:223\nsigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).startRunnable.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/manager/internal.go:681"}
{"level":"error","ts":1632418932.3093493,"logger":"controller-runtime.manager","msg":"error received after stop sequence was engaged","error":"no matches for kind \"AWSPCAIssuer\" in version \"awspca.cert-manager.io/v1beta1\"","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).engageStopProcedure.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/manager/internal.go:530"}
{"level":"error","ts":1632418932.3094308,"logger":"controller-runtime.manager","msg":"error received after stop sequence was engaged","error":"leader election lost","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).engageStopProcedure.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/manager/internal.go:530"}
{"level":"error","ts":1632418932.3094597,"logger":"setup","msg":"problem running manager","error":"no matches for kind \"AWSPCAClusterIssuer\" in version \"awspca.cert-manager.io/v1beta1\"","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/log.(*DelegatingLogger).Error\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/log/deleg.go:144\nmain.main\n\t/workspace/main.go:139\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:225"}

[Feature Request]: Allow passing in environmental variables

Describe why this change is needed

Currently you can not set ENV parameters for the helm chart.

Allowing such setting would enable customers to set required settings such as corporate proxy settings via https_proxy/no_proxy, so they can reach ACM API endpoints.

Describe solutions and alternatives considered (optional)

I tried to contribute the below method a few months ago, but you needed tests writing and I don't have IAM access within our corperation to create PCA resources.

values.yaml

env:
  #var1: myvar1
  #var2: myvar2

deployment.yaml

          env:
          {{- range $key, $value := .Values.env }}
          - name: {{ $key }}
            value: "{{ $value }}"
          {{- end }}

example values

env:
  http_proxy: http://corpproxy:3128
  https_proxy: http://corpproxy:3128
  no_proxy: "localhost.127.0.0.1,instance-data,10.0.0.0/8,169.254.169.254,100.64.0.0/10,.corptld.plc"



### Is there anything else you would like to add?

We've been using this method for months now without issues and the application appears to honour the proxy env parameters.

Create certificate secret with expected cert-manager content

As discussed in #10 the ca.crt contained the intermediate CA certificate that was used to issue the requested certificate. This was updated in #11 to have the requested certificate in tls.crt and intermediate + root CAs in ca.crt.

Based on cert-manager certificate documentation the expected format of the secret should be

When a certificate is issued by intermediates of the CA and the Issuer knows the intermediates, the content of tls.crt will be a resulting certificate followed by a certificate chain. The certificate chain doesn’t include a root CA certificate, as it is stored in ca.crt.

I've created #14 as an attempt to address the above.

As an added bonus pem.Encode will add a \n after each certificate in ca.crt which fixes applications that expect a concatenated PEM file (tls.crt + ca.crt + tls.key). I'm running into this with the default OpenShift ingress controller (HAProxy) for example.

[Bug]: AWS PCA Issuer Pod does not assume AWS IAM Role using Helm Deployment

Describe the expected outcome

When deploying the pod using a helm chart and manually creating the ServiceAccount, the pod should inherit the permissions assigned to the IAM role specified in the manually created ServiceAccount and be able to requested certificates.

Describe the actual outcome

The Pod appears to not inherit the correct permissions and fails to get the certificate.

cert-manager  The certificate request has failed to complete and will be retried: failed to request certificate from 
PCA: operation error ACM PCA: IssueCertificate, https response error StatusCode: 400, RequestID: e1e844fb-1af5-4d63-9872-bfe9e627537e, api error 
AccessDeniedException: User: arn:aws:sts::xxxxxx:assumed-role/xxxxxxxx/xxxxxx is not authorized to perform: acm-pca:IssueCertificate 
on resource: arn:aws:acm-pca:eu-west-2:xxxx:certificate-authority/xxxxx because no resource-based policy 
allows the acm-pca:IssueCertificate action

It appears that the pod has the correct ServiceAccount role as it is getting the token?

Steps to reproduce

Deployment of the PCA Issuer Helm Chart using terraform

resource "helm_release" "eks_aws_pca" {
  name       = "${var.site_name}-aws-pca"
  chart      = "aws-privateca-issuer"
  repository = "https://cert-manager.github.io/aws-privateca-issuer"
  version    = var.aws_pca_helm_version
  namespace  = local.aws_pca_namespace

  set {
    name  = "installCRDs"
    value = "true"
  }

  set {
    name  = "image.repository"
    value = var.aws_privateca_issuer_image
  }

  set {
    name  = "image.tag"
    value = var.aws_privateca_issuer_tag
  }

  set {
    name  = "serviceAccount.create"
    value = false
  }

  set {
    name  = "serviceAccount.name"
    value = module.eks_aws_pca_irsa.service_account_name
  }

  set {
    name  = "approverRole.namespace"
    value = local.cert_manager_namespace
  }

  set {
    name  = "approverRole.serviceAccountName"
    value = local.cert_manager_service_account_name
  }

}

Deploying version 1.0.0 of the Helm Chart and Version 1.0.0 of the aws-privateca container

Relevant log output

No response

Version

v1.0.0

Have you tried the following?

Check the Troubleshooting section
Search open issues

Category

Authentication Issue

Severity

Severity 2

Unable to create certificate using ingress annotations - MalformedCSRException: CSR must mark the SAN extension critical when it has an empty subject

Kubernetes: 1.19 (v1.19.6-eks-49a6c0)
aws-pca-issuer: Helm release 0.1.0
certmanager: 1.3.1

I have a working setup of this addon but currently unable to use it to auto-generate certificate using annotations such as below:

apiVersion: extensions/v1beta1
# apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: myprivate-ca
  name: podinfo
  namespace: podinfo
spec:
  rules:
  - host: podinfo.infrastructure.com
    http:
      paths:
      - path: /
        backend:
          serviceName: podinfo
          servicePort: 80
  tls:
    - hosts:
      - podinfo.infrastructure.com
      secretName: podinfo10

Creating certificate using the CRD works, e.g with:

kind: Certificate
apiVersion: cert-manager.io/v1
metadata:
  name: podinfo-cert
  namespace: podinfo
spec:
  commonName: podinfo.infrastructure.com
  dnsNames:
    - podinfo.infrastructure.com
  issuerRef:
    group: awspca.cert-manager.io
    kind: AWSPCAClusterIssuer
    name: myprivate-ca
  renewBefore: 360h0m0s
  secretName: podinfo-cert

The error I get when using the ingress approach to autogenerate the certificate is:

2021-04-28T18:42:12.883939412Z {"level":"error","ts":1619635332.883713,"logger":"controllers.CertificateRequest","msg":"failed to request certificate from PCA","certificaterequest":"podinfo/podinfo3-rxrmp","error":"MalformedCSRException: CSR must mark the SAN extension critical when it has an empty subject.","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\ngithub.com/jniebuhr/aws-pca-issuer/pkg/controllers.(*CertificateRequestReconciler).Reconcile\n\t/workspace/pkg/controllers/certificaterequest_controller.go:171\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

The approach of using ingress works with other issuers as we currently use vault issuer but we want to migrate to AWS ACM.
Does this work for anyone?

Integration with cert-manager, istio-csr fails pod to pod mTLS.

Describe the expected outcome

Pods in the cluster should be able to talk mTLS with each other and have the same root cert dispatched by cert-manager through aws pca.

Describe the actual outcome

kubectl -n foo exec -it deploy/sleep -c sleep -- curl http://httpbin.foo:8000/headers -v

Trying xxxx:8000...
Connected to httpbin.foo (xxxx.155) port 8000 (#0)

GET /headers HTTP/1.1
Host: httpbin.foo:8000
User-Agent: curl/7.87.0-DEV
Accept: /

Mark bundle as not supporting multiuse
< HTTP/1.1 503 Service Unavailable
< content-length: 218
< content-type: text/plain
< date: Fri, 20 Jan 2023 02:48:23 GMT
< server: envoy
<
Connection #0 to host httpbin.foo left intact
upstream connect error or disconnect/reset before headers. retried and the latest reset reason: connection failure, transport failure reason: TLS error: 268435581:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED

Steps to reproduce

Trying to move from self signed cert in istio mesh to ACM PCA certs for istio workloads and with everything seemingly successful (setup), the test apps (sleep, httbin) give a 503 while talking to each other in the same namespace. Im using cert-manager, istio-csr, awspcaissuer add on and istio istio mesh in EKS.

The cert manager, istio csr, awspca/aws-privateca-issuer are all installed using helm and successfully running

awspca issuer (cluster issuer) is created, root cert also created with the following:

kind: Certificate
apiVersion: cert-manager.io/v1
metadata:
  name: istio-ca
  namespace: istio-system
spec:
  isCA: true
  commonName: istio-ca
  duration: 2160h0m0s
  issuerRef:
    group: awspca.cert-manager.io
    kind: AWSPCAClusterIssuer
    name: awspcaclusterissuer
  secretName: istio-ca
  subject:
     organizations:
     - cluster.local
     - cert-manager
   privateKey:
     algorithm: "RSA"
     size: 2048

istio is installed with overlay, everything looks good in sample apps, it comes up with new certs, the CSR are approved but the actual request fails

kubectl -n foo exec -it deploy/sleep -c sleep -- curl http://httpbin.foo:8000/headers -v
*   Trying 192.20.179.155:8000...
* Connected to httpbin.foo (172.20.179.155) port 8000 (#0)
> GET /headers HTTP/1.1
> Host: httpbin.foo:8000
> User-Agent: curl/7.87.0-DEV
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 503 Service Unavailable
< content-length: 218
< content-type: text/plain
< date: Fri, 20 Jan 2023 02:55:07 GMT
< server: envoy
<
* Connection #0 to host httpbin.foo left intact
upstream connect error or disconnect/reset before headers. retried and the latest reset reason: connection failure, transport failure reason: TLS error: 268435581:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED

Relevant log output

No response

Version

istio - 1.14.1
latest for cert-manager, istio-csr, aws-privateca-issuer

Have you tried the following?

Check the Troubleshooting section
Search open issues

Category

Supported Workflow Broken

Severity

Severity 3

Add in unit testing

Currently working on adding unit testing, will update this issue when I complete.

Referenced signer resource does not exist

I tried searching online for this error but I am unable to locate any, what exactly is the issue here and what should I do to resolve it?

"error"="admission webhook \"webhook.cert-manager.io\" denied the request: spec.issuerRef: Forbidden: referenced signer resource does not exist: {pca-issuer AWSPCAClusterIssuer awspca.cert-manager.io/v1beta1}"

[Bug]: Can's issue certificate from a shared private CA

Describe the expected outcome

AWSPCAClusterIssuer should be able to provision certificate

Describe the actual outcome

Our private CA authority is hosted in the networking account and has been shared with other accounts(let's call it dev-1) here. We can manually generate the certs by logging into the dev-1 account.

Now when I create an AWSPCAClusterIssuer, the CA arn I am specifying contains the network account's id (which is obvious), and when I try to create the cert using this certificate it errors out with the below error,

"level":"error","ts":1657793954.0080278,"logger":"controllers.CertificateRequest","msg":"failed to request certificate from PCA","certificaterequest":"default/example-com-f9v6w","error":"operation error ACM PCA: IssueCertificate, https response error StatusCode: 400, RequestID: b6cd6e9b-9e6a-42b9-a96f-38806a3ffffb, api error AccessDeniedException: User: arn:aws:sts::XXXXXXXX:assumed-role/shared-dev-1-cert-manager/1657793931500012209 is not authorized to perform: acm-pca:IssueCertificate on resource: arn:aws:acm-pca:eu-west-2:YYYYYYY:certificate-authority/68275bf7-8147-4620-ba7d-4fecf3f234e0 because no resource-based policy allows the acm-pca:IssueCertificate action","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}

Steps to reproduce

Install certmanager on eks
Try to use a shared private CA for cert creation

Relevant log output

No response

Version

v1.2.2

Have you tried the following?

Check the Troubleshooting section
Search open issues

Category

Authentication Issue

Severity

Severity 2

aws-pca-issuer master doesn't work with kiam

As you requested in #24 I compiled the latest code (jniebuhr/aws-pca-issuer@07e5bed) and deployed it to our test cluster. The original issue was resolved but switching the AWS SDK 2 broke the functionality when KIAM or some similar is in use which is quite common in the AWS world. I think this can be caused the AWS SDK 2 by default enforces to use IMDSv2. The 0.2.1 release doesn't suffer from this nor any of our other "KIAM-handled" services. The KIAM is configured to allow access to every metadata endpoints.

The logs from the aws-pca-issuer:

2021-05-27T09:07:37.731866000Z {"level":"error","ts":1622106457.731583,"logger":"controllers.GenericIssuer","msg":"failed to sts.GetCallerIdentity","genericissuer":"/acmpca","error":"operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: 7a2a4f4b-3c04-4207-9601-e5986b0140dc, api error MissingAuthenticationToken: Request is missing Authentication Token","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\ngithub.com/jniebuhr/aws-pca-issuer/pkg/controllers.(*GenericIssuerReconciler).Reconcile\n\t/workspace/pkg/controllers/genericissuer_controller.go:122\ngithub.com/jniebuhr/aws-pca-issuer/pkg/controllers.(*AWSPCAClusterIssuerReconciler).Reconcile\n\t/workspace/pkg/controllers/awspcaclusterissuer_controller.go:57\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}
2021-05-27T09:07:37.732050000Z {"level":"error","ts":1622106457.7316792,"logger":"controller-runtime.manager.controller.awspcaclusterissuer","msg":"Reconciler error","reconciler group":"awspca.cert-manager.io","reconciler kind":"AWSPCAClusterIssuer","name":"acmpca","namespace":"","error":"operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: 7a2a4f4b-3c04-4207-9601-e5986b0140dc, api error MissingAuthenticationToken: Request is missing Authentication Token","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:302\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

The uswitch/kiam#359 could be useful to understand the issue better.

The KIAM logs says that the requested IAM role was fetched successfully when the aws-pca-issuer pod was in the Pending phase - KIAM fetches roles in advance for performance reasons.

time="2021-05-27T09:04:54Z" level=info msg="fetched credentials" credentials.access.key=ASIATTQ4YQR7GCJ5EJLJ credentials.expiration="2021-05-27T10:01:01Z" credentials.role="<Role Name>" generation.metadata=0 pod.iam.role="<Role Name>" pod.name=cert-manager-acmpca-5f99df9b5c-x7tq5 pod.namespace=cert-manager pod.status.ip= pod.status.phase=Pending resource.version=18948857

As there isn't any shell in the container I can't easily exec into it and check what curl/aws says.

[Feature Request]: Remove region parameter on issuer

Describe why this change is needed

The region field on AWSPCA Issuers is redundent and only allows breaking the issuer.

I might have it wrong but it seems that the region must always match the region of the specified PCA arn.
Any region value that is not the region from the PCA arn, will always break.

Therefore it seems that having the parameter at all is a bug, as it's only function is to allow the user to break the issuer.

Describe solutions and alternatives considered (optional)

Controller should derive the API client region from the CA arn itself.

Is there anything else you would like to add?

Thank you for the very usefull tooling!!

Operation error STS: GetCallerIdentity issue with Unit Tests issue with Region

Trying to run local unit tests but seeing this error in aws-privateca-issuer-controller-manager pod

{"level":"error","ts":1631654584.4164567,"logger":"controllers.GenericIssuer","msg":"failed to sts.GetCallerIdentity","genericissuer":"/pca-cluster-issuer-ec","error":"operation error STS: GetCallerIdentity, failed to resolve service endpoint, an AWS region is required, but was not found","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\ngithub.com/cert-manager/aws-privateca-issuer/pkg/controllers.(*GenericIssuerReconciler).Reconcile\n\t/workspace/pkg/controllers/genericissuer_controller.go:92\ngithub.com/cert-manager/aws-privateca-issuer/pkg/controllers.(*AWSPCAClusterIssuerReconciler).Reconcile\n\t/workspace/pkg/controllers/awspcaclusterissuer_controller.go:57\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

Signature algorithm falsely assumes CA private key type

When we tried to deploy aws-pca-issuer and we hit by the following issue:

2021-05-25T11:33:33.261830000Z {"level":"error","ts":1621942413.261467,"logger":"controllers.CertificateRequest","msg":"failed to request certificate from PCA","certificaterequest":"opa/akunszt-test-9jl58","error":"InvalidArgsException: Signing algorithm SHA384WITHECDSA cannot be used with key type RSA_2048. Valid signing algorithms are: SHA256WITHRSA, SHA384WITHRSA, SHA512WITHRSA.","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\ngithub.com/jniebuhr/aws-pca-issuer/pkg/controllers.(*CertificateRequestReconciler).Reconcile\n\t/workspace/pkg/controllers/certificaterequest_controller.go:171\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

This was caused that the signatureAlgorithm function determines the signature algorithm from the certificate request, not from the signing CA.
https://github.com/jniebuhr/aws-pca-issuer/blob/f2e355d766e625d3c55b287d2a260bbdff5d61ff/pkg/aws/pca.go#L161

We have a CA in ACMPCA with an RSA_2048 key and we wanted to issue an ECDSA certificate. This doesn't work in aws-pca-issuer but it works if you properly set the signing algorithm.

We're using this to issue a certificate using ACMPCA from the command line - it's a snippet from a far bigger script, this is just the bare logic.

# the algorithm for the local key
LOCAL_CURVE_NAME="secp384r1"

# the algorithm used by the PCA to sign the certificate
# - if the PCA has an EC key then this should be SHA512WITHECDSA
# - it the PCA has an RSA key then this should be SHA512WITHRSA
PCA_SIGNING_ALGORITHM="SHA512WITHRSA"

### zillion lines of code to create a private key and a CSR, check a lot of things, etc...

aws acm-pca issue-certificate \
                        --certificate-authority-arn "$PCA_ARN" \
                        --csr file:///dev/stdin \
                        --region "$AWS_REGION" \
                        --signing-algorithm "$PCA_SIGNING_ALGORITHM" \
                        --template-arn "arn:aws:acm-pca:::template/$TEMPLATE" \
                        --validity "Value=$EXPIRES_IN,Type=DAYS" \
                        --query CertificateArn \
                        --output text < "csr.pem"

We use the same algorithm everywhere but for a general solution you can fetch the signing CA private key easily with:

aws acm-pca describe-certificate-authority --region "$REGION" --certificate-authority-arn "$ARN" --query "CertificateAuthority.CertificateAuthorityConfiguration.KeyAlgorithm" --output text

The aws-pca-issuer should fetch that and then use that information to construct the signing algorithm. Another solution could be to set that in the AWSPCAClusterIssuer/AWSPCAIssuer resources. It's rarely changes on it's own as it requires to recreate the CA on the AWS side.

A little background information why we wanted to do this:

You can use the ACMPCA to sign certificates in ACM only if the CA uses an RSA-2048 key. It's a known limitation of ACM at the moment. We have one regional CA - as they're quite costly - which we can use to issue certificates from the AWS web console in ACM or from Kubernetes using Certificate resources. This is why we have to use the RSA-2048 for the CA but we would like to use ECDSA for the certificates issued by the CA to make them a bit more future proof.

[Bug]: Helm Chart fails linting due to whitespace

Describe the expected outcome

When performing linting using helm lint on the Helm Chart, linting should succeed. Additionally, to ensure issues do not occur with changes to the Helm Chart, linting should be included as part of the testing actions.

Describe the actual outcome

When performing linting using helm lint on the Helm Chart, the command fails due to an invalid YAML document separator. After debugging this further, this is caused by incorrect template directives causing issues with whitespace, particularly on line 15 in templates/rbac.yaml. The dash at the end of that line moves that line to the line above. Removing the right dash in the directive, fixes the issue.

Steps to reproduce

This can be reproduced by either using helm or helmfile binaries to performing the linting. If linting using the helmfile binary, the below steps are basically automated.

Add the Helm repository

$ helm repo add awspca https://cert-manager.github.io/aws-privateca-issuer
$ helm repo update

Pull the Helm Chart locally

$ helm pull awspca/aws-privateca-issuer -d ./awspca --untar

Lint the chart

$ helm lint awspca/aws-privateca-issuer

Relevant log output

$ helm lint awspca/aws-privateca-issuer

==> Linting awspca/aws-privateca-issuer
[INFO] Chart.yaml: icon is recommended
[ERROR] templates/rbac.yaml: unable to parse YAML: invalid Yaml document separator: apiVersion: rbac.authorization.k8s.io/v1

Error: 1 chart(s) linted, 1 chart(s) failed

Version

Helm Chart Version: 1.2.2
Helm Application Version: 1.2.2

Have you tried the following?

Check the Troubleshooting section
Search open issues

Category

Build Issues

Severity

Severity 3

AWS PCA shared cannot generate certificate

Currently the aws-privateca-issuer cannot mange to use AWS PCA shared from another account via RAM because when you share a PCA, acm-pca:IssueCertficate requests a condition: StringEquals acm-pca:TemplateArn: "arn:aws:acm-pca:::template/EndEntityCertificate/V1" and there is no way to have that permission without condition.

[Feature Request]: can we support for arm platform?

Describe why this change is needed

we are migrating to arm platform, can we support for arm platform use multi-arch docker image?

Describe solutions and alternatives considered (optional)

No response

Is there anything else you would like to add?

No response

append ca bundle ( chainPem ) to caPem insted to actual certificate i.e. certPem

https://github.com/jniebuhr/aws-pca-issuer/blob/cd313e77bde92960ab17aef85e22848d87f8cff9/pkg/aws/pca.go#L123

Good Morning,

I am not sure reason behind the current solution i.e. append ca bundle ( chainPem ) with actual certificate ( certPem
) issued to FQDN .

currently, i am not proposing any changes as it may be it's kubernetes focused ? just want to show my observation.

the openshift route has three separate keys:

    key: |-                      
      -----BEGIN PRIVATE KEY-----
      [...]
      -----END PRIVATE KEY-----
    certificate: |-              
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----
    caCertificate: |-            
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----

ex. Assume, below is our chain looks like )

D ( root )
  C ( issuer of B)
     B ( issuer of A )
        A ( actual certificate issued to requested FQDN)

Current version:

	certPem := []byte(*getOutput.Certificate + "\n")
	chainPem := []byte(*getOutput.CertificateChain)
	certPem = append(certPem, chainPem...)

	caParams := acmpca.GetCertificateAuthorityCertificateInput{
		CertificateAuthorityArn: aws.String(p.arn),
	}
	caOutput, err := svc.GetCertificateAuthorityCertificate(&caParams)
	if err != nil {
		return nil, nil, err
	}

	caPem := []byte(*caOutput.Certificate)

	return certPem, caPem, nil

in above case, cert-manager creating secret like below: i.e. tks.crt contains the CA bundle.

ca.crt: contains certificate of B
tls.crt: contains certificates of A, B, C, D
tls.key: contains private key of A

Modified version:

	certPem := []byte(*getOutput.Certificate + "\n")
	chainPem := []byte(*getOutput.CertificateChain)
	caPem := chainPem
	//certPem = append(certPem, chainPem...)

	//caParams := acmpca.GetCertificateAuthorityCertificateInput{
	//	CertificateAuthorityArn: aws.String(p.arn),
	//}
	//caOutput, err := svc.GetCertificateAuthorityCertificate(&caParams)
	//if err != nil {
	//	return nil, nil, err
	//}

	//caPem := []byte(*caOutput.Certificate)

	return certPem, caPem, nil

in above case, cert-manager creating secret like below: completely separating CA bundle with tls.crt. and adding whole bundle to ca.crt

ca.crt: contains certificate of B, C, D
tls.crt: contains certificate of A
tls.key: contains private key of A

[Bug]: `aws-privateca-issuer` doesn't works with `csi-cert-manager` for mTLS.

Describe the expected outcome

aws-privateca-issuer shold work with csi-cert-manager for mTLS without any issuse.

Describe the actual outcome

aws-privateca-issuer is not working with csi-cert-manager for mTLS and showing following error:

Steps to reproduce

Issuer.yaml

apiVersion: awspca.cert-manager.io/v1beta1
kind: AWSPCAClusterIssuer
metadata:
  name: demo-test-root-ca
spec:
  arn: <CA_ARN>
  region: us-east-1

node_server.yaml

---
apiVersion: v1
kind: Service
metadata:
  labels: {app: hello-mtls-aws-pca}
  name: hello-mtls-aws-pca
  namespace: default
spec:
  type: ClusterIP
  ports:
  - port: 443
    targetPort: 443
  selector: {app: hello-mtls-aws-pca}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-mtls-aws-pca
  labels: {app: hello-mtls-aws-pca}
  namespace: default
spec:
  replicas: 1
  selector: {matchLabels: {app: hello-mtls-aws-pca}}
  template:
    metadata:
      labels: {app: hello-mtls-aws-pca}
    spec:
      containers:
      - name: hello-mtls-aws-pca
        image: test/cert-manager-server-node:latest
        imagePullPolicy: IfNotPresent
        resources: {requests: {cpu: 10m, memory: 20Mi}}
        volumeMounts:
        - mountPath: "/tls"
          name: tls
      volumes:
      - name: tls
        csi:
          readOnly: true
          driver: csi.cert-manager.io
          volumeAttributes:
            csi.cert-manager.io/issuer-kind: AWSPCAClusterIssuer
            csi.cert-manager.io/issuer-name: demo-test-root-ca
            csi.cert-manager.io/issuer-group: awspca.cert-manager.io
            csi.cert-manager.io/dns-names: hello-mtls-aws-pca.default.svc.cluster.local
            csi.cert-manager.io/duration: "20h"
            csi.cert-manager.io/renew-before: "8h"

Create AWSPCAClusterIssuer and Deployment using above files. Wait for deployment pod coming up. It is waiting for mounting volume and volume mount failing due to no validity mentioned in CertificateRequest.

Relevant log output

{"level":"error","ts":1636429708.7008724,"logger":"controllers.CertificateRequest","msg":"failed to request certificate from PCA","certificaterequest":"default/db9e779d-10f6-467b-acc2-341bcd30dd74","error":"operation error ACM PCA: IssueCertificate, https response error StatusCode: 400, RequestID: 07a63afe-f788-4047-8dbe-3857c1557c61, api error ValidationException: 1 validation error detected: Value '0' at 'validity.value' failed to satisfy constraint: Member must have value greater than or equal to 1","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\ngithub.com/cert-manager/aws-privateca-issuer/pkg/controllers.(*CertificateRequestReconciler).Reconcile\n\t/workspace/pkg/controllers/certificaterequest_controller.go:171\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

E1109 04:22:06.405118       1 manager.go:506] manager "msg"="Failed to issue certificate, retrying after applying exponential backoff" "error"="waiting for request: request \"8961b4c0-3f33-4b30-bbdb-43dedae0a51a\" has failed: failed to request certificate from PCA: operation error ACM PCA: IssueCertificate, https response error StatusCode: 400, RequestID: c6c1fdb3-9a12-424f-bef4-245b7af6c249, api error ValidationException: 1 validation error detected: Value '0' at 'validity.value' failed to satisfy constraint: Member must have value greater than or equal to 1" "volume_id"="csi-fbd8716d6f985bae4fefa59ee13848f66cc8608bb4e4563ae69394424940f6b4"



### Version

0.3.1 (Latest)

### Have you tried the following?

- [X] Check the [Troubleshooting section](../#troubleshooting)
- [X] Search open [issues](https://github.com/cert-manager/aws-privateca-issuer/issues)

### Category

Supported Workflow Broken

### Severity

Severity 1

topologySpreadConstraints support in helm chart

Describe why this change is needed

It would be nice to have topologySpreadConstraints support in the official helm chart

/kind feature

Describe solutions and alternatives considered (optional)

No response

Is there anything else you would like to add?

No response

Unable to issue certificate when namespace name + certificate name > 35 characters

Use case:

I would like to issue a certificate where the combined number of characters of the namespace and certificate name is more than 35 characters.

For example:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: my-very-long-certificate-name-that-has-quite-a-few-characters
  namespace: example

Expected behavior:

A certificate should be issued.

Actual behavior:

A certificate is not issued, and I receive an the following error in the logs for aws-privateca-issuer:

{"level":"error","ts":1624317003.908817,"logger":"controllers.CertificateRequest","msg":"failed to request certificate from PCA","certificaterequest":"example/my-very-long-certificate-name-that-has-quite-a-few-c-8qbnl","error":"operation error ACM PCA: IssueCertificate, https response error StatusCode: 400, RequestID: c73883c0-2ae8-42f1-9369-ca1705b36119, api error ValidationException: 1 validation error detected: Value example/my-very-long-certificate-name-that-has-quite-a-few-c-8qbnl at idempotencyToken failed to satisfy constraint: Member must have length less than or equal to 36","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error
        /go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132
github.com/cert-manager/aws-privateca-issuer/pkg/controllers.(*CertificateRequestReconciler).Reconcile
        /workspace/pkg/controllers/certificaterequest_controller.go:172
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.UntilWithContext
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

References:

The idempotency token token is limited to 36 characters
The token string used for the API call is a concatenation of the namespace name and certificate name:

aws-privateca-issuer/pkg/aws/pca.go

Line 103 in b5f3ba4

idempotencyToken := cr.ObjectMeta.Namespace + "/" + cr.ObjectMeta.Name
Kubernetes object names can be much more than this limit.

aws-pca-issuer tries to create non-PCA certificates

I don't know if it's an issue in cert-manager or in aws-pca-issuer. If it's the former then I'll open an issue there. I just experienced this only with aws-pca-issuer.

When aws-pca-issuer is running then it tries to handle Certificate resources even if the issuerRef points to a "plain" Issuer resource.

In the opa namespace I wanted to create a self-signed CA and a Certificate signed by that CA. The self-signed CA is created but aws-pca-issuer complains about that it doesn't support CA certificates. There aren't any reference to PCA in the YAML, so it shouldn't be even aware of it or it should be skipped silently.

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: ca
  namespace: opa
spec:
  commonName: OpenPolicyAgent CA
  duration: 87600h
  isCA: true
  issuerRef:
    kind: ClusterIssuer
    name: self-signed
  privateKey:
    algorithm: ECDSA
    encoding: PKCS8
    rotationPolicy: Always
    size: 384
  renewBefore: 168h
  secretName: tls-ca
  subject:
    organizations:
      - blah

The log I saw in the aws-pca-issuer.

2021-05-25T14:59:48.961908000Z {"level":"info","ts":1621954788.961414,"logger":"controllers.CertificateRequest","msg":"AWSPCA does not support CA certificates","certificaterequest":"opa/ca-qn2gk"}

The Certificate couldn't be created, it complains about that it can't find the issuer. Also I saw that aws-pca-issuer tried to look for an AWSPCAClusterIssuer which is - of course - doesn't exist as I created an Issuer only. Again there isn't anything PCA related in the certificate at all.

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: server
  namespace: opa
spec:
  commonName: OpenPolicyAgent Server
  dnsNames:
    - opa
    - opa.opa.svc
    - localhost
  duration: 43800h
  ipAddresses:
    - "::1"
    - 127.0.0.1
  isCA: false
  issuerRef:
    kind: Issuer
    name: opa-ca
  privateKey:
    algorithm: ECDSA
    encoding: PKCS8
    rotationPolicy: Always
    size: 384
  renewBefore: 168h
  secretName: tls-server
  subject:
    organizations:
      - blah
  usages:
    - server auth

The log I saw in the aws-pca-issuer pod:

2021-05-25T14:59:50.471630000Z {"level":"error","ts":1621954790.471369,"logger":"controllers.CertificateRequest","msg":"failed to retrieve Issuer resource","certificaterequest":"opa/server-x952x","error":"AWSPCAClusterIssuer.awspca.cert-manager.io \"opa-ca\" not found","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\ngithub.com/jniebuhr/aws-pca-issuer/pkg/controllers.(*CertificateRequestReconciler).Reconcile\n\t/workspace/pkg/controllers/certificaterequest_controller.go:150\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}
2021-05-25T14:59:50.485633000Z {"level":"error","ts":1621954790.4854755,"logger":"controller-runtime.manager.controller.certificaterequest","msg":"Reconciler error","reconciler group":"cert-manager.io","reconciler kind":"CertificateRequest","name":"server-x952x","namespace":"opa","error":"AWSPCAClusterIssuer.awspca.cert-manager.io \"opa-ca\" not found","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:302\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

When I scale down the aws-pca-issuer deployment to zero and re-create the certificate it works.

We're using v0.2.1 with disableApprovedCheck: true as we have cert-manager:1.2.0.

[Feature Request]: Alternative installation method to helm ie manifest to be applied using kubectl

Describe why this change is needed

Currently the only method to install the ca issuer is using helm and we would like to only use native kubectl to install it.

Describe solutions and alternatives considered (optional)

run helm with --dry-run -o yaml but this requires the installation of helm

Is there anything else you would like to add?

No response

no matches for kind "Certificate" in version "cert-manager.io/v1"

Authentication failing on EKS using IAM role assigned to service account

Setup

We're deploying aws-privateca-issuer via Helm (Image: ghcr.io/jniebuhr/aws-pca-issuer:v0.3.0) on an EKS cluster (k8s v1.20).
I'm trying to pass credentials to the aws-privateca-issuer by using an IAM role for its service account, following this AWS guide.
When I describe the pod, I can see that the role has been assigned successfully and the credentials are mounted on the pod:

    Environment:
      AWS_DEFAULT_REGION:           ap-southeast-1
      AWS_REGION:                   ap-southeast-1
      AWS_ROLE_ARN:                 arn:aws:iam::123456789012:role/correct-role
      AWS_WEB_IDENTITY_TOKEN_FILE:  /var/run/secrets/eks.amazonaws.com/serviceaccount/token
    Mounts:
      /var/run/secrets/eks.amazonaws.com/serviceaccount from aws-iam-token (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from aws-pca-issuer-1628064934-aws-privateca-issuer-token-mmrrp (ro)

Problem

aws-privateca-issuer does not seem to support this setup, as in the logs, it's complaining about a missing token:

{"level":"error","ts":1628242137.8225927,"logger":"controller-runtime.manager.controller.awspcaclusterissuer","msg":"Reconciler error","reconciler group":"awspca.cert-manager.io","reconciler kind":"AWSPCAClusterIssuer","name":"poc-ca","namespace":"","error":"operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: ae89e38d-60c9-4b8c-9280-f81052238788, api error MissingAuthenticationToken: Request is missing Authentication Token","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:302\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

key part:
operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: ae89e38d-60c9-4b8c-9280-f81052238788, api error MissingAuthenticationToken: Request is missing Authentication Token

Question

Is this supported in general? The readme states that:

Access to AWS can also be configured using an EC2 instance role.

How would we go about doing that?

[Bug]: Image Tag in Helm Chart doesn't match Contianer Image Tag

Describe the expected outcome

aws-privateca-issuer pods are able to pull correct image at correct image tag, without having to specify an image tag.
public.ecr.aws/k1n1h4h4/cert-manager-aws-privateca-issuer:v1.2.4 vs public.ecr.aws/k1n1h4h4/cert-manager-aws-privateca-issuer:1.2.4

Describe the actual outcome

aws-privateca-issuer pods attempt to pull image public.ecr.aws/k1n1h4h4/cert-manager-aws-privateca-issuer:1.2.4, which fails.

Steps to reproduce

No response

Relevant log output

No response

Version

1.2.4

Have you tried the following?

Check the Troubleshooting section
Search open issues

Category

Other

Severity

Severity 3

Support client/server key usages on issued certificates

Usage case:

I would like to use aws-pca-issuer to issue certificates that can only be used for TLS clients or servers.

Expected behavior:

Specify KeyUsage: [ "server auth" ] or KeyUsage: [ "client auth" ] on a certificate resource and be issued a certificate that is limited to those usages.

Actual behavior:

KeyUsage on the certificate resource is ignored and a certificate is issued with both client and server key usages.

This is because no TemplateArn is specified to IssueCertificate() so all certificates use the default EndEntityCertificate template which includes both client and server key usages.

References:

AWS region is required, but was not found

We're deploying aws-privateca-issuer using Helm on EKS 1.18 with the following config but somehow it is unable to read the region from AWSPCAClusterIssuer spec

NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                           APP VERSION
awspca          cert-manager    1               2021-09-22 23:39:06.1956528 +0530 IST   deployed        aws-privateca-issuer-0.1.2      v0.3.1

Deploying the issuer using the following config
Note: We are using secrets and not KIAM

apiVersion: awspca.cert-manager.io/v1beta1
kind: AWSPCAClusterIssuer
metadata:
  name: awspca
  namespace: cert-manager
spec:
  region: us-east-1
  arn: arn:aws:acm-pca:us-east-1:xxxxxxxxxx:certificate-authority/xxxxxxx-xxxx-xxxx-9526-xxxxxxxxxx
  secretRef:
    namespace: cert-manager
    name: pca-access

ERROR

{"level":"error","ts":1632339059.6446667,"logger":"controller-runtime.manager.controller.awspcaclusterissuer","msg":"Reconciler error","reconciler group":"awspca.cert-manager.io","reconciler kind":"AWSPCAClusterIssuer","name":"awspca","namespace":"","error":"operation error STS: GetCallerIdentity, failed to resolve service endpoint, an AWS region is required, but was not found","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:302\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

Everything starts working when we pass AWS_REGION into the deployment container env variables


    spec:
      containers:
      - args:
        - --leader-elect
        command:
        - /manager
        env:
        - name: AWS_REGION
          value: us-east-1
        image: ghcr.io/jniebuhr/aws-pca-issuer:v0.3.1
        imagePullPolicy: IfNotPresent

issuer fails to validate

Any idea why the issuer would never validate?

Here are my manifests:

apiVersion: awspca.cert-manager.io/v1beta1
kind: AWSPCAIssuer
metadata:
  name: awspcaissuer
  namespace: cert-manager
spec:
  arn: {{ .Values.pcaArn }}
  region: {{ .Values.region }}
---
apiVersion: v1
kind: Namespace
metadata:
  name: {{ .Chart.Name }}
  labels:
    name: {{ .Chart.Name }}
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
  annotations:
    eks.amazonaws.com/role-arn: {{ .Values.roleArn }}
  name: {{ .Chart.Name }}
  namespace: {{ .Chart.Name }}

Here are the values I passed to the aws-pricateca-issuer helm chart and cert-manager helm chart:

cert-manager:
  installCRDs: true
  serviceAccount:
    create: false
    name: cert-manager
aws-privateca-issuer:
  serviceAccount:
    create: false
    name: cert-manager

Here's the output when I try to describe the issuer:

$ kubectl describe -n cert-manager awspcaissuers.awspca.cert-manager.io

Name:         awspcaissuer
Namespace:    cert-manager
Labels:       argocd.argoproj.io/instance=cert-manager
Annotations:  <none>
API Version:  awspca.cert-manager.io/v1beta1
Kind:         AWSPCAIssuer
Metadata:
  Creation Timestamp:  2021-08-03T14:23:45Z
  Generation:          1
  Managed Fields:
    API Version:  awspca.cert-manager.io/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
        f:labels:
          .:
          f:argocd.argoproj.io/instance:
      f:spec:
        .:
        f:arn:
        f:region:
    Manager:         argocd-application-controller
    Operation:       Update
    Time:            2021-08-03T14:23:45Z
  Resource Version:  45566948
  Self Link:         /apis/awspca.cert-manager.io/v1beta1/namespaces/cert-manager/awspcaissuers/awspcaissuer
  UID:               d5a200ab-0ff9-4f77-858d-1d6e89c2b0f7
Spec:
  Arn:     arn:aws:acm-pca:us-east-1:734116470960:certificate-authority/3c01fb76-eced-49a2-8076-3bf2110807a0
  Region:  us-east-1
Events:    <none>

You'll see no events ever fire, including the one where the issuer is validated.

Here's my certificate:

$ kubectl describe -A certificate
Name:         cms-devops
Namespace:    default
Labels:       argocd.argoproj.io/instance=cms-devops
Annotations:  <none>
API Version:  cert-manager.io/v1
Kind:         Certificate
Metadata:
  Creation Timestamp:  2021-08-03T13:09:57Z
  Generation:          1
  Managed Fields:
    API Version:  cert-manager.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
        f:labels:
          .:
          f:argocd.argoproj.io/instance:
      f:spec:
        .:
        f:commonName:
        f:dnsNames:
        f:duration:
        f:issuerRef:
          .:
          f:group:
          f:kind:
          f:name:
        f:privateKey:
          .:
          f:algorithm:
          f:size:
        f:renewBefore:
        f:secretName:
        f:usages:
    Manager:      argocd-application-controller
    Operation:    Update
    Time:         2021-08-03T13:09:57Z
    API Version:  cert-manager.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:conditions:
        f:lastFailureTime:
    Manager:         controller
    Operation:       Update
    Time:            2021-08-03T14:09:59Z
  Resource Version:  45559101
  Self Link:         /apis/cert-manager.io/v1/namespaces/default/certificates/cms-devops
  UID:               884b894b-acf8-4136-8075-d0719934055b
Spec:
  Common Name:  cms-devops.william-sandbox.internal
  Dns Names:
    cms-devops.william-sandbox.internal
  Duration:  2160h0m0s
  Issuer Ref:
    Group:  awspca.cert-manager.io
    Kind:   AWSPCAClusterIssuer
    Name:   root-ca
  Private Key:
    Algorithm:   RSA
    Size:        2048
  Renew Before:  360h0m0s
  Secret Name:   cms-devops-tls-app-secret
  Usages:
    server auth
    client auth
Status:
  Conditions:
    Last Transition Time:  2021-08-03T14:09:59Z
    Message:               The certificate request has failed to complete and will be retried: issuer is not ready
    Observed Generation:   1
    Reason:                Failed
    Status:                False
    Type:                  Issuing
    Last Transition Time:  2021-08-03T13:09:57Z
    Message:               Issuing certificate as Secret does not exist
    Observed Generation:   1
    Reason:                DoesNotExist
    Status:                False
    Type:                  Ready
  Last Failure Time:       2021-08-03T14:09:59Z
Events:
  Type     Reason     Age                From          Message
  ----     ------     ----               ----          -------
  Normal   Issuing    36m (x2 over 96m)  cert-manager  Issuing certificate as Secret does not exist
  Warning  Failed     36m (x2 over 96m)  cert-manager  The certificate request has failed to complete and will be retried: issuer is not ready
  Normal   Generated  36m                cert-manager  Stored new private key in temporary Secret resource "cms-devops-pj7gp"
  Normal   Requested  36m                cert-manager  Created new CertificateRequest resource "cms-devops-2dfh5"

You can see that the the certificate never completes because the issuer is not ready. Is something misconfigured?

Known Issue: Certificates with Validity Duration under 24h failing to Issue

There is currently an issue where certificate requests with validity duration < 24h will fail to issue because of a division by 24 and cast to int64, resulting in 0 for the validity field. This results in an api error from PCA because PCA requires validity is not 0.

Please see #69 for more details.

While we work to cut a release to fix this change (#70), please use validity durations greater than 24h.

[Feature Request]: Have tls server name (sni) set for outbound https connections

Describe why this change is needed

In an Istio enabled environment when egress filtering is enabled, Istio uses the hostname / sni to do egress hostname matching.

If there is no tls server name / sni then Istio can't match the oubound tcp port 443 connection so it would block it.

Describe solutions and alternatives considered (optional)

Istio sidecar resource can allow all but that defeats the purpose of having Istio perform egress filtering.

Is there anything else you would like to add?

No response

Dependency Dashboard

This issue provides visibility into Renovate updates and their statuses. Learn more

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Update module k8s.io/client-go to v1

Check this box to trigger a request for Renovate to run again on this repository

MissingAuthenticationToken: Request is missing Authentication Token

I wanted to upgrade to the latest v0.3.0 but when I tried I noticed I started seeing exceptions in the logs.

We are using AWS IAM roles as annotations on the serviceAccount to handle the authorization. This works in v0.2.1 and I've looked through the code but I see no differences between 0.2.1 and 0.3.0 that would cause this.

{"level":"error","ts":1626803861.6337998,"logger":"controllers.GenericIssuer","msg":"failed to sts.GetCallerIdentity","genericissuer":"/aws-issuer","error":"operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: dd011a20-5b5f-4191-9996-cd709af162f9, api error MissingAuthenticationToken: Request is missing Authentication Token","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\ngithub.com/cert-manager/aws-privateca-issuer/pkg/controllers.(*GenericIssuerReconciler).Reconcile\n\t/workspace/pkg/controllers/genericissuer_controller.go:122\ngithub.com/cert-manager/aws-privateca-issuer/pkg/controllers.(*AWSPCAClusterIssuerReconciler).Reconcile\n\t/workspace/pkg/controllers/awspcaclusterissuer_controller.go:57\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}
{"level":"error","ts":1626803861.6339357,"logger":"controller-runtime.manager.controller.awspcaclusterissuer","msg":"Reconciler error","reconciler group":"awspca.cert-manager.io","reconciler kind":"AWSPCAClusterIssuer","name":"aws-issuer","namespace":"","error":"operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: dd011a20-5b5f-4191-9996-cd709af162f9, api error MissingAuthenticationToken: Request is missing Authentication Token","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:302\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

CertificateRequest resource shows no events

I installed aws-privateca-issuer as per documentation.
I created a new AWSPCAIssuer, everything seems to be fine, all resources are ready, no issues reported.

Status:
  Conditions:
    Last Transition Time:  2021-06-27T22:06:19Z
    Message:               Issuer verified
    Reason:                Verified
    Status:                True
    Type:                  Ready
Events:
  Type    Reason    Age   From                     Message
  ----    ------    ----  ----                     -------
  Normal  Verified  38m   awspcaissuer-controller  Issuer verified
  Normal  Verified  11m   awspcaissuer-controller  Issuer verified

When I create a Certificate using my AWSPCAIssuer, the CertificateRequest is generated, but there are no events logged and it is stuck in unknown state.

Any hints how to start to debug/resolve this?

Access via IAM roles for service account

Describe why this change is needed

Currently awspca configuration needs creating a dedicated IAM user with proper access and define a secret with access key - but since it is used in EKS it would be better to use dedicated service account with assigned IAM policy. It would be more natural way I think to integrate it. How do you think? (https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html)

Describe solutions and alternatives considered (optional)

No response

Is there anything else you would like to add?

No response

[Feature Request]: List the chart repository on artifacthub.io

Describe why this change is needed

My team finds it helpful to be able to search for helm charts on artifacthub.io, which also displays version information and security reports for the images used by a chart. The main chart for cert-manager is already listed, however I believe that published via a separate repository. Could you submit the repository for this plugin to artifacthub.io also?

Describe solutions and alternatives considered (optional)

We can continue to just directly look at this repository for chart information and updates, so this is a nice-to-have, not essential.

Is there anything else you would like to add?

No response

aws-pca-issuer doesn't work in Kubernetes versions bellow 1.16

Most probably this is caused by the version of apiextensions.k8s.io because it looks like the go code cannot identify CRDs:

2021-07-30T14:31:32.031Z INFO controllers.GenericIssuer sts.GetCallerIdentity {"genericissuer": "/test", "arn": "arn:aws:sts::410807025456:assumed-role/k8s_certissuer_awsprivateca/1627655491949926463", "account": "410807025456", "user_id": "AROAV7JQBX4YAPTSZ6F75:1627655491949926463"}
2021-07-30T14:31:32.031Z INFO controllers.GenericIssuer Calling StoreProvisioner {"genericissuer": "/test"}
2021-07-30T14:31:32.032Z DEBUG controller-runtime.manager.events Normal {"object": {"kind":"AWSPCAClusterIssuer","name":"test","uid":"3c9fd077-f103-11eb-b462-067082f90d8f","apiVersion":"awspca.cert-manager.io/v1beta1","resourceVersion":"89368672"}, "reason": "Verified", "message": "Issuer verified"}
2021-07-30T14:31:32.033Z ERROR controller-runtime.manager.controller.awspcaclusterissuer Reconciler error {"reconciler group": "awspca.cert-manager.io", "reconciler kind": "AWSPCAClusterIssuer", "name": "test", "namespace": "", "error": "the server could not find the requested resource (put awspcaclusterissuers.awspca.cert-manager.io test)"}
github.com/go-logr/zapr.(*zapLogger).Error
/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:302
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.UntilWithContext
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99