Giter Club home page Giter Club logo

aws-privateca-issuer's Introduction

Logo

Build Status Build Status Latest version

AWS Private CA Issuer

AWS Private CA is an AWS service that can setup and manage private CAs, as well as issue private certifiates.

cert-manager is a Kubernetes add-on to automate the management and issuance of TLS certificates from various issuing sources. It will ensure certificates are valid and up to date periodically, and attempt to renew certificates at an appropriate time before expiry.

This project acts as an addon (see https://cert-manager.io/docs/configuration/external/) to cert-manager that signs off certificate requests using AWS Private CA.

Setup

Install cert-manager first (https://cert-manager.io/docs/installation/kubernetes/).

Then install AWS PCA Issuer using Helm:

helm repo add awspca https://cert-manager.github.io/aws-privateca-issuer
helm install awspca/aws-privateca-issuer --generate-name

You can check the chart configuration in the default values file.

Configuration

As of now, the only configurable settings are access to AWS. So you can use AWS_REGION, AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY.

Alternatively, you can supply arbitrary secrets for the access and secret keys with the accessKeyIDSelector and secretAccessKeySelector fields in the clusterissuer and/or issuer manifests.

Access to AWS can also be configured using an EC2 instance role or [IAM Roles for Service Accounts] (https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html).

A minimal policy to use the issuer with an authority would look like follows:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "awspcaissuer",
      "Action": [
        "acm-pca:DescribeCertificateAuthority",
        "acm-pca:GetCertificate",
        "acm-pca:IssueCertificate"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:acm-pca:<region>:<account_id>:certificate-authority/<resource_id>"
    }
  ]
}

Usage

This operator provides two custom resources that you can use.

Examples can be found in the examples and samples directories.

AWSPCAIssuer

This is a regular namespaced issuer that can be used as a reference in your Certificate CRs.

AWSPCAClusterIssuer

This CR is identical to the AWSPCAIssuer. The only difference being that it's not namespaced and can be referenced from anywhere.

Usage with cert-manager Ingress Annotations

The cert-manager.io/cluster-issuer annotation cannot be used to point at a AWSPCAClusterIssuer. Instead, use cert-manager.io/issuer:. Please see this issue for more information.

Disable Approval Check

The AWSPCA Issuer will wait for CertificateRequests to have an approved condition set before signing. If using an older version of cert-manager (pre v1.3), you can disable this check by supplying the command line flag -disable-approved-check to the Issuer Deployment.

Authentication

Please note that if you are using KIAM for authentication, this plugin has been tested on KIAM v4.0. IRSA is also tested and supported.

There is a custom AWS authentication method we have coded into our plugin that allows a user to define a Kubernetes secret with AWS Creds passed in, example here. The user applies that file with their creds and then references the secret in their Issuer CRD when running the plugin, example here.

Supported workflows

AWS Private Certificate Authority(PCA) Issuer Plugin supports the following integrations and use cases:

Mapping Cert-Manager Usage Types to AWS PCA Template Arns

The code for the translation can be found here.

Depending on which UsageTypes are set in the Cert-Manager certificate, different AWS PCA templates will be used. This table shows how the UsageTypes are being translated into which template to use when making an IssueCertificate request:

Cert-Manager Usage Type(s) AWS PCA Template ARN
CodeSigning acm-pca:::template/CodeSigningCertificate/V1
ClientAuth acm-pca:::template/EndEntityClientAuthCertificate/V1
ServerAuth acm-pca:::template/EndEntityServerAuthCertificate/V1
OCSPSigning acm-pca:::template/OCSPSigningCertificate/V1
ClientAuth, ServerAuth acm-pca:::template/EndEntityCertificate/V1
Everything Else acm-pca:::template/BlankEndEntityCertificate_CSRPassthrough/V1

Understanding/Running the tests

Running the Unit Tests

Running make test will run the written unit test

If you run into an issue like

/home/linuxbrew/.linuxbrew/Cellar/go/1.17/libexec/src/net/cgo_linux.go:13:8: no such package located

This can be fixed with a

brew install gcc@5

Running the End-To-End Tests

NOTE: Running these tests will incur charges in your AWS Account.

Running make e2etest will take the current code artifacts and transform them into a Docker image that will run on a kind cluster and ensure that the current version of the code still works with the Supported Workflows

The easiest way to get the test to run would be to use the follow make targets: make cluster && make install-eks-webhook && make e2etest

Getting make cluster to run

make cluster will create a kind cluster on your machine that has Cert-Manager installed as well as the aws-pca-issuer plugin (using the HEAD of the current branch)

Before running make cluster we will need to do the following:

- Have the following tools on your machine:

- (Optional) You will need a AWS IAM User to test authentication via K8 secrets. You can provide an already existing user into the test via export PLUGIN_USER_NAME_OVERRIDE=<IAM User Name>. This IAM User should have a policy attached to it that follows with the policy listed in Configuration. This user will be used to test authentication in the plugin via K8 secrets.

- An S3 Bucket with BPA disabled in us-east-1. After creating the bucket run export OIDC_S3_BUCKET_NAME=<Name of bucket you just created>

- You will need AWS credentials loaded into your terminal that, via the CLI, minimally allow the following actions via an IAM policy:

  • acm-pca:* : This is so that Private CA's maybe be created and deleted via the appropriate APIs for testing
  • If you did not provider a user via PLUGIN_USER_NAME_OVERRIDE, the test suite can create a user for you. This will require the following permissions: iam:CreatePolicy,iam:CreateUser, and iam:AttachUserPolicy
  • iam:CreateAccessKey and iam:DeleteAccessKey: This allow us to create and delete access keys to be used to validate that authentication via K8 secrets is functional. If the user was set via $PLUGIN_USER_NAME_OVERRIDE
  • s3:PutObject and s3::PutObjectAcl these can be scoped down to the s3 bucket you created above

- An AWS IAM OIDC Provider. Before creating the OIDC provider, set a temporary value for $OIDC_IAM_ROLE (export OIDC_IAM_ROLE=arn:aws:iam::000000000000:role/oidc-kind-cluster-role and run make cluster && make install-eks-webhook && make kind-cluster-delete). This needs to be done otherwise you may see an error complaining about the absence of a file .well-known/openid-configuration. Running these commands helps bootstrap the S3 bucket so that the OIDC provider can be created. Set the provider url of the OIDC provider to be $OIDC_S3_BUCKET_NAME.s3.us-east-1.amazonaws.com/cluster/my-oidc-cluster. Set the audience to be sts.amazonaws.com.

- An IAM role that has a trust relationship with the IAM OIDC Provider that was just created. An inline policy for this role can be grabbed from Configuration except you can't scope it to a particular CA since those will be created during the test run. This role will be used to test authentication in the plugin via IRSA. The trust relationship should look something like:

{  
  "Version": "2012-10-17",  
  "Statement": [  
	{  
      "Effect": "Allow",  
	  "Principal": {  
	    "Federated": "${OIDC_ARN}"  
	   },  
	   "Action": "sts:AssumeRoleWithWebIdentity",  
	   "Condition": {  
	     "StringEquals": {  
	       "${OIDC_URL}:sub": "system:serviceaccount:aws-privateca-issuer:aws-privateca-issuer-sa"  
	     }  
	   }  
	 }  
   ]  
}

After creating this role run export OIDC_IAM_ROLE=<IAM role arn you created above>

- make cluster recreate the cluster with all the appropriate enviornment variables set

- make install-eks-webhook will install a webhook in that kind cluster that will enable the use of IRSA

- make e2etest will run end-to-end test against the kind cluster created via make cluster.

- After you update controller code locally, an easy way to redeploy the new controller code and re-run end-to-end test is to run:: make upgrade-local && make e2etest

Getting IRSA to work on Kind was heavily inspired by the following blog: https://reece.tech/posts/oidc-k8s-to-aws/

If you want to also test that cross account issuers are working, you will need:

- A seperate AWS account that has a role that trust the caller who kicks off the end-to-end test via the CLI, the role will need a policy with the following permissions

  • acm-pca:*: This is so the test can create a Private CA is the other account
  • ram:GetResourceShareAssociations, ram:CreateResourceShare, and ram:DeleteResourceShare: These allow the creation of a CA that can be shared with the source (caller) account
  • After creating this role you will need to run export PLUGIN_CROSS_ACCOUNT_ROLE=<name of the role you created above>. If you do not do this, you will see a message about cross account testing being skipped due to this enviornment variable not being set.

Soon these test should be automatically run on each PR, but for the time being each PR will have a core-collaborator for the project run the tests manually to ensure no regressions on the supported workflows

Contributing to the End-to-End test

The test are fairly straightforward, they will take a set of "issuer templates" (Base name for a aws-pca-issuer as well as a AWSIssuerSpec) and a set of "certificate templates" (Base name for type of certificate as well as a certificate spec). The tests will then take every certificate spec and apply it to each issuer spec. The test will ensure all issuers made from issuer specs reach a ready state as well as ensure that each certificate issued off a issuer reaches a ready state. The issuers with the different certificates is verified to be working for both cluster and namespace issuers.

For the most part, updating end-to-end will be updating these "issuer specs" and "certificate specs" which reside within e2e/e2e_test.go. If the test need updating beyond that, the core logic for the test is also embedded in e2e/e2e_test.go. The other files within the e2e folder are mainly utilities that shouldn't require frequent update

Other Tests

  1. Test to ensure that the workflow laid out in the blog Setting up end-to-end TLS encryption on Amazon EKS with the new AWS Load Balancer Controller is functional. To run the test: make cluster && make install-eks-webhook && make blog-test

  2. Test that pulls down the latest release via Helm, checks that the plugin was installed correctly, with the correct version, then gets deleted correctly. To run the test make cluster && make install-eks-webhook && make helm-test

Troubleshooting

  1. Check the secret with the AWS credentials: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY values have to be base64 encoded.

  2. If the generated CertificateRequest shows no events, it is very likely that you're using an older version of cert-manager which doesn't support approval check. Disable approval check at the issuer deployment.

Help & Feedback

For help, please consider the following venues (in order):

Contributing

We welcome community contributions and pull requests.

See our contribution guide for more information on how to report issues, set up a development environment, and submit code.

We adhere to the Amazon Open Source Code of Conduct.

aws-privateca-issuer's People

Contributors

abohne avatar amibhi avatar anbaig avatar aveega avatar baiakbar avatar bmsiegel avatar bpotaczek avatar bryanstenson-okta avatar cert-manager-prow[bot] avatar cnatter avatar dave-shawley avatar dependabot[bot] avatar divyansh-gupta avatar hamidhasan avatar jakexks avatar jetstack-bot avatar jniebuhr avatar jongwooo avatar joshvanl avatar karlhungus avatar kit837 avatar kollabpr avatar marshields avatar paramsethi avatar renovate-bot avatar renovate[bot] avatar secustor avatar solonish avatar thatarchguy avatar thatsmrtalbot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aws-privateca-issuer's Issues

[Bug]: Using the default version causes image pull failures

Describe the expected outcome

When using the chart without specifying a version number, the latest version should be deployed.

Describe the actual outcome

The chart attempts to pull a malformed image version and the deployment fails.

The image it's trying to pull is: cert-manager-aws-privateca-issuer:vv1.2.3

There's code here to specify the default version should be prefixed with a v
The helm chart downloads with a v in the version number by default:

Chart.yaml:

appVersion: v1.2.3

Hence 2 v's.

The workaround is to specify the chart version explicitly

Steps to reproduce

  1. Attempt to use the chart but do not specifying a version to deploy the latest version of the aws-privateca-issuer.
  2. Deployment should fail pulling the image due to the malformed version

Relevant log output

No response

Version

1.2.3

Have you tried the following?

Category

Supported Workflow Broken

Severity

Severity 2

aws-pca-issuer master doesn't work with kiam

As you requested in #24 I compiled the latest code (jniebuhr/aws-pca-issuer@07e5bed) and deployed it to our test cluster. The original issue was resolved but switching the AWS SDK 2 broke the functionality when KIAM or some similar is in use which is quite common in the AWS world. I think this can be caused the AWS SDK 2 by default enforces to use IMDSv2. The 0.2.1 release doesn't suffer from this nor any of our other "KIAM-handled" services. The KIAM is configured to allow access to every metadata endpoints.

The logs from the aws-pca-issuer:

2021-05-27T09:07:37.731866000Z {"level":"error","ts":1622106457.731583,"logger":"controllers.GenericIssuer","msg":"failed to sts.GetCallerIdentity","genericissuer":"/acmpca","error":"operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: 7a2a4f4b-3c04-4207-9601-e5986b0140dc, api error MissingAuthenticationToken: Request is missing Authentication Token","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\ngithub.com/jniebuhr/aws-pca-issuer/pkg/controllers.(*GenericIssuerReconciler).Reconcile\n\t/workspace/pkg/controllers/genericissuer_controller.go:122\ngithub.com/jniebuhr/aws-pca-issuer/pkg/controllers.(*AWSPCAClusterIssuerReconciler).Reconcile\n\t/workspace/pkg/controllers/awspcaclusterissuer_controller.go:57\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}
2021-05-27T09:07:37.732050000Z {"level":"error","ts":1622106457.7316792,"logger":"controller-runtime.manager.controller.awspcaclusterissuer","msg":"Reconciler error","reconciler group":"awspca.cert-manager.io","reconciler kind":"AWSPCAClusterIssuer","name":"acmpca","namespace":"","error":"operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: 7a2a4f4b-3c04-4207-9601-e5986b0140dc, api error MissingAuthenticationToken: Request is missing Authentication Token","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:302\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

The uswitch/kiam#359 could be useful to understand the issue better.

The KIAM logs says that the requested IAM role was fetched successfully when the aws-pca-issuer pod was in the Pending phase - KIAM fetches roles in advance for performance reasons.

time="2021-05-27T09:04:54Z" level=info msg="fetched credentials" credentials.access.key=ASIATTQ4YQR7GCJ5EJLJ credentials.expiration="2021-05-27T10:01:01Z" credentials.role="<Role Name>" generation.metadata=0 pod.iam.role="<Role Name>" pod.name=cert-manager-acmpca-5f99df9b5c-x7tq5 pod.namespace=cert-manager pod.status.ip= pod.status.phase=Pending resource.version=18948857

As there isn't any shell in the container I can't easily exec into it and check what curl/aws says.

[Feature Request]: Unable to Issue a CA certificate with the desired pathlen constraint value

Describe the expected outcome

  • I created a Root CA in AWS Private CA with the template: RootCACertificate/V1
  • Created a Subordinate CA with the help of the same Root CA. Template used for Subordinate: SubordinateCACertificate_PathLen3/V1
  • Verified that the certificate of Subordinate CA has the pathlen:3 constraint i.e.
            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:3
  • Next, I created a CA certificate (using Certificate of cert-manager.io/v1). The generated CA certificate has the following constraints-
            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:0

Expected a CA certificate with pathlen:2

Describe the actual outcome

The generated CA certificate had a pathlen:0 constraint, instead of the expected pathlen:2 constraint.

Is this happening due to this section in pca.go?

	if spec.IsCA {
		return prefix + "acm-pca:::template/SubordinateCACertificate_PathLen0/V1"
	}

Steps to reproduce

  1. Create an EKS Cluster.

  2. Install Cert Manager v1.10.0

# https://cert-manager.io/docs/installation/helm/
CERT_MANAGER_VERSION=v1.10.0
helm repo add jetstack https://charts.jetstack.io
helm repo update

helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version ${CERT_MANAGER_VERSION} \
  --set installCRDs=true \
  --wait
  1. Create a Root CA and a Subordinate CA in AWS Private CA
    You could run this script to create a Root CA and a Subordinate CA- https://raw.githubusercontent.com/find-arka/k8s-misc/main/create-ca-hierarchy-aws-pca.sh

With the output from the script, save the Intermediate CA ARN in an environment variable-

export CA_ARN=arn:aws:acm-pca:REDACTED:REDACTED-AC:certificate-authority/REDACTED
  1. Setup an IAM Policy to access the CA
cat <<EOF > AWSPCAIssuerPolicyTest.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "awspcaissuer",
      "Action": [
        "acm-pca:DescribeCertificateAuthority",
        "acm-pca:GetCertificate",
        "acm-pca:IssueCertificate"
      ],
      "Effect": "Allow",
      "Resource": "${CA_ARN}"
    }
  ]
}
EOF

POLICY_ARN=$(aws iam create-policy \
    --policy-name AWSPCAIssuerPolicyTest \
    --policy-document file://AWSPCAIssuerPolicyTest.json \
    --output json | jq -r '.Policy.Arn')

echo "POLICY_ARN = ${POLICY_ARN}"
  1. Setup a k8s Service Account and an associated IAM role to access the subordinate CA.
CURRENT_CLUSTER="Please put your cluster name here"
echo "${POLICY_ARN}"
echo "${REGION}"
echo "${CURRENT_CLUSTER}"

# Currently, we are installing the plugin in the same namespace as cert-manager
export PCA_NAMESPACE=cert-manager
# latest version https://github.com/cert-manager/aws-privateca-issuer/releases
export AWSPCA_ISSUER_TAG=v1.2.2

# Enable the IAM OIDC Provider for the cluster
eksctl utils associate-iam-oidc-provider \
    --cluster=${CURRENT_CLUSTER} \
    --region=${REGION} \
    --approve;

# Create IAM role bound to a service account
eksctl create iamserviceaccount --cluster=${CURRENT_CLUSTER} \
    --region=${REGION} \
    --namespace=${PCA_NAMESPACE} \
    --attach-policy-arn=${POLICY_ARN} \
    --override-existing-serviceaccounts \
    --tags "created-by=${USER},team=${TEAM},purpose=customer-support" \
    --name=aws-pca-issuer \
    --role-name "ServiceAccountRolePrivateCA-${CURRENT_CLUSTER}" \
    --approve;

# Install AWS Private CA Issuer Plugin 
# https://github.com/cert-manager/aws-privateca-issuer/#setup
helm repo add awspca https://cert-manager.github.io/aws-privateca-issuer
helm repo update
helm install aws-pca-issuer awspca/aws-privateca-issuer \
    --namespace ${PCA_NAMESPACE} \
    --set image.tag=${AWSPCA_ISSUER_TAG} \
    --set serviceAccount.create=false \
    --set serviceAccount.name=aws-pca-issuer \
    --kube-context ${CURRENT_CLUSTER} \
    --wait;

# Verify deployment status
kubectl --context ${CURRENT_CLUSTER} -n ${PCA_NAMESPACE} \
    rollout status deploy/aws-pca-issuer-aws-privateca-issuer;
  1. Create the Issuer-
# edit the var if your CA is in a different region
export CA_REGION="YOUR CA REGION"
export CA_ARN="arn:aws:acm-pca:redacted:redacted:certificate-authority/redacted"

cat << EOF | kubectl apply -f -
apiVersion: awspca.cert-manager.io/v1beta1
kind: AWSPCAClusterIssuer
metadata:
  name: my-cluster-issuer
spec:
  arn: ${CA_ARN}
  region: ${CA_REGION}
EOF
  1. Create a Certificate object
cat << EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: my-internal-ca
spec:
  commonName: my-internal-ca
  dnsNames:
    - "*.internal.my-org.ca"
  isCA: true
  duration: 2160h #90d
  secretName: my-internal-ca-cert-manager
  subject:
    organizations:
    - cluster.local
    - cert-manager
  issuerRef:
    group: awspca.cert-manager.io
    kind: AWSPCAClusterIssuer
    name: my-cluster-issuer
EOF

Verify:

kubectl get Certificate my-internal-ca

Expected output:

NAME             READY   SECRET                        AGE
my-internal-ca   True    my-internal-ca-cert-manager   4s
  1. extract the secret to read the value-
kubectl get secret my-internal-ca-cert-manager -o yaml | yq -r '.data."tls.crt"' | base64 -d > my-internal-ca-cert-manager-tls-crt.pem
kubectl get secret my-internal-ca-cert-manager -o yaml | yq -r '.data."ca.crt"' | base64 -d > my-internal-ca-cert-manager-ca-crt.pem

my-internal-ca-cert-manager-tls-crt.pem has the CA cert chained with the Issuer. Extract the top section from the pem file and copy it to a different file. I named it generated-ca-cert.pem

openssl x509 -in generated-ca-cert.pem -noout -text | grep -A3 Constraint
            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:0
            X509v3 Authority Key Identifier:
                5C:F9:5F:9D:CF:86:DD:56:94:64:36:C4:REDACTED
  1. Get the Subordinate cert and verify that the pathlen 3 constraint is present-
ROOT_CAARN="arn:aws:acm-pca:REDACTED:REDACTED:certificate-authority/REDACTED"
SUBORDINATE_CERTARN="arn:aws:acm-pca:REDACTED:REDACTED:certificate-authority/REDACTED/certificate/REDACTED"

aws acm-pca get-certificate \
    --certificate-authority-arn "${ROOT_CAARN}" \
    --certificate-arn "${SUBORDINATE_CERTARN}" \
    --output json | jq -r '.Certificate' > "intermediate-cert-common-purpose.pem"

openssl x509 -in intermediate-cert-common-purpose.pem -noout -text | grep -A3 Constraint
            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:3
            X509v3 Authority Key Identifier:
                0B:97:66:22:D3:3A:FF:7D:51:10:2F:46:D1:F8:E8:E9:1D:4E:64:CA

Subordinate CA cert has pathlen:3 but the generated CA cert from that CA cert doesn't have pathlen:2, instead it has pathlen:0

Relevant log output

N/A.
Have already attached the expected output along with the commands in the above section.

Version

Cert Manager -> v1.10.0
aws-privateca-issuer-> v1.2.2
Kubernetes -> 1.22
Amazon EKS platform version -> eks.6

Have you tried the following?

Category

Supported Workflow Broken

Severity

Severity 3

[Bug]: Helm chart use latest image instead of appVersion

Describe the expected outcome

Installing the Helm chart should use the image matching the appVersion.

This will allow a reproducible deployment.

Describe the actual outcome

The latest image get always used, which will lead to unexpected upgrade in case the pod gets restarted.

Steps to reproduce

helm template pca-issuer awspca/aws-privateca-issuer --version 1.2.0

Output:

...
     containers:
        - name: aws-privateca-issuer
          image: "public.ecr.aws/k1n1h4h4/cert-manager-aws-privateca-issuer:latest"
...

Note:
I'm aware of the possibility to set image.tag to "" in the values file to fallback to the Chart.AppVersion.

Relevant log output

No response

Version

= 1.0.0

Have you tried the following?

Category

Other

Severity

Severity 4

[Bug]: Image Tag in Helm Chart doesn't match Contianer Image Tag

Describe the expected outcome

aws-privateca-issuer pods are able to pull correct image at correct image tag, without having to specify an image tag.
public.ecr.aws/k1n1h4h4/cert-manager-aws-privateca-issuer:v1.2.4 vs public.ecr.aws/k1n1h4h4/cert-manager-aws-privateca-issuer:1.2.4

Describe the actual outcome

aws-privateca-issuer pods attempt to pull image public.ecr.aws/k1n1h4h4/cert-manager-aws-privateca-issuer:1.2.4, which fails.

Steps to reproduce

No response

Relevant log output

No response

Version

1.2.4

Have you tried the following?

Category

Other

Severity

Severity 3

AWS PCA shared cannot generate certificate

Currently the aws-privateca-issuer cannot mange to use AWS PCA shared from another account via RAM because when you share a PCA, acm-pca:IssueCertficate requests a condition: StringEquals acm-pca:TemplateArn: "arn:aws:acm-pca:::template/EndEntityCertificate/V1" and there is no way to have that permission without condition.

CertificateRequest resource shows no events

I installed aws-privateca-issuer as per documentation.
I created a new AWSPCAIssuer, everything seems to be fine, all resources are ready, no issues reported.

Status:
  Conditions:
    Last Transition Time:  2021-06-27T22:06:19Z
    Message:               Issuer verified
    Reason:                Verified
    Status:                True
    Type:                  Ready
Events:
  Type    Reason    Age   From                     Message
  ----    ------    ----  ----                     -------
  Normal  Verified  38m   awspcaissuer-controller  Issuer verified
  Normal  Verified  11m   awspcaissuer-controller  Issuer verified

When I create a Certificate using my AWSPCAIssuer, the CertificateRequest is generated, but there are no events logged and it is stuck in unknown state.

Any hints how to start to debug/resolve this?

[Bug]: Bash end of file errors with new testing

Describe the expected outcome

make cluster runs with no errors locally on MacOS running Bash 5.x

Describe the actual outcome

make cluster

RUNNING=$(docker inspect -f '{{.State.Running}}' "kind-registry" 2>/dev/null || true)
if [ "$RUNNING" != 'true' ]; then
bash: -c: line 2: syntax error: unexpected end of file

Steps to reproduce

MacOS 11.6

$ bash --version
bash --version
GNU bash, version 5.1.8(1)-release (x86_64-apple-darwin20.3.0)

Relevant log output

No response

Version

0.3.1 (Latest)

Have you tried the following?

Category

Broken Testing Infrastructure

Severity

Severity 4

Signature algorithm falsely assumes CA private key type

When we tried to deploy aws-pca-issuer and we hit by the following issue:

2021-05-25T11:33:33.261830000Z {"level":"error","ts":1621942413.261467,"logger":"controllers.CertificateRequest","msg":"failed to request certificate from PCA","certificaterequest":"opa/akunszt-test-9jl58","error":"InvalidArgsException: Signing algorithm SHA384WITHECDSA cannot be used with key type RSA_2048. Valid signing algorithms are: SHA256WITHRSA, SHA384WITHRSA, SHA512WITHRSA.","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\ngithub.com/jniebuhr/aws-pca-issuer/pkg/controllers.(*CertificateRequestReconciler).Reconcile\n\t/workspace/pkg/controllers/certificaterequest_controller.go:171\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

This was caused that the signatureAlgorithm function determines the signature algorithm from the certificate request, not from the signing CA.
https://github.com/jniebuhr/aws-pca-issuer/blob/f2e355d766e625d3c55b287d2a260bbdff5d61ff/pkg/aws/pca.go#L161

We have a CA in ACMPCA with an RSA_2048 key and we wanted to issue an ECDSA certificate. This doesn't work in aws-pca-issuer but it works if you properly set the signing algorithm.

We're using this to issue a certificate using ACMPCA from the command line - it's a snippet from a far bigger script, this is just the bare logic.

# the algorithm for the local key
LOCAL_CURVE_NAME="secp384r1"

# the algorithm used by the PCA to sign the certificate
# - if the PCA has an EC key then this should be SHA512WITHECDSA
# - it the PCA has an RSA key then this should be SHA512WITHRSA
PCA_SIGNING_ALGORITHM="SHA512WITHRSA"

### zillion lines of code to create a private key and a CSR, check a lot of things, etc...

aws acm-pca issue-certificate \
                        --certificate-authority-arn "$PCA_ARN" \
                        --csr file:///dev/stdin \
                        --region "$AWS_REGION" \
                        --signing-algorithm "$PCA_SIGNING_ALGORITHM" \
                        --template-arn "arn:aws:acm-pca:::template/$TEMPLATE" \
                        --validity "Value=$EXPIRES_IN,Type=DAYS" \
                        --query CertificateArn \
                        --output text < "csr.pem"

We use the same algorithm everywhere but for a general solution you can fetch the signing CA private key easily with:

aws acm-pca describe-certificate-authority --region "$REGION" --certificate-authority-arn "$ARN" --query "CertificateAuthority.CertificateAuthorityConfiguration.KeyAlgorithm" --output text

The aws-pca-issuer should fetch that and then use that information to construct the signing algorithm. Another solution could be to set that in the AWSPCAClusterIssuer/AWSPCAIssuer resources. It's rarely changes on it's own as it requires to recreate the CA on the AWS side.

A little background information why we wanted to do this:

You can use the ACMPCA to sign certificates in ACM only if the CA uses an RSA-2048 key. It's a known limitation of ACM at the moment. We have one regional CA - as they're quite costly - which we can use to issue certificates from the AWS web console in ACM or from Kubernetes using Certificate resources. This is why we have to use the RSA-2048 for the CA but we would like to use ECDSA for the certificates issued by the CA to make them a bit more future proof.

[Feature Request]: Create `Kind: AWSPCAClusterIssuers` resource

Describe why this change is needed

Save me from creating this with a local helm chart

Describe solutions and alternatives considered (optional)

An alternative is to install the current helm chart, and then kubectl apply -f this file, or helm install a local chart with this.

apiVersion: awspca.cert-manager.io/v1beta1
kind: AWSPCAClusterIssuer
metadata:
  name: demo-test-root-ca
spec:
  arn: arn:aws:acm-pca:us-east-2:snip:certificate-authority/snip
  region: us-east-2

Is there anything else you would like to add?

Original source of the above yaml https://aws.amazon.com/blogs/security/tls-enabled-kubernetes-clusters-with-acm-private-ca-and-amazon-eks-2/

[Feature Request]: can we support for arm platform?

Describe why this change is needed

we are migrating to arm platform, can we support for arm platform use multi-arch docker image?

Describe solutions and alternatives considered (optional)

No response

Is there anything else you would like to add?

No response

Known Issue: STS GetCallerIdentity failing because of a region not specified bug

There is currently a known issue with the plugin that is preventing certificate issuance due to STS GetCallerIdentity failing because of a region not specified bug, regardless of whether a region was specified or not (#54). There is an existing pull request to fix this (#53), but we are holding off on accepting any pull requests until our testing is redesigned. To fix this issue until then, please checkout the cleanup branch by running

git fetch -a
git checkout cleanup

Also, please be sure you are using the plugin with an IAM user, as that is the most reliable workflow https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html#Using_CreateAccessKey
This user must have minimum permissions listed here: https://github.com/cert-manager/aws-privateca-issuer#configuration

export AWS_SECRET_ACCESS_KEY=<Secret Access Key you generated>
export AWS_ACCESS_KEY_ID=<Access Key you generated>

CA Certificates are not currently supported via the PCA external issuer

As the code says at:
https://github.com/jniebuhr/aws-pca-issuer/blob/e5e5d80ede895ecc079ffa3bb3f2e0821b8d6c3b/pkg/controllers/certificaterequest_controller.go#L136

Is it an intentional limitation? Or is it still in progress? If yes than do you have any timeline for it?

We would like to issue a sub-ordinate CA signed by ACMPCA. We can do that using aws or the web console but it would be nice if aws-pca-issuer supports this too.

Create certificate secret with expected cert-manager content

As discussed in #10 the ca.crt contained the intermediate CA certificate that was used to issue the requested certificate. This was updated in #11 to have the requested certificate in tls.crt and intermediate + root CAs in ca.crt.

Based on cert-manager certificate documentation the expected format of the secret should be

When a certificate is issued by intermediates of the CA and the Issuer knows the intermediates, the content of tls.crt will be a resulting certificate followed by a certificate chain. The certificate chain doesnโ€™t include a root CA certificate, as it is stored in ca.crt.

I've created #14 as an attempt to address the above.

As an added bonus pem.Encode will add a \n after each certificate in ca.crt which fixes applications that expect a concatenated PEM file (tls.crt + ca.crt + tls.key). I'm running into this with the default OpenShift ingress controller (HAProxy) for example.

Consider bumping to v2 of the go SDK

In private EKS clusters, VPC endpoints are typically created for regional endpoints.

The AWS sdk in v1 defaults to using the global sts endpoint so the issuer doesn't work without setting an additional env var: AWS_STS_REGIONAL_ENDPOINTS: 'regional'. If using the v2 sdk it uses regional by default as documented here.

[Feature Request]: List the chart repository on artifacthub.io

Describe why this change is needed

My team finds it helpful to be able to search for helm charts on artifacthub.io, which also displays version information and security reports for the images used by a chart. The main chart for cert-manager is already listed, however I believe that published via a separate repository. Could you submit the repository for this plugin to artifacthub.io also?

Describe solutions and alternatives considered (optional)

We can continue to just directly look at this repository for chart information and updates, so this is a nice-to-have, not essential.

Is there anything else you would like to add?

No response

[Feature Request]: Remove region parameter on issuer

Describe why this change is needed

The region field on AWSPCA Issuers is redundent and only allows breaking the issuer.

I might have it wrong but it seems that the region must always match the region of the specified PCA arn.
Any region value that is not the region from the PCA arn, will always break.

Therefore it seems that having the parameter at all is a bug, as it's only function is to allow the user to break the issuer.

Describe solutions and alternatives considered (optional)

Controller should derive the API client region from the CA arn itself.

Is there anything else you would like to add?

Thank you for the very usefull tooling!!

[Bug]: AWS PCA Issuer Pod does not assume AWS IAM Role using Helm Deployment

Describe the expected outcome

When deploying the pod using a helm chart and manually creating the ServiceAccount, the pod should inherit the permissions assigned to the IAM role specified in the manually created ServiceAccount and be able to requested certificates.

Describe the actual outcome

The Pod appears to not inherit the correct permissions and fails to get the certificate.

cert-manager  The certificate request has failed to complete and will be retried: failed to request certificate from 
PCA: operation error ACM PCA: IssueCertificate, https response error StatusCode: 400, RequestID: e1e844fb-1af5-4d63-9872-bfe9e627537e, api error 
AccessDeniedException: User: arn:aws:sts::xxxxxx:assumed-role/xxxxxxxx/xxxxxx is not authorized to perform: acm-pca:IssueCertificate 
on resource: arn:aws:acm-pca:eu-west-2:xxxx:certificate-authority/xxxxx because no resource-based policy 
allows the acm-pca:IssueCertificate action

It appears that the pod has the correct ServiceAccount role as it is getting the token?

Screenshot 2021-12-22 at 10 40 10

Steps to reproduce

Deployment of the PCA Issuer Helm Chart using terraform

resource "helm_release" "eks_aws_pca" {
  name       = "${var.site_name}-aws-pca"
  chart      = "aws-privateca-issuer"
  repository = "https://cert-manager.github.io/aws-privateca-issuer"
  version    = var.aws_pca_helm_version
  namespace  = local.aws_pca_namespace

  set {
    name  = "installCRDs"
    value = "true"
  }

  set {
    name  = "image.repository"
    value = var.aws_privateca_issuer_image
  }

  set {
    name  = "image.tag"
    value = var.aws_privateca_issuer_tag
  }

  set {
    name  = "serviceAccount.create"
    value = false
  }

  set {
    name  = "serviceAccount.name"
    value = module.eks_aws_pca_irsa.service_account_name
  }

  set {
    name  = "approverRole.namespace"
    value = local.cert_manager_namespace
  }

  set {
    name  = "approverRole.serviceAccountName"
    value = local.cert_manager_service_account_name
  }

}

Deploying version 1.0.0 of the Helm Chart and Version 1.0.0 of the aws-privateca container

Relevant log output

No response

Version

v1.0.0

Have you tried the following?

Category

Authentication Issue

Severity

Severity 2

aws-pca-issuer tries to create non-PCA certificates

I don't know if it's an issue in cert-manager or in aws-pca-issuer. If it's the former then I'll open an issue there. I just experienced this only with aws-pca-issuer.

When aws-pca-issuer is running then it tries to handle Certificate resources even if the issuerRef points to a "plain" Issuer resource.

In the opa namespace I wanted to create a self-signed CA and a Certificate signed by that CA. The self-signed CA is created but aws-pca-issuer complains about that it doesn't support CA certificates. There aren't any reference to PCA in the YAML, so it shouldn't be even aware of it or it should be skipped silently.

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: ca
  namespace: opa
spec:
  commonName: OpenPolicyAgent CA
  duration: 87600h
  isCA: true
  issuerRef:
    kind: ClusterIssuer
    name: self-signed
  privateKey:
    algorithm: ECDSA
    encoding: PKCS8
    rotationPolicy: Always
    size: 384
  renewBefore: 168h
  secretName: tls-ca
  subject:
    organizations:
      - blah

The log I saw in the aws-pca-issuer.

2021-05-25T14:59:48.961908000Z {"level":"info","ts":1621954788.961414,"logger":"controllers.CertificateRequest","msg":"AWSPCA does not support CA certificates","certificaterequest":"opa/ca-qn2gk"}

The Certificate couldn't be created, it complains about that it can't find the issuer. Also I saw that aws-pca-issuer tried to look for an AWSPCAClusterIssuer which is - of course - doesn't exist as I created an Issuer only. Again there isn't anything PCA related in the certificate at all.

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: server
  namespace: opa
spec:
  commonName: OpenPolicyAgent Server
  dnsNames:
    - opa
    - opa.opa.svc
    - localhost
  duration: 43800h
  ipAddresses:
    - "::1"
    - 127.0.0.1
  isCA: false
  issuerRef:
    kind: Issuer
    name: opa-ca
  privateKey:
    algorithm: ECDSA
    encoding: PKCS8
    rotationPolicy: Always
    size: 384
  renewBefore: 168h
  secretName: tls-server
  subject:
    organizations:
      - blah
  usages:
    - server auth

The log I saw in the aws-pca-issuer pod:

2021-05-25T14:59:50.471630000Z {"level":"error","ts":1621954790.471369,"logger":"controllers.CertificateRequest","msg":"failed to retrieve Issuer resource","certificaterequest":"opa/server-x952x","error":"AWSPCAClusterIssuer.awspca.cert-manager.io \"opa-ca\" not found","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\ngithub.com/jniebuhr/aws-pca-issuer/pkg/controllers.(*CertificateRequestReconciler).Reconcile\n\t/workspace/pkg/controllers/certificaterequest_controller.go:150\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}
2021-05-25T14:59:50.485633000Z {"level":"error","ts":1621954790.4854755,"logger":"controller-runtime.manager.controller.certificaterequest","msg":"Reconciler error","reconciler group":"cert-manager.io","reconciler kind":"CertificateRequest","name":"server-x952x","namespace":"opa","error":"AWSPCAClusterIssuer.awspca.cert-manager.io \"opa-ca\" not found","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:302\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

When I scale down the aws-pca-issuer deployment to zero and re-create the certificate it works.

We're using v0.2.1 with disableApprovedCheck: true as we have cert-manager:1.2.0.

Referenced signer resource does not exist

I tried searching online for this error but I am unable to locate any, what exactly is the issue here and what should I do to resolve it?

"error"="admission webhook \"webhook.cert-manager.io\" denied the request: spec.issuerRef: Forbidden: referenced signer resource does not exist: {pca-issuer AWSPCAClusterIssuer awspca.cert-manager.io/v1beta1}"

[Bug]: Helm Chart fails linting due to whitespace

Describe the expected outcome

When performing linting using helm lint on the Helm Chart, linting should succeed. Additionally, to ensure issues do not occur with changes to the Helm Chart, linting should be included as part of the testing actions.

Describe the actual outcome

When performing linting using helm lint on the Helm Chart, the command fails due to an invalid YAML document separator. After debugging this further, this is caused by incorrect template directives causing issues with whitespace, particularly on line 15 in templates/rbac.yaml. The dash at the end of that line moves that line to the line above. Removing the right dash in the directive, fixes the issue.

Steps to reproduce

This can be reproduced by either using helm or helmfile binaries to performing the linting. If linting using the helmfile binary, the below steps are basically automated.

  1. Add the Helm repository
$ helm repo add awspca https://cert-manager.github.io/aws-privateca-issuer
$ helm repo update
  1. Pull the Helm Chart locally
$ helm pull awspca/aws-privateca-issuer -d ./awspca --untar
  1. Lint the chart
$ helm lint awspca/aws-privateca-issuer

Relevant log output

$ helm lint awspca/aws-privateca-issuer

==> Linting awspca/aws-privateca-issuer
[INFO] Chart.yaml: icon is recommended
[ERROR] templates/rbac.yaml: unable to parse YAML: invalid Yaml document separator: apiVersion: rbac.authorization.k8s.io/v1

Error: 1 chart(s) linted, 1 chart(s) failed

Version

Helm Chart Version: 1.2.2
Helm Application Version: 1.2.2

Have you tried the following?

Category

Build Issues

Severity

Severity 3

Add in unit testing

Currently working on adding unit testing, will update this issue when I complete.

[Known Issue]: Incorrect Certificate Validity for certificates with specified validity duration. (v1.0.0/v1.1.0)

During the release of v1.0.0 in Nov 2021, the plugin would add 30 days as the validity period for every certificate as a default for certificates with no validity duration specified, with this a bug was introduced which would add 30 days to the validity period on top of the specified validity duration; this was carried forward to v1.1.0 as well. This issue has been rectified in v1.2.0 (For more information please look in to PR. If the incorrect validity is a breaking issue then revoke the old certificates, move to version v1.2.0, and issue new certs.

Known Issue: Certificates with Validity Duration under 24h failing to Issue

There is currently an issue where certificate requests with validity duration < 24h will fail to issue because of a division by 24 and cast to int64, resulting in 0 for the validity field. This results in an api error from PCA because PCA requires validity is not 0.

Please see #69 for more details.

While we work to cut a release to fix this change (#70), please use validity durations greater than 24h.

MissingAuthenticationToken: Request is missing Authentication Token

I wanted to upgrade to the latest v0.3.0 but when I tried I noticed I started seeing exceptions in the logs.

We are using AWS IAM roles as annotations on the serviceAccount to handle the authorization. This works in v0.2.1 and I've looked through the code but I see no differences between 0.2.1 and 0.3.0 that would cause this.

{"level":"error","ts":1626803861.6337998,"logger":"controllers.GenericIssuer","msg":"failed to sts.GetCallerIdentity","genericissuer":"/aws-issuer","error":"operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: dd011a20-5b5f-4191-9996-cd709af162f9, api error MissingAuthenticationToken: Request is missing Authentication Token","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\ngithub.com/cert-manager/aws-privateca-issuer/pkg/controllers.(*GenericIssuerReconciler).Reconcile\n\t/workspace/pkg/controllers/genericissuer_controller.go:122\ngithub.com/cert-manager/aws-privateca-issuer/pkg/controllers.(*AWSPCAClusterIssuerReconciler).Reconcile\n\t/workspace/pkg/controllers/awspcaclusterissuer_controller.go:57\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}
{"level":"error","ts":1626803861.6339357,"logger":"controller-runtime.manager.controller.awspcaclusterissuer","msg":"Reconciler error","reconciler group":"awspca.cert-manager.io","reconciler kind":"AWSPCAClusterIssuer","name":"aws-issuer","namespace":"","error":"operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: dd011a20-5b5f-4191-9996-cd709af162f9, api error MissingAuthenticationToken: Request is missing Authentication Token","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:302\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

topologySpreadConstraints support in helm chart

Describe why this change is needed

It would be nice to have topologySpreadConstraints support in the official helm chart

/kind feature

Describe solutions and alternatives considered (optional)

No response

Is there anything else you would like to add?

No response

aws-pca-issuer doesn't work in Kubernetes versions bellow 1.16

Most probably this is caused by the version of apiextensions.k8s.io because it looks like the go code cannot identify CRDs:

2021-07-30T14:31:32.031Z INFO controllers.GenericIssuer sts.GetCallerIdentity {"genericissuer": "/test", "arn": "arn:aws:sts::410807025456:assumed-role/k8s_certissuer_awsprivateca/1627655491949926463", "account": "410807025456", "user_id": "AROAV7JQBX4YAPTSZ6F75:1627655491949926463"}
2021-07-30T14:31:32.031Z INFO controllers.GenericIssuer Calling StoreProvisioner {"genericissuer": "/test"}
2021-07-30T14:31:32.032Z DEBUG controller-runtime.manager.events Normal {"object": {"kind":"AWSPCAClusterIssuer","name":"test","uid":"3c9fd077-f103-11eb-b462-067082f90d8f","apiVersion":"awspca.cert-manager.io/v1beta1","resourceVersion":"89368672"}, "reason": "Verified", "message": "Issuer verified"}
2021-07-30T14:31:32.033Z ERROR controller-runtime.manager.controller.awspcaclusterissuer Reconciler error {"reconciler group": "awspca.cert-manager.io", "reconciler kind": "AWSPCAClusterIssuer", "name": "test", "namespace": "", "error": "the server could not find the requested resource (put awspcaclusterissuers.awspca.cert-manager.io test)"}
github.com/go-logr/zapr.(*zapLogger).Error
/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:302
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.UntilWithContext
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99

Operation error STS: GetCallerIdentity issue with Unit Tests issue with Region

Trying to run local unit tests but seeing this error in aws-privateca-issuer-controller-manager pod

{"level":"error","ts":1631654584.4164567,"logger":"controllers.GenericIssuer","msg":"failed to sts.GetCallerIdentity","genericissuer":"/pca-cluster-issuer-ec","error":"operation error STS: GetCallerIdentity, failed to resolve service endpoint, an AWS region is required, but was not found","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\ngithub.com/cert-manager/aws-privateca-issuer/pkg/controllers.(*GenericIssuerReconciler).Reconcile\n\t/workspace/pkg/controllers/genericissuer_controller.go:92\ngithub.com/cert-manager/aws-privateca-issuer/pkg/controllers.(*AWSPCAClusterIssuerReconciler).Reconcile\n\t/workspace/pkg/controllers/awspcaclusterissuer_controller.go:57\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

[Feature Request]: Have tls server name (sni) set for outbound https connections

Describe why this change is needed

In an Istio enabled environment when egress filtering is enabled, Istio uses the hostname / sni to do egress hostname matching.

If there is no tls server name / sni then Istio can't match the oubound tcp port 443 connection so it would block it.

Describe solutions and alternatives considered (optional)

Istio sidecar resource can allow all but that defeats the purpose of having Istio perform egress filtering.

Is there anything else you would like to add?

No response

Unable to issue certificate when namespace name + certificate name > 35 characters

Use case:

I would like to issue a certificate where the combined number of characters of the namespace and certificate name is more than 35 characters.

For example:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: my-very-long-certificate-name-that-has-quite-a-few-characters
  namespace: example

Expected behavior:

A certificate should be issued.

Actual behavior:

A certificate is not issued, and I receive an the following error in the logs for aws-privateca-issuer:

{"level":"error","ts":1624317003.908817,"logger":"controllers.CertificateRequest","msg":"failed to request certificate from PCA","certificaterequest":"example/my-very-long-certificate-name-that-has-quite-a-few-c-8qbnl","error":"operation error ACM PCA: IssueCertificate, https response error StatusCode: 400, RequestID: c73883c0-2ae8-42f1-9369-ca1705b36119, api error ValidationException: 1 validation error detected: Value example/my-very-long-certificate-name-that-has-quite-a-few-c-8qbnl at idempotencyToken failed to satisfy constraint: Member must have length less than or equal to 36","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error
        /go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132
github.com/cert-manager/aws-privateca-issuer/pkg/controllers.(*CertificateRequestReconciler).Reconcile
        /workspace/pkg/controllers/certificaterequest_controller.go:172
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.UntilWithContext
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

References:

Support client/server key usages on issued certificates

Usage case:

I would like to use aws-pca-issuer to issue certificates that can only be used for TLS clients or servers.

Expected behavior:

Specify KeyUsage: [ "server auth" ] or KeyUsage: [ "client auth" ] on a certificate resource and be issued a certificate that is limited to those usages.

Actual behavior:

KeyUsage on the certificate resource is ignored and a certificate is issued with both client and server key usages.

This is because no TemplateArn is specified to IssueCertificate() so all certificates use the default EndEntityCertificate template which includes both client and server key usages.

References:

Authentication failing on EKS using IAM role assigned to service account

Setup

We're deploying aws-privateca-issuer via Helm (Image: ghcr.io/jniebuhr/aws-pca-issuer:v0.3.0) on an EKS cluster (k8s v1.20).
I'm trying to pass credentials to the aws-privateca-issuer by using an IAM role for its service account, following this AWS guide.
When I describe the pod, I can see that the role has been assigned successfully and the credentials are mounted on the pod:

    Environment:
      AWS_DEFAULT_REGION:           ap-southeast-1
      AWS_REGION:                   ap-southeast-1
      AWS_ROLE_ARN:                 arn:aws:iam::123456789012:role/correct-role
      AWS_WEB_IDENTITY_TOKEN_FILE:  /var/run/secrets/eks.amazonaws.com/serviceaccount/token
    Mounts:
      /var/run/secrets/eks.amazonaws.com/serviceaccount from aws-iam-token (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from aws-pca-issuer-1628064934-aws-privateca-issuer-token-mmrrp (ro)

Problem

aws-privateca-issuer does not seem to support this setup, as in the logs, it's complaining about a missing token:

{"level":"error","ts":1628242137.8225927,"logger":"controller-runtime.manager.controller.awspcaclusterissuer","msg":"Reconciler error","reconciler group":"awspca.cert-manager.io","reconciler kind":"AWSPCAClusterIssuer","name":"poc-ca","namespace":"","error":"operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: ae89e38d-60c9-4b8c-9280-f81052238788, api error MissingAuthenticationToken: Request is missing Authentication Token","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:302\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

key part:
operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: ae89e38d-60c9-4b8c-9280-f81052238788, api error MissingAuthenticationToken: Request is missing Authentication Token

Question

Is this supported in general? The readme states that:

Access to AWS can also be configured using an EC2 instance role.

How would we go about doing that?

[Feature Request]: Integration with cert-manager 1.10

Describe why this change is needed

It appears that the AWS PCA Issuer integration with cert-manager version 1.10 does not quite work. After upgrading to cert-manager 1.10 and attempting to use the latest version of AWS PCA Cluster Issuer resources, these are the messages I see (this works with cert-manager 1.9.1):

E1121 15:03:27.073943       1 checks.go:55] cert-manager/controller/certificaterequests-issuer-selfsigned/handleSecretReference "msg"="failed to determine affected certificate requests" "error"="invalid value \"AWSPCAClusterIssuer\" for issuerRef.kind. Must be empty, \"Issuer\" or \"ClusterIssuer\"" "resource_kind"="Secret" "resource_name"="external-secrets-webhook" "resource_namespace"="my-platform" "resource_version"="v1"

Key thing to point out is "error"="invalid value \"AWSPCAClusterIssuer\" for issuerRef.kind. Must be empty, \"Issuer\" or \"ClusterIssuer\"".

Here is my certificate definition:

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: external-secrets-cert
  namespace: my-platform
spec:
  commonName: external-secrets-webhook.my-platform.svc
  dnsNames:
    - external-secrets-webhook.my-platform.svc
    - external-secrets-webhook
    - external-secrets-webhook.svc.cluster.local
    - external-secrets-webhook.cluster.local
    - localhost
  duration: 2160h0m0s
  ipAddresses:
    - 127.0.0.1
  issuerRef:
    kind: ClusterIssuer
    name: intermediate-ca
  renewBefore: 360h0m0s
  secretName: external-secrets-webhook

And my ClusterIssuer/AWS PCA ClusterIssuer:

---
apiVersion: awspca.cert-manager.io/v1beta1
kind: AWSPCAClusterIssuer
metadata:
  name: root-ca
spec:
  arn: REDACTED
  region: us-gov-west-1
---
kind: Certificate
apiVersion: cert-manager.io/v1
metadata:
  name: intermediate-ca
  namespace: my-platform
spec:
  commonName: intermediate-ca
  duration: 2160h0m0s
  isCA: true
  privateKey:
    algorithm: RSA
    size: 4096
  issuerRef:
    group: awspca.cert-manager.io
    kind: AWSPCAClusterIssuer
    name: root-ca
  renewBefore: 360h0m0s
  secretName: intermediate-ca
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: intermediate-ca
spec:
  ca:
    secretName: intermediate-ca

Describe solutions and alternatives considered (optional)

No response

Is there anything else you would like to add?

Looking for some guidance to see if this has been tested with 1.10.

[Bug]: Can's issue certificate from a shared private CA

Describe the expected outcome

AWSPCAClusterIssuer should be able to provision certificate

Describe the actual outcome

Our private CA authority is hosted in the networking account and has been shared with other accounts(let's call it dev-1) here. We can manually generate the certs by logging into the dev-1 account.

Now when I create an AWSPCAClusterIssuer, the CA arn I am specifying contains the network account's id (which is obvious), and when I try to create the cert using this certificate it errors out with the below error,

"level":"error","ts":1657793954.0080278,"logger":"controllers.CertificateRequest","msg":"failed to request certificate from PCA","certificaterequest":"default/example-com-f9v6w","error":"operation error ACM PCA: IssueCertificate, https response error StatusCode: 400, RequestID: b6cd6e9b-9e6a-42b9-a96f-38806a3ffffb, api error AccessDeniedException: User: arn:aws:sts::XXXXXXXX:assumed-role/shared-dev-1-cert-manager/1657793931500012209 is not authorized to perform: acm-pca:IssueCertificate on resource: arn:aws:acm-pca:eu-west-2:YYYYYYY:certificate-authority/68275bf7-8147-4620-ba7d-4fecf3f234e0 because no resource-based policy allows the acm-pca:IssueCertificate action","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}

Steps to reproduce

Install certmanager on eks
Try to use a shared private CA for cert creation

Relevant log output

No response

Version

v1.2.2

Have you tried the following?

Category

Authentication Issue

Severity

Severity 2

[Feature Request]: Ability to set PathLength for Sub CA certificates

Describe why this change is needed

With this PR #55 we are introducing the ability to issue subCA certificate when the isCA flag is toggled. This is a feature request to add the ability to modify the subCA template ARN to use something besides a path length of zero.

Describe solutions and alternatives considered (optional)

This could possibly be done as a flag set on the issuer, that when set, is evaluated and used to set the appropriate template arn when issuing the subCA certificate.

Is there anything else you would like to add?

No response

AWS region is required, but was not found

We're deploying aws-privateca-issuer using Helm on EKS 1.18 with the following config but somehow it is unable to read the region from AWSPCAClusterIssuer spec

NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                           APP VERSION
awspca          cert-manager    1               2021-09-22 23:39:06.1956528 +0530 IST   deployed        aws-privateca-issuer-0.1.2      v0.3.1

Deploying the issuer using the following config
Note: We are using secrets and not KIAM

apiVersion: awspca.cert-manager.io/v1beta1
kind: AWSPCAClusterIssuer
metadata:
  name: awspca
  namespace: cert-manager
spec:
  region: us-east-1
  arn: arn:aws:acm-pca:us-east-1:xxxxxxxxxx:certificate-authority/xxxxxxx-xxxx-xxxx-9526-xxxxxxxxxx
  secretRef:
    namespace: cert-manager
    name: pca-access

ERROR

{"level":"error","ts":1632339059.6446667,"logger":"controller-runtime.manager.controller.awspcaclusterissuer","msg":"Reconciler error","reconciler group":"awspca.cert-manager.io","reconciler kind":"AWSPCAClusterIssuer","name":"awspca","namespace":"","error":"operation error STS: GetCallerIdentity, failed to resolve service endpoint, an AWS region is required, but was not found","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:302\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

Everything starts working when we pass AWS_REGION into the deployment container env variables


    spec:
      containers:
      - args:
        - --leader-elect
        command:
        - /manager
        env:
        - name: AWS_REGION
          value: us-east-1
        image: ghcr.io/jniebuhr/aws-pca-issuer:v0.3.1
        imagePullPolicy: IfNotPresent

issuer fails to validate

Any idea why the issuer would never validate?

Here are my manifests:

apiVersion: awspca.cert-manager.io/v1beta1
kind: AWSPCAIssuer
metadata:
  name: awspcaissuer
  namespace: cert-manager
spec:
  arn: {{ .Values.pcaArn }}
  region: {{ .Values.region }}
---
apiVersion: v1
kind: Namespace
metadata:
  name: {{ .Chart.Name }}
  labels:
    name: {{ .Chart.Name }}
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
  annotations:
    eks.amazonaws.com/role-arn: {{ .Values.roleArn }}
  name: {{ .Chart.Name }}
  namespace: {{ .Chart.Name }}

Here are the values I passed to the aws-pricateca-issuer helm chart and cert-manager helm chart:

cert-manager:
  installCRDs: true
  serviceAccount:
    create: false
    name: cert-manager
aws-privateca-issuer:
  serviceAccount:
    create: false
    name: cert-manager

Here's the output when I try to describe the issuer:

$ kubectl describe -n cert-manager awspcaissuers.awspca.cert-manager.io

Name:         awspcaissuer
Namespace:    cert-manager
Labels:       argocd.argoproj.io/instance=cert-manager
Annotations:  <none>
API Version:  awspca.cert-manager.io/v1beta1
Kind:         AWSPCAIssuer
Metadata:
  Creation Timestamp:  2021-08-03T14:23:45Z
  Generation:          1
  Managed Fields:
    API Version:  awspca.cert-manager.io/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
        f:labels:
          .:
          f:argocd.argoproj.io/instance:
      f:spec:
        .:
        f:arn:
        f:region:
    Manager:         argocd-application-controller
    Operation:       Update
    Time:            2021-08-03T14:23:45Z
  Resource Version:  45566948
  Self Link:         /apis/awspca.cert-manager.io/v1beta1/namespaces/cert-manager/awspcaissuers/awspcaissuer
  UID:               d5a200ab-0ff9-4f77-858d-1d6e89c2b0f7
Spec:
  Arn:     arn:aws:acm-pca:us-east-1:734116470960:certificate-authority/3c01fb76-eced-49a2-8076-3bf2110807a0
  Region:  us-east-1
Events:    <none>

You'll see no events ever fire, including the one where the issuer is validated.

Here's my certificate:

$ kubectl describe -A certificate
Name:         cms-devops
Namespace:    default
Labels:       argocd.argoproj.io/instance=cms-devops
Annotations:  <none>
API Version:  cert-manager.io/v1
Kind:         Certificate
Metadata:
  Creation Timestamp:  2021-08-03T13:09:57Z
  Generation:          1
  Managed Fields:
    API Version:  cert-manager.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
        f:labels:
          .:
          f:argocd.argoproj.io/instance:
      f:spec:
        .:
        f:commonName:
        f:dnsNames:
        f:duration:
        f:issuerRef:
          .:
          f:group:
          f:kind:
          f:name:
        f:privateKey:
          .:
          f:algorithm:
          f:size:
        f:renewBefore:
        f:secretName:
        f:usages:
    Manager:      argocd-application-controller
    Operation:    Update
    Time:         2021-08-03T13:09:57Z
    API Version:  cert-manager.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:conditions:
        f:lastFailureTime:
    Manager:         controller
    Operation:       Update
    Time:            2021-08-03T14:09:59Z
  Resource Version:  45559101
  Self Link:         /apis/cert-manager.io/v1/namespaces/default/certificates/cms-devops
  UID:               884b894b-acf8-4136-8075-d0719934055b
Spec:
  Common Name:  cms-devops.william-sandbox.internal
  Dns Names:
    cms-devops.william-sandbox.internal
  Duration:  2160h0m0s
  Issuer Ref:
    Group:  awspca.cert-manager.io
    Kind:   AWSPCAClusterIssuer
    Name:   root-ca
  Private Key:
    Algorithm:   RSA
    Size:        2048
  Renew Before:  360h0m0s
  Secret Name:   cms-devops-tls-app-secret
  Usages:
    server auth
    client auth
Status:
  Conditions:
    Last Transition Time:  2021-08-03T14:09:59Z
    Message:               The certificate request has failed to complete and will be retried: issuer is not ready
    Observed Generation:   1
    Reason:                Failed
    Status:                False
    Type:                  Issuing
    Last Transition Time:  2021-08-03T13:09:57Z
    Message:               Issuing certificate as Secret does not exist
    Observed Generation:   1
    Reason:                DoesNotExist
    Status:                False
    Type:                  Ready
  Last Failure Time:       2021-08-03T14:09:59Z
Events:
  Type     Reason     Age                From          Message
  ----     ------     ----               ----          -------
  Normal   Issuing    36m (x2 over 96m)  cert-manager  Issuing certificate as Secret does not exist
  Warning  Failed     36m (x2 over 96m)  cert-manager  The certificate request has failed to complete and will be retried: issuer is not ready
  Normal   Generated  36m                cert-manager  Stored new private key in temporary Secret resource "cms-devops-pj7gp"
  Normal   Requested  36m                cert-manager  Created new CertificateRequest resource "cms-devops-2dfh5"

You can see that the the certificate never completes because the issuer is not ready. Is something misconfigured?

[Bug]: `aws-privateca-issuer` doesn't works with `csi-cert-manager` for mTLS.

Describe the expected outcome

aws-privateca-issuer shold work with csi-cert-manager for mTLS without any issuse.

Describe the actual outcome

aws-privateca-issuer is not working with csi-cert-manager for mTLS and showing following error:

Steps to reproduce

Issuer.yaml

apiVersion: awspca.cert-manager.io/v1beta1
kind: AWSPCAClusterIssuer
metadata:
  name: demo-test-root-ca
spec:
  arn: <CA_ARN>
  region: us-east-1

node_server.yaml

---
apiVersion: v1
kind: Service
metadata:
  labels: {app: hello-mtls-aws-pca}
  name: hello-mtls-aws-pca
  namespace: default
spec:
  type: ClusterIP
  ports:
  - port: 443
    targetPort: 443
  selector: {app: hello-mtls-aws-pca}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-mtls-aws-pca
  labels: {app: hello-mtls-aws-pca}
  namespace: default
spec:
  replicas: 1
  selector: {matchLabels: {app: hello-mtls-aws-pca}}
  template:
    metadata:
      labels: {app: hello-mtls-aws-pca}
    spec:
      containers:
      - name: hello-mtls-aws-pca
        image: test/cert-manager-server-node:latest
        imagePullPolicy: IfNotPresent
        resources: {requests: {cpu: 10m, memory: 20Mi}}
        volumeMounts:
        - mountPath: "/tls"
          name: tls
      volumes:
      - name: tls
        csi:
          readOnly: true
          driver: csi.cert-manager.io
          volumeAttributes:
            csi.cert-manager.io/issuer-kind: AWSPCAClusterIssuer
            csi.cert-manager.io/issuer-name: demo-test-root-ca
            csi.cert-manager.io/issuer-group: awspca.cert-manager.io
            csi.cert-manager.io/dns-names: hello-mtls-aws-pca.default.svc.cluster.local
            csi.cert-manager.io/duration: "20h"
            csi.cert-manager.io/renew-before: "8h"

Create AWSPCAClusterIssuer and Deployment using above files. Wait for deployment pod coming up. It is waiting for mounting volume and volume mount failing due to no validity mentioned in CertificateRequest.

Relevant log output

{"level":"error","ts":1636429708.7008724,"logger":"controllers.CertificateRequest","msg":"failed to request certificate from PCA","certificaterequest":"default/db9e779d-10f6-467b-acc2-341bcd30dd74","error":"operation error ACM PCA: IssueCertificate, https response error StatusCode: 400, RequestID: 07a63afe-f788-4047-8dbe-3857c1557c61, api error ValidationException: 1 validation error detected: Value '0' at 'validity.value' failed to satisfy constraint: Member must have value greater than or equal to 1","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\ngithub.com/cert-manager/aws-privateca-issuer/pkg/controllers.(*CertificateRequestReconciler).Reconcile\n\t/workspace/pkg/controllers/certificaterequest_controller.go:171\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}
E1109 04:22:06.405118       1 manager.go:506] manager "msg"="Failed to issue certificate, retrying after applying exponential backoff" "error"="waiting for request: request \"8961b4c0-3f33-4b30-bbdb-43dedae0a51a\" has failed: failed to request certificate from PCA: operation error ACM PCA: IssueCertificate, https response error StatusCode: 400, RequestID: c6c1fdb3-9a12-424f-bef4-245b7af6c249, api error ValidationException: 1 validation error detected: Value '0' at 'validity.value' failed to satisfy constraint: Member must have value greater than or equal to 1" "volume_id"="csi-fbd8716d6f985bae4fefa59ee13848f66cc8608bb4e4563ae69394424940f6b4"


### Version

0.3.1 (Latest)

### Have you tried the following?

- [X] Check the [Troubleshooting section](../#troubleshooting)
- [X] Search open [issues](https://github.com/cert-manager/aws-privateca-issuer/issues)

### Category

Supported Workflow Broken

### Severity

Severity 1

Unable to create certificate using ingress annotations - MalformedCSRException: CSR must mark the SAN extension critical when it has an empty subject

Kubernetes: 1.19 (v1.19.6-eks-49a6c0)
aws-pca-issuer: Helm release 0.1.0
certmanager: 1.3.1

I have a working setup of this addon but currently unable to use it to auto-generate certificate using annotations such as below:

apiVersion: extensions/v1beta1
# apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: myprivate-ca
  name: podinfo
  namespace: podinfo
spec:
  rules:
  - host: podinfo.infrastructure.com
    http:
      paths:
      - path: /
        backend:
          serviceName: podinfo
          servicePort: 80
  tls:
    - hosts:
      - podinfo.infrastructure.com
      secretName: podinfo10

Creating certificate using the CRD works, e.g with:

kind: Certificate
apiVersion: cert-manager.io/v1
metadata:
  name: podinfo-cert
  namespace: podinfo
spec:
  commonName: podinfo.infrastructure.com
  dnsNames:
    - podinfo.infrastructure.com
  issuerRef:
    group: awspca.cert-manager.io
    kind: AWSPCAClusterIssuer
    name: myprivate-ca
  renewBefore: 360h0m0s
  secretName: podinfo-cert

The error I get when using the ingress approach to autogenerate the certificate is:

2021-04-28T18:42:12.883939412Z {"level":"error","ts":1619635332.883713,"logger":"controllers.CertificateRequest","msg":"failed to request certificate from PCA","certificaterequest":"podinfo/podinfo3-rxrmp","error":"MalformedCSRException: CSR must mark the SAN extension critical when it has an empty subject.","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\ngithub.com/jniebuhr/aws-pca-issuer/pkg/controllers.(*CertificateRequestReconciler).Reconcile\n\t/workspace/pkg/controllers/certificaterequest_controller.go:171\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"}

The approach of using ingress works with other issuers as we currently use vault issuer but we want to migrate to AWS ACM.
Does this work for anyone?

[Feature Request]: Allow passing in environmental variables

Describe why this change is needed

Currently you can not set ENV parameters for the helm chart.

Allowing such setting would enable customers to set required settings such as corporate proxy settings via https_proxy/no_proxy, so they can reach ACM API endpoints.

Describe solutions and alternatives considered (optional)

I tried to contribute the below method a few months ago, but you needed tests writing and I don't have IAM access within our corperation to create PCA resources.

values.yaml

env:
  #var1: myvar1
  #var2: myvar2

deployment.yaml

          env:
          {{- range $key, $value := .Values.env }}
          - name: {{ $key }}
            value: "{{ $value }}"
          {{- end }}

example values

env:
  http_proxy: http://corpproxy:3128
  https_proxy: http://corpproxy:3128
  no_proxy: "localhost.127.0.0.1,instance-data,10.0.0.0/8,169.254.169.254,100.64.0.0/10,.corptld.plc"



### Is there anything else you would like to add?

We've been using this method for months now without issues and the application appears to honour the proxy env parameters.

pod keeps crashing

Followed the below guide:
https://aws.amazon.com/blogs/containers/setting-up-end-to-end-tls-encryption-on-amazon-eks-with-the-new-aws-load-balancer-controller/

Logs:

I0923 17:41:51.050435       1 request.go:655] Throttling request took 1.019788438s, request: GET:https://172.20.0.1:443/apis/policy/v1beta1?timeout=32s
{"level":"info","ts":1632418911.1048965,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":":8080"}
{"level":"info","ts":1632418911.1053565,"logger":"setup","msg":"starting manager"}
I0923 17:41:51.105568       1 leaderelection.go:243] attempting to acquire leader lease aws-pca-issuer/b858308c.awspca.cert-manager.io...
{"level":"info","ts":1632418911.105594,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}
I0923 17:42:08.605338       1 leaderelection.go:253] successfully acquired lease aws-pca-issuer/b858308c.awspca.cert-manager.io
{"level":"info","ts":1632418928.6055086,"logger":"controller-runtime.manager.controller.certificaterequest","msg":"Starting EventSource","reconciler group":"cert-manager.io","reconciler kind":"CertificateRequest","source":"kind source: /, Kind="}
{"level":"info","ts":1632418928.6055412,"logger":"controller-runtime.manager.controller.awspcaclusterissuer","msg":"Starting EventSource","reconciler group":"awspca.cert-manager.io","reconciler kind":"AWSPCAClusterIssuer","source":"kind source: /, Kind="}
{"level":"info","ts":1632418928.6055186,"logger":"controller-runtime.manager.controller.awspcaissuer","msg":"Starting EventSource","reconciler group":"awspca.cert-manager.io","reconciler kind":"AWSPCAIssuer","source":"kind source: /, Kind="}
{"level":"info","ts":1632418928.7065413,"logger":"controller-runtime.manager.controller.certificaterequest","msg":"Starting Controller","reconciler group":"cert-manager.io","reconciler kind":"CertificateRequest"}
I0923 17:42:09.656073       1 request.go:655] Throttling request took 1.045965413s, request: GET:https://172.20.0.1:443/apis/discovery.k8s.io/v1beta1?timeout=32s
{"level":"error","ts":1632418929.7086253,"logger":"controller-runtime.source","msg":"if kind is a CRD, it should be installed before calling Start","kind":"AWSPCAClusterIssuer.awspca.cert-manager.io","error":"no matches for kind \"AWSPCAClusterIssuer\" in version \"awspca.cert-manager.io/v1beta1\"","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/log.(*DelegatingLogger).Error\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/log/deleg.go:144\nsigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/source/source.go:117\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:167\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:223\nsigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).startRunnable.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/manager/internal.go:681"}
{"level":"info","ts":1632418929.708644,"logger":"controller-runtime.manager.controller.certificaterequest","msg":"Starting workers","reconciler group":"cert-manager.io","reconciler kind":"CertificateRequest","worker count":1}
{"level":"info","ts":1632418929.7087255,"logger":"controller-runtime.manager.controller.certificaterequest","msg":"Stopping workers","reconciler group":"cert-manager.io","reconciler kind":"CertificateRequest"}
{"level":"error","ts":1632418932.3092573,"logger":"controller-runtime.source","msg":"if kind is a CRD, it should be installed before calling Start","kind":"AWSPCAIssuer.awspca.cert-manager.io","error":"no matches for kind \"AWSPCAIssuer\" in version \"awspca.cert-manager.io/v1beta1\"","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/log.(*DelegatingLogger).Error\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/log/deleg.go:144\nsigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/source/source.go:117\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:167\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:223\nsigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).startRunnable.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/manager/internal.go:681"}
{"level":"error","ts":1632418932.3093493,"logger":"controller-runtime.manager","msg":"error received after stop sequence was engaged","error":"no matches for kind \"AWSPCAIssuer\" in version \"awspca.cert-manager.io/v1beta1\"","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).engageStopProcedure.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/manager/internal.go:530"}
{"level":"error","ts":1632418932.3094308,"logger":"controller-runtime.manager","msg":"error received after stop sequence was engaged","error":"leader election lost","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).engageStopProcedure.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/manager/internal.go:530"}
{"level":"error","ts":1632418932.3094597,"logger":"setup","msg":"problem running manager","error":"no matches for kind \"AWSPCAClusterIssuer\" in version \"awspca.cert-manager.io/v1beta1\"","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/log.(*DelegatingLogger).Error\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/log/deleg.go:144\nmain.main\n\t/workspace/main.go:139\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:225"}

Access via IAM roles for service account

Describe why this change is needed

Currently awspca configuration needs creating a dedicated IAM user with proper access and define a secret with access key - but since it is used in EKS it would be better to use dedicated service account with assigned IAM policy. It would be more natural way I think to integrate it. How do you think? (https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html)

Describe solutions and alternatives considered (optional)

No response

Is there anything else you would like to add?

No response

append ca bundle ( chainPem ) to caPem insted to actual certificate i.e. certPem

https://github.com/jniebuhr/aws-pca-issuer/blob/cd313e77bde92960ab17aef85e22848d87f8cff9/pkg/aws/pca.go#L123

Good Morning,

I am not sure reason behind the current solution i.e. append ca bundle ( chainPem ) with actual certificate ( certPem
) issued to FQDN .

currently, i am not proposing any changes as it may be it's kubernetes focused ? just want to show my observation.

the openshift route has three separate keys:

    key: |-                      
      -----BEGIN PRIVATE KEY-----
      [...]
      -----END PRIVATE KEY-----
    certificate: |-              
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----
    caCertificate: |-            
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----

ex. Assume, below is our chain looks like )

D ( root )
  C ( issuer of B)
     B ( issuer of A )
        A ( actual certificate issued to requested FQDN)

Current version:

	certPem := []byte(*getOutput.Certificate + "\n")
	chainPem := []byte(*getOutput.CertificateChain)
	certPem = append(certPem, chainPem...)

	caParams := acmpca.GetCertificateAuthorityCertificateInput{
		CertificateAuthorityArn: aws.String(p.arn),
	}
	caOutput, err := svc.GetCertificateAuthorityCertificate(&caParams)
	if err != nil {
		return nil, nil, err
	}

	caPem := []byte(*caOutput.Certificate)

	return certPem, caPem, nil

in above case, cert-manager creating secret like below: i.e. tks.crt contains the CA bundle.

ca.crt: contains certificate of B
tls.crt: contains certificates of A, B, C, D
tls.key: contains private key of A

Modified version:

	certPem := []byte(*getOutput.Certificate + "\n")
	chainPem := []byte(*getOutput.CertificateChain)
	caPem := chainPem
	//certPem = append(certPem, chainPem...)

	//caParams := acmpca.GetCertificateAuthorityCertificateInput{
	//	CertificateAuthorityArn: aws.String(p.arn),
	//}
	//caOutput, err := svc.GetCertificateAuthorityCertificate(&caParams)
	//if err != nil {
	//	return nil, nil, err
	//}

	//caPem := []byte(*caOutput.Certificate)

	return certPem, caPem, nil

in above case, cert-manager creating secret like below: completely separating CA bundle with tls.crt. and adding whole bundle to ca.crt

ca.crt: contains certificate of B, C, D
tls.crt: contains certificate of A
tls.key: contains private key of A

Integration with cert-manager, istio-csr fails pod to pod mTLS.

Describe the expected outcome

Pods in the cluster should be able to talk mTLS with each other and have the same root cert dispatched by cert-manager through aws pca.

Describe the actual outcome

kubectl -n foo exec -it deploy/sleep -c sleep -- curl http://httpbin.foo:8000/headers -v

  • Trying xxxx:8000...
  • Connected to httpbin.foo (xxxx.155) port 8000 (#0)

GET /headers HTTP/1.1
Host: httpbin.foo:8000
User-Agent: curl/7.87.0-DEV
Accept: /

  • Mark bundle as not supporting multiuse
    < HTTP/1.1 503 Service Unavailable
    < content-length: 218
    < content-type: text/plain
    < date: Fri, 20 Jan 2023 02:48:23 GMT
    < server: envoy
    <
  • Connection #0 to host httpbin.foo left intact
    upstream connect error or disconnect/reset before headers. retried and the latest reset reason: connection failure, transport failure reason: TLS error: 268435581:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED

Steps to reproduce

Trying to move from self signed cert in istio mesh to ACM PCA certs for istio workloads and with everything seemingly successful (setup), the test apps (sleep, httbin) give a 503 while talking to each other in the same namespace. Im using cert-manager, istio-csr, awspcaissuer add on and istio istio mesh in EKS.

The cert manager, istio csr, awspca/aws-privateca-issuer are all installed using helm and successfully running

awspca issuer (cluster issuer) is created, root cert also created with the following:

kind: Certificate
apiVersion: cert-manager.io/v1
metadata:
  name: istio-ca
  namespace: istio-system
spec:
  isCA: true
  commonName: istio-ca
  duration: 2160h0m0s
  issuerRef:
    group: awspca.cert-manager.io
    kind: AWSPCAClusterIssuer
    name: awspcaclusterissuer
  secretName: istio-ca
  subject:
     organizations:
     - cluster.local
     - cert-manager
   privateKey:
     algorithm: "RSA"
     size: 2048

istio is installed with overlay, everything looks good in sample apps, it comes up with new certs, the CSR are approved but the actual request fails

kubectl -n foo exec -it deploy/sleep -c sleep -- curl http://httpbin.foo:8000/headers -v
*   Trying 192.20.179.155:8000...
* Connected to httpbin.foo (172.20.179.155) port 8000 (#0)
> GET /headers HTTP/1.1
> Host: httpbin.foo:8000
> User-Agent: curl/7.87.0-DEV
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 503 Service Unavailable
< content-length: 218
< content-type: text/plain
< date: Fri, 20 Jan 2023 02:55:07 GMT
< server: envoy
<
* Connection #0 to host httpbin.foo left intact
upstream connect error or disconnect/reset before headers. retried and the latest reset reason: connection failure, transport failure reason: TLS error: 268435581:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED

Relevant log output

No response

Version

istio - 1.14.1
latest for cert-manager, istio-csr, aws-privateca-issuer

Have you tried the following?

Category

Supported Workflow Broken

Severity

Severity 3

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.