circleci-public / aws-eks-orb Goto Github PK

What would you like to be added

Ability to pass a values file to install-helm-chart, similar to how helm install -f works. That allows environment-specific values files like values.production.yaml to be installed using this orb.

Why is this needed

We're currently able to override specific values using values-to-override, which works if you only want to override just a few values but doesn't scale well if you have separate values files for reach environment.

Orb version

1.0.0

What happened

• CircleCI Link (not sure if accessible)

• When running step "Install the AWS IAM Authenticator for Kubernetes" from the "update-kubeconfig-with-authenticator" command, I got the following error:

curl: (22) The requested URL returned error: 404 Not Found

After further investigation, I found that the latest v0.5.2 release, which came out 2 hours previously, doesn't have any content at the download url that would be constructed from this script. If I pin the version to v0.5.1, things work properly.

Expected behavior

• I shouldn't need to pin the version to have this code work - it should have installed the AWS IAM Authenticator and finished the job.

There is not an update-container-image command as is mentioned in the docs: https://circleci.com/orbs/registry/orb/circleci/aws-eks#jobs-update-container-image

Example YAML:

version: 2.1

orbs:
  aws-ecr: circleci/[email protected]
  aws-eks: circleci/[email protected]

jobs:
  build_and_push_image:
    executor: aws-ecr/default
    steps:
      - aws-ecr/build-and-push-image:
          repo: me/myimage
          tag: "${CIRCLE_SHA1}"
  deploy_to_kubernetes:
    executor: aws-eks/python3
    steps:
      - aws-eks/update-container-image:
          cluster-name: mycluster
          namespace: myns
          container-image-updates: "mydeployment=${AWS_ECR_ACCOUNT_URL}/me/myimage:${CIRCLE_SHA1}"

workflows:
  build_push_and_deploy:
    jobs:
      - build_and_push_image
      - deploy_to_kubernetes:
          requires:
            - build_and_push_image

In CircleCI:

#!/bin/sh -eo pipefail
# Error calling workflow: 'build_push_and_deploy'
# Error calling job: 'deploy_to_kubernetes'
# Cannot find a definition for command named aws-eks/update-container-image
# 
# -------
# Warning: This configuration was auto-generated to show you the message above.
# Don't rerun this job. Rerunning will have no effect.
false
Exited with code 1

Validator:

$ circleci config validate
Error: Error calling workflow: 'build_push_and_deploy'
Error calling job: 'deploy_to_kubernetes'
Cannot find a definition for command named aws-eks/update-container-image

eks-orbs is not working for windows application. After deploying the windows docker image to ECR, the build is failed while installing KOPS.

Orb version

2.0

What happened

#!/bin/sh -eo pipefail
# Error calling workflow: 'deployment'
# Error calling job: 'aws-eks/update-container-image'
# Cannot find a definition for executor named python
# 
# -------
# Warning: This configuration was auto-generated to show you the message above.
# Don't rerun this job. Rerunning will have no effect.

Expected behavior

Should proceed without any errors

Here is the example of configs that I used

version: '2.1'
orbs:
  aws-eks: circleci/[email protected]  
workflows:
  deployment:
    jobs:
      - aws-eks/update-container-image:
          cluster-name: staging-eks
          aws-region: us-east-1
          container-image-updates: 'nginx=nginx:1.9.1'
          resource-name: deployment/nginx-deployment
          namespace: citestns
          show-kubectl-command: true
          watch-rollout-status: true
          get-rollout-status: true

the same configs with orb version 1.2.0 works without any problems

Orb version

aws-eks: circleci/[email protected]
kubernetes: circleci/[email protected]

What happened

We just started getting this error when running kubectl commands:

error: exec plugin: invalid apiVersion "client.authentication.k8s.io/v1alpha1"

  my-job:
    docker:
      - image: 'cimg/python:3.10'
    parameters:
      cluster-name:
        type: string
    steps:
      - kubernetes/install
      - aws-eks/update-kubeconfig-with-authenticator:
          cluster-name: << parameters.cluster-name >>
      - run:
          command: |
            kubectl version
          name: My Job

Expected behavior

The kubectl command should work, but instead its erroring on every command.

https://circleci.com/gh/CircleCI-Public/circleci-demo-aws-eks/44

Error: unknown flag: --storage-class
Usage: eksctl create cluster [flags]

Orb requires common variables like region to be passed as parameter, with an empty default overriding environmant values.

Default should be ${AWS_REGION}

Orb version

Unable to specify a release tag. Tested on 1.0.0. 1.2.0, 2.0.0. 2.1.1

Working on 1.0.0 and 1.2.0

NOT working in 2.0.0 and 2.1.1

What happened

#!/bin/bash -eo pipefail
#!/bin/bash
if which aws-iam-authenticator > /dev/null; then
echo "AWS IAM Authenticator for Kubernetes is already installed"
exit 0
fi
PLATFORM="linux"
if uname | grep -q 'Darwin'
then
PLATFORM="darwin"
fi
FILENAME="aws-iam-authenticator"
VERSION=$(curl -Ls --fail --retry 3 -o /dev/null -w "%{url_effective}" "https://github.com/kubernetes-sigs/aws-iam-authenticator/releases/latest" | sed 's:.*/::' | sed 's/v//g')
if [ -n "${PARAM_RELEASE_TAG}" ]; then
export RELEASE_TAG=${!PARAM_RELEASE_TAG}
VERSION="${RELEASE_TAG}"
if [ "${VERSION}" == "0.3.0" ]; then
FILENAME="heptio-authenticator-aws"
fi
fi

DOWNLOAD_URL="https://github.com/kubernetes-sigs/aws-iam-authenticator/releases/download/v${VERSION}/${FILENAME}_${VERSION}_${PLATFORM}_amd64"

curl -L --fail --retry 3 -o aws-iam-authenticator "$DOWNLOAD_URL"
chmod +x ./aws-iam-authenticator

if [ "$(id -u)" -ne 0 ] && which sudo > /dev/null ; then
SUDO="sudo"
fi
$SUDO mv ./aws-iam-authenticator /usr/local/bin/aws-iam-authenticator

/bin/bash: line 13: v0.5.1: invalid variable name

Exited with code exit status 1
CircleCI received exit code 1

Expected behavior

Install and finish the job successfully

What would you like to be added

I would like to use the update-kubeconfig-with-authenticator command, but the only way to pass configuration like AWS_REGION is through parameters. It would be a great addition if the orb commands can also be configured through env vars.

Why is this needed

We have configured a context per environment and pass the context of the correct environment to the jobs. Through the context, all correct env vars will be populated.

Orb version

8.2.1

What happened

When trying to authenticate with OIDC in AWS, it tries to assume role and it's missing the role session name. So it cannot assume the correct role.

Expected behavior

It should allow for setting the role session name, and assume role properly. Other orbs like aws-ecr require a role-session-name.

Version 1.0.1 of the aws-eks orb pins version 0.5.1 of aws-iam-authenticator as the default when no specific version is specified. See: #33
Now that kubernetes-sigs/aws-iam-authenticator#344 is resolved, we can reverse the changes in the PR

Orb version

2.1.1

What happened

The install-aws-iam-authenticator command is failing our builds because the aws-iam-authenticator project pushed a release that does not contain binary assets.

#!/bin/bash -eo pipefail
if which aws-iam-authenticator > /dev/null; then
  echo "AWS IAM Authenticator for Kubernetes is already installed"
  exit 0
fi

PLATFORM="linux"
if [ -n "$(uname | grep "Darwin")" ]; then
  PLATFORM="darwin"
fi

RELEASE_TAG=""
RELEASE_URL="https://api.github.com/repos/kubernetes-sigs/aws-iam-authenticator/releases/latest"

FILENAME="aws-iam-authenticator"
VERSION=$(curl -Ls --fail --retry 3 -o /dev/null -w %{url_effective} "https://github.com/kubernetes-sigs/aws-iam-authenticator/releases/latest" | sed 's:.*/::')
if [ -n "${RELEASE_TAG}" ]; then
  VERSION="${RELEASE_TAG}"
  if [ "${VERSION}" == "v0.3.0" ]; then
    FILENAME="heptio-authenticator-aws"
  fi
fi

# extract version number
VERSION_NUMBER=$(echo $VERSION | cut -c 2-)

DOWNLOAD_URL="https://github.com/kubernetes-sigs/aws-iam-authenticator/releases/download/${VERSION}/${FILENAME}_${VERSION_NUMBER}_${PLATFORM}_amd64"

curl -L --fail --retry 3 -o aws-iam-authenticator "$DOWNLOAD_URL"

chmod +x ./aws-iam-authenticator

SUDO=""
if [ $(id -u) -ne 0 ] && which sudo > /dev/null ; then
  SUDO="sudo"
fi

$SUDO mv ./aws-iam-authenticator /usr/local/bin/aws-iam-authenticator

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404 

Exited with code exit status 22
CircleCI received exit code 22

Expected behavior

Revert to a known good build. It would be ideal if the release version of the aws-iam-authenticator could be specified in a parameter.

Orb version

What happened

Getting:
curl: (3) URL using bad/illegal format or missing URL

Reason:

DOWNLOAD_URL=$(curl -s --retry 5 "${RELEASE_URL}" \
    | grep "${PLATFORM}" | awk '/browser_download_url/ {print $2}' | sed 's/"//g')
echo $DOWNLOAD_URL
https://github.com/kubernetes-sigs/aws-iam-authenticator/releases/download/v0.5.5/aws-iam-authenticator_0.5.5_linux_amd64
https://github.com/kubernetes-sigs/aws-iam-authenticator/releases/download/v0.5.5/aws-iam-authenticator_0.5.5_linux_arm64

^ this now returns 2 urls instead of one, which breaks curl

### Expected behavior

<!--- what should happen, ideally? -->

Orb version

2.1.0

What happened

Calling the image definition for python or python3 fails as the executors folder no longer exists in this version. Failing with:

Cannot find a definition for executor named aws-eks/python

Build links available on request (please Slack me)

Expected behavior

Orb should launch an executor using a Python image.

Orb version

2.1.1

What happened

The install-aws-iam-authenticator command is failing our builds because the aws-iam-authenticator project pushed a release that does not contain binary assets.

'''
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 404

Exited with code exit status 22
CircleCI received exit code 22
'''

Expected behavior

Revert to a known good build. It would be ideal if the release version of the aws-iam-authenticator could be check 404 error when we download aws-iam-authenticator and downgrade version

Orb version

circleci/[email protected]

What happened

We noticed today our deployments are failing. We've been using the Orb aws-eks: circleci/[email protected] to update container images on the EKS cluster without issues until today. We are seeing the kubectl command includes an argument --dry-run, which is not set in the CI configuration. We tried setting up a variable through context and project level, but that doesn't seem to work either.

https://app.circleci.com/pipelines/github/b-eee/sleep-partners/156/workflows/39411f04-e7cf-4eed-a7f9-fd18be342bac/jobs/275

Expected behavior

Should be able to update the container image with dry-run argument is deactivated or set to none.

Orb version

  orbs:
  aws-eks: circleci/[email protected]
  kubernetes: circleci/[email protected]

What happened

While creating a new AWS EKS cluster using this example config file, it generates an error at the install-eksctl step. Below is the output from the CircleCI failed build.

#!/bin/bash -eo pipefail
if which eksctl > /dev/null; then
  echo "eksctl is already installed"
  exit 0
fi

mkdir -p eksctl_download
curl --silent --location --retry 5 "https://github.com/weaveworks/eksctl/releases/download/latest_release/eksctl_$(uname -s)_amd64.tar.gz" \
  | tar xz -C eksctl_download
chmod +x eksctl_download/eksctl
SUDO=""
if [ $(id -u) -ne 0 ] && which sudo > /dev/null ; then
  SUDO="sudo"
fi
$SUDO mv eksctl_download/eksctl /usr/local/bin/
rmdir eksctl_download


gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

Exited with code exit status 2
CircleCI received exit code 2

See the snapshot of the error below.
You can see the files in this commit.

Expected behavior

The example should install eksctl and create a cluster successfully.

I'm using the https://github.com/CircleCI-Public/aws-eks-orb, version and specifically the aws-eks/[email protected] task which triggers the aws-cli/install command.

Over the past day or so, it has started failing with the following error:

#!/bin/bash -eo pipefail
CLUSTER_NAME="redacted"
AWS_REGION="eu-central-1"
AWS_PROFILE=""
KUBECONFIG_FILE_PATH=""
ROLE_ARN="redacted"
CLUSTER_CONTEXT_ALIAS=""
DRY_RUN="false"
VERBOSE="true"
if [ -n "${CLUSTER_NAME}" ]; then
  set -- "$@" --name "${CLUSTER_NAME}"
fi
if [ -n "${AWS_REGION}" ]; then
  set -- "$@" --region "${AWS_REGION}"
fi
if [ -n "${AWS_PROFILE}" ]; then
  set -- "$@" --profile "${AWS_PROFILE}"
fi
if [ -n "${KUBECONFIG_FILE_PATH}" ]; then
  set -- "$@" --kubeconfig "${KUBECONFIG_FILE_PATH}"
fi
if [ -n "${ROLE_ARN}" ]; then
  set -- "$@" --role-arn "${ROLE_ARN}"
fi
if [ -n "${CLUSTER_CONTEXT_ALIAS}" ]; then
  set -- "$@" --alias "${CLUSTER_CONTEXT_ALIAS}"
fi
if [ "${DRY_RUN}" == "true" ]; then
  set -- "$@" --dry-run
fi
if [ "${VERBOSE}" == "true" ]; then
  set -- "$@" --verbose
fi
aws eks update-kubeconfig "$@"
Traceback (most recent call last):
  File "/root/.local/bin/aws", line 6, in <module>
    from aws.main import main
  File "/root/.local/lib/python3.7/site-packages/aws/main.py", line 23
    print '%(name)s: %(endpoint)s' % {
                                 ^
SyntaxError: invalid syntax
Exited with code 1

This error takes place when the aws library has been installed via pip instead of the awscli library.

Having checked the code that was run in the install aws sdk task, it was clear that this is the issue. The code that was run is:

#!/bin/bash -eo pipefail
export PIP=$(which pip pip3 | head -1)
if [[ -n $PIP ]]; then
  if which sudo > /dev/null; then
    sudo $PIP install awscli --upgrade
  else
    # This installs the AWS CLI to ~/.local/bin. Make sure that ~/.local/bin is in your $PATH.
    $PIP install aws --upgrade --user
  fi
elif [[ $(which unzip curl | wc -l) -eq 2 ]]; then
  cd
  curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
  unzip awscli-bundle.zip
  if which sudo > /dev/null; then
    sudo ~/awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
  else
    # This installs the AWS CLI to the default location (~/.local/lib/aws) and create a symbolic link (symlink) at ~/bin/aws. Make sure that ~/bin is in your $PATH.
    awscli-bundle/install -b ~/bin/aws
  fi
  rm -rf awscli-bundle*
  cd -
else
  echo "Unable to install AWS CLI. Please install pip."
  exit 1
fi

You can see the issue on line 8 - when sudo is available, it installs the awscli library, however when sudo isn't available, it installs the aws library.

I've tried tracing it back into this repository and also the aws-eks-orb repo, and I can't find any reference to this code. It looks like the aws-eks-orb is using aws-cli: circleci/[email protected]

I'm a bit stuck for suggestions as to why this is occurring, but I was recommended to register the issue here. More information including screenshots of a comparison between the code running in my builds and the code in the repository can be found in this thread: https://twitter.com/mxkrmr/status/1162465650163552258

What would you like to be added

Currently the latest version is fetched but there is no attempt to check if it is a working binary

Why is this needed

Improve the resiliency of the command. See: #32

Describe Request:

Migrate to Orb Tools v11.1

Supporting Documentation Links:

https://github.com/CircleCI-Public/orb-tools-orb/blob/master/MIGRATION.md

Orb version

1.0.0

What happened

After updating to 1.0.0 env vars are no longer getting picked up, so we get the following response when trying to authenticate:

#!/bin/bash -eo pipefail
CLUSTER_NAME="${EKS_CLUSTER_NAME}"
AWS_REGION=""
AWS_PROFILE=""
KUBECONFIG_FILE_PATH=""
ROLE_ARN=""
CLUSTER_CONTEXT_ALIAS=""
DRY_RUN="false"
VERBOSE="false"
if [ -n "${CLUSTER_NAME}" ]; then
  set -- "$@" --name "${CLUSTER_NAME}"
fi
if [ -n "${AWS_REGION}" ]; then
  set -- "$@" --region "${AWS_REGION}"
fi
if [ -n "${AWS_PROFILE}" ]; then
  set -- "$@" --profile "${AWS_PROFILE}"
fi
if [ -n "${KUBECONFIG_FILE_PATH}" ]; then
  set -- "$@" --kubeconfig "${KUBECONFIG_FILE_PATH}"
fi
if [ -n "${ROLE_ARN}" ]; then
  set -- "$@" --role-arn "${ROLE_ARN}"
fi
if [ -n "${CLUSTER_CONTEXT_ALIAS}" ]; then
  set -- "$@" --alias "${CLUSTER_CONTEXT_ALIAS}"
fi
if [ "${DRY_RUN}" == "true" ]; then
  set -- "$@" --dry-run
fi
if [ "${VERBOSE}" == "true" ]; then
  set -- "$@" --verbose
fi
aws eks update-kubeconfig "$@"

Invalid endpoint: https://eks..amazonaws.com

Exited with code exit status 255
CircleCI received exit code 255

Expected behavior

env vars should be recognized

Example of .circleci.yml

    <<: *defaults
    steps:
      - checkout
      - attach_workspace:
          at: .
      - run:
          name: Load the environment variables in to bash env
          command: source ./setup-env.sh
      - run:
          name: Load Build Image for Deployment
          command: docker load -i Dockerfile.tar
      - aws-ecr/ecr-login:
          region: AWS_DEFAULT_REGION
      - aws-ecr/push-image:
          account-url: AWS_ECR_ACCOUNT_URL
          repo: app
          tag: "$CIRCLE_SHA1,latest"
      - kubernetes/install-kubectl
      - aws-eks/update-kubeconfig-with-authenticator:
          cluster-name: ${EKS_CLUSTER_NAME}

Orb version

1.1.0

What happened

Installation Failed due to incorrect URL
"https://github.com/weaveworks/eksctl/releases/download/latest_release/eksctl_$(uname -s)_amd64.tar.gz"

Expected behavior

eks-ctl install should work as expected

What would you like to be added

Folks, I'm seeing an install of awscli 1.x after explicitly installing awscli 2.0. Can a check be done first for existing awscli install?

orbs:
  aws-cli: circleci/[email protected]
  aws-eks: circleci/[email protected]

jobs:
  test:
    executor: aws-eks/python3
    steps:
      - checkout
      - aws-cli/setup
      - aws-eks/update-kubeconfig-with-authenticator:
          cluster-name: dl-k8s
          install-kubectl: true

Why is this needed

Overhead of additional aws install is large and unnecessary.

What would you like to be added

In the v2 rewrite it looks like helm support has disappeared from this orb, but not from the usage examples. Would be good to have helm support reinstated. Would also be really useful if the helm support included optionally running helm test against the deployed chart, but I guess that needs a separate ticket on the helm orb.

Why is this needed

The steps to setup kubeconfig from eks, install the helm client and run helm upgrade (and then run helm test) make sense to bundle together in an orb to reduce build config clutter

This orb is woefully out of date, needs some updating.

This may be an issue with the kubernetes orb, but in the latest version of the eks orb (0.2.2), supplying a list of multiple images delimited by spaces fails to update multiple images. Instead, it attempts to set the first container's image to the entire string that comes after the first = character.

For instance, here's my circleci config.yml snippet:

          name: Staging Deploy
          requires:
            - Deploy to staging
          aws-region: us-east-2
          cluster-name: staging
          resource-name: deployment.v1.apps/my-app-staging
          container-image-updates: my-app=my-org/my-app:$CIRCLE_SHA1 my-app-job-runner=my-org/my-app:$CIRCLE_SHA1
          record: true

and here is the outcome:

Containers:
  my-app:
    Container ID:
    Image:          my-org/my-app:f3c65ebb0602966922eeceb92af37cdf6222d375 my-app-job-runner=my-org/my-app:f3c65ebb060296692
2eeceb92af37cdf6222d375

which of course leads to kubernetes image pull failures:

Warning  InspectFailed   10s (x5 over 26s)  kubelet, ip-192-168-9-21.us-east-2.compute.internal  Failed to apply default image tag "my-org/my-app:f3c65ebb0602966922eeceb92af37cdf6222d375 my-app-job-runner=my-org/my-app:f3c65ebb0602966922eeceb92af37cdf6222d375": couldn't parse image reference "my-org/my-app:f3c65ebb0602966922eeceb92af37cdf6222d375 my-app-job-runner=my-org/my-app:f3c65ebb0602966922eeceb92af37cdf6222d375": invalid reference format

I've tried wrapping the line in quotes or no quotes, but it doesn't seem to make any difference.