Maintenance: redoing the testing structure for the orb.

Orb version:

3.0

What happened:

install command does not configure profile region properly:

- sam/install:
    aws-region: MY_REGION
    profile-name: myprofile

- run: cat ~/.aws/config

- run: aws configure list --profile myprofile

- run: sam package --profile myprofile ...

cat ~/.aws/config
[default]
region = *********

aws configure list --profile myprofile
      Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile                    myprofile           manual    --profile
access_key     **************** shared-credentials-file    
secret_key     **************** shared-credentials-file    
    region                *********              env    ['MY_REGION', 'AWS_DEFAULT_REGION']


sam package --profile myprofile ...
Traceback (most recent call last):
  File "samcli/__main__.py", line 12, in <module>
  File "click/core.py", line 829, in __call__
  File "click/core.py", line 782, in main
  File "click/core.py", line 1259, in invoke
  File "click/core.py", line 1066, in invoke
  File "click/core.py", line 610, in invoke
  File "samcli/lib/cli_validation/image_repository_validation.py", line 76, in wrapped
  File "click/decorators.py", line 73, in new_func
  File "click/core.py", line 610, in invoke
  File "samcli/lib/telemetry/metric.py", line 152, in wrapped
  File "samcli/lib/telemetry/metric.py", line 121, in wrapped
  File "samcli/lib/utils/version_checker.py", line 42, in wrapped
  File "samcli/lib/telemetry/metric.py", line 84, in wrapped
  File "samcli/cli/main.py", line 90, in wrapper
  File "samcli/commands/package/command.py", line 176, in cli
  File "samcli/commands/package/command.py", line 225, in do_cli
  File "samcli/commands/package/package_context.py", line 105, in run
  File "boto3/__init__.py", line 91, in client
  File "boto3/session.py", line 263, in client
  File "botocore/session.py", line 838, in create_client
  File "botocore/client.py", line 85, in create_client
  File "botocore/client.py", line 308, in _get_client_args
  File "botocore/args.py", line 73, in get_client_args
  File "botocore/args.py", line 153, in compute_client_args
  File "botocore/args.py", line 218, in _compute_endpoint_config
  File "botocore/args.py", line 301, in _resolve_endpoint
  File "botocore/client.py", line 382, in resolve
  File "botocore/regions.py", line 134, in construct_endpoint
  File "botocore/regions.py", line 148, in _endpoint_for_partition
botocore.exceptions.NoRegionError: You must specify a region.

Expected behavior:

The install command should configure the profile region correctly so it would be detected by further sam CLI commands.

Workaround:

Use aws-cli/setup version 1.4.0+ directly to configure the profile's region:

orbs:
  aws-cli: circleci/[email protected]
  sam: circleci/[email protected]

...
    steps:
      - aws-cli/setup:
          aws-region: MY_REGION
          profile-name: myprofile
          configure-profile-region: true
          configure-default-region: false

      - sam/install:
          configure-default-region: false

Additional Information:

This seems to have been a problem in the aws-cli orb, fixed in this PR that was merged in v1.4.0. It might be that updating the this orb to use aws-cli 1.40 (or later) will fix the problem.

Describe Request:

When executing sam build, I would like to pass environment variables to a build container.
In my case, I need to pass environment variables for the artifactory private maven repository to a container.

Examples:

There are 2 ways to pass environment variables in aws sam CLI.
I think it's simple to support taking the container-env-var-file option compared to the other way.

To build with environment variables passed to the build container from the command line
$ sam build --use-container --container-env-var Function1.GITHUB_TOKEN=<token1> --container-env-var GLOBAL_ENV_VAR=<global-token>

To build with environment variables passed to the build container from a file
$ sam build --use-container --container-env-var-file <env-file.json>

Supporting Documentation Links:

• https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-build.html
• Container environment variable file

Orb version:

3.1.0

Here is my config.yml

# Use the latest 2.1 version of CircleCI pipeline process engine.
# See: https://circleci.com/docs/2.0/configuration-reference
version: 2.1

# Orbs are reusable packages of CircleCI configuration that you may share across projects, enabling you to create encapsulated, parameterized commands, jobs, and executors that can be used across multiple projects.
# See: https://circleci.com/docs/2.0/orb-intro/
orbs:
  sam: circleci/[email protected]

workflows:
  test_and_deploy:
    jobs:
      - sam/deploy:
          name: deploy-production
          image-repositories: myorganisationid1234567.dkr.ecr.ap-northeast-1.amazonaws.com/my-project-name
          stack-name: my-project-name
          template: ./template.yaml
          filters:
            branches:
              only: main

What happened:

I defined a statemachine resource in a SAM template.yaml as below:

  StateMachine:
    Type: AWS::Serverless::StateMachine 
    Properties:
      Name: GetAllTickerEarningsCallsStateMachine
      DefinitionUri: ./statemachine/statemachine.asl.json
      Role: !GetAtt StateMachineRole.Arn

I can run sam build and sam deploy locally to deploy the statemachine. I have problem deploying it using this orb on CircleCI, got the following error message:

Error: Unable to upload artifact ../../statemachine/statemachine.asl.json referenced by DefinitionUri parameter of StateMachine resource.

S3 Bucket not specified, use --s3-bucket to specify a bucket name, or use --resolve-s3 to create a managed default bucket, or run sam deploy --guided

I tried to set s3-bucket option in config.yml but it failed because I already set image-repositories

Expected behavior:

It is expected to deploy the statemachine as it does locally. sam deploy creates a managed s3 bucket (name like aws-sam-cli-managed-default-samclisourcebucket-1n1tv1h0em72e) for deploy use

Additional Information:

It seems it can have both s3_bucket and image_repositories options
https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html

Orb version

1.0.1

What happened

Orb command build requires a template parameter and always pass it to the package step, where the sam package command packs files base on CodeUri in the given template instead of the folder generated in the build step.

Expected behavior

sam package runs against the built files.

Possible solution:

1. Pass ./.aws-sam/build/<<parameters.template>> or <<parameters.build-dir>>/<<parameters.template>> to sam package
2. Suggest users build requirements layer instead of relying on requirements.txt, more like a workaround for now.

Orb version:

4.0.0

What happened:

When we use the additional-args parameter, it seems to be ignored. When we don't use it, a '' is appended to the sam deploy command which can break the parameter-overrides parameter (see #50).

Expected behavior:

It should be the opposite. additional-args parameter should be used when used and should be ignored when not used.

Additional Information:

It seems to be due to the if [ -z "$SAM_PARAM_ADDITIONAL_ARGS" ]; then in the deploy script which should have a -n instead of a -z

Orb version

1.0.2

What happened

CircleCI Step to install SAM CLI failed. Here is the log: https://gist.github.com/kmeelu/b6483118368abe63de8a8fd6b8f1c74e

I'm not sure whether the follow up is with this Orb or SAM CLI, but pinging here in case you're aware.

Expected behavior

SAM CLI installs as expected, allowing us to deploy lambdas.

Orb version: 3.1.0

What happened:

S3 Bucket not specified, use --s3-bucket to specify a bucket name, or use --resolve-s3 to create a managed default bucket, or run sam deploy --guided

Exited with code exit status 1

Expected behavior:

Have the option to use --resolve-s3

Orb version:

3.0.0

What happened:

Failed with exception, worked fine with version 2.0.0

#!/bin/bash -eo pipefail
sam validate -t ./functions/template.yaml \
--region $AWS_DEFAULT_REGION \
--profile default \
--debug

Traceback (most recent call last):
  File "samcli/__main__.py", line 12, in <module>
  File "click/core.py", line 829, in __call__
  File "click/core.py", line 782, in main
  File "click/core.py", line 1257, in invoke
  File "click/core.py", line 700, in make_context
  File "click/core.py", line 1048, in parse_args
  File "click/core.py", line 1630, in handle_parse_result
  File "click/core.py", line 123, in invoke_param_callback
  File "samcli/cli/options.py", line 42, in callback
  File "samcli/cli/context.py", line 78, in region
  File "samcli/cli/context.py", line 178, in _refresh_session
  File "boto3/__init__.py", line 35, in setup_default_session
  File "boto3/session.py", line 80, in __init__
  File "boto3/session.py", line 120, in _setup_loader
  File "botocore/session.py", line 685, in get_component
  File "botocore/session.py", line 924, in get_component
  File "botocore/session.py", line 158, in <lambda>
  File "botocore/session.py", line 241, in get_config_variable
  File "botocore/configprovider.py", line 313, in get_config_variable
  File "botocore/configprovider.py", line 410, in provide
  File "botocore/configprovider.py", line 471, in provide
  File "botocore/session.py", line 330, in get_scoped_config
  File "botocore/session.py", line 226, in _build_profile_map
  File "botocore/session.py", line 357, in full_config
  File "botocore/configloader.py", line 106, in load_config
  File "botocore/configloader.py", line 147, in raw_config_parse
  File "botocore/vendored/six.py", line 92, in __get__
  File "botocore/vendored/six.py", line 115, in _resolve
  File "botocore/vendored/six.py", line 82, in _import_module
ModuleNotFoundError: No module named 'configparser'
[11478] Failed to execute script __main__

Exited with code exit status 255
CircleCI received exit code 255

Expected behavior:

Pass without exception

Additional Information:

My circleci config is very simple

version: 2.1

orbs:
  sam: circleci/[email protected]

workflows:
  main:
    jobs:
      - sam/deploy:
          name: deploy-production
          s3-bucket: production
          stack-name: production
          template: ./functions/template.yaml
          debug: true
          filters:
            branches:
              only: master

Orb version: 5.0.0

What happened:

Deploying a stack like this

- aws-sam-serverless/deploy:
    auth:
      - aws-cli/setup:
          role_arn: 'arn:aws:iam::**************:role/circleci-oidc-role'
          role_session_name: "deploy-local"
    arguments: '--resolve-image-repos --config-env local'
    template: template.yml
    stack_name: sample-stack
    region: eu-west-1

results in an error message:

Error: No such option: --resolve-image-repos --config-env local Did you mean --resolve-image-repos?

I haven't tested if this issue also appears for build_arguments

Expected behavior:

Orb should be able to accept multiple arguments, either by:

• space separated arguments for arguments
• a list of arguments for arguments

Orb version:

circleci/[email protected]

What happened:

environment variables cannot be passed to some string parameters.

for instance:

 deploy-dev:                               
   executor: sam/default                   
   steps:                                  
     - checkout                            
     - sam/install                         
     - sam/deploy:                         
         stack-name: "myapp-dev"      
         template: template.yaml           
         s3-bucket: $S3_ARTIFACT_BUCKET

produces an error Invalid bucket name "$S3_ARTIFACT_BUCKET": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$" or be an ARN matching the regex "^arn:(aws).*:(s3|s3-object-lambda):[a-z\-0-9]+:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-]{1,63}$|^arn:(aws).*:s3-outposts:[a-z\-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9\-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9\-]{1,63}$"
when S3_ARTIFACT_BUCKET is set via context.

Expected behavior:

Deploys, using the --s3-bucket parameter set via context variable S3_ARTIFACT_BUCKET.

Additional Information:

This appears to be a function of how this orb uses params to set env variables then builds commands via the env variables, thus not resolving them if they are references to shell variables themselves.

I'm relatively new to circleci and wasn't sure if this was just me misusing circleci's configuration, but testing this with a few other orbs works correctly. And it even appears in the documents for aws-sam-serverless to work for image-repositories.

Of note, that image-repositories uses an eval echo inside the evaluation of it, which presumably in turn causes the appropriate shell variable to be interpolated.

image-repositories uses this eval in the array evaluation, template has its own eval echo construct, but stack-name s3-bucket, profile-name, and parameter-overrides do not appear to have this additional expansion, which means those values cannot inherit from a context or project environment variable.

I think resolving this is as simple as adding the eval echo wrappers to the other possible parameters, and I can't think of any scenarios where this might break things.

Orb version

What happened

Expected behavior

It should be
https://circleci.com/orbs/registry/orb/circleci/aws-sam-serverless
Rather than
https://circleci.com/orbs/registry/orb/circleci/aws-serverless

What would you like to be added

Upgrade dependency to aws-cli-1.1.0.

Why is this needed

Use awscli2, amongst other improvements and a "stable" release: https://circleci.com/orbs/registry/orb/circleci/aws-cli

Orb version

1.0.2

What happened

I'm trying to deploy this AWS SAM template, which contains an AWS lambda function using the dotnetcore2.1 runtime.

When running the build with this the following config (see code here)

      - aws-serverless/deploy:
          name: deploy-backend-staging
          # requires: and filters: settings removed for brevity
          stack-name: chameleon-game-test
          template: ./template.yaml
          s3-bucket: aws-sam-cli-managed-default-samclisourcebucket-1hoa4cv2sc075

I got the following error (link to build here):

Starting Build inside a container
Building resource 'ChameleonFunction'

Build Failed
Error: We do not support building .NET Core Lambda functions within a container. Try building without the container. Most .NET Core functions will build successfully.

This led me to add the use-container: false parameter (the config with that change is here), but this resulted in the following error (link to build here)

Building resource 'ChameleonFunction'

Build Failed
Error: DotnetCliPackageBuilder:Resolver - No dotnet cli executable found!

Is there some extra config I need to include the dotnet cli and allow the orb to build dotnet Lambda Functions?

Expected behavior

That the build should successfully build and deploy the Lambda Function (as defined in the SAM template) to AWS.

Orb version: circleci/[email protected]

What happened:

version: 2.1

orbs:
  #orb
  aws-serverless: circleci/[email protected]

workflows:
  build_and_deploy:
    jobs:
      - aws-serverless/deploy:
          name: deploy-staging
          s3-bucket: bucket
          stack-name: staging-serverless-actions
          template: template.yaml

Executing this created a cloudformation in "REVIEW_IN_PROGRESS" vs pushing it all the way through. Curious if the CF needs to be initially ran locally prior to being run through circleci.

Describe Request:

The sam local start-api tool has a --warm-containers EAGER option. It makes sense to give the users the ability to use it in CircleCI to reuse already warmed lambdas. This way we could have faster executions.

Examples:

Supporting Documentation Links:

https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-local-start-api.html

Describe Request:

Presently, this orb's install script hard-codes the CPU architecture to be used. This installation won't work on an ARM-based execution environment for CircleCI.

Examples:

https://github.com/CircleCI-Public/aws-sam-serverless-orb/blob/master/src/scripts/install.sh

Supporting Documentation Links:

• https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html (arm64 install is slightly different to x86_64).
• https://circleci.com/docs/using-arm/

Describe Request:

In my deployment job I want to specify sam deploy options like shown in the SAM CLI documentation.

Examples:

sam deploy --config-file samconfig-devqa.toml

Supporting Documentation Links:

https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-deploy.html

Describe Request:

This orb doesn't allow using the parameter --parallel to build multiple lambdas at the same time

Examples:

sam build --template template.yaml --parallel

Supporting Documentation Links:

https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-build.html

Orb version:

Unsure - probably either 6.0.0 or 5.1.0

What happened:

The version for this orb on https://circleci.com/developer/orbs/orb/circleci/aws-sam-serverless is v6.0.0 but the releases here on GitHub are at latest v5.1.0 (https://github.com/CircleCI-Public/aws-sam-serverless-orb/releases).

Trying to use circleci/[email protected] in my CI config errors with:

Cannot find circleci/[email protected] in the orb registry. Check that the namespace, orb name and version are correct.

Using version 6.0 in my config works fine, though I'm not sure what this version incorporates and whether it contains the fix in v5.1.0.

Expected behavior:

Versions should be consistent and clear as to which to use.

Additional Information:

What would you like to be added

sam deploy supports the --no-fail-on-empty-changeset option. I want this orb to support that too.

Right now, if there's no change for sam to deploy, it fails loudly with an exit code 1, which in turn causes CircleCI to mark the job as failed. This option would silence that, as having nothing to deploy shouldn't fail the build.

Why is this needed

This issue in AWS goes into more detail on why they added the --no-fail-on-empty-changeset option to sam deploy to start with. Basically failing on no change doesn't make sense in a CI/CD pipeline.

I can make a PR to solve this issue. One thing I want to get feedback though is whether to make --no-fail-on-empty-changeset the default. Personally I think that's the sensible thing to do for a CI/CD pipeline. Having said that, I can also see it could be confusing/bad form to change default behavior.

Orb version:

4.0.0

What happened:

When specifying parameter-overrides and not specifying additional-args, CLI will happen error.

Error message:

+ sam deploy --stack-name example-stack --region us-east-1 --s3-bucket example-bucket --profile default --parameter-overrides ParameterKey=dummy,ParameterValue=dummy ''
Usage: sam deploy [OPTIONS]
Try 'sam deploy -h' for help.

Error: Invalid value for '--parameter-overrides':   is not in valid format. It must look something like 'ParameterKey=KeyPairName,ParameterValue=MyKey ParameterKey=InstanceType,ParameterValue=t1.micro' or 'KeyPairName=MyKey InstanceType=t1.micro'

Reproduced workflow:

version: 2.1

orbs:
  aws-cli: circleci/[email protected]
  aws-sam: circleci/[email protected]

jobs:
  lambda-edge:
    docker:
      - image: cimg/node:16.18
    steps:
      - aws-cli/setup
      - aws-sam/install
      - checkout
      - aws-sam/deploy:
          parameter-overrides: ParameterKey=dummy,ParameterValue=dummy
          s3-bucket: example-bucket
          stack-name: example-stack

workflows:
  version: 2
  sample:
    jobs:
      - lambda-edge

Expected behavior:

sam command works fine

Additional Information:

Just my guess, but setting $SAM_PARAM_ADDITIONAL_ARGS may cause this error.
If I specify additional-args parameters, the command line works fine.

if [ -n "$SAM_PARAM_PARAMETER_OVERRIDES" ]; then
    set -- "$@" --parameter-overrides "$(eval "echo $SAM_PARAM_PARAMETER_OVERRIDES")"
fi
if [ -z "$SAM_PARAM_ADDITIONAL_ARGS" ]; then
    set -- "$@" "$SAM_PARAM_ADDITIONAL_ARGS"  # <--- This may add '' to $@
fi

set -x
sam deploy "$@"
set +x

Orb version: 2.1.0

What happened:

Last month, AWS released Container Image Support for Lambda functions which allows us to define Dockerfiles, build images using these, and run functions within them.

So I've written template.yaml to include the container image definition:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Resources:
  HelloFunction:
    Type: AWS::Serverless::Function
    Properties:
      PackageType: Image
      ImageUri: <removed>.dkr.ecr.eu-west-1.amazonaws.com/<removed>:latest
    Metadata:
      DockerContext: .
      Dockerfile: Dockerfile
      DockerTag: latest

(where <removed>s were actual values.)

and .circle/xconfig.yml as:

version: 2.1
orbs:
  aws-sam: circleci/[email protected]
workflows:
  build_and_deploy:
    jobs:
      - aws-sam/deploy:
          s3-bucket: hello
          stack-name: hello
          template: ./template.yaml

Obviously s3-bucket isn't needed as SAM deploys container images to ECR rather than S3. What's needed instead of s3-bucket is something like image-repository.

If I tried running the above on CircleCI, it fails at the sam package stage:

Error: Missing option '--image-repository' or '--image-repositories'

I tried adding a samconfig.yoml with:

version=0.1

[default.package.parameters]
image_repository = "<removed>.dkr.ecr.eu-west-1.amazonaws.com/<removed>"

[default.deploy.parameters]
image_repository = "<removed>.dkr.ecr.eu-west-1.amazonaws.com/<removed>"

Then the package stage worked but deploy didn't.

Expected behavior:

It packages and deploys the container image.