Giter Club home page Giter Club logo

aws-serverlessrepo-python's Introduction

AWS Serverless Application Repository - Python

A Python library with convenience helpers for working with the AWS Serverless Application Repository.

Installation

Simply use pip to install the library:

pip install serverlessrepo

Basic Usage

The serverlessrepo module provides a simple interface for publishing applications and managing application permissions. To get started, import the serverlessrepo module:

import serverlessrepo

Publish Applications

publish_application(template, sar_client=None)

Given an AWS Serverless Application Model (SAM) template, it publishes a new application using the specified metadata in AWS Serverless Application Repository. If the application already exists, it updates metadata of the application and publishes a new version if specified in the template.

For example:

import boto3
import yaml
from serverlessrepo import publish_application

sar_client = boto3.client('serverlessrepo', region_name='us-east-1')

with open('template.yaml', 'r') as f:
    template = f.read()
    # if sar_client is not provided, we will initiate the client using region inferred from aws configurations
    output = publish_application(template, sar_client)

    # Alternatively, pass parsed template as a dictionary
    template_dict = yaml.loads(template)
    output = publish_application(template_dict, sar_client)

The output of publish_application has the following structure:

{
    'application_id': 'arn:aws:serverlessrepo:us-east-1:123456789012:applications/test-app',
    'actions': ['CREATE_APPLICATION'],
    'details': {
        'Author': 'user1',
        'Description': 'hello',
        'Name': 'hello-world',
        'SemanticVersion': '0.0.1',
        'SourceCodeUrl': 'https://github.com/hello'}
    }
}

There are three possible values for the actions field:

  • ['CREATE_APPLICATION'] - Created a new application.
  • ['UPDATE_APPLICATION'] - Updated metadata of an existing application.
  • ['UPDATE_APPLICATION', 'CREATE_APPLICATION_VERSION'] - Updated metadata of an existing application and created a new version, only applicable if a new SemanticVersion is provided in the input template.

details has different meaning based on the actions taken:

  • If a new application is created, it shows metadata values used to create the application.
  • If application is updated, it shows updated metadata values.
  • If application is updated and new version is created, it shows updated metadata values as well as the new version number.

update_application_metadata(template, application_id, sar_client=None)

Parses the application metadata from the SAM template and only updates the metadata.

For example:

import boto3
import yaml
from serverlessrepo import update_application_metadata

sar_client = boto3.client('serverlessrepo', region_name='us-east-1')

with open('template.yaml', 'r') as f:
    template = f.read()
    application_id = 'arn:aws:serverlessrepo:us-east-1:123456789012:applications/test-app'
    # if sar_client is not provided, we will initiate the client using region inferred from aws configurations
    update_application_metadata(template, application_id, sar_client)

    # Alternatively, pass parsed template as a dictionary
    template_dict = yaml.loads(template)
    update_application_metadata(template_dict, application_id, sar_client)

Manage Application Permissions

make_application_public(application_id, sar_client=None)

Makes an application public so that it's visible to everyone.

make_application_private(application_id, sar_client=None)

Makes an application private so that it's only visible to the owner.

share_application_with_accounts(application_id, account_ids, sar_client=None)

Shares the application with specified AWS accounts.

Examples

import boto3
from serverlessrepo import (
    make_application_public,
    make_application_private,
    share_application_with_accounts
)

application_id = 'arn:aws:serverlessrepo:us-east-1:123456789012:applications/test-app'
sar_client = boto3.client('serverlessrepo', region_name='us-east-1')

# Share an application publicly
make_application_public(application_id, sar_client)

# Make an application private
make_application_private(application_id, sar_client)

# Share an application with other AWS accounts
share_application_with_accounts(application_id, ['123456789013', '123456789014'], sar_client)

Development

  • Fork the repository, then clone to your local:
    • git clone https://github.com/<username>/aws-serverlessrepo-python.git
  • Set up the environment: make init
    • It installs Pipenv to manage package dependencies. Then it creates a virtualenv and installs dependencies from Pipfile (including dev).
  • Install new packages: pipenv install [package names]
    • Pipenv will automatically update Pipfile and Pipfile.lock for you.
    • Add new dependencies to setup.py install_requires if they are needed for consumers of this library.
  • Verify that everything works: make build
    • You can run make test separately to verify that tests pass.
    • Check code style with make flake and make lint.
  • Make code changes, run all verifications again before sending a Pull Request: make pr

License

This library is licensed under the Apache 2.0 License.

aws-serverlessrepo-python's People

Contributors

gruebel avatar jfuss avatar jpeddicord avatar paoptu023 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aws-serverlessrepo-python's Issues

Better BadRequestException for Non Support Resources

SAM CLI was cut this issue, which outputs a stack trace when Unsupported Resources are within the template that is being published.

I think it makes sense to include some additional error handling within this library that provides a better error that the boto3 error that gets returned. This is similar to wrapping of the Botocore ClientError that this library already wraps.

Proposal

Add a UnsupportedResourcesError in exceptions.py. We can then catch botocore.errorfactory.BadRequestException in this try: except: block and wrap it in the UnsupportedResourcesError with a message of "The template contains unsupported resource types. [XX::XX::XX]", which can be parsed out of the exception.

version conflict for using default way of pipenv

Hi, I find the last version of serverlessrepo 0.1.6 using pipenv package, but If I using pip install that it will conflict with other tools like localstack.

serverlessrepo 0.1.6 has requirement six~=1.11.0, but you'll have six 1.12.0 which is incompatible.  *********error here***********
Installing collected packages: six, airspeed, localstack
  Found existing installation: six 1.10.0
    Uninstalling six-1.10.0:
      Successfully uninstalled six-1.10.0
  Found existing installation: airspeed 0.5.5.dev20160812
    Uninstalling airspeed-0.5.5.dev20160812:
      Successfully uninstalled airspeed-0.5.5.dev20160812
Successfully installed airspeed-0.5.10 localstack-0.8.10 six-1.12.0

the last version localstack require six 1.12.0 while the serverlessrepo still lock the six version 1.11.0. Can you make your Pipfile.lock file using

"six": {
            "hashes": [
                "sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9",
                "sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb"
            ],
            "version": "==1.11.0"
        },

to

"version": ">=1.11.0"

since it will block six or other dependence upgrade later on.

sam publish duplicate semantic version issue

Description

When using sam publish command to publish a new version of a SAR app, if I forget to update the semantic version in the template metadata, it succeeds leading the user to think they published a new version.

Steps to reproduce

We use this script when publishing to SAR:
https://github.com/awslabs/aws-athena-query-federation/blob/master/tools/publish.sh

It has the exact commands that get run. The steps to reproduce are:

  1. use the script to publish an initial semantic version to SAR
  2. deploy an instance of that app
  3. make a code change to the app but forget to change the semantic version
  4. publish the code change using the script
  5. attempt to deply an update to the previous app instance.

You'll find that step #4 doesn't throw and error but that after step 5 which also won't error, the deployed app will still be running the code from step 1.

Expected result

Step #4 should somehow throw an error or helpful information to indicate a new version is not published so that we can stop before running step 5.

Missing required parameter in input: "ApplicationId"

When trying to run sam publish -t packaged.yaml to update an application that already exists I get the following error:

Traceback (most recent call last):
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/serverlessrepo/publish.py", line 45, in publish_application
    response = sar_client.create_application(**request)
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/botocore/client.py", line 635, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.ConflictException: An error occurred (ConflictException) when calling the CreateApplication operation: Application with name NewRelic-log-ingestion already exists

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/kolanos/.virtualenvs/aws/bin/sam", line 8, in <module>
    sys.exit(cli())
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 96, in wrapped
    raise exception  # pylint: disable=raising-bad-type
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 62, in wrapped
    return_value = func(*args, **kwargs)
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/samcli/commands/publish/command.py", line 54, in cli
    do_cli(ctx, template_file, semantic_version)  # pragma: no cover
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/samcli/commands/publish/command.py", line 77, in do_cli
    publish_output = publish_application(template_data)
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/serverlessrepo/publish.py", line 57, in publish_application
    sar_client.update_application(**request)
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/botocore/client.py", line 608, in _make_api_call
    api_params, operation_model, context=request_context)
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/botocore/client.py", line 656, in _convert_to_request_dict
    api_params, operation_model)
  File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/botocore/validate.py", line 297, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Missing required parameter in input: "ApplicationId"

I'm running serverlessrepo==0.1.10. But tried with 0.1.9 and got the same error.

The following line appears to fail to parse an application ID in serverlessrepo.publish:publish_application:

application_id = parse_application_id(error_message)

Can't use pip install serverlessrepo anymore

$ pip install serverlessrepo
Collecting serverlessrepo
  Could not find a version that satisfies the requirement serverlessrepo (from versions: )
No matching distribution found for serverlessrepo
$ python --version
Python 3.5.2

First started to fail 2 days ago: https://travis-ci.org/sblask/dotfiles/builds
Seems to be a sudden incompatibility to Python 3.5 which is the standard Python in Ubuntu 16.04 (LTS)

Fn::Sub or Fn::Join not working for SAR template

Description:

Don't know if this is a CloudFormation bug or a SAM bug but since the error appears to be coming from the SAM CLI stack I'm making it here.
I'm unable to utilize CloudFormation functions for parameters in my template under the AWS::ServerlessRepo::Application Metadata section to deploy my application to the Serverless Application Repo (SAR).

Steps to reproduce the issue:
Here's my template.yaml

AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31
Description: Lambda layer to house all shared code with backend apps
Parameters:
  DeploymentEnvironment:
    Type: String
    Default: dev

Resources:
  NodeModulesLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      LayerName: !Sub 'serverless-shared-node-modules-${DeploymentEnvironment}-grant'
      ContentUri: ./
      CompatibleRuntimes:
        - nodejs8.10
        - nodejs10.x
      # could specify Delete if we want to clean up layers?
      RetentionPolicy: Retain

Outputs:
  LayerVersion:
    Description: Node Modules Layer ARN Reference
    Value: !Ref NodeModulesLayer

Metadata:
  AWS::ServerlessRepo::Application:
    Name: !Sub serverless-shared-node-modules-${DeploymentEnvironment}-grant
    Description: >
      Lambda layer to house all Node-based shared code with backend/serverless apps
    Author: grant
    SemanticVersion: 1.0.0
    ReadmeUrl: README.md
    SourceCodeUrl: <url>
  1. I created an s3 bucket named lambda-layers-dev
  2. Run the following commands from the terminal:
    sam package --template-file template.yaml --s3-bucket lambda-layers-dev --output-template-file packaged.yaml
    sam publish --template packaged.yaml

Observed result:
I get the following errors stack trace:

Uploading to d533bb334dcca7b4904d6e56c7e319a3  107675 / 107675.0  (100.00%)
Successfully packaged artifacts and wrote output template to file packaged.yaml.
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file <path>lambda-layers/packaged.yaml --stack-name <YOUR STACK NAME>
$ sam publish --template packaged.yaml
2019-10-25 10:24:56 Found credentials in shared credentials file: ~/.aws/credentials
Traceback (most recent call last):
  File "/usr/local/bin/sam", line 11, in <module>
    load_entry_point('aws-sam-cli==0.23.0', 'console_scripts', 'sam')()
  File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 93, in wrapped
    raise exception  # pylint: disable=raising-bad-type
  File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 62, in wrapped
    return_value = func(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/samcli/commands/publish/command.py", line 52, in cli
    do_cli(ctx, template, semantic_version)  # pragma: no cover
  File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/samcli/commands/publish/command.py", line 75, in do_cli
    publish_output = publish_application(template_data)
  File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/serverlessrepo/publish.py", line 45, in publish_application
    response = sar_client.create_application(**request)
  File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/botocore/client.py", line 634, in _make_api_call
    api_params, operation_model, context=request_context)
  File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/botocore/client.py", line 682, in _convert_to_request_dict
    api_params, operation_model)
  File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/botocore/validate.py", line 297, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter Name, value: OrderedDict([('Fn::Sub', 'serverless-shared-node-modules-${DeploymentEnvironment}-ga892')]), type: <class 'collections.OrderedDict'>, valid types: <class 'str'>
  1. I've tried Fn::Sub:, Fn::Join: and !Sub. All throw the same error.

Expected result:
When I change in my template the Name property under the ServerlessRep to just a string with no CF functions the layer publishes fine.

I referenced this page for the params for that section:
https://docs.aws.amazon.com/en_pv/serverless-application-model/latest/developerguide/serverless-sam-template-publishing-applications-metadata-properties.html

Sorry if this is me not understanding/finding some of the documentation and if so let me know what I missed. Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.