Giter Club home page Giter Club logo

action-s3-cloudfront-smart-deploy's Introduction

linkertinker
S3/Cloudfront Smart Invalidation

AWS Cloudfront invalidation requests cost ๐Ÿ’ฐ once you're past the free limit (1000 / account). Issuing wildcard /_ invalidations is not an option because it causes total cache revalidation and negatively impacts performance. This action provides a smart invalidation algorithm which just works. It issues as few invalidation requests as possible whilst preserving user cache and preventing useless cache drops.

Usage

- name: Upload changes to S3 and issue Cloudfront invalidations
  uses: vladcosorg/action-s3-cloudfront-smart-deploy@v1
  with:
    source: local/path/to/dir
    target: s3://my-bucket-name/
    distribution: DOAJN11MNDAND

Configuration

Key Description Required Default Value Type Example
source Path to sync the files from Yes โ— N/A path or S3 bucket URI relative/path/to/dir
/absolute/path
s3://my-bucket-name/
target Target s3 bucket to sync to Yes โ— N/A S3 bucket URI s3://my-bucket-name/
s3args See here No N/A string --exact-timestamps --delete
distribution Cloudfront distribution ID. No N/A string DOAJN11MNDAND
cfargs See here No N/A string --debug
invalidation-strategy Invalidation strategy See description here No BALANCED BALANCED or PRECISE or FRUGAL FRUGAL
balanced-limit Maximum amount of invalidation requests when using BALANCED strategy No 5 positive number or Ininity 10

Invalidation strategies

BALANCED โœ… recommended

This strategy prioritizes issuing as many precise invalidations as possible (within set limits). If this is not possible, it falls back to a hybrid mode which would issue a mix of targeted invalidations and wildcard invalidations. And finally, if there are too many invalidations, it falls back to wildcard approach, BUT the wildcards are as specific as possible, so that the consumers of the app would redownload as little as possible.

For example if the value of balanced-limit is set to 5, then it will issue up to 5 invalidation requests. The action will never exceed this value.

If the balanced-limit value is too low to perform all necessary precise invalidations, then it will resort to the wildcard approach, partially or completely.

The generated wildcards will try to minimize the number of invalidated files by narrowing its scope.

linkertinker linkertinker linkertinker

FRUGAL

It's a shortcut to the balanced-limit set to 1. It means that the action would always issues at most 1 invalidation request that is going to contain a scoped wildcard if you have more than 1 file that needs to be invalidated.

๐ŸŸ  Attention: This option, whilst very economical towards your AWS invalidation quota, would affect many unrelated paths. Use with caution.

PRECISE

It's a shortcut to the balanced-limit set to Infinity. It means that the action would always issue precise invalidation requests, potentially

๐Ÿงจ Warning: This option could potentially cost you a significant amount of money, because it will issue 1 invalidation request per 1 changed files. If you have lots of changed files that are frequently deployed to S3, think again before using this option.

Full example

jobs:
  deploy:
    name: Upload to Amazon S3
    runs-on: ubuntu-latest
    # These permissions are needed to interact with GitHub's OIDC Token endpoint.
    permissions:
      id-token: write
      contents: read
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Configure AWS credentials from Test account
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::111111111111:role/my-github-actions-role-test
          aws-region: us-east-1
      - name: Upload changes to S3 and issue Cloudfront invalidations
        uses: vladcosorg/action-s3-cloudfront-smart-deploy@v1
        with:
          source: local/path/to/dir
          target: s3://my-bucket-name/
          distribution: DOAJN11MNDAND

Motivation

The available actions are using a simple yet inefficient approach that invalidates the changes using a precise 1 file -> 1 invalidation request approach, which potentially can result in a quite large monthly bill, provided that your project is updated frequently and has a lot of files (exactly the case at my company). Another approach is to issue general, root invalidations like /* which would cause the consumers of your app to redownload the assets which did not actually change.

This action features a BALANCED approach which is as precise and as economical as you want it to be.

License

This project is distributed under the MIT license.

action-s3-cloudfront-smart-deploy's People

Contributors

mxcl avatar myappchetzof[bot] avatar vladcos avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

action-s3-cloudfront-smart-deploy's Issues

local build/test

re #32 or any other contributions, what do you recommend for building locally?
i've cloned it to modify and test in a project i'm working on.
i've had to disable a bunch of tests and after using npx projen build, have to delete the generated .gitignore so that i can vendor the built action.
can you make it easier to build and use locally?

support query string for edge-routed content

i have a cloudfront distro with a viewer request edge lambda on the cache behavior that switches between two s3 origins based on some behavior, and to differentiate and to be able to invalidate one or the other, the lambda will add a query string. can you support invalidating with a query string?

s3args issue

Hi,

Thanks for the action. it works great. except when I I use the s3args option.

Here is my config

      - name: Upload changes to S3 and issue Cloudfront invalidations
        uses: vladcosorg/action-s3-cloudfront-smart-deploy@v1
        with:
          source: dist/
          target: "s3://${{ secrets.S3_BUCKET}}/"
          s3args: --delete --exact-timestamps
          distribution: ${{ secrets.CLOUDFRONT_DISTRIBUTION_ID}}

But it dose not remove the files I dont have at source, or newly built files.
And looked at my github action log it still runs using the default flags.

/usr/local/bin/aws s3 sync dist/ s3://***/ --no-progress --size-only

also here's a screenshot of my github action log

Screenshot 2024-01-29 at 15 42 10

quoting of invalidation args

i haven't dug in enough but i don't know that the exec/spawn call uses a shell, so when i do the invalidation with * in it, it complains. i think maybe you don't need to map/convert the paths to a quoted path, unless in some cases it would be running under a shell. what i can say is that i took that out of runner.ts so that it's just

  await getExecOutput('aws', [
    'cloudfront',
    'create-invalidation',
    '--distribution-id',
    distribution,
    '--paths',
    ...invalidationCandidates,
    ...cfargs,
  ])

and stopped getting the error:

/usr/local/bin/aws cloudfront create-invalidation --distribution-id E2Q69Q44OA19XZ --paths '/browser/*'

An error occurred (InvalidArgument) when calling the CreateInvalidation operation: Your request contains one or more invalid invalidation paths.

(this one has single quotes because i tried that first thinking the shell was trying to glob using the *, but now i think that aws just balked at there being quotes in it)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.