Giter Club home page Giter Club logo

middleman-cloudfront's Introduction

Middleman CloudFront Build Status Dependency Status Code Climate

A deploying tool for middleman which allows you to interact with Amazon CloudFront. Some of its features are:

  • CloudFront cache invalidation;
  • Ability to call it from command line and after middleman build;
  • Ability to filter files which are going to be invalidated by regex;

Usage

Installation

Add this to Gemfile:

gem "middleman-cloudfront"

Then run:

bundle install

Configuration

Edit config.rb and add:

activate :cloudfront do |cf|
  cf.access_key_id = 'I'
  cf.secret_access_key = 'love'
  cf.distribution_id = 'cats'
  # cf.filter = /\.html$/i  # default is /.*/
  # cf.after_build = false  # default is false
end

On Amazon use following parameters inside your IAM policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1409254980000",
      "Effect": "Allow",
      "Action": [
        "cloudfront:CreateInvalidation",
        "cloudfront:GetDistribution"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}

Running

If you set after_build to true cache would be automatically invalidated after build:

bundle exec middleman build

Otherwise you should run it through commandline interface like so:

bundle exec middleman invalidate

or from within Middleman, optionally specifying a list of files to invalidate:

# Invalidate automatic selection of files from build directory
invalidate

# Invalidate explicit list of files
invalidate %w(/index.html /images/example.png)

S3 + Cloudfront deploying

In real world this gem shouldn't be used alone, but as a part of your deployment solution. As for me I use it with middleman-sync and my configuration file looks like this:

configure :build do
  # so there would be no need in invalidationg css-js files on cdn
  activate :asset_hash
end

activate :sync do |sync|
  sync.fog_provider = 'AWS'
  sync.fog_directory = '...'
  sync.fog_region = 'us-west-1'
  sync.aws_access_key_id = ENV['AWS_ACCESS_KEY']
  sync.aws_secret_access_key = ENV['AWS_SECRET']
  sync.existing_remote_files = 'delete'
  sync.gzip_compression = true
end

activate :cloudfront do |cf|
  cf.access_key_id = ENV['AWS_ACCESS_KEY']
  cf.secret_access_key = ENV['AWS_SECRET']
  cf.distribution_id = '...'
  cf.filter = /\.html$/i
end

And when I want to deploy my site I do:

AWS_ACCESS_KEY= AWS_SECRET= bundle exec middleman sync
AWS_ACCESS_KEY= AWS_SECRET= bundle exec middleman invalidate

If you use middleman-s3_sync for deployment, you can use its after_s3_sync hook to automatically invalidate updated files after syncing:

after_s3_sync do |files_by_status|
  invalidate files_by_status[:updated]
end

NOTE: The after_s3_sync hook only works with middleman-s3_sync v3.x and below. It has been removed in v4.0.

middleman-cloudfront's People

Contributors

andrewkvalheim avatar andrusha avatar bostonaholic avatar breezeight avatar chriseidhof avatar joshukraine avatar karlfreeman avatar manuelmeurer avatar query-string avatar rosskevin avatar tmaier avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

middleman-cloudfront's Issues

Middleman v4.0 support

Middleman is currently on its way to release verison 4.0. Unfortunately, middleman-cloudfront still has a dependency which restricts versions > 3.

You can find recent middleman alpha releases here: https://rubygems.org/gems/middleman-core (4.0.0.alpha.6 is the most recent one as of this writing)

I would really appreciate it, if this very helpful gem would introduce support for version 4.0

gem old dependency

Hey @andrusha ! Thanks for the gem! First time using it and looks awesome, although I ran into this issue. Could you pls update the dependencies? Thanks a lot!

Bundler could not find compatible versions for gem "middleman-core":

  In snapshot (Gemfile.lock):

    middleman-core (= 4.1.9)

  In Gemfile:

    middleman (>= 4.0.0) was resolved to 4.1.9, which depends on
      middleman-core (= 4.1.9)

    middleman-cloudfront (~> 0.2.1) was resolved to 0.2.1, which depends on
      middleman-core (~> 3.0)

    .......

listen gem version conflict

Fetching gem metadata from http://rubygems.org/..
Bundler could not find compatible versions for gem "listen":
  In Gemfile:
    middleman-cloudfront (>= 0) ruby depends on
      listen (~> 0.4.7) ruby

    guard-livereload (>= 0) ruby depends on
      listen (0.7.2)

My ruby version: ruby-1.9.3-p362

RubyGems dependencies not correct

Despite the correct gemspec for 0.2.1 here on Github, the 0.2.1 on rubygems.org is still stuck at middleman-core ~> 3.0 for some reason. I'm not sure how that might have happened, as you commit history seems to indicate you update the dependency before bumping the version!

In any case, it's breaking Bundler for automatic installs. You might want to bump the version and release it again.

I'll do a local install from this repo for the time being.

Undefined method `invalidations'

When I run invalidate after build, I get the following error message:

## Invalidating files on CloudFront
Invalidating 6 files. It might take 10 to 15 minutes until all files are invalidated.
Please check the AWS Management Console to see the status of the invalidation.
/Users/juanjo/.rvm/gems/ruby-2.1.2/gems/middleman-cloudfront-0.1.0/lib/middleman-cloudfront/commands.rb:64:in `invalidate': undefined method `invalidations' for nil:NilClass (NoMethodError)
    from /Users/juanjo/.rvm/gems/ruby-2.1.2/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
    from /Users/juanjo/.rvm/gems/ruby-2.1.2/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
    from /Users/juanjo/.rvm/gems/ruby-2.1.2/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
    from /Users/juanjo/.rvm/gems/ruby-2.1.2/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
    from /Users/juanjo/.rvm/gems/ruby-2.1.2/gems/middleman-core-3.3.7/lib/middleman-core/cli.rb:72:in `method_missing'
    from /Users/juanjo/.rvm/gems/ruby-2.1.2/gems/thor-0.19.1/lib/thor/command.rb:29:in `run'
    from /Users/juanjo/.rvm/gems/ruby-2.1.2/gems/thor-0.19.1/lib/thor/command.rb:126:in `run'
    from /Users/juanjo/.rvm/gems/ruby-2.1.2/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
    from /Users/juanjo/.rvm/gems/ruby-2.1.2/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
    from /Users/juanjo/.rvm/gems/ruby-2.1.2/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
    from /Users/juanjo/.rvm/gems/ruby-2.1.2/gems/middleman-core-3.3.7/lib/middleman-core/cli.rb:20:in `start'
    from /Users/juanjo/.rvm/gems/ruby-2.1.2/gems/middleman-core-3.3.7/bin/middleman:18:in `<top (required)>'
    from /Users/juanjo/.rvm/gems/ruby-2.1.2/bin/middleman:23:in `load'
    from /Users/juanjo/.rvm/gems/ruby-2.1.2/bin/middleman:23:in `<main>'
    from /Users/juanjo/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15:in `eval'
    from /Users/juanjo/.rvm/gems/ruby-2.1.2/bin/ruby_executable_hooks:15:in `<main>'

I am using:

middleman-3.3.7
middleman-cloudfront-0.1.0
middleman-s3_sync-3.0.35

And this is an excerpt of my config.rb:

aws_config = YAML::load(File.open('aws.yml'))


activate :s3_sync do |s3_sync|
  s3_sync.bucket                = aws_config['s3_bucket']
  s3_sync.region                = aws_config['aws_region']
  s3_sync.aws_access_key_id     = aws_config['access_key_id']
  s3_sync.aws_secret_access_key = aws_config['secret_access_key']
  s3_sync.delete                = true
  s3_sync.after_build           = true
end

activate :cloudfront do |cf|
  cf.access_key_id              = aws_config['access_key_id']
  cf.secret_access_key          = aws_config['secret_access_key']
  cf.distribution_id            = aws_config['cloud_front_dist_id']
  cf.filter                     = /\.html$/i
  cf.after_build                = false
end

fog gem version conflict when using with middleman-sync

Fetching gem metadata from http://rubygems.org/..
Bundler could not find compatible versions for gem "fog":
  In Gemfile:
    middleman-cloudfront (>= 0) ruby depends on
      fog (~> 1.9.0) ruby

    middleman-sync (~> 3.0.9) ruby depends on
      fog (1.10.1)

undefined method

I've been running across this error since upgrading to cloud front 0.1.1, was hoping someone might have insight into the error. Thanks!

middleman-cloudfront/commands.rb:70:ininvalidate': undefined method invalidations' for nil:NilClass (NoMethodError)

Option to not wait until completion?

When I try to invalidate, I always get a timeout after a while:

11:36:21 ~/code/myapp (master) $ bundle exec middleman invalidate
## Invalidating files on CloudFront
Please wait while Cloudfront is reloading 15 paths, it might take up to 10 minutes
/Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/fog-1.11.1/lib/fog/core/wait_for.rb:10:in `wait_for': The specified wait_for timeout (600 seconds) was exceeded (Fog::Errors::TimeoutError)
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/fog-1.11.1/lib/fog/core/model.rb:64:in `wait_for'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/middleman-cloudfront-0.0.5/lib/middleman-cloudfront/commands.rb:34:in `block in invalidate'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/middleman-cloudfront-0.0.5/lib/middleman-cloudfront/commands.rb:31:in `each_slice'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/middleman-cloudfront-0.0.5/lib/middleman-cloudfront/commands.rb:31:in `invalidate'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/thor-0.15.4/lib/thor/task.rb:27:in `run'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/thor-0.15.4/lib/thor/invocation.rb:120:in `invoke_task'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/thor-0.15.4/lib/thor.rb:275:in `dispatch'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/thor-0.15.4/lib/thor/base.rb:425:in `start'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/middleman-core-3.1.0.rc.1/lib/middleman-core/cli.rb:77:in `method_missing'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/thor-0.15.4/lib/thor/task.rb:29:in `run'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/thor-0.15.4/lib/thor/task.rb:126:in `run'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/thor-0.15.4/lib/thor/invocation.rb:120:in `invoke_task'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/thor-0.15.4/lib/thor.rb:275:in `dispatch'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/thor-0.15.4/lib/thor/base.rb:425:in `start'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/middleman-core-3.1.0.rc.1/lib/middleman-core/cli.rb:22:in `start'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/gems/middleman-core-3.1.0.rc.1/bin/middleman:18:in `<top (required)>'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/bin/middleman:23:in `load'
    from /Users/manuel/.rbenv/versions/2.0.0-p195/lib/ruby/gems/2.0.0/bin/middleman:23:in `<main>'

When I check the AWS console, I see that the invalidation is there, it's "InProgress", and it's invalidating the correct paths.

Would you be open to adding an option that lets Middleman::Cli::CloudFront#invalidate return immediately, reporting the invalidation status, instead of waiting for ready??

Add something like a dry-run

It would be great if I could call bundle exec middleman invalidate --dry-run

which would then just output list_files(options.filter) (see commands.rb#L58).

This would allow me to see if my filter regex is correct or if I did some mistake

Uses configure :development block

When I run invalidate after build, I get the following error message

       ## Invalidating files on CloudFront
       == Unknown Extension: livereload
       == Locales: de (Default de)
       == Unknown Extension: livereload
       == Locales: de (Default de)
       == Unknown Extension: livereload
       == Locales: de (Default de)
       == Unknown Extension: livereload
       == Locales: de (Default de)
x       Invalidating 252 files. It might take 10 to 15 minutes until all files are invalidated.
       Please check the AWS Management Console to see the status of the invalidation.

livereload is unknown, as I have it in the development group of my Gemfile.
At least when I run invalidate with middleman build it should respect the :build block and ignore the :development block.
Second, I wonder why it tries to load the configuration four times.

An excerpt of my config.rb

if ENV['AWS_ACCESS_KEY_ID'] && ENV['AWS_SECRET_ACCESS_KEY'] && ENV['AWS_CF_DISTRIBUTION_ID']
  activate :cloudfront do |cf|
    cf.access_key_id = ENV['AWS_ACCESS_KEY_ID']
    cf.secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
    cf.distribution_id = ENV['AWS_CF_DISTRIBUTION_ID']
    cf.after_build = true
  end
end

configure :development do
  # Reload the browser automatically whenever files change
  activate :livereload,
           no_swf: true
  config[:file_watcher_ignore] += [%r{\.idea/}]

  activate :google_analytics do |ga|
    ga.tracking_id = false
  end
end

# Build-specific configuration
configure :build do
  # ...
end

Make compatible with Middleman v3 and v4

After merging #23 (and doing some cleanup), the master branch currently only works with Middleman v4. I would like it to work with Middleman v3 and v4 and then release 1.0. ๐Ÿ˜„
It shouldn't be too hard, we probably only need to make sure to use the v3 syntax for loading the Extension and Invalidate classes (see lib/middleman-bootstrap-navbar/extension.rb here)
First I would like to add Appraisal to test different Middleman versions and started to do so in the appraisals branch.
Since I'm no good at Cucumber, I'm having a hard time writing the cukes though. ๐Ÿ˜„
If someone could help me out here, that would be fantastic.
@andrusha do you have time?

Invalidation using wildcard

I got an email from Amazon:

Amazon CloudFront's invalidation feature, which allows you to remove an object from the CloudFront cache before it expires, now supports the * wildcard character. You can add a * wildcard character at the end of an invalidation path to remove all objects that match this path. In the past, when you wanted to invalidate multiple objects, you had to list every object path separately. Now, you can easily invalidate multiple objects using the * wildcard character.

Using the * wildcard character in the invalidation path is useful for many use cases. For example, if you're doing a new deployment, you can now just use /* to invalidate the entire distribution. Similarly, if you want to invalidate all objects for a specific end user, you can invalidate the content in a directory, for example, /enduser-x-data/*.

This new capability can also help you lower your cost of invalidating multiple objects. Invalidation paths that include the * wildcard (representing multiple objects) incur the same charge as an invalidation path that represents a single object. As before, the first 1,000 invalidation paths each month are provided at no additional charge and above this level, there is a $0.005 charge per invalidation path. But now you pay $0.005 for the /directory-name/* invalidation path even if this path matches thousands of objects. For more information on invalidation pricing, see Amazon CloudFront Pricing.

Maybe we should rework this to make sure we can invalidate with a wildcard (e.g. after an s3 sync?). I haven't tested this yet, though.

Only invalidate updated files

middleman build logs the following

       identical  build/opensearch.xml
       update  build/blog/2013/04/01/index.html
       update  build/blog/2013/03/index.html
       update  build/impressum/index.html

It would be great, if middleman-cloudfront would only invalidate the files market with update

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.