Giter Club home page Giter Club logo

waffle_gcs's Introduction

Waffle GCS

Build Status

Google Cloud Storage for Waffle

What's Waffle?

Waffle (formerly Arc) is a file uploading library for Elixir. It's main goal is to provide "plug and play" file uploading and retrieval functionality for any storage provider (e.g. AWS S3, Google Cloud Storage, etc).

What's Waffle GCS?

Waffle GCS provides an integration between Waffle and Google Cloud Storage. It is (in my opinion) the spiritual successor to arc_gcs. If you want to easily upload and retrieve files using Google Cloud Storage as your provider, and you also use Waffle, then this library is for you.

What's different from arc_gcs?

The major three differences are:

  1. Transitions from Arc to Waffle.
  2. Uses the official Google Cloud API client for Elixir rather than constructing XML requests and sending them over HTTP.
  3. (In Progress) Implements the v4 URL signing process in addition to the existing v2 process.

Because Google now officially builds client libraries for Elixir, it is more maintainable to use those libraries rather than relying on the older XML API. The v2 URL signing process is also being deprecated in favor of the v4 process. Although Google will give plenty of advance notice if/when the v2 process is becoming unsupported, it's again more maintainable to use Google best practices to ensure future compatibility.

Installation

Add it to your mix dependencies:

defp deps do
  [
    ...,
    {:waffle_gcs, "~> 0.1"}
  ]
end

Configuration

All configuration values are stored under the :waffle app key. E.g.

config :waffle,
  storage: Waffle.Storage.Google,
  bucket: "gcs-bucket",
  storage_dir: "uploads/waffle"

Note: a valid bucket name is a required config. This can either be a hard-coded string (e.g. "gcs-bucket") or a system env tuple (e.g. {:system, "WAFFLE_BUCKET"}). You can also override this in your definition module (e.g. def bucket(), do: "my-bucket").

Authentication is done through Goth which requires credentials (https://github.com/peburrows/goth#installation).

URL Signing

If your bucket/object permissions do not allow for public access, you will need to use signed URLs to give people access to the uploaded files. This is done by either passing signed: true to CloudStorage.url/4 or by setting it in the configs (config :waffle, signed: true). The default expiration time is one hour but can be changed by setting the :expires_in config/option value. The value is the number of seconds the URL should remain valid for after generation.

GCS object headers

You can specify custom object headers by defining gcs_object_headers/2 in your definition which returns keyword list or map. E.g.

def gcs_object_headers(_version, {_file, _scope}) do
  [contentType: "image/jpeg"]
end

The list of all the supported attributes can be found here: https://hexdocs.pm/google_api_storage/GoogleApi.Storage.V1.Model.Object.html.

waffle_gcs's People

Contributors

tyler-eon avatar almirsarajcic avatar iangreenleaf avatar montebrown avatar

Stargazers

Anthony Kim avatar  avatar Nelson Estevão avatar Felipe Menegazzi avatar Oscar Olivera avatar kimihito avatar Klemen Sever avatar Thibaut Decaudain avatar James Rissler avatar Jeroen Visser avatar Guillaume Paquet avatar Nick C avatar Andrey Chernyshev avatar Franco Catena avatar  avatar justed avatar Olga Novikova avatar Andrew Molchanov avatar Oleg Balbekov avatar Aleksey Glukhov avatar  avatar  avatar Mikhail Morgunov avatar Boris Kuznetsov avatar

Watchers

James Rissler avatar Boris Kuznetsov avatar  avatar

waffle_gcs's Issues

Generates wrong URL for CDN

If I have CDN in front of GCS bucket and have this in my config:

config :waffle,
  storage: Waffle.Storage.Google.CloudStorage,
  bucket: "bucket_name",
  asset_host: "cdn-domain.com"

Avatar.url("image.jpg") will generate URLs that have bucket name in the path, e. g. https://cdn-domain.com/bucket_name/uploads/image.jpg instead of https://cdn-domain.com/uploads/image.jpg.

Storage adapter Waffle.Storage.S3 generates URLs correctly without bucket name in the path. GCS adapter should replicate that behaviour.

Is this project dead?

There has been many attempts to fix some issues with it, but non merged for a while now. If it's dead and we need to make a new one, please tell us.

Unable to generate a signed URL on GAE with default service account

Hey there,

I have opened the issue on Goth here peburrows/goth#107 but it may very well be an issue with waffle_gcs or with my code / config entirely. I am sorry if this is wrong place.

In short: I am using waffle_gcs on GAE, with a default service account and I am not providing any config, just relying on Goth picking up the defaults.

The service account is a Storage Admin, so has full access to uploading files to buckets. And indeed, uploading itself works. The app, however, crashes every time we are trying to generate signed URL. The issue seems to be with this line:

https://github.com/tyler-eon/waffle_gcs/blob/master/lib/waffle/storage/google/url_v2.ex#L82

which returns :error. Any ideas you may have I appreciate...

Cannot pass headers

When the image gets uploaded, when I open image URL it gets downloaded instead of being shown in the browser, because the Content-Type header is missing.

For S3 adapter uploader definition can have s3_object_headers/2 function that returns keywords list. Example:

def s3_object_headers(_version, {_file, _scope}) do
  [content_type: "image/jpeg"]
end

Waffle GCS adapter currently doesn't have this option, but Arc GCS adapter does (https://github.com/martide/arc_gcs/blob/ad6826124ac176fb52e00869f49a19cde860e227/lib/arc/storage/gcs.ex#L133).

Add support for recent goth versions.

At this moment, this package is not compatible with the new versions of Goth. We should use Goth.fetch/1 instead of Goth.Token.for_scope/1

Deprecation:

warning: Goth.Token.for_scope/1 is deprecated. Use Goth.fetch/1 instead

I'll be happy to contribute in this project to add this support.
Are you open to receive new PRs? Is the library still maintained?

Doesn't work with updated Waffle

When I run mix deps.get I get the following error message:

Resolving Hex dependencies...

Failed to use "waffle" (version 1.1.0) because
  waffle_gcs (versions 0.1.0 and 0.1.1) requires ~> 0.0.3
  mix.exs specifies ~> 1.1.0

** (Mix) Hex dependency resolution failed, change the version requirements of your dependencies or unlock them (by using mix deps.update or mix deps.unlock). If you are unable to resolve the conflicts you can try overriding with {:dependency, "~> 1.0", override: true}

Needs an update for waffle from version 0.0.3 to 1.1.0.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.