Giter Club home page Giter Club logo

dayron's Introduction

Dayron

Build Status Deps Status Inline docs Coverage Status Twitter

Dayron is a flexible library to interact with RESTful APIs and map resources to Elixir data structures. It works similar of Ecto.Repo but, instead of retrieving data from a database, it has underlying http clients to retrieve data from external HTTP servers.

Installation

  1. Add Dayron to your list of dependencies in mix.exs:
def deps do
  [{:dayron, "~> 0.1"}]
end
  1. Ensure Dayron is started before your application:
def application do
  [applications: [:dayron]]
end
  1. Then run mix deps.get in your shell to fetch the dependencies.

Getting Started

Dayron requires a configuration entry with at least the external url attribute:

# In your config/config.exs file
config :my_app, MyApp.RestRepo,
  url: "http://api.example.com"

Then you must define MyApp.RestRepo somewhere in your application with:

# Somewhere in your application
defmodule MyApp.RestRepo do
  use Dayron.Repo, otp_app: :my_app
end

Defining Models

With modules and structs

Dayron Models are simple modules with use Dayron.Model to implement the required protocol. The resource option defines the path to be used by the HTTP client to retrieve data. A struct must be defined with defstruct to allow json responses mapping.

defmodule MyApp.User do
  # api requests to http://api.example.com/users
  use Dayron.Model, resource: "users"

  # struct defining model attributes
  defstruct name: "", age: 0
end

Reusing Ecto Models

Dayron Models can work together with Ecto Models, allowing data loading from Database and External APIs just selecting the desired Repo. The defined schema will be used by Dayron when parsing server responses. If no resource option is present, the schema source is used as resource name.

defmodule MyApp.User do
  use Ecto.Schema
  use Dayron.Model

  # dayron requests to http://api.example.com/users
  schema "users" do
    field :name, :string
    field :age, :integer, default: 0
  end
end

Retrieving Data

After defining the configuration and model, you are allowed to retrieve data from an external API in a similar way when compared to an Ecto Repo. The example below presents a UsersController where an index action retrieves a list of users from the server, and a show action retrieves a single User:

defmodule MyApp.UsersController do
  use MyApp.Web, :controller

  alias MyApp.User
  alias MyApp.RestRepo

  def index(conn, params)
    conn
    |> assign(:users, RestRepo.all(User))
    |> render("index.html")
  end

  def show(conn, %{"id" => id}) do
    case RestRepo.get(User, id) do
      nil  -> put_status(conn, :not_found)
      user -> render conn, "show.html", user: user
    end
  end

Generating modules

You can generate Dayron modules using the task dayron.gen.model

mix dayron.gen.model User users name age:integer

The first argument is the module name followed by the resource path. The generated model will contain:

  • a model file in lib/your_app/models
  • a test file in test/your_app/models

Both the model and the test path can be configured using the Dayron generators config.

config :dayron, :generators,
  models_path: "web/models",
  models_test_path: "test/models"

The model fields are given using name:type syntax where types can be one of the following:

:array, :integer, :float, :boolean, :string

Omitting the type makes it default to :string

Extra Configuration

Request Headers

Using the configuration you're allowed to set headers that will be sent on every HTTP request made by Dyron. In the configuration example below, the access-token header is sent on every request:

# In your config/config.exs file
config :my_app, MyApp.Dayron,
  url: "https://api.example.com",
  headers: ["access-token": "my-api-token"]

HTTP Client Adapter

Currently the only adapter available is HTTPoisonAdapter, which uses HTTPoison and hackney to manage HTTP requests.

You can also create your own adapter implementing the Dyron.Adapter behavior, and changing the configuration to something like:

# In your config/config.exs file
config :my_app, MyApp.Dayron,
  url: "https://api.example.com",
  adapter: MyDayronAdapter

Important links

Contributing

Pull request are very wellcome, but before opening a new one, please open an issue first.

If you want to send us a pull request, get the project working in you local:

$ git clone https://github.com/inaka/Dayron.git
$ cd Dayron
$ mix deps.get
$ mix test

Create a branch with the issue name and once you're ready (new additions and tests passing), submit your pull request!

Building docs

$ MIX_ENV=docs mix docs

Contact Us

If you find any bugs or have a problem while using this library, please open an issue in this repo (or a pull request).

You can also check all of our open-source projects at inaka.github.io.

Copyright and License

Copyright (c) 2016, Inaka.

Dayron source code is licensed under the Apache 2 License.

dayron's People

Contributors

alemata avatar amilkr avatar flaviogranero avatar jcypret avatar piton4eg avatar rrrene avatar tiagoengel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dayron's Issues

Add associations/embed data mapping

  • Research how Ecto manage the data mapping into associations and embed resources.
  • Implement association fetch
  • Implement embed data mapping

Fulfil the open-source checklist

General Items

Exhibition

  • There is a blog post about it
  • It's shared on social networks
  • It's shared on reddit
  • It's shared on hacker news with a title like Show HN: description
  • It has a landing page built in github pages

For Libraries

  • It provides a sample application
  • Examples of use are documented in the README or linked from there

For Erlang/Elixir Projects

Dynamic atom keys

The default HTTPoison adapter implementation decodes and creates atom keys, which may lead to unsolvable memory leaks.

I see here that __from_json__/2 assumes (in the default implementation) atom keys, but it doesn't have to.

Would it be worth revising that method default implementation to handle string keys, so that we avoid the problem altogether? If I'm not mistaken, we can leverage the schema as well at that point and generate it from the definition.

Accept Ecto.Changesets

Add support to Ecto.Changeset, casting field values, running validations before sending requests

Circuit breaker integration

When dealing with external services an important part of the equation is circuit breaking logic to avoid long timeouts during outages or network congestion. Is anything like this already planned?

Supporting different json list structure when retrieving data

Hey there,

Currently, it seems that Dayron only supports data retrieval of endpoints that look like this:

[
    { "id": 0, "name": "Elem 1"},
    { "id": 1, "name": "Elem 2"}
]

Unfortunately I have to work with REST APIs that output data in a different way quite often. Some APIs add some additional metadata and therefore store the main result list in a specific property. Classic example:

{
"data": [
    { "id": 0, "name": "Elem 1"},
    { "id": 1, "name": "Elem 2"}
],
"count": 2
}

I made quick changes on my fork of Dayron that support this kind of data structure, those are tested in the model_test.ex file. I have also adressed this topic a bit in the README.MD file, as I feel it would have saved me a bit of time to have this specified in the readme.

Would you be interested in a pull request?

Note that some of our services also use the JSONApi.org specification, which is even more complex. Full support of JSONApi would probably be a big work (though if you are looking to build nested data mapping, you should check out how they handle it, it's quite interesting). However, I have also written small changes that allow Dayron to extract data from a JSONApi response. It's still quite early, it feels too unsure to push this publicly, but tell me if you're interested on this topic as well, as I will probably investigate it further.

Thanks

Get request encode empty body to "" - Throw 403 forbidden response from some API

Hi,

I encountered an issue while using your lib to consume API.ai's api. A GET request is actually sending double quotes in body. After hours of debugging (I thought something was wrong with the headers sent), I found that we encode an empty body with Poison.encode even for a GET request.

This behavior causes API.ai to throw a 403 forbidden response.

I'm doing my first step in Elixir and functional programming, so I lack deep knowledge in how to solve in good manner this issue. Below is my attempt:

adapters/httpoison_adapter.ex, line ~24:

     def process_request_body(""), do: "" #tried to return nil but it's not catched by hackney library
     
     def process_request_body(body), do: Poison.encode!(body)

You can check this behavior using http://requestb.in/ and looking at the raw body of your GET request. it should be empty, it's not.

Looking for your advice and a proper fix!

Allow specifying top-level attribute, possibly overriding `__from_json_list__: 2`

I'm currently using Dayron to interact with a jsonapi.org spec API (and it's working great! ๐Ÿ˜„), but ran into an issue deserializing the data. The JSON-API spec returns either a top level data key or a top-level errors keys. This runs into an issue with Dayron because in Dayron.Model the first method that gets called is __from_json_list__/2 (

def __from_json_list__(data, opts) when is_list(data) do
) which assumes the array of results is at the root level.

So, when JSON-API returns an array of records nested inside of the data attribute, Dayron sees there is no list at the root level, then it tries to apply the attributes to a struct, which results in an empty struct being returned.

I was able to correct his behavior by mapping the data key instead:

def __from_json_list__(%{data: data}, opts) do
  Enum.map(data, &__from_json__(&1, opts))
end

This also had the benefit of allowing me to match on the error key as well:

def __from_json_list__(%{errors: errors}, opts) do
  # ...
end

But this is not currently allowed by the Dayron library. I had to adjust the Dayron.Model to allow the __from_json_list/2` to be overridable, otherwise the library models took precedence:

# lib/dayron/model.ex#96
defoverridable [__url_for__: 1, __from_json__: 2, __from_json_list__: 2]

With the overridable change to the library and the custom __from_json_list__/2 method on the model, I was able to achieve the desired result.

Support a pipeline of "plugs" to change request/response

The idea here is have a kind of plug and pipelines support. It would be something like faraday middlewares from ruby.
With the solution, we would be able to change the request data before it's sent, and change the response data after the request but before the mapping to a struct.
I did a quick research if we could use Plug structure for that, but in my opinion it does not support all response data or error handling we need.

Documentation code links do not work

It seems that hexdoc.pm links the documentation with the source code in github, but those links are not working because of missing version tags.

  • v0.1.0 has no tag.
  • v0.1.1 tag is name does not follow the expectaction (it's missing the initial v).

Better integration with Ecto.Schema

Since Ecto 2.0 is more data-centric we should research how to allow Ecto Schema functions and structures to provide data mapping of Dayron requests. It may allows easy associations/embed support.
Research if embedded_schema is the way to go.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.