beam-community / jsonapi Goto Github PK

The JSONAPI Spec documents define how errors should be handled, but I don't see helpers in this module. Should I create a pull request, or is it already handled and I missed it?

I know that additional keys are optional, but it might be nice to bake in a couple of conventions.

Option to not include data could help speed things up.

• self
• links
• included

code-corps/code-corps-api#1139

I'll work on a PR if everyone agrees this is a good idea!

Is there a way to customize the links that get built?

My resource is set up in the router.ex like this...

 scope "/api", Handiman do
   pipe_through :api

   resources "/users", UserController, except: [:new, :edit, :create]
   resources "/register", RegistrationController, only: [:create]
   resources "/rounds", RoundController, except: [:new, :edit]
  end

so my URLs will look like /api/rounds but when the links are created for 'self' for example, api is not included and the URL is useless.

https://www.w3.org/Protocols/rfc1341/4_Content-Type.html

%{"data" => %{"id" => nil, "type" => "foo"}} is the relationship linkage object when data is not preloaded, even for a one-to-one relationship where the id is known.

I don't know exactly what should be there (it's not really clear in the spec) but it seems not quite right to set id to nil regardless of what the correct action is.

Is ist possible to define custom field names?
I am working with a legacy database that has inconsistent column names that I don't want to expose over the api, e.g. I want to transform columnName to column_name.

Additionally I would like to combine multiple database columns to a single key: value pair.

I think it would be great to get some phoenix generators in here. This is my "best case" api

mix jsonapi.gen.resource User users name:string email:string bio:string number_of_pets:integer
* creating priv/repo/migrations/20150409213440_create_user.exs
* creating web/models/user.ex
* creating test/models/user_test.exs
* creating web/controllers/user_controller.ex
* creating web/views/user_view.ex
* creating test/controllers/user_controller_test.exs

Add the resource to your browser scope in web/router.ex:

    resources "/users", UserController

and then update your repository by running migrations:

    $ mix ecto.migrate

And then

mix jsonapi.gen.resource Post posts title:string body:string belongs_to:User
* creating priv/repo/migrations/20150409213440_create_post.exs
* creating web/models/post.ex
* creating test/models/post_test.exs
* creating web/controllers/post_controller.ex
* creating web/views/post_view.ex
* creating test/controllers/post_controller_test.exs

Add the resource to your browser scope in web/router.ex:

    resources "/posts", PostController
    #TODO Figure out all the different LINKS routes we can insert here

and then update your repository by running migrations:

    $ mix ecto.migrate

Then you could do a
GET /posts?include=user&fields=title,body,user.name&order=-posts.created_at
And it would work out of the box.

According to the spec, an empty to-many relationship should be represented as an empty array in relationships. So something like this:

%{
  id: 1,
  text: "Hello",
  body: "Hello world",
  author: %{id: 2, username: "jason"},
  best_comments: []
}

would have a relationships field something like this:

{
  relationships: %{
    author: %{
      data: %{id: "2", type: "user"},
      links: %{related: "/user/2", self: "/mytype/1/relationships/author"}
    },
    best_comments: %{
      data: [],
      links: %{
        self: "/mytype/1/relationships/best_comments"
      }
    }
  }
}

Currently, the serializer omits an empty to-many relationship, so it looks like this:

{
  relationships: %{
    author: %{
      data: %{id: "2", type: "user"},
      links: %{related: "/user/2", self: "/mytype/1/relationships/author"
    }
  }
}

The test for handling an empty relationship, specifies this behavior, but it seems to be at odds with the spec.

Would you accept a PR to serialize empty to-many relationships as empty lists? The change would definitely change responses for users of the library, but in a good way I think, bringing responses more in line with the spec.

Hey, I'm trying to use this library for parsing json data that i fetch from an external json api.
I couldn't find a proper example for this so i tried this myself.

defmodule JsonApi do
  use Tesla

  plug Tesla.Middleware.BaseUrl, "www.example.com"
  plug Tesla.Middleware.Headers, %{"Authorization" => "hidden"}
  plug JSONAPI.QueryParser

  def get do
    get("/asset-types/locomotives/")
  end
end

the issue is that tesla has 3 arguments for plugs, and jsonapi only expects 2.

is there any chance for a Tesla compatible plug for requesting json?
or is there a better way to parse json requested from external api(without phoenix)?

If I define methods in my **view.ex file, I want to include them in the attributes hash. Once could imagine combining first + last or some type of conditional logic for a specific field at the view layer.

I was wondering your thoughts were on this and if it is something you want me to take a stab at...

defmodule MyApp.ArticleSerializer do

  def fields, do: [:body, :excerpt]

  def excerpt(article, _conn) do
    [first | _ ] = String.split(article.body, ".")
    first
  end
end

Potential Issue

Currently it seems a view is required to process any type of relationship.

ticket_view.ex

def relationships, do: [ticket_categories: MyProject.TicketCategoryView]

Right now, the below is needed because encode_data is called recursively through all the relationships and asks for each relationships view.

ticket_category_view.ex

def relationships, do: [category: MyProject.CategoryView, ticket: MyProject.TicketView]

example payload

{
  "relationships": {
    "ticket-categories": {
      "data": [{"id": "1234", "type": "ticket-category"}]
    }
  }
}

Brainstorming a solution

Since I'm most likely missing something, the first step for me would be to understand the need for a relationship's view. In this case, it seems what might be ok to do is something like.

[ticket_categories: %{type: "ticket-category", field: "ticket_category_id"}]

Then one option during serialization of the data would have the lib do Map.get(ticket_data, field) to get the id for the relationship instead of recursively moving down the relationship tree.

Would appreciate some thoughts. I would love to open a PR!

Travis or Circle or w/e we can get to work.

http://jsonapi.org/format/#document-links
Unless I'm reading something wrong, is it not required to show a self link under index views as well as show views?

Seems like a change here would fix the issue:
https://github.com/jeregrine/jsonapi/blob/master/lib/jsonapi/serializer.ex#L25

The use of String.to_atom is considered unsafe because it can fill the global atom table. Places where String.to_atom now could be changed to String.to_existing_atom, which should only allow atoms for known relationships and fields that have already been loaded, in JSONAPI.QueryParser and elsewhere. If using String.to_existing_atom doesn't work, because all the fields and relationships might not be an atom anywhere else, you would have to keep all the data as Strings until the application using the library can validate if it is safe to convert to atoms. Either of these should mitigate the Denial of Service potential of flooding of the atom table.

Hi. Let's say I have a Product, with a tag_ids field of type {:array, :string}.

I'd like to construct a normal-appearing tags relationship in my ProductView, such that the tags derive not from a regular has_many relationship, but rather any Tag whose uuid primary_key is in that product.tag_ids array.

Is there any reasonable way to do this?

I am implementing a search endpoint, where the search may return different sorts of resources, as in this scenario. I don't think this is currently supported, is it?

Do you think this library should heterogenous collections , given that json:api supports them?

I'm not sure yet what would be the best approach to implementing, although I'd be happy to try a few things and see what works.

In the meantime can require multiple requests to perform a search as a workaround.

Cheers!

Hi,

Could we add this library to the http://jsonapi.org/implementations/ webpage?
It would add an elixir solution and help users find this library.

What do you think ?

Any chance we can get a newer example application? A few things are not so clear, like how to use the ErrorView in phoenix etc...

Seems that according to jsonapi spec and ember data jsonapi,

some_thing , should become some-thing.

In the jsonapi.org page, it has the example:
....

attributes": {
      "first-name": "Dan",
      "last-name": "Gebhardt",
      "twitter": "dgeb"
    },

....

If i manage to learn how to do this, i can do it on the weekend and write some tests.

I imagine defaulting to ecto/phoenix similar to https://github.com/phoenixframework/phoenix_ecto/blob/master/lib/phoenix_ecto/json.ex#L1 and otherwise expecting the user to configure a module that defines some functions

defmodule App.JSONAPI do
   def loaded?(assoc) #checks if the association is loaded. This is normally passed off to Ecto
   def index_url(model, conn, params) # Returns a index url for the model, connection and params
   def show_url(model, conn, params) # Returns a show url for the model, connection and params
end

I think the format of relationship self links in incorrect (although I'm no expert on JSONAPI). In your unit test, you have a PostView, which has a author relationship to a UserView. You define the relationship as such:

def relationships(), do: [author: {JSONAPISerializerTest.UserView, :include}]

I would expect the relationship author self link to be: mytype/1/relationships/author, but instead it is serialised as mytype/1/relationships/user. Essentially, I think it's using the type of the relationship at the end, and not the name.

The JSONAPI spec page has a very similar example, and there, when they mention a sample self link to an "author", which one presumes is of type "user", they specify the relationship as /articles/1/relationships/author

Hi,

When in branch overhaul, commenting the includes and relationships if the views, make all tests pass.
When enabling includes and relationships in views ( in this case, users has one company, company has many users ), the following error appears:

[error] #PID<0.327.0> running JsonapiOverhaul.Endpoint terminated
Server: localhost:4000 (http)
Request: GET /api/companies/1
** (exit) an exception was raised:
    ** (UndefinedFunctionError) undefined function: Enum.empty/1
        (elixir) Enum.empty(nil)
        lib/jsonapi/serializer.ex:51: anonymous fn/6 in JSONAPI.Serializer.encode_data/4
        (elixir) lib/enum.ex:1102: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
        lib/jsonapi/serializer.ex:17: JSONAPI.Serializer.serialize/3
        (phoenix) lib/phoenix/view.ex:341: Phoenix.View.render_to_iodata/3
        (phoenix) lib/phoenix/controller.ex:628: Phoenix.Controller.do_render/4
        (jsonapi_overhaul) web/controllers/company_controller.ex:1: JsonapiOverhaul.CompanyController.action/2
        (jsonapi_overhaul) web/controllers/company_controller.ex:1: JsonapiOverhaul.CompanyController.phoenix_controller_pipeline/2
        (jsonapi_overhaul) lib/phoenix/router.ex:255: JsonapiOverhaul.Router.dispatch/2
        (jsonapi_overhaul) web/router.ex:1: JsonapiOverhaul.Router.do_call/2
        (jsonapi_overhaul) lib/jsonapi_overhaul/endpoint.ex:1: JsonapiOverhaul.Endpoint.phoenix_pipeline/1
        (jsonapi_overhaul) lib/plug/debugger.ex:92: JsonapiOverhaul.Endpoint."call (overridable 3)"/2
        (jsonapi_overhaul) lib/phoenix/endpoint/render_errors.ex:34: JsonapiOverhaul.Endpoint.call/2
        (plug) lib/plug/adapters/cowboy/handler.ex:15: Plug.Adapters.Cowboy.Handler.upgrade/4
        (cowboy) src/cowboy_protocol.erl:442: :cowboy_protocol.execute/4

This could be related to the fact that on the tests, i am not filling out the relationships.

Thanks!

Travis CI output:

==> plug
warning: the dependency :plug requires Elixir "~> 1.2.3 or ~> 1.3" but you are running on v1.1.1
Compiled lib/plug/mime.ex
Compiled lib/plug/html.ex
Compiled lib/plug/conn/adapter.ex
Compiled lib/plug.ex
Compiled lib/plug/crypto.ex
Compiled lib/plug/conn/unfetched.ex
Compiled lib/plug/logger.ex
Compiled lib/plug/supervisor.ex
Compiled lib/plug/request_id.ex
Compiled lib/plug/crypto/key_generator.ex
Compiled lib/plug/session/store.ex
Compiled lib/plug/conn/wrapper_error.ex
Compiled lib/plug/error_handler.ex
Compiled lib/plug/session.ex
Compiled lib/plug/session/ets.ex

== Compilation error on file lib/plug/crypto/message_verifier.ex ==
** (CompileError) lib/plug/crypto/message_verifier.ex:105: function '<-'/2 undefined
    (stdlib) lists.erl:1336: :lists.foreach/2
    (stdlib) erl_eval.erl:657: :erl_eval.do_apply/6

could not compile dependency :plug, "mix compile" failed. You can recompile this dependency with "mix deps.compile plug", update it with "mix deps.update plug" or clean it with "mix deps.clean plug"

Would moving to elixir v1.3 from v1.1.1 in .travis.yml do the trick?

This might have a bad title, apologies.

So, we just deployed our serialization layer redone in JSONAPI! Its great.
However, we noticed one weird tidbit and wondered if you thought it would be something that might be able to be handled at the library level, or if there may be something in our configuration that we need to work around.

What we are having occur is that our host on our conn is getting set as 0.0.0.0 due to the way we run the binary in production. We obviously would like the links we send to be real/working, and reflect the actual hostname of the machine etc on which the things are hosted on.

We could make a simple plug that sets the hostname from a configuration and have no changes here, but I havent researched if changing that in our conn could impact anything else.

It could be an option to add a host argument to url_for/2 and customize the behaviour.

Tbqh I am not sure what the best way to handle things would be, but it didnt seem like it would break any current behaviour or really change the exposed API if we added the option to pass your own host. But this also could be a bit of a band-aid and we really just should configure out own things better.

Thoughts?

Thanks Again for the awesome work!

I've read over the docs and have done my best to grok the code, but I'm a bit confused on how to specify links.

Suppose I want to include a link for a Book's many Pages, I might want something like this:

{
  "data": {
    "id": 123,
    "title": "Curious George"
  },
  "relationships": {
    "pages": {
      "self": "/api/book/123/relationship/pages"
// snip

Setting up my BookView isn't quite clear to me. If I override links/2 it doesn't seem to do anything. Overriding relationships/0 as so doesn't seem to work:

def relationships do
  [pages: PageView]
end

I think I may be missing something obvious, but I could use a hand.

I've using the QueryParser plug and it works fine, but there is something that is somewhat of an issue, since the options, namely view are fetched in the init function, they need to be passed at compile time. This makes it harder to use Plug.Builder and have another plug extend the QueryParser.

I'd like to be able to do:

defmodule SuperQueryParser do
  use Plug.Builder

  plug JSONAPI.QueryParser

  def init(opts) do
    opts
  end

  def call(conn, opts) do
    conn = conn |> super(opts)
    ...
  end
end

defmodule App.SomeController do
  use App, :controller

  plug SuperQueryParser, view: App.SomeView
end

This would mean moving build_config to the call function instead. Is this something desirable or is there any other way to achieve the same behaviour?

Thank you for jsonapi! Unfortunately, the current version on hex.pm (0.3.0) does not contain the latest significant change (removal of PhoenixView, which is a bit confusing. Could you upload a new release?

I was surprised that nothing is produced for a relationship that is not loaded. I expected that

• if I have a belongs_to field on my Ecto schema, and
• the relationships/0 function in my view includes that field, and
• the association is "not loaded",

then I would get a resource identifier object that only contains the type and id of the related resource. Even though the association is not loaded, there is still enough information to generate the resource identifier.

Would you accept a PR that, instead of omitting "not loaded" associations from relationships, adds a minimal "resource identifier" object for them? Or are there other considerations I am missing? Thanks!

Handy to know exactly what is supposed to be in the hex.pm package. :)

Right now we handle the includes case in the query parser but not filtering fields attributes on the response. We can handle this automagically.

The get_view_for_type function in QueryParser module maps type to module name. However, it assumes the view for the module has the same plural-ness (??) as the type, which often isn't the case.

For instance, a view for a user might live in a module called UserView, but the URL to access users might be http://example.com/users. This implies the type field should also be users. Now you end up mapping users to UsersView, which is incorrect.

So I am putting together this gist of ideas on how we can better structure the library to be less ad-hoc. I realize now that only doing a view layer really limits you to what's possible with JSONAPI. So I've put together a sort of happy path in this gist https://gist.github.com/jeregrine/b57827f36e4d70116540. Please comment here so the conversation stays in one place.

The basic idea is we start at the top and build up a JSONAPI configuration that can be used to validate the request, build queries, build data, and render data. This also gives us a base data structure we can use to build simple scaffolding so you can get spun up with jsonapi/phoenix/ecto very quickly, and without complex and opaque macro's.

So let me know what you think! All 7 of you watchers :)

Hi,
Let's say you have a map of:
%{"foo" => "bar"}

Then doing:

fields do
 [:foo]
end

Won't work, even if you replace :foo with "foo".

As atoms aren't garbage collected, I hope we can get this supported rather that converting everything to atoms:
elixir-lang/elixir#2915 (comment)

It would be helpful if the README and/or wiki included a section on installing this. As an Elixir newcomer, I'm not sure how mix resolves packages (or from where), let alone how a git repo corresponds to a package name. The only help in this was examining the example project's mix.exs file https://github.com/alexjp/jsonapi-testing/blob/master/mix.exs and adapting it to

{:jsonapi, git: "https://github.com/jeregrine/jsonapi", tag: "v0.6.0"}

but this is just educated guesswork.

A few extra details to get the ball rolling would go a long way in onboarding users and would be much appreciated. Thanks!

As far as I could tell the QueryParser will correctly parse pagination data, but there is not way to then add pagination links (next, prev, first and/or last) to the links object at the collection level, as per the JSON:API spec.

Am I missing something or is it really a feature that is missing? If so, would you be willing to take a PR on that?

Hi @jeregrine,

I have a nested resource in phoenix and my parameter is not called id, but foo_id. Would it be possible to make that configurable?

cheers,
malte

I'm working with an API where I have a field data_schema that is an arbitrary JSON object. The API uses the dashed key format, so it shows up in requests/responses as data-schema. This is all fine until someone includes an underscore in one of their JSON object keys, such as __finalized__. So my incoming data

%{
  "data" => %{
    "attributes" => %{
      "data-schema" => %{
        "properties" => %{
          "__finalized__" => %{"title" => "Finalized", "type" => "boolean"},
        },
        "type" => "object"
      },
    },
    "relationships" => %{},
    "type" => "schemas"
  }
}

gets transformed into

%{
  "data" => %{
    "attributes" => %{
      "data-schema" => %{
        "properties" => %{
          "--finalized--" => %{"title" => "Finalized", "type" => "boolean"}
        },
        "type" => "object"
      },
      "inserted-at" => "2018-09-14T21:03:25.829318",
      "updated-at" => "2018-09-14T21:03:25.829328"
    },
    "id" => "27304",
    "relationships" => %{},
    "type" => "schemas"
  },
  "included" => [],
  "meta" => nil
}

(this is surprising to the people who used the leading underscores to indicate "don't mess with this") Ideally, the value of data-schema would not be touched at all. I'm not sure how we could accomplish that simply. But I wonder if it makes sense to add an underscore option that specifies field prefixes or patterns that would not be underscored/dashed? Any thoughts?

Would be nice to allow to set meta for each resources. Something like this I imagine:

# message_view.ex
defmodule Foo.MessageView do
  def meta
    %{ count: Message.count, unread_count: Message.unread_count_for(params[:id]) }
  end
end

Not sure how we can pass the "params" in there though. Any thoughts?

In this file, it forces the headers to conform to the JSONAPI media type improperly.

To quote:

Servers MUST send all JSON API data in response documents with the header Content-Type: application/vnd.api+json without any media type parameters.

The server MUST respond with the proper headers, the library handles this okay.

Servers MUST respond with a 415 Unsupported Media Type status code if a request specifies the header Content-Type: application/vnd.api+json with any media type parameters.

This seems to be worded intentionally, such that if a request specifies the Content-Type: application/vnd.api+json header with any additional parameters, it should return a 415. The server itself is not required to force the client to send the Content-Type: application/vnd.api+json header (just to enforce that it's done properly if the client does send it), though the client is required by the spec to send it.

Servers MUST respond with a 406 Not Acceptable status code if a request’s Accept header contains the JSON API media type and all instances of that media type are modified with media type parameters.

This also seems to be worded intentionally, such that if a request specifies an Accept header with the proper type, but ALL instances of that media type are modified (meaning it can have multiple media types, and even multiple declarations of the JSONAPI media type), throw a 406. Again, the server is not required to force the client to send the Accept: application/vnd.api+json header (just to enforce that it's done properly if the client does send it -- and only to enforce it if they use the application/vnd.api+json media type, and each time it's specified it has modified media type parameters), and actually, the client isn't required to send this header at all, they're just required to not modify the media type with parameters on at least one instance of it if they do pass it.

Just wanted to clarify that. Great library!

I imaging this working like

plug JSONAPI.Controller.QueryValidator, %{
  filter: %{
    name: %{ allows_list: true, allow_null: false},
  },
  sort: [:created_at, :name]
  includes: [:image, :posts]
}

Currently, meta is included in each document payload. How about including it as a top level document member? I'd be happy to make a PR.

http://jsonapi.org/format/#document-top-level
http://jsonapi.org/format/#document-meta

According to the includes spec on jsonapi (1), invalid includes should only return 400.

https://github.com/jeregrine/jsonapi/blob/242499eaf42362fa0f1e6602ecc08f9baa7feaad/lib/jsonapi/query_parser.ex#L183

Looks to be due to this line. Based on whether or not the atom exists in the app, the line raises an error only on certain invalid includes

@doomspork @jeregrine Thanks for the awesome attention to this lib! I think you were thinking about cutting a new release after this PR: #92

Hi, thanks for your work on this, it's been useful.

I have two questions about the options of how relationships are rendered:

1. currently, it seems that returning [ relationship: RelationshipType ] from relationships will automatically include the resource in the response, which is not ideal. This seems to be coming from here: https://github.com/jeregrine/jsonapi/blob/master/lib/jsonapi/serializer.ex#L68.

2. one of the problems I've encountered already is with has_many relationships with large numbers of relations. even including the resource identifiers (type, id) for many relations can bloat pretty quickly to the point that you might be looking for pagination of the data portion of the relationship:

"relationships": {
      "comments": {
        "links": {
          "self": "http://example.com/articles/1/relationships/comments",
          "related": "http://example.com/articles/1/comments"
        },
        "data": [
          { "type": "comments", "id": "5" },
          { "type": "comments", "id": "12" },
          //and many many more...
        ]
      }
    }

so it would be nice to have an option to not include the data, which is legit JSONAPI as far as I understand (http://jsonapi.org/format/#document-resource-object-relationships).

"relationships": {
      "comments": {
        "links": {
          "self": "http://example.com/articles/1/relationships/comments",
          "related": "http://example.com/articles/1/comments"
        },
      }
    }

I'm happy to put some work into this, but I'm still learning Elixir so I'll be a bit slow and possibly not writing idiomatic code.

@jeregrine curious to get your thoughts on whether this is a feature we want or something I should keep in my application. There are a few instances in which we need to include a field in our view so it can be posted or patched but we don't want it included in the response (think passwords). For this I've implemented a hidden/0 that works like fields/0 but is used to hide certain values from the response.

Is this something you'd be open to considering? If so I'll open a PR.