omohokcoj / filterable Goto Github PK
View Code? Open in Web Editor NEWFiltering from incoming params in Elixir/Ecto/Phoenix with easy to use DSL.
Home Page: https://hex.pm/packages/filterable
License: MIT License
Filtering from incoming params in Elixir/Ecto/Phoenix with easy to use DSL.
Home Page: https://hex.pm/packages/filterable
License: MIT License
filterable/lib/filterable/ecto/helpers.ex
Line 72 in 125bb36
Should be changed to something useful ;)
Thanks,
Philipp
As documented in Erlang, an atom is never garbage collected. One should therefore take extreme precaution when converting a binary to an atom from uncontrolled input.
Using the following filter,
@options cast: :atom
filter sort_by(query, value, conn), do: #...
When called with a random value, each random value will be converted to an atom, regardless of whether or not it is indeed a valid value for this parameter.
A default BEAM configuration will have a limit of 1,048,576 atoms. Since BEAM applications tend to have long up times, such a limit could be reached quite easily simply from API consumers mistakes.
Since sometimes, not checking for the validity of the value when casting to an atom is sometimes desired, allow it through an option like cast: :atom_unchecked
.
To allow one to validate the value before it is cast, cast: {:atom, [:value1, :value2, value3]}
could be used.
To prevent breaking changes to users of this library, we could keep the present cast: :atom
option, but display a deprecation warning until the next major release. After displaying the warning, it can simply forward it's arguments to cast: :atom_unchecked
.
Please fix your readme. I tried your Phoenix controller
and Separate module
both of them don't compile.
Phoenix: 1.5.4
Elixir: 1.10.4
Hello,
When using top_param
I noticed that the filter_values does not retain the same level of nesting. Example
@options top_param: :q, cast: &String.downcase/1
filter name(list, name), do: ...
When running this filter I get back
{:ok, list, %{name: "search". ... }%}
This feels a bit inconsistent, I rather have it return back the original nesting of top_param
{:ok, list, %{q: %{name: "search"}, ...}%}
This way it's much easier writing an UI and when you want to indicate which filter fields are used (especially when using multiple search fields)
Hi,
I added the following code for filtering on a boolean column archived
defmodule MyApp.Registrations.RegistrationFilters do
use Filterable.DSL
use Filterable.Ecto.Helpers
import Ecto.Query, warn: false
field(:archived)
end
when I try the following, it works:
{:ok, query, filters} = RegistrationFilters.apply_filters(query, %{archived: true})
with result being:
{:ok,
#Ecto.Query<from r0 in MyApp.Registrations.Registration, as: :registration,
where: r0.archived == ^true, order_by: [desc: r0.inserted_at]>,
%{archived: true}}
but if I run the same with filtering on false
, the clause does not get added in query. It does show up in filters though:
{:ok, query, filters} = RegistrationFilters.apply_filters(query, %{archived: false})
returns:
{:ok,
#Ecto.Query<from r0 in MyApp.Registrations.Registration, as: :registration,
order_by: [desc: r0.inserted_at]>, %{archived: false}}
Are we supposed to use some different format for the boolean values?
Best regards,
Gorav
can I use filterable to convert the ? = pattern to SEO friendly / ?
Hello,
When implementing a frontend to filterable that allows to search / order / paginate on Lists and ecto records, I noticed that there is no way to find out the total amount of records when a search + ordering is performed.
This is needed when rendering pagination helpers, you don't want to show a next link when you know there won't be any records to show on the next page.
It would be helpful if this gets returned in the response in apply_filters
.
Note that I'm using a regular list instead of ecto models
defmodule Filter do
use Filterable.DSL
@options param: :q, cast: &String.downcase/1
filter name(list, value) do
list |> Enum.filter(&(String.contains?(String.downcase(&1["name"]), value)))
end
# Basic copy past of what paginateable generates + adaption for Lists
@options param: [:page, :per_page], default: [page: 1, per_page: 10], cast: :integer, share: false
filter paginate(list, %{page: page, per_page: per_page}) do
list |> Enum.slice((page - 1) * per_page, per_page)
end
@options param: [:sort, :order], default: [order: :desc], cast: :atom, share: false
filter sort(list, %{sort: field, order: :asc}) do
list |> Enum.sort_by(&(&1[Atom.to_string(field)]), &<=/2)
end
filter sort(list, %{sort: field, order: :desc}) do
list |> Enum.sort_by(&(&1[Atom.to_string(field)]), &>=/2)
end
end
Add a new @option
flag that indicates that this filter is the pagination filter and should be run at the end. This way you can store the total amount of items currently returned by other filters. Then this value can be returned when apply_filters
is run.
{:ok, result, filter_values, total} = Filter.apply_filters(items, params)
Hi, I am trying to use filterable in a phoenix project, I have installed version 0.6.0. When i compile the project I got the next Error:
== Compilation error in file lib/project/movement/movement_filters.ex ==
** (CompileError) lib/project/movement/movement_filters.ex:5: undefined function filterable/1
(stdlib) erl_eval.erl:677: :erl_eval.do_apply/6
(elixir) lib/kernel/parallel_compiler.ex:198: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/6
The code of movement_filters is a copy of the code in the documentation
defmodule PostFilters do
use Filterable.DSL
use Filterable.Ecto.Helpers
filterable do
filter title(query, value, _conn) do
query |> where(title: ^value)
end
@options cast: :integer
filter stars(query, value, _conn) do
query |> where(stars: ^value)
end
end
end
Thanks.
After considerable trial and error, these are the correct instructions for this to work:
// patient_filters.ex
defmodule PatientFilters do
use Filterable.DSL
use Filterable.Ecto.Helpers
use Filterable
import Ecto.Query
field(:name)
field(:gender)
paginateable(per_page: 10)
filterable do
@options param: :name
filter name(query, value) do
query |> where([u], ilike(u.name, ^"%#{value}%"))
end
@options param: :gender
filter gender(query, value) do
query |> where(gender: ^value)
end
end
end
And in controller:
def index(conn, _params) do
with {:ok, query, filter_values} <- PatientFilters.apply_filters(Patient, _params),
patients <- Repo.all(query),
do: render(conn, "index.json", patients: patients, meta: filter_values)
end
Hello,
I'm having a difficult time grasping when a filter is triggered when giving some params.
Given following search params
search = %{q: %{name: "playpass", barcode: "1234"}, sort: "name", order: "desc", page: 1, per_page: 6}
list = [%{"barcode" => "915580318","name" => "PlayPass Scanout"}, ...]
With the following filters
@options param: [q: :name], cast: &String.downcase/1
filter name(list, %{name: name}) do
IO.inspect "SEARCHING NAME"
list
end
@options param: [q: :barcode], cast: &String.downcase/1
filter barcode(list, %{barcode: barcode}) do
IO.inspect "SEARCHING barcode"
list
end
When running the filters, nothing is printed in the console, but I expect both to be run. So some questions that pop up that I couldn't find in the readme:
@options
relate to the filter name and the params? I do notice that the original nesting is not kept, meaning everything is flattendThanks! I really like this project since it does not assume anything about your framework or how you want to filter data
https://github.com/omohokcoj/filterable/blob/master/lib/filterable/params.ex#L11 - this could cause BEAM atoms limit overflow.
kudos to @brodeuralexis for reporting this issue.
Hey @omohokcoj - really enjoying my time with filterable, has been much smoother than trying to work with another DSL on top of SQL.
Is it possible to use filterable on top of a query rather than just a schema? My use case is that I'd like to get all my join tables and computed columns up front and then filter on those, rather than duplicating the joins in the filterable macro.
i.e. right now, the order of operations is
Fic.apply_filters(conn) |> Fic.get_all_join_tables
and I'd like to do
Fic.get_all_join_tables |> Fic.apply_filters(conn)
My get_all_join_tables query looks like this:
def get_all_join_tables(query) do
from f in query,
left_join: reviews in assoc(f, :reviews),
group_by: f.id,
preload: [
:submitter,
:author,
:genres,
reviews: :submitter,
],
select: %{
f |
review_count: fragment("count(?) as review_count", reviews.id),
review_avg: fragment("coalesce(?::float, 0) as review_avg", avg(reviews.rating))
}
end
And one of my filterable blocks look like this...
@options top_param: :search, cast: :integer
filter review_avg(query, value, _conn) do
from f in query,
left_join: reviews in assoc(f, :reviews),
having: avg(reviews.rating) > ^value
end
Ideally, I wouldn't have to left_join reviews again and could build off the work I did on the get_all_join_tables.
Any suggestions? Thx again for open sourcing this.
This error happens when I try to cast a value that doesn't match any of the atoms in a list.
filterable do
field :status, cast: {:atom, [:accepted, :pending, :rejected]}
end
[info] GET /v1/trees
[debug] Processing with PetreeApiWeb.TreeController.index/2
Parameters: %{"status" => "asdf"}
Pipelines: [:api]
[info] Sent 500 in 1617ms
[error] #PID<0.466.0> running PetreeApiWeb.Endpoint (connection #PID<0.465.0>, stream id 1) terminated
Server: 127.0.0.1:4000 (http)
Request: GET /v1/trees?status=asdf
** (exit) an exception was raised:
** (FunctionClauseError) no function clause matching in Filterable.Params.cast_error_message/1
(filterable 0.7.3) lib/filterable/params.ex:229: Filterable.Params.cast_error_message([value: "asdf", cast: {:atom, [:accepted, :pending, :rejected]}])
(filterable 0.7.3) lib/filterable/params.ex:191: Filterable.Params.cast/3
(filterable 0.7.3) lib/filterable/params.ex:18: Filterable.Params.filter_value/2
(filterable 0.7.3) lib/filterable.ex:107: anonymous fn/4 in Filterable.filter_values/3
(filterable 0.7.3) lib/filterable/utils.ex:8: anonymous fn/3 in Filterable.Utils.reduce_with/3
(elixir 1.10.4) lib/enum.ex:3686: Enumerable.List.reduce/3
(elixir 1.10.4) lib/enum.ex:2161: Enum.reduce_while/3
(filterable 0.7.3) lib/filterable.ex:93: Filterable.apply_filters/4
(petree_api 0.1.0) lib/petree_api_web/controllers/tree_controller.ex:16: PetreeApiWeb.TreeController.index/2
(petree_api 0.1.0) lib/petree_api_web/controllers/tree_controller.ex:1: PetreeApiWeb.TreeController.action/2
(petree_api 0.1.0) lib/petree_api_web/controllers/tree_controller.ex:1: PetreeApiWeb.TreeController.phoenix_controller_pipeline/2
(phoenix 1.5.6) lib/phoenix/router.ex:352: Phoenix.Router.__call__/2
(petree_api 0.1.0) lib/petree_api_web/endpoint.ex:1: PetreeApiWeb.Endpoint.plug_builder_call/2
(petree_api 0.1.0) lib/plug/debugger.ex:132: PetreeApiWeb.Endpoint."call (overridable 3)"/2
(petree_api 0.1.0) lib/petree_api_web/endpoint.ex:1: PetreeApiWeb.Endpoint.call/2
(phoenix 1.5.6) lib/phoenix/endpoint/cowboy2_handler.ex:65: Phoenix.Endpoint.Cowboy2Handler.init/4
(cowboy 2.8.0) /app/deps/cowboy/src/cowboy_handler.erl:37: :cowboy_handler.execute/2
(cowboy 2.8.0) /app/deps/cowboy/src/cowboy_stream_h.erl:300: :cowboy_stream_h.execute/3
(cowboy 2.8.0) /app/deps/cowboy/src/cowboy_stream_h.erl:291: :cowboy_stream_h.request_process/3
(stdlib 3.13) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
We are currently using Filterable in our app successfully with controllers and separate filter modules. We're at a point now where this would also be really useful when fetching data through Phoenix channels for async requests.
Is this possible now or are there plans to add it in the future?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.