Giter Club home page Giter Club logo

searchjoy's Introduction

Searchjoy

Search analytics made easy

See it in action

Screenshot

  • view searches in real-time
  • track conversion rate week over week
  • monitor the performance of top searches

Works with any search platform, including Elasticsearch, OpenSearch, Sphinx, and Solr

💘 An amazing companion to Searchkick

Build Status

Installation

Add this line to your application’s Gemfile:

gem "searchjoy"

And run the generator. This creates a migration to store searches.

rails generate searchjoy:install
rails db:migrate

Next, add the dashboard to your config/routes.rb.

mount Searchjoy::Engine, at: "searchjoy"

Be sure to protect the endpoint in production - see the Authentication section for ways to do this.

Track Searches

Track searches by creating a record in the database.

Searchjoy::Search.create(
  search_type: "Item", # typically the model name
  query: "apple",
  results_count: 12,
  user_id: 1
)

With Searchkick, you can use the track option to do this automatically.

Item.search("apple", track: {user_id: 1})

If you want to track more attributes, add them to the searchjoy_searches table.

add_column :searchjoy_searches, :source, :string

Then, pass the values to the track option.

Item.search("apple", track: {user_id: 1, source: "web"})

Track Conversions

First, choose a conversion metric. At Instacart, an item added to the cart from the search results page counts as a conversion.

Next, when a user searches, keep track of the search id. With Searchkick, you can get the id with:

results = Item.search("apple", track: true)
results.search.id

When a user converts, find the record and call convert.

search = Searchjoy::Search.find(params[:id])
search.convert

Better yet, record the model that converted.

search.convert(item)

Authentication

Don’t forget to protect the dashboard in production.

Devise

In your config/routes.rb:

authenticate :user, ->(user) { user.admin? } do
  mount Searchjoy::Engine, at: "searchjoy"
end

Basic Authentication

Set the following variables in your environment or an initializer.

ENV["SEARCHJOY_USERNAME"] = "andrew"
ENV["SEARCHJOY_PASSWORD"] = "secret"

Data Retention

Data should only be retained for as long as it’s needed. Delete older data with:

Searchjoy::Search.where("created_at < ?", 1.year.ago).find_in_batches do |searches|
  search_ids = searches.map(&:id)
  Searchjoy::Conversion.where(search_id: search_ids).delete_all
  Searchjoy::Search.where(id: search_ids).delete_all
end

You can use Rollup to aggregate important data before you do.

Searchjoy::Search.rollup("Searches")

Delete data for a specific user with:

user_id = 123
search_ids = Searchjoy::Search.where(user_id: user_id).pluck(:id)
Searchjoy::Conversion.where(search_id: search_ids).delete_all
Searchjoy::Search.where(id: search_ids).delete_all

Customize

To customize, create an initializer config/initializers/searchjoy.rb.

Change the time zone

Searchjoy.time_zone = "Pacific Time (US & Canada)" # defaults to Time.zone

Change the number of top searches shown

Searchjoy.top_searches = 500 # defaults to 100

Link to the search results

Searchjoy.query_url = ->(search) { Rails.application.routes.url_helpers.items_path(q: search.query) }

Add additional info to the query in the live stream

Searchjoy.query_name = ->(search) { "#{search.query} #{search.city}" }

Show the conversion name in the live stream

Searchjoy.conversion_name = ->(model) { model.name }

Upgrading

1.0

Searchjoy now supports multiple conversions per search 🎉

Before updating the gem, create a migration with:

create_table :searchjoy_conversions do |t|
  t.references :search
  t.references :convertable, polymorphic: true, index: {name: "index_searchjoy_conversions_on_convertable"}
  t.datetime :created_at
end

Deploy and run the migration, then update the gem.

You can optionally backfill the conversions table

Searchjoy.backfill_conversions

And optionally remove convertable from searches

remove_reference :searchjoy_searches, :convertable, polymorphic: true

You can stay with single conversions (and skip all the previous steps) by creating an initializer with:

Searchjoy.multiple_conversions = false

History

View the changelog

Contributing

Everyone is encouraged to help improve this project. Here are a few ways you can help:

To get started with development and testing:

git clone https://github.com/ankane/searchjoy.git
cd searchjoy
bundle install
bundle exec rake test

searchjoy's People

Contributors

amalrik avatar ankane avatar askehansen avatar ckorhonen avatar dora0825 avatar dwightwatson avatar endorfin avatar kshnurov avatar patrick-nits avatar tegon avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

searchjoy's Issues

Searchjoy causing error when Searchkick upgraded

I believe Searchkick automatically upgraded to 0.8.0 and I started getting this error when running the console:

Running rails console attached to terminal... up, run.9005
/app/vendor/bundle/ruby/1.9.1/gems/searchjoy-0.0.6/lib/searchjoy.rb:29:in alias_method': undefined methodsearch' for module Searchkick::Search' (NameError) from /app/vendor/bundle/ruby/1.9.1/gems/searchjoy-0.0.6/lib/searchjoy.rb:29:inmodule:Search'

Is this an incompatibility with Searchjoy or was it caused by something else?

Unused Eager Loading detected

Bullet is getting this on the searchjoy module:
/admin/searchjoy/searches/recent
Unused Eager Loading detected
Searchjoy::Search => [:convertable]
Remove from your finder: :includes => [:convertable]

Should you remove it? Im wrong?

On Migration (index too long)

ArgumentError: Index name 'index_searchjoy_searches_on_convertable_id_and_convertable_type' on table 'searchjoy_searches' is too long; the limit is 62 characters

Index name too long SQLite

StandardError: An error has occurred, this and all later migrations canceled:

-- add_index(:searchjoy_searches, [:search_type, :normalized_query, :created_at], {:name=>"index_searchjoy_searches_on_search_type_and_normalized_query_an"})
rake aborted!

Index name 'index_searchjoy_searches_on_search_type_and_normalized_query_an' on table 'searchjoy_searches' is too long; the limit is 62

User track with multiple models

Hi @ankane. Thanks for this gem!

I have several devise models on my app, and I want to save it in the search table regardless of the model. However I see that the migration does not have a polymorphic reference. I could solve this by enabling this option in the migration and overwriting the model according to the Rails engines guide. I've taken a look at the code and can't see where else the user relationship is used, in case something else needs to be overwritten. I would like to ask you if there is any special reason for not giving polymorphism support to the user that it may be bypassing.

Thanks!

Searchkick Conversions

I'm trying to feed the conversions feature of Searchkick (https://github.com/ankane/searchkick#keep-getting-better) with Searchjoy data.

I'm tracking searches with the track: true convenience method, which is working great:

@podcasts = Podcast.search(params[:q], track: true)

But, I don't see anything in the documentation on how to add Searchjoy data to the Searchkick index. The Searchkick docs recommend creating a Search model but that already exists inside Searchjoy.

I'm using the follow which seems to work fine:

  def search_data
    {
      title: title,
      summary: summary,
      conversions: Searchjoy::Search.where(convertable_id: self.id).where(convertable_type: "Podcast").group("query").count
    }
  end

Is this correct or should I be using the polymorphic association (convertable) to connect my model to Searchjoy::Search?

NameError in Searchjoy::Searches#stream after rails4 upgrade

We just upgraded to rails4 and I get the following error now while loading the searchjoy view:

ActionView::Template::Error (undefined local variable or method `searchjoy' for #<#<Class:0x007fa62c068fe0>:0x007fa62c260578>):
    4: 
    5: <script>
    6:   function fetchRecentSearches() {
    7:     $("#stream").load("<%= searchjoy.searches_recent_path %>");
    8:     setTimeout(fetchRecentSearches, 5 * 1000);
    9:   }
   10:   $( function() {
  searchjoy (0.0.8) app/views/searchjoy/searches/stream.html.erb:7:in `___sers_ddddd__rvm_gems_ruby_______rails__x_gems_searchjoy_______app_views_searchjoy_searches_stream_html_erb__88024631391264746_70175841129020'

Any help is much appreciated.

-Jitesh

Missing tests

Hi, I notice the gem doesn't have tests yet.
I can tackle this task myself, which flavor do you prefer? rspec or mini-test?

Duplicate tracking when Searchkick execute => false

I was debugging some tracking and found out that if I have a Searchkick query with execute: false and then later on, I use the multi search like this: Searchkick.multi_search([query]) I got 3 entries on Searchjoy just from that one query.

`method_missing': undefined method `assets' for #<Rails::Application::Configuration:0x00007fc2af60afd0> (NoMethodError)

It does not work if a rails project is created with --skip-sprockets. @ankane

1: from /Users/xxx/.rbenv/versions/2.6.0/lib/ruby/gems/2.6.0/gems/searchjoy-0.4.1/lib/searchjoy/engine.rb:7:in `block in <class:Engine>'

/Users/xxxx/.rbenv/versions/2.6.0/lib/ruby/gems/2.6.0/gems/railties-6.0.0/lib/rails/railtie/configuration.rb:97:in `method_missing': undefined method `assets' for #<Rails::Application::Configuration:0x00007fc2af60afd0> (NoMethodError)

Searchjoy::Search convert method undefined.

Using searchjoy 0.3 and searchkick 3.0 in a rails 4.2 application.

I've installed and run db:migrate successfully, but when I was searching I was getting an error that said WARNING: Can't mass-assign protected attributes for Searchjoy search fields :user_id, :search_type, :query etc. and the rows would remain with an id but no user_id etc filled in.

To solve this I created an initializer, searchjoy.rb with the following:

module Searchjoy
  Search.class_eval do
    attr_accessible :user_id, :search_type, :query, :normalized_query, :results_count, :convertable_id, :convertable_type, :converted_at
  end
end

to add the needed permissions attr_accessible to the fields. This then allowed proper writes to the searchjoy_searches table.

Subsequently I realized that searchkick intelligent search would not work without conversions. So, I set up the two models with conversions, as described by searchkick and searchjoy documentation:
eg:

class User < ActiveRecord::Base
  has_many :searches, class_name: "Searchjoy::Search", as: :convertable

  searchkick conversions: [:conversions], word_start: [:name], callbacks: :queue
...
   def search_data
      {
         name: name,
        organization_id: organization_id,
        group_type: group_type,
        active_users: active_users.invited.count,
        conversions: searches.group(:query).uniq.count(:user_id)
     }
  end

(same for class Model < ActiveRecord::Base)
Every search turns up in the database, but whenever I call search.convert I get the following error:
undefined method 'convert' for Searchjoy::Search
I checked the class name and it is a Searchjoy::Search that I'm calling the function on; and I checked what methods are available, and there is no convert method available. I am wondering if there is some confusion between the searchjoy and searchkick documentation or whether there is something missing, or it's a bug.

Multiple conversions per search

Hi,

is there a built-in way for searches to be able to convert more than once?
Right now, if I have Searchkick create the SearchJoy search via track and then call .convert(model) on that search, no other model can convert that search. But it makes sense for multiple search results to convert a single search. For example when I search for tortilla chips, I might not just add the unfavored version to the cart, but the spicy one as well ;-)

If there is no built-in way to do this with Searchkick & SearchJoy right now, I guess a possible workaround would be to not have Searchkick create any searches and only create a new search and convert it immediately once the actual conversion takes place.

Fails with Sprockets 4.0.0

Sprockets 4.0.0 was just released, and rails assets:precompile fails, but when I comment this gem, it works.

My environment -

  • Rails 6.0.0
  • Sprockets 4.0.0
  • Ruby 2.6.5
  • Searchjoy 0.4.1

Errors when using Searchkick with option execute => false

When creating queries with searchkick using the option "execute: false" in combination with searchjoy, the Track module tries to append the Search object to the Searchkick::Query, which doesn´t have the respective attribute, thus leading to errors.

Relevant references:

  • searchjoy/lib/searchjoy/track.rb:8
  • searchjoy/lib/searchjoy.rb:41

Additionally, when creating queries with searchkick using the option "index_name", Search objects without search_type are created. This leads to errors in SearchesController#set_searches_types

Relevant references:

  • searchjoy/app/controllers/searchjoy/searches_controller.rb:41

Tested with:

  • searchkick 1.2.1
  • searchjoy 0.0.10

Undefined error running generate

Running ruby 1.9.3 on 3.1 Rails and receiving this error when trying to run the generate searchjoy:install step:

gems/searchjoy-0.0.6/lib/searchjoy/engine.rb:3:in <class:Engine>': undefined methodisolate_namespace' for Searchjoy::Engine:Class (NoMethodError)

MassAssignmentSecurity::Error

Not sure if I'm just missing the obvious but when I try to track searches I receive:

ActiveModel::MassAssignmentSecurity::Error: Can't mass-assign protected attributes: search_type, query, results_count

I tried tracking via searchkick and adding manually.

Searchjoy::Search.create(
  search_type: "Item", 
  query: "apple",
  results_count: 12
)

How to use with autocomplete

Hello @ankane, thanks as usual for your amazing work
A question: how would you suggest using searchjoy with autocomplete-as-you-type feature? Tracking each character is too much, maybe with a timeout? Something like

if user.searches.where('created_at > ?', 5.seconds.ago)
  # update search
else
  # search and track
end

exception from groupdate gem

when click post button will report wrong number of arguments (given 4, expected 2) exception from groupdate gem.

image

Routing error

Hi,
i see strange behaviour of searchjoy gem. Just added gem, but there are errors in logs .

server  | Started GET "/searchjoy/searches/recent" for ::1 at 2019-05-12 23:30:00 +0300
server  |    (2.0ms)  SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC
server  |   ↳ /Users/alec/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/log_subscriber.rb:98
server  |
server  | ActionController::RoutingError (No route matches [GET] "/searchjoy/searches/recent"):
server  |
server  | actionpack (5.2.3) lib/action_dispatch/middleware/debug_exceptions.rb:65:in `call'
server  | web-console (3.7.0) lib/web_console/middleware.rb:135:in `call_app'
server  | web-console (3.7.0) lib/web_console/middleware.rb:30:in `block in call'
server  | web-console (3.7.0) lib/web_console/middleware.rb:20:in `catch'
server  | web-console (3.7.0) lib/web_console/middleware.rb:20:in `call'
server  | actionpack (5.2.3) lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
server  | railties (5.2.3) lib/rails/rack/logger.rb:38:in `call_app'
server  | railties (5.2.3) lib/rails/rack/logger.rb:26:in `block in call'
server  | activesupport (5.2.3) lib/active_support/tagged_logging.rb:71:in `block in tagged'
server  | activesupport (5.2.3) lib/active_support/tagged_logging.rb:28:in `tagged'
server  | activesupport (5.2.3) lib/active_support/tagged_logging.rb:71:in `tagged'
server  | railties (5.2.3) lib/rails/rack/logger.rb:26:in `call'
server  | actionpack (5.2.3) lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
server  | request_store (1.4.1) lib/request_store/middleware.rb:19:in `call'
server  | actionpack (5.2.3) lib/action_dispatch/middleware/request_id.rb:27:in `call'
server  | rack (2.0.7) lib/rack/method_override.rb:22:in `call'
server  | rack (2.0.7) lib/rack/runtime.rb:22:in `call'
server  | activesupport (5.2.3) lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
server  | actionpack (5.2.3) lib/action_dispatch/middleware/executor.rb:14:in `call'
server  | actionpack (5.2.3) lib/action_dispatch/middleware/static.rb:127:in `call'
server  | rack (2.0.7) lib/rack/sendfile.rb:111:in `call'
server  | webpacker (4.0.2) lib/webpacker/dev_server_proxy.rb:29:in `perform_request'
server  | rack-proxy (0.6.5) lib/rack/proxy.rb:57:in `call'
server  | railties (5.2.3) lib/rails/engine.rb:524:in `call'
server  | puma (3.12.1) lib/puma/configuration.rb:227:in `call'
server  | puma (3.12.1) lib/puma/server.rb:660:in `handle_request'
server  | puma (3.12.1) lib/puma/server.rb:474:in `process_client'
server  | puma (3.12.1) lib/puma/server.rb:334:in `block in run'
server  | puma (3.12.1) lib/puma/thread_pool.rb:135:in `block in spawn_thread'

PS: routes.rb does contains

  authenticate :user, lambda { |u| u.has_role? :admin } do
    mount Searchjoy::Engine, at: "searchjoy"
  end

Conversion tracking one-to-many query-to-conversions

Is one-to-many query-to-conversions supported ootb?

We're using searchkick and currently investigating if we should include searchjoy for conversion tracking. Our search has a one search many result behaviour pattern.

It looks like by default searchkick creates the searchjoy record. Can we overwrite this behaviour easily? Possibly add a join table and move convertable poly assoc to there?

Rails API Only Support

I think a good addition to this gem would be Rails 5 API only support. As it stands now, this gem requires Sprockets and ActionView to be included in your project. I would love to be able to remove both from my application, as I use Searchjoy for tracking searches and conversions, but don't use the interface provided to view this information.

It would be an even bigger bonus if the option made the Searchjoy routes return JSON instead of an HTML page, but that can always be added at a later time.

Multiple queries

Is it possible to log a multiple queried search? For example if someone has to choose a search term and a category.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.