Elasticsearch Completion Suggest about chewy HOT 13 CLOSED

try index.suggest(mysuggest: { text: 'roy', completion: { field: 'suggest' }}).suggest

from chewy.

That works, but the server is still also performing the empty query, and then I think Chewy is doing some stuff behind the scenes with the corresponding (irrelevant) active record objects, which is slowing down the completion significantly.

ConsoleIndex Search (8.7ms) {:body=>{:suggest=>{:mysuggest=>{:text=>"r", :completion=>{:field=>"suggest"}}}}, :index=>"console", :type=>["job", "table", "column"]}
Completed 200 OK in 146ms (Views: 0.3ms | ActiveRecord: 126.2ms)

Another question I have: is there any way to load the active record objects relevant to the completion suggest query?

from chewy.

Chewy will do nothing until you'll ask. Could you please double check your conclusions? Maybe. it is just rails views are slow? Search itself seems to be quite fast - 8.7ms

Yeah, Chewy doesn't use separate suggestions api, it uses cooperative query + suggest api. It query is empty, ES returns suggestions only, which might be accessed via query.suggest method called paramless. If there was a query in addition - ES returns found objects for the query, which might be accessed via query.to_a or if you want AR objects - query.load. Any of this 3 methods executed at first triggers request to ES performing. Other method calls on the same query object will use first call results. So there will no be additional requests to ES. Lazy loading, dude :)

from chewy.

When I submit this query (the same query produced by "index.suggest(mysuggest: { text: 'roy', completion: { field: 'suggest' }})") to the ES server:

GET /console/job,table,column/_search [{"suggest":{"mysuggest":{"text":"roy","completion":{"field":"suggest"}}}}]

{
   "took": 2,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hits": {
      "total": 612,
      "max_score": 1,
      "hits": [
          ...
          (10 hits here)
          ...
        ]
   },
   "suggest": {
      "mysuggest": [
         {
            "text": "roy",
            "offset": 0,
            "length": 3,
            "options": [
               {
                  "text": "Roy Lane",
                  "score": 1
               }
            ]
         }
      ]
   }
}

I'm pretty sure that with an empty query ES returns everything in the index. What I'm confused by is why the rails server is telling me "ActiveRecord 126.2ms" when really it should not be doing anything with AR if all it was doing is getting the completion suggest results. I suspected that it had to be Chewy doing this work (maybe wrapping the returned objects from blank query in the chewy wrapper class?) since the actual query is so fast and I'm not calling either of the to_a or load functions on my suggest, but really I'm not sure what's going on. The rails views says it only took 0.3ms so I don't think it would be that, but again I'm new to ruby/rails so not entirely sure how everything works together.

from chewy.

I'm pretty sure that with an empty query ES returns everything in the index

Yep, I was wrong. But it returns first 10 documents only by default.

I suspected that it had to be Chewy doing this work (maybe wrapping the returned objects from blank query in the chewy wrapper class?)

There must be evidences in log. If there is nothing AR in log - then Chewy does nothing. Chewy wraps results only on query.load (or query.preload) method call.

Actually, try to reload page. If there is nothing AR in log - it seems to be just system AR queries such as getting database schema for models for the first time.

from chewy.

You're right: there is some other method in my app that is being called at each completion suggest call that is doing a really slow DB query.

On a semi-related note: is there any way to get chewy to load objects for suggestion results? "index.suggest(mysuggest: { text: 'roy', completion: { field: 'suggest' }}).suggest" returns a ruby array, not a chewy wrapper. The reason why I'm asking is because I need to implement document level permissions for my autocomplete results.

Probably the best way to do this, however, would be to just store the permissions in a payload attribute in the ES index via the ES search completion mapping like this example from the ES website http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-suggesters-completion.html

curl -X PUT localhost:9200/music/song/_mapping -d '{
  "song" : {
        "properties" : {
            "name" : { "type" : "string" },
            "suggest" : { "type" : "completion",
                          "index_analyzer" : "simple",
                          "search_analyzer" : "simple",
                          "payloads" : true
            }
        }
    }
}'

How can I specify a mapping with sub-fields like this in chewy?

I currently have something like this...
define_type Job.includes(:user) do
field :user, value: -> { user.name }
field :type_name
field :type_description
field :name, analyzer: 'trigrams'
field :group_name
field :id, type: 'integer'
field :suggest, value: -> { name }, type: 'completion'
end

from chewy.

is there any way to get chewy to load objects for suggestion results?

Unfortunately, no. Didn't even think about it. Moreover, I didn't use ES suggester at all yet, sorry.

How can I specify a mapping with sub-fields like this in chewy?

field :field, type: 'object' do
  field :subfield, type: 'integer'
end

Or you can even use templates, there is a DSL in Chewy.

from chewy.

In ES, specifying a mapping and inserting into the index are separated into different requests. In Chewy, they are kind of combined into one thing (the index definition) and I'm a bit confused. In order to specify a mapping like this:

"suggest" : { "type" : "completion",
                          "index_analyzer" : "simple",
                          "search_analyzer" : "simple",
                          "payloads" : true
 }

Where payloads is set to true in the mapping, but also each insert into this type has a "payload" value specified by some function (maybe it is just the document id). How do I write something like that in Chewy? Also, the fields index_analyzer and search_analyzer only need to be set during the mapping and are not present during an index insert at all. How do I tell Chewy to give those values when specifying the mapping but not when inserting into index?

(this is my last question and then I will close this issue haha)

from chewy.

Ah, got it now. All the options passed to field except value are used for mapping. So you can do something like:

field :suggest, type: 'completion', index_analyzer: 'simple', search_analyzer: 'simple', payloads: true

If you are not specifying value explicitly - it will call job.suggest method implicitly while indexing and use returned value to put in index. So you need to define this method like this:

class Job < ActiveRecord::Base
  ...
  def suggest
    { input: name, output: name, payload: { job_id : id }, weight: i_dont_know_some_job_weight_or_maybe_not }
  end
  ...
end

from chewy.

great, thanks for your help!

from chewy.

Not working for me!

from chewy.

@jondavidford While using Chewy how are you able to get the ElasticSearch requests? Can we enable logs or something so that we would be able to see the requests? It will be extremely helpful, please help.

from chewy.

@suryapandian sorry, but it's been 5 years since I looked at any of this stuff. I don't remember very well how I got it to work in the end.

from chewy.