A particular search term returns a result count, but the wrappers are empty about chewy HOT 6 OPEN

I'm finding more and more search terms where this is the case. (Despite recreating the indexes again.)

What might be the reason a ModelIndex class doesn't populate the .wrappers or work with .each some of the time? (In some search terms, the first page (25 results) is displayed correctly, but any subsequent pages, though they exist according to the total, have no .wrappers or .each, but are correct and accessible via .first(n).

from chewy.

I have upgraded my app to Rails 6.1 and Ruby 3.2.1, but the behavior is unchanged.

Would it be possible for @pyromaniac or one of the other maintainers to offer thoughts on what might be causing this behavior, where results are correct, .total_pages is correct, but .wrappers is [] on page 2 and onwards with Kaminari?

from chewy.

Further finding:

when a search has, say, 285 result pages (at 25 results per page), some pages load fine, including 1, 2, and 285 (the last one), and others with empty .wrappers as above (pages 281, 284). This behavior is consistent for this particular search.

(my index is fresh and up to date, verified with rake chewy:reset)

from chewy.

Unfortunately, the persistence of this issue is forcing me to ditch Chewy in favor of plain Elasticsearch. :(

from chewy.

In a last-ditch effort, I traced into Chewy and into elastic-transport, and have found the problem:

"type"=>"illegal_argument_exception", "reason"=>"The length [1134481] of field [fulltext] in doc[7882]/index[manifestations_1678539023000] exceeds the [index.highlight.max_analyzed_offset] limit [1000000]. To avoid this error, set the query parameter [max_analyzed_offset] to a value less than index setting [1000000] and this will tolerate long field values by truncating them."}

This Elastic error was indeed introduced in 7.x, so explains this happening after my upgrade. It being a function of document size explains the apparent randomness of its occurrence, but consistently for particular search terms.

It seems to me that this error is somehow ignored or missed by Chewy, which just returns those pages without results. So while I will solve my issue by changing the setting, I do believe there is a Chewy issue here as well.

Thoughts?

from chewy.

(for anyone who's run into this and is looking for a quick fix, adding the max_analyzed_offset parameter to the highlight method solves the issue.)

index.highlight(max_analyzed_offset: 999000, fields: {fulltext: {}})

from chewy.