NativeQuery performance CPU usage about examine HOT 5 CLOSED

@Shazwazza i've done some extra investigation and tried to simplify the above query.
I've changed my code a bit and my query now looks like this:

+(FullTextContent_nl:ventilatie* FullTextContent:ventilatie* nodeName_nl:ventilatie* nodeName:ventilatie*) +(__NodeTypeAlias:contentpage __NodeTypeAlias:labitem __NodeTypeAlias:newsitem __NodeTypeAlias:project __NodeTypeAlias:technicalcommittee __NodeTypeAlias:standardsantennaitem __NodeTypeAlias:jobitem __NodeTypeAlias:tool __NodeTypeAlias:contact __NodeTypeAlias:technologiesdetail)

I've noticed that our CPU usage is also spiking when using this query in de examine management tab in our externalIndex to about 80-90%.

from examine.

Hi, you'll need to do some more investigating on this one to determine the underlying problem.

• What happens when you only search non 'FullTextContent' fields?
• What happens when you only do a very simple search?
• Is the problem directly related to the FullTextContent or not?
• Is the problem related to building the query or executing the query?
• Can you create a dump file for analysis of where the CPU bottleneck is?
• Can you replicate this from scratch with a simple setup?

from examine.

Hi @Shazwazza ,

I've done some more investigation.
We've got some pages with a lot of content (like a lot lot :-) ).

The FullTextSearch package creates an extra property in the externalIndex with a string that is generated based on the content of that page.

We notice that some pages have a fullTextSearch value in examine of 1 million+.
I'm guessing that this can cause some serious issues in performance when quering the fullTextSearch property with an examine query?

And if so - What would be the ideal max length (if there is any) that a examine property value can have?

Thanks a lot!

from examine.

You should also check if the fulltextsearch project specifies LuceneSearchOptions.AllowLeadingWildcard = true. If that is the case, this is most likely the cause for the performance degredation. Lucene specifically says that enabling this

//     When set, * or ? are allowed as the first character of a PrefixQuery and WildcardQuery.
//     Note that this can produce very slow queries on big indexes.

from examine.

I will close this issue since I don't think this is an issue with Examine but more the usage of Examine via the FullTextSearch package. As for what is an ideal max length in a field, you'll need to investigate lucene docs/forums regarding that. I assume there is some performance penalties for indexing values of such huge sizes.

from examine.