Examine GetFieldInternalQuery error about examine HOT 20 CLOSED

Hi @jplatfordquba
if there's going to be any chance of figuring out the issue we need some more information. Examples:

• What precise Examine version in use.
• Examples of queries that caused the issue and expected results of these queries (Not the data but maybe amount of elements returned)
• Do you have any custom code to Examine/Lucene related code. I.E custom index, custom searcher or similar.

from examine.

Hi @nikcio

• Examine version 3.0.0
• All attempts to use pages with Examine used for searching errored, regardless of query. Essentially the only thing it was filtering for were document types (News doc types for the News search for example).
• No custom indexes, just the ExternalIndex default to Umbraco

Snippet below:

var criteria = searcher.CreateQuery(IndexTypes.Content, BooleanOperation.And);
IBooleanOperation examineQuery;
examineQuery = criteria.GroupedOr(new string[] { "__NodeTypeAlias" }, searchableNodeTypes);
var results = examineQuery.Execute();

from examine.

@jplatfordquba the value / type of searchableNodeTypes needs to be included to help diagnose. Also the query string that can be obtained from the debugger by inspecting the value of examineQuery.

from examine.

We are seeing the same problem as @jplatfordquba. It is the exact same symptoms with the exact same stack trace in Umbraco 10.6.1. All queries will work fine for days, but all of a sudden queries that worked fine before now fail. Every query we do will fail until we restart the app pool and then everything will work fine again. We run two load balanced server and the failure will randomly happen on one or the other.

from examine.

As this error is coming from Lucene code, I might suggest reporting this issue on the Lucene repo https://github.com/apache/lucenenet

But like @nzdev mentions, without all of the information its pretty hard to diagnose the issue without steps to replicate.

from examine.

Unfortunately we are unable to determine what steps are needed to replicate it. It happens pretty randomly.

from examine.

Is it generally from the same query? If so, can you share the entire query and how you are creating it? I wonder if we can setup a test to continuously execute it and see if it eventually fails.

from examine.

Is it generally from the same query? If so, can you share the entire query and how you are creating it? I wonder if we can setup a test to continuously execute it and see if it eventually fails.

We query the index in a couple different areas of our site and once it starts failing it fails in all of them. This is probably the most straight forward way we query it. This is for a general index page so no user input is involved, all the parameters are statically determined. Note this code is slightly simplified.

if (!_examineManager.TryGetIndex(UmbracoIndexes.ExternalIndexName, out var index) || !(index is IUmbracoIndex umbIndex))
{
throw new InvalidOperationException($"No index found by name ExternalIndex or is not of type {typeof(IUmbracoIndex)}");
}
QueryOptions queryOptions = new QueryOptions(0, 3000);
string[] docTypes = docTypeList.ToArray(); // Gets populated with a list of doc types
var resultsAsSearchItems = index
.Searcher
.CreateQuery(IndexTypes.Content)
.Field("__Published_en", "y")
.And()
.GroupedOr(new[] { "__NodeTypeAlias" }, docTypes)
.Execute(queryOptions);

This is the stack trace it produces when the problem occurs. We have a more complex query that produces the exact stack trace that @jplatfordquba reported

System.NullReferenceException: Object reference not set to an instance of an object.
at Lucene.Net.QueryParsers.Classic.ParseException.Initialize(Token currentToken, Int32[][] expectedTokenSequences, String[] tokenImage)
at Lucene.Net.QueryParsers.Classic.QueryParser.GenerateParseException()
at Lucene.Net.QueryParsers.Classic.QueryParser.Jj_consume_token(Int32 kind)
at Lucene.Net.QueryParsers.Classic.QueryParser.Clause(String field)
at Lucene.Net.QueryParsers.Classic.QueryParser.Query(String field)
at Lucene.Net.QueryParsers.Classic.QueryParser.TopLevelQuery(String field)
at Lucene.Net.QueryParsers.Classic.QueryParserBase.Parse(String query)
at Examine.Lucene.Search.LuceneSearchQueryBase.GetFieldInternalQuery(String fieldName, IExamineValue fieldValue, Boolean useQueryParser)
at Examine.Lucene.Search.LuceneSearchQuery.Search(QueryOptions options)

from examine.

Another interesting piece of information on this. We removed IndexTypes.Content from CreateQuery(IndexTypes.Content) since in our scenario it really wasn't needed and when we did this searches no longer fail. I was able to confirm this since we missed that change in one place and when the problem re-occurred it only failed on the function where we didn't remove it, it worked on in other areas.

from examine.

Ok thanks for the feedback! At least this gives us something to try to use to test/replicate and then figure out. Will need to setup some tests to see if/how we can try go get this issue to occur.

from examine.

Seems like this is also reported in the CMS Github issue tracker - if you need more details @Shazwazza

umbraco/Umbraco-CMS#12646 (comment)

from examine.

Thanks all - Since this happens intermittently this sounds like an issue with something holding state and not being thread safe. My feeling is that maybe its a QueryParser instance being re-used and the QueryParser is not thread safe. I can see in the docs that "Note that QueryParser is not thread-safe." so potentially could be that.

But that said, there is no concurrency test that I can seem to write that will replicate this locally.

Could it be that folks are trying to re-use the instance of IQuery?:

IQuery criteria = searcher.CreateQuery(IndexTypes.Content, BooleanOperation.And);

If that is attempted to be re-used, things will break because it is not thread safe. Each instance of IQuery creates a new instance of the query parser.

from examine.

I think I have an idea of where/how this is happening but I'm unsure how this is being caused outside of the Examine Source. Based on the call stack, I can see that this goes into (in bold)

at Lucene.Net.QueryParsers.Classic.QueryParserBase.Parse(String query)
at Examine.Lucene.Search.LuceneSearchQueryBase.GetFieldInternalQuery(String fieldName, IExamineValue fieldValue, Boolean useQueryParser)

The only way this occurs if if the boolean useQueryParser is false. But there is nowhere in the Examine code that can make this false. But I discovered that behind that query parser is a static instance of a query parser and as I mentioned above this is not thread safe. So my assumption is this is the problem, but I don't know how it gets caused.

Changing to a non-static instance is easy, so I can make that change and publish it as part of 3.2 (in hopes that it 'just works')

from examine.

I've pushed Examine 3.2.0 RTM, hopefully that resolves the issue.

from examine.

I've pushed Examine 3.2.0 RTM, hopefully that resolves the issue.

Thanks for the update. We are tied to the version that Umbraco is using so we don't have any way to test this.

from examine.

You can update to the latest Examine externally of Umbraco, unless you are on an older version

from examine.

Umbraco version 13.1.1

Lucene.Net.QueryParsers.Classic.ParseException: Cannot parse '(id: OR __Path:-1*,,)': Encountered " "OR "" at line 1, column 5.
Was expecting one of:
...
"(" ...
"" ...
...
...
...
...
...
"[" ...
"{" ...
...

---> Lucene.Net.QueryParsers.Classic.ParseException: Encountered " "OR "" at line 1, column 5.
Was expecting one of:
...
"(" ...
"*" ...
...
...
...
...
...
"[" ...
"{" ...
...

at Lucene.Net.QueryParsers.Classic.QueryParser.Jj_consume_token(Int32 kind)
at Lucene.Net.QueryParsers.Classic.QueryParser.Clause(String field)
at Lucene.Net.QueryParsers.Classic.QueryParser.Query(String field)
at Lucene.Net.QueryParsers.Classic.QueryParser.Clause(String field)
at Lucene.Net.QueryParsers.Classic.QueryParser.Query(String field)
at Lucene.Net.QueryParsers.Classic.QueryParserBase.Parse(String query)
--- End of inner exception stack trace ---
at Lucene.Net.QueryParsers.Classic.QueryParserBase.Parse(String query)
at Examine.Lucene.Search.LuceneSearchQueryBase.NativeQuery(String query)
at Umbraco.Cms.Infrastructure.Examine.DeliveryApiContentIndex.PerformDeleteFromIndex(IEnumerable1 itemIds, Action1 onComplete)
at Umbraco.Cms.Infrastructure.Examine.Deferred.DeliveryApiContentIndexHandleContentChanges.Reindex(IContent content, IIndex index)
at Umbraco.Cms.Infrastructure.Examine.Deferred.DeliveryApiContentIndexHandleContentChanges.b__7_0(CancellationToken _)
at Umbraco.Cms.Infrastructure.HostedServices.QueuedHostedService.BackgroundProcessing(CancellationToken stoppingToken)

from examine.

I have this error when I save and publish.

from examine.

@biapar What Examine version are you using and have you tried the latest version?

from examine.

3.2.0

from examine.