Indexing of new records will crash if the name of the Indexing Queue Configuration is different from tablename about ext-solr HOT 14 CLOSED

If I read the manual right here: https://forge.typo3.org/projects/extension-solr/wiki/Tx_solrindex

"plugin.tx_solr.index.queue
Defines a set of table indexing configurations. By convention the name of the indexing configuration also represents the table name. You can name the indexing configuration differently though by explicitly defining the table as a parameter within the indexing configuration. That's useful when indexing records from one table with different configuration - different single view pages / URLs for example."

So is this really a Bug?

from ext-solr.

Additionally there is a difference between tt_news and ext:news or?

from ext-solr.

And have you seen that we have static templates for those two news extensions?

https://github.com/TYPO3-Solr/ext-solr/blob/master/Configuration/TypoScript/Examples/IndexQueueNews/setup.txt

https://github.com/TYPO3-Solr/ext-solr/blob/master/Configuration/TypoScript/Examples/IndexQueueTtNews/setup.txt

from ext-solr.

Okay, I've realized, that it works fine with tt_news but I have this problem with recordsets of my own extension. I thought there will be the same behaviour with other tables.

So, I have to look out, why I have this problem with my own extension-records.
At the time I can see that my own new extension-records are dealt in the hook like an updated record but with a $uid value "_NEW". That's why I thought it is better to use the $recordUid.

I don't know if this is correct or if I have missed some configurations.
I will check it next week!

from ext-solr.

So, next try with new information.
At this time I could reproduce the behaviour with a tt_news-record!

Last time it looked as it would work fine, but it worked fine because of an additionalWhereClause (plugin.tx_solr.index.queue.news.additionalWhereClause = uid IN (SELECT mm.uid_local FROM tt_news_cat_mm mm WHERE mm.uid_foreign IN(1))) in my SOLR-Configuration for a news element, that checked the affiliation to a tt_news category. This affiliation is not set at the time the hook processDatamap_afterDatabaseOperations is called, that's why the new tt_news record wasn't added by this hook to the index queue automatically.

But without this additionalWhereClause the new tt_news record was added automatically to the index queue by this hook and was inserted into the tx_solr_indexqueue_item table with the wrong indexing_configuration name ("tt_news" instead of "news").

That means, it is not a specific configuration error with my own extension records and it seems to be a real bug!

from ext-solr.

Some additional information for better understanding.

In my previous comments I mean with "news" the [indexingConfiguration] name, I configured in my SOLR-configuration for my tt_news table records.
I've choosed the indexingConfiguration "news" because I have another indexingConfiguration ("dates") for the same tt_news table but with different additionalWhereClause to split my SOLR index into "news" and "dates".

And only with the constellation "the name of the Indexing Queue Configuration is different from tablename" the indexing queue is not correct inserted.

In other words - it will work fine with this static template, because of there is no explicit table defined, that is different from the indexingConfiguration name:
https://github.com/TYPO3-Solr/ext-solr/blob/master/Configuration/TypoScript/Examples/IndexQueueTtNews/setup.txt

But you should be able to reproduce the error with this following static template, because "the name of the Indexing Queue Configuration is different from tablename" ("news" vs. "tx_news_domain_model_news"):
https://github.com/TYPO3-Solr/ext-solr/blob/master/Configuration/TypoScript/Examples/IndexQueueNews/setup.txt

from ext-solr.

Might actually be a bug, feels like it at least. What versions are you on; TYPO3 and the solr extension?

from ext-solr.

svenleemaier i think your problem is more that u use for queue name as type :D so u write
plugin.tx_solr.index.queue.news.fields.type = news

but thats wrong. The type field should everytime hold the value for the table
plugin.tx_solr.index.queue.news.fields.type = tx_news

and create an additional field for your semantic type name

plugin.tx_solr.index.queue.fields.type_stringS = TEXT
plugin.tx_solr.index.queue.fields.type_stringS.value = news

and use this field for building ur templates & facets.

Because if you make the type field different from the table name, many hooks will not work (delete item from solr after deleting item in backend and so on)
Because if u delete an item in backend there will be a direct request to solr server with the shema
delete Where uid = {record ID) & type = {record tablename}

from ext-solr.

Br4inbukz, I don't use any "type" variable and I can use the "[indexingConfiguration]" (if you mean that as "type") for different configurations from the same table as it is discribed in the manual of the solr extension for TYPO3: https://forge.typo3.org/projects/extension-solr/wiki/Tx_solrindex#queueindexingConfigurationtable

Ingo, I reproduced the problem with the current stable 3.0.1 version of this extension from the TYPO3 extension repository and with the current 3.0.1-dev version from here on a TYPO3 6.2.12 / 6.2.14 LTS.
But it is not working correctly since the following commit: 6aebf4c
Before the $configurationName parameter wasn't passed to the indexQueue->updateItem function.

from ext-solr.

I could reproduce this issue, the index configuration can not be determined if it's a new record and so the table name is used.

Since $uid, used in processDatamap_afterDatabaseOperations(), contains the NEW... id the record can not be fetched and getIndexingConfigurationName() returns the wrong indexing configuration name. Using the already translated uid in $recordUid fixes the problem, as already mentioned.

from ext-solr.

What happend? Why it was removed from the 3.1 milestone?
This issue could be reproduced and a solution is already mentioned.

from ext-solr.

@svenleemaier Thanks bringing this up again and sorry for the delay. Since it is allways hard to find a balance between releasing a new version immediate or close to a TYPO3 release and to get all issues done, we decided to do the release last week and not to pospone it anymore.

But we will take care of thise issue and it would not be lost. Currently we are preparing the release of the "addon's" for solr to have one recommended, combination there.

If you have time, you could speed it up by investigating the problem and contribute a pull request. I think i could also have a look on this issue in the next days, but when you start before please just let me know.

from ext-solr.

@svenleemaier I've checked reproduced the issue and also the fix looks valid. I am currently preparing a regression test and a fix for it.

from ext-solr.

@svenleemaier
@dkd-friedrich
I've applied the fix and created a test and pull request, it would be nice when you have time and could shortly test it and give your +1 or -1

#180

from ext-solr.