MKQA dataset about leaderboard HOT 8 CLOSED

Hey, sorry for late reply. Despite how much we would like to include more datasets, MKQA is more of a open domain question answering dataset than a KGQA dataset. Therefore, I do not think we would integrate MKQA. What do you think @RicardoUsbeck?

from leaderboard.

I think this is a very interesting special case. In particular, MKQA has answers grounded in a graph. Similar to FreebaseQA. Even though it has no KGQA logical form, e.g. a SPARQL query or a lambda expression, it still is valid. So, adding it makes sense if it fits the other criteria.

@alexanderpanchenko feel free to submit a pull request.

from leaderboard.

Thank you! From what I understood, part of the queries are answered from KG. But if the question is non-answerable with KG then there is no answer in this form.

Regarding pull request: in case you are not planning to implement yourself but is ready to include we may try to do it on our side, yes.

from leaderboard.

Then it makes sense to add this dataset.

@alexanderpanchenko It would be faster if you add the MKQA. We would publish an instruction video soon, you can check it out. :)

from leaderboard.

fixed by #49

from leaderboard.

Thank you. We will consider implementing it during fall 2022.

from leaderboard.

Implementing?

from leaderboard.

Thanks for integrating it: so far I did not had capacity to do it. It's great you managed to find resources to include it (and RuBQ too).

from leaderboard.