Is the result of ComplEx model on FB15K reproducible? about pytorch-biggraph HOT 1 CLOSED

The only way to exactly reproduce the metrics in the paper is to use the examples/fb15k.py script. That is because that script uses a "special" evaluation method (filtered MRR), whereas the standard torchbiggraph_eval command used unfiltered MRR. There's no way to have torchbiggraph_eval do filtered MRR as it doesn't scale.

Just to clarify: the training you performed did probably produce good embeddings, as good as in the paper, it's just that the evaluation didn't capture this.

You can still use examples/fb15k.py to perform the evaluation on the embeddings you learned: if you put it in the right location (see README) the script will detect and use it rather than re-train from scratch.

I'm closing this but feel free to reopen if I got anything wrong or if you're still facing issues.

from pytorch-biggraph.