fnielsen / wembedder Goto Github PK

View Code? Open in Web Editor NEW

49.0 8.0 11.0 168 KB

Wikidata embedding

Home Page: https://tools.wmflabs.org/wembedder

License: Other

Python 68.70% HTML 5.74% JavaScript 0.47% Jupyter Notebook 25.09%

wikidata wikidata-dump embedding-python embeddings gensim-word2vec

wembedder's Introduction

Wembedder

Embedding with Wikidata.

There is a web application and web service available:

python app.py

When the server is running and the model is loaded the webservice should appear at http://127.0.0.1:5000/

Note you will need to download model(s) from Zenodo for the webservice to start.

Pre-trained models

Wembedder currently works with specially trained Gensim models. Pretrained models are published on Zenodo:

Wembedder wikidata-20170613-truthy-BETA-cbow-size=100-window=1-min_count=20: Wikidata embedding Gensim CBOW trained on truthy dump with a 100-dimensional subspace, a window of 1 and a minimum count of 20. https://zenodo.org/record/823195
Wembedder wikidata-20170613-truthy-BETA-cbow-size=100-window=1-min_count=20-iter=25: Wikidata embedding Gensim CBOW trained on truthy dump with a 100-dimensional subspace, a window of 1 and a minimum count of 20. Trained with 25 iterations. https://zenodo.org/record/827339

The models need to be unpacked under ~/wembedder_data/models/.

Wikimedia Toolforge

The canonical webservice runs from Wikimedia Toolforge at https://wembedder.toolforge.org/

Reference

Finn Årup Nielsen, Wembedder: Wikidata entity embedding web service, arXiv:1710.04099, http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/7011/pdf/imm7011.pdf

wembedder's People

Contributors

Stargazers

Watchers

Forkers

stevenlol benjamesbabala yifangma bigrlab alpes2012 bodhisattwawiki waldenn marcohassan luke-feng zeta1999 rpatil524

wembedder's Issues

Update to Python3.9 on Toolforge

Unexpected "is out-of-vocabulary" error

https://tools.wmflabs.org/wembedder/most-similar/Q11989#language=en
currently gives "Q11989 is out-of-vocabulary", and I do not see anything in
https://www.wikidata.org/wiki/Q11989
that would justify that.

Link to paper

API similarity gives Internal Server Error

https://wembedder.toolforge.org/api/similarity/Q5/Q9035 gives "Internal Server Error

The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application."