acbull / hice Goto Github PK

Do you have Python scripts for the NER and POS tasks? Also, for these tasks, were the same embeddings and HiCE models used, or were they trained on a new set of embeddings?

Hi,

I am currently trying to replicate the results in the paper, but my numbers seem to be off. Is there any hyperparameter difference from the defaults in code?

Here is the line I trained the model with:
python3 HiCE-master/train.py --cuda 0 --use_morph --adapt --w2v_dir "herbelot_and_baroni_w2v/wiki_all.sent.split.model" --corpus_dir "HiCE-master/data/wikitext-103/" --save_dir "/my_directory/" --chimera_dir "HiCE-master/data/chimeras/"

While the Upperbound seems to be correct, both the Baseline:Additive and HiCE seem to be off (HiCE less so, but it is). Do you have any insight as to why? Here are my results:

Baseline: Additive
0.2871964621624245
0.30641486173408883
0.30354669461624095

Upper Bound: Ground Truth Embedding
0.41732506349077386
0.4366845012259888
0.4409890833288333

Test with 2 shot: Cosine: 0.4728; Spearman: 0.3509

Test with 4 shot: Cosine: 0.5198; Spearman: 0.3842

Test with 6 shot: Cosine: 0.5483; Spearman: 0.4007

hi，thank u for ur excellent work. In ur paper, u listed Top-5 similar words. I'm wondering how to get them? R they selected from the whole dictionary? And compared by the cosine similarity of embedding from wiki_all.model?