Running with unknown languages (how to disable eval) about muse HOT 6 CLOSED

The Voynich manuscript is difficult because the amount of monolingual data is very small. But here you have no dictionary as you say so you cannot evaluate anything. In the all_eval function of evaluator.py you can probably just comment out the line self.word_translation(to_log) and it should be fine.

from muse.

Thanks for your suggestion! It is true that the Voynich is a pretty small text, but I'm interested in seeing what happens for some of the most frequently occurring words.

I've tried commenting out self.word_translation(to_log) and I am left with an error thrown from self.dist_mean_cosine(to_log):

INFO - 04/17/18 12:25:02 - 0:07:29 - Cross-lingual word similarity score average: nan
Traceback (most recent call last):
File "unsupervised.py", line 137, in
evaluator.all_eval(to_log)
File "/gpfs/loomis/home.grace/fas/frank/wcm24/voynich2vec/MUSE/src/evaluation/evaluator.py", line 192, in all_eval
self.dist_mean_cosine(to_log)
File "/gpfs/loomis/home.grace/fas/frank/wcm24/voynich2vec/MUSE/src/evaluation/evaluator.py", line 172, in dist_mean_cosine
s2t_candidates = get_candidates(src_emb, tgt_emb, _params)
File "/gpfs/loomis/home.grace/fas/frank/wcm24/voynich2vec/MUSE/src/dico_builder.py", line 38, in get_candidates
scores = emb2.mm(emb1[i:min(n_src, i + bs)].transpose(0, 1)).transpose(0, 1)
ValueError: result of slicing is an empty tensor

I'm a little confused about what cosine distance this next line is evaluating, but it seems like something that I shouldn't be commenting out (?). Indeed, when I try to, I get a bunch of KeyErrors since the value it finds is being referenced elsewhere.

from muse.

This cosine distance is trying to reflect how aligned are your embedding spaces and is helpful to select the best mapping when you do not have access to a cross-lingual lexicon.

Can you try to put:
--dico_build S2T
as a parameter?

from muse.

When I run it with --dico_build S2T, I get the same "result of slicing is an empty tensor" error message.

EDIT:
The error is coming from the kNN code in get_candidates. Perhaps if I try changing dico_method, I can get around it?

    # nearest neighbors
    if params.dico_method == 'nn':

        # for every source word
        for i in range(0, n_src, bs):

            # compute target words scores
            scores = emb2.mm(emb1[i:min(n_src, i + bs)].transpose(0, 1)).transpose(0, 1)
            best_scores, best_targets = scores.topk(2, dim=1, largest=True, sorted=True)

            # update scores / potential targets
            all_scores.append(best_scores.cpu())
            all_targets.append(best_targets.cpu())

        all_scores = torch.cat(all_scores, 0)
        all_targets = torch.cat(all_targets, 0)

from muse.

Also, I should have mentioned before that the way I am running unsupervised.py is:

python unsupervised.py --src_emb ../models/voynich.vec --tgt_emb ../models/secretaSecretorum.vec
    --n_refinement 5 --emb_dim 100 --dis_most_frequent 100 --dico_build S2T

from muse.

Update:
The issue came from the fact that _params.dico_max_rank is being reset in line 169 of src/evaluate/evaluator.py:

_params.dico_max_rank = 10000

I commented this out so that _params.dico_max_rank is still set to 0, and the error was resolved.

from muse.