RuntimeError: cuda runtime error (2) : out of memory at /pytorch/torch/lib/THC/generic/THCStorage.cu:58 about muse HOT 5 CLOSED

You are right! When I decrease the size of the dictionary to 5k pairs, there is no any error. Thank you very much! @glample

from muse.

Hi. Can you provide the entire backtrace? What line in the code triggers the error exactly?

from muse.

INFO - 01/19/18 08:48:09 - 0:00:52 - Found 25201 pairs of words in the dictionary (13620 unique). 33812 other pairs contained at least one unknown word (21885 in lang1, 27008 in lang2)
THCudaCheck FAIL file=/pytorch/torch/lib/THC/generic/THCStorage.cu line=58 error=2 : out of memory
Traceback (most recent call last):
File "./unsupervised.py", line 136, in
evaluator.all_eval(to_log)
File "/home/jack/codes/MUSE/src/evaluation/evaluator.py", line 192, in all_eval
self.word_translation(to_log)
File "/home/jack/codes/MUSE/src/evaluation/evaluator.py", line 95, in word_translation
method=method
File "/home/jack/codes/MUSE/src/evaluation/word_translation.py", line 133, in get_word_translation_accuracy
scores.sub_(average_dist1[dico[:, 0]][:, None] + average_dist2[None, :])
File "/home/jack/.local/lib/python3.5/site-packages/torch/tensor.py", line 293, in add
return self.add(other)
RuntimeError: cuda runtime error (2) : out of memory at /pytorch/torch/lib/THC/generic/THCStorage.cu:58

Thank you very much! @glample

from muse.

Are you using a dictionary different than the train / test we provide? 25k pairs seems quite a lot. What happens if you use the dictionaries we provide? Maybe try that and see if you still have the issue.

That being said, the code should still run even with a big dictionary. If you send it to me I can try to have a look.

from muse.

No, I used a Chinese-Inner Mogolian dictionary of my own as test . Because there is no language pairs I need in the website.
And I used the unsupervied.py to train the model.
So, should I first use the dictionaries provided with the default parameters?
Thank you very much!

from muse.