Comments (5)
You are right! When I decrease the size of the dictionary to 5k pairs, there is no any error. Thank you very much! @glample
from muse.
Hi. Can you provide the entire backtrace? What line in the code triggers the error exactly?
from muse.
INFO - 01/19/18 08:48:09 - 0:00:52 - Found 25201 pairs of words in the dictionary (13620 unique). 33812 other pairs contained at least one unknown word (21885 in lang1, 27008 in lang2)
THCudaCheck FAIL file=/pytorch/torch/lib/THC/generic/THCStorage.cu line=58 error=2 : out of memory
Traceback (most recent call last):
File "./unsupervised.py", line 136, in
evaluator.all_eval(to_log)
File "/home/jack/codes/MUSE/src/evaluation/evaluator.py", line 192, in all_eval
self.word_translation(to_log)
File "/home/jack/codes/MUSE/src/evaluation/evaluator.py", line 95, in word_translation
method=method
File "/home/jack/codes/MUSE/src/evaluation/word_translation.py", line 133, in get_word_translation_accuracy
scores.sub_(average_dist1[dico[:, 0]][:, None] + average_dist2[None, :])
File "/home/jack/.local/lib/python3.5/site-packages/torch/tensor.py", line 293, in add
return self.add(other)
RuntimeError: cuda runtime error (2) : out of memory at /pytorch/torch/lib/THC/generic/THCStorage.cu:58
Thank you very much! @glample
from muse.
Are you using a dictionary different than the train / test we provide? 25k pairs seems quite a lot. What happens if you use the dictionaries we provide? Maybe try that and see if you still have the issue.
That being said, the code should still run even with a big dictionary. If you send it to me I can try to have a look.
from muse.
No, I used a Chinese-Inner Mogolian dictionary of my own as test . Because there is no language pairs I need in the website.
And I used the unsupervied.py to train the model.
So, should I first use the dictionaries provided with the default parameters?
Thank you very much!
from muse.
Related Issues (20)
- why unsupervised can achieve Word alignment?
- Can some one give the dictionary tree of the whole project? Like in the data/crosslingual or monlingual/.. HOT 5
- non-parallel chinese traditional - english
- evaluate.py error
- openssl ssl_read ssl_error_syscall errno 110
- Reproducing Results in Table 1 HOT 1
- IndexError: index out of range in self
- AttributeError: 'Namespace' object has no attribute 'dico_max_rank'
- Assertion Error while using the unsupervised way.
- Tokenization issue in to-En bilingual dictionaries
- They hated the kid HOT 1
- Bad outcome in ja-en task HOT 1
- Rush Shhh INPUT aUTOMATION
- ValueError: too many values to unpack (expected 2) in unsupervised.py
- Will pytorch's deprecation of volatile affect the result?
- [ML Question] Is it possible somehow to translate two or three words ?
- Tried on GloVe?
- self-mapped english words in dictionaries
- ValueError: Function has keyword-only parameters or annotations, use inspect.signature() API which can support them HOT 3
- demo notebook references unavailable private files
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from muse.