KeyError in validation_next_word_loop when running main.py about mdvc HOT 4 CLOSED

Hi @yxinli92,
cc: @v-iashin
I tried to train the model on my own and also stumble across this problem. I noticed that in the latest version of PyTorch/text module there is an issue with the unknown token being used. Please refer PyTorch/Text Unknown token for more details. In short, if you specify unknown token explicitly at least I don't reproduce this issue,

self.ASR_SUBTITLES_FIELD = data.ReversibleField( tokenize='spacy', init_token=self.start_token, eos_token=self.end_token, pad_token=self.pad_token, lower=True, batch_first=True, unk_token='<unk>')

If you already solved it, then please ignore it. I just wanted to point the root cause for other if they stumble across this problem.

from mdvc.

Hi Xinli,

Thanks for reporting.

I installed the env on another machine with 1080Ti and I couldn't reproduce the problem after training it for 6 epochs.

I also found that conda saves the spacy model in the environment under pip package but fails to install it and all other packages which are expected to be installed after (torchtext in our case). I fixed it in 7873bea.

Anyway, let's see why do you have such a problem. It seems that it is related to text-processing parts. Please share:

1. When do you get this error? How many epochs have you run it for?
2. Which version of torchtext, spacy are you using?

from mdvc.

Assuming the problem was local. Please reopen if you think otherwise and provide more details.

from mdvc.

@VP-0822 This is a valuable comment. Thanks for sharing.

from mdvc.