Problem in prepare_data.py about nmt-chatbot HOT 8 CLOSED

I have the same issue

from nmt-chatbot.

Hi.

Are you sure you are using latest version of code? I was using prepare_data.py a lot lately in many ways without any issue.
Did you modify files or folder structure in any way?

As for underscores - something is odd here - underscores shouldn't be printed. Did you get that repo recursively (it will include our modified fork of nmt)?

from nmt-chatbot.

I didn't get that repo recursively by the first way. Now I had get it recursively and the underscore is gone but I still can't get prepare_data.py running from Git Bash and cmd. I am running prepare_data.py from python and everything seems working normally. I think the underscore is related to the BPE/WPM-like (subword) tokenization. Thanks by the way.

from nmt-chatbot.

Underscores - yes, it's related but not that exactly. NMT is modified by us and includes some fixes.

I'm not really sure what do you mean by run prepare_data.py in Git Bash or in Python. Can you please show me screenshots of all those 3 ways?

from nmt-chatbot.

Here are the screenshot:
Git Bash

CMD

It worked normally when I run from python.

from nmt-chatbot.

Ok, thank you.
I think that your problem might actually be space character in folder name (New folder). Could you please either remove space or replace it with underscore for example and let me know if that helped?

from nmt-chatbot.

I probably found an issue in our code.
I'm testing it now.

from nmt-chatbot.

In case you wonder - this should be fixed now (with latest version of code)

from nmt-chatbot.