Why do we get different embeddings for target language? about muse HOT 5 CLOSED

If the target embeddings in the generated source file are different, it can be for 2 reasons:

1. You set the parameter normalize_embeddings to something that will renormalize the loaded embeddings (either recenter them / divide them by their norm, etc) so they will be modified
2. The embeddings are normalized before they are written to the disk at the end of the experiment.

Now that you are mentioning this I'm not sure we were right about 2), it might not be best for downstream tasks. A simple fix is simply to uncomment these 2 lines:
https://github.com/facebookresearch/MUSE/blob/master/src/trainer.py#L249-L250
After this, you should obtain the original embeddings on the target side.

from muse.

I just checked now. Thanks, @glample . You are right. This is happening because of the normalization.

from muse.

These two https://github.com/facebookresearch/MUSE/blob/master/src/trainer.py#L255-L256

from muse.

sorry, what are the two lines to be uncommented? https://github.com/facebookresearch/MUSE/blob/master/src/trainer.py#L249-L250 has been changed since your comment

from muse.

These two https://github.com/facebookresearch/MUSE/blob/master/src/trainer.py#L255-L256

thx for the confirmation. Will comment those

from muse.