New benchmarks for the m2m/mbart models after the switch away from Fairseq to huggingface transformers in EasyNMT 2.0 about easynmt HOT 3 OPEN

Hi @kgravenreuth
We hosted the models using floating point with 16 bits (fp16). In version 2, we switch to use huggingface transformers. They host the models using fp32, hence, the models are twice as large.

Performance in terms of translation accuracy should be the same.

Will it be faster or slower: The size has here little impact, because the models can internally transferred between these two type of storing floating point numbers.

But it can be that the huggingface implementation is faster or slower then the previous fairseq implementation. Hope I can update the numbers soon

from easynmt.

Sadly huggingface models are much slower than the fairseq models.

I think a difference is the usage of FP32 vs. FP16 as fairseq is doing. Will check if converting the model to FP16 will speed up the computation.

from easynmt.

Hello. I'm wondering if you have any further infomation.. we'd like to run the m2m-100 model and I'm considering going back to the fairseq code that was removed with v2.0.0.

from easynmt.