SpeechT5：how much epoch is set about speecht5 HOT 5 CLOSED

We used max updates to limit the epoch of pre-training and fine-tuning.

from speecht5.

OK, thanks a lot!!!

from speecht5.

Sorry, I have another question. When I want to use 8 GPUs training, I just change the parameters '--distributed-world-size 8' and finally the utilization rate of each GPU is similar(all 8 GPUs can work properly). However, I find that using 8 GPUs to train for one update is even slower than using one GPU. Do I need to modify other parameters ？Have you ever encountered such problem?

from speecht5.

Actually, there are samples of --max-tokens in each GPU and one update requires a forward on each GPU with communication between them. Thus, the time cost for one update is higher than that using a single GPU due to additional communication overhead.

from speecht5.

OK, thanks for your reply!!

from speecht5.