Low-complexity FreqCodec requires a lot of VRAM about funcodec HOT 3 CLOSED

Sorry for the late response. If the number of residual layers is same, the smaller model should require less VRAM. However, the model have a larger number of residual layers, the situation would be different. Not that the number of residual layers would affect each block in the Encoder. Compare with the larger model (4.50M params, 2.18 GFlops), the total NN layers of the smaller model (0.52M params, 0.34 GFlops) will be enlarged 3 times. As a result, we need more VRAM to save intermediate hidden states for back-propagation. I think this is the reason why a larger model model can be trained but a smaller model can't.

from funcodec.

If only the smaller model matches the task on your hand, I recommend you use more GPUs or reduce the batch size (from 32 to 16) or reduce the speech_max_length (from 40800 to 20400).

from funcodec.

Thanks for your insight! I reduced the batch size to 16 for now, which is sufficient for my current use case.

from funcodec.