The test report compare to alpaca-lora. about mlora HOT 4 CLOSED

how about the gpu memory usage comparison?

from mlora.

how about the gpu utils information?

from mlora.

Test report about: alpaca_data_en_52k dataset on vicuna-7b-v1.1 (GPU: A100)

Method1: Using the same configuration file and data, fine-tune two datasets simultaneously.
Method2: Using the same configuration file and data, fine-tune only one dataset.

Method1: time cost: 12h45min, gpu memory cost: 23.69GB
Method2: time cost: 6h10min, gpu memory cost: 17.54GB

from mlora.

Test report about: alpaca_data_en_52k dataset on vicuna-7b-v1.1 (GPU: A100)

Method1: Using the same configuration file and data, fine-tune two datasets simultaneously. Method2: Using the same configuration file and data, fine-tune only one dataset.

Method1: time cost: 12h45min, gpu memory cost: 23.69GB Method2: time cost: 6h10min, gpu memory cost: 17.54GB

Does alpaca-lora and multi-lora-fine-tune set the group_by_length = true? Due to the random group of the dataset, the training time may be different, even in alpaca-lora, the training time varies greatly.

from mlora.