ofa-sys / diverseevol Goto Github PK

/DiverseEvol/configs/config_example_kc_dolly.yml
的设置中model_name_or_path: decapoda-research/llama-7b-hf 如果换成其他模型的路径会报各种各样的错
train是否只支持llama模型的训练？

Hello, thanks for your work~
I have some confusions about the experiments.

1. Why you train three epochs?(maybe this is unfair)
2. you set n_query = 100, and you report just 15 steps, so it just use 1500 samples in total, why don't you report the subsequent experimental results？

For my experiment, I use same model and dataset llama2-7b,dolly-15k, and I set n_query=500, train_epoch=1.
I test three rounds(rd-3,rd-15,rd-20), but the result is not good as report in your paper.
rd=3:

RS: 0.6028708133971292
Win: 0.0125
Tie: 0.0125
Lose: 0.975

rd=15:

RS: 0.6
Win: 0.0125
Tie: 0.0
Lose: 0.9875

rd=20:

RS: 0.7141887304820095
Win: 0.025
Tie: 0.0
Lose: 0.975

Could u give me an explanation or some guidance?
any reply will be appreciated