yihedeng9 / stic Goto Github PK

Describe the issue

Hi there. A nice work! When I tried to reproduce the result of STIC, I did not see the improvement of STIC-stage1-preference optimization. The training setting is the same as yours. I tried two different versions of LLAVA 1.6 (vicuna-7b and mistral-7b) on the Science QA test dataset. Here I report the results:

llava-mistral-7b on Science-QA test without STIC-stage1

llava-mistral-7b on Science-QA test with STIC-stage1 (use your provided weight of lora)

Here somehow, we saw the results but it is not consistent with the paper (approximately 60).

And I tried STIC-stage1 in the llava-vicuna-7b. Here I saw no improvement. we did not change the trainingdate and setting.
llava-vicuna-7b (original)

llava-vicuna-7b after STIC-stage1 (trained on 4 48G L20 in our envs)

Here I also share the training loss log and lora setting here. They look normal

How should I do to get improvement in preference optimization? It really helps me. Thank you