Hi,
I've gone through your preprint at https://arxiv.org/abs/2404.17835, nice work!
Looking at your code here, it is not clear to me what labels you assign at the prepended isntructions at training-time. All I see is the token-based classification finetuning pipeline as provided by https://github.com/4AI/LS-LLaMA, where you prepend the DBR tokens witht the corresponding labels. Could you provide any detail?
Thanks for sharing the repo, very interesting