Chat model template? about llm_finetuning HOT 8 CLOSED

@LoopControl Thanks for the reference. The support for sequence of chats will be updated soon.

from llm_finetuning.

@LoopControl Update for conversation-based training data has been added in 7cc1995

Please try with finetune.py {params} --prompt_template_name "sharegpt"
I have tested with this data https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/blob/main/ShareGPT_V3_unfiltered_cleaned_split.json

from llm_finetuning.

@LoopControl Update for conversation-based training data has been added in 7cc1995

Please try with finetune.py {params} --prompt_template_name "sharegpt" I have tested with this data https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/blob/main/ShareGPT_V3_unfiltered_cleaned_split.json

I finetuned a chat model with qlora and it worked well with this code, thanks @taprosoft !

Some notes:

1. I couldn't get the training data to load when val_set_size was set to 0.0 - it kept erroring out saying something to the effect of key "train" not found. I'm guessing because it was expecting the json file to already be split into "train" and "test" datasets? (Setting val_set_size to 0.05 made it work however)
2. I modified the finetune.py directly to set the lora_r and lora_alpha values. Would be nice to have an option to do this via a commandline parameter instead (unless I'm missing one that already exists).
3. I had to rewrite my JSON training data to use "USER" and "ASSISTANT" for the from field of the sharegpt format so I could match the Vicuna-ish prompt setup. The sharegpt template seems to insert the "from = 'human'/'gpt'" values directly in the training input otherwise.

The command I used to run this on a 24GB P40:

python finetune.py \
    --base_model 'huggyllama/llama-7b' \
    --data_path 'training-data/train.json' \
    --output_dir 'output_lora' \
    --batch_size 4 \
    --micro_batch_size 1 \
    --train_on_inputs True \
    --num_epochs 2 \
    --learning_rate 2e-4 \
    --cutoff_len 256 \
    --group_by_length \
    --val_set_size 0.05 \
    --eval_steps 0 \
    --logging_steps 5 \
    --save_steps 100 \
    --gradient_checkpointing 1 \
    --mode 4 \
    --prompt_template_name "sharegpt"

from llm_finetuning.

@LoopControl Yes the template currently don't support it. I suppose you can re-convert the data or make some small modification to the prompt code in:

It can offer greater control to your need.

from llm_finetuning.

Looking forward to it thanks!

from llm_finetuning.

@LoopControl Sound great!

1. About the train key issue, you are right that it expects the dataset to be splitted already in train and test. Normally for Alpaca instruction dataset it is designed this way.
2. lora_r and lora_alpha are also exposed in the command line. You can use finetune.py --help to see the comprehensive list.
3. You can modify templates/sharegpt.json which specify how to construct the chat prompt (the part "chat":):

{
    "description": "A chat prompt template",
    "input": "conversations",
    "user": "from",
    "text": "value",
    "prompt": "A conversation between a helpful AI model and the user",
    "chat": "{user}: {text}\n"
}

from llm_finetuning.

Could you expand on how to modify the prompt? I'm having the same issue using human gpt

from llm_finetuning.

@taprosoft Thanks for the info on 1 + 2.

For 3: When the "from" field is "human", Vicuna format is "USER" and when "from" = "gpt" it's "ASSISTANT".

Unless I'm missing something, I don't think the template supports using that if/else format (assuming we're supposed to modify the "chat" to change based on the "from" value)?

from llm_finetuning.