Giter Club home page Giter Club logo

llm_finetuning's Introduction

update information

    2024-04-23 support qwen2
    2024-04-22 简化配置
    2023-11-27 yi modle_type change to llama
    2023-11-15 support load custom model , only modify config/constant_map.py
    2023-10-09 support accelerator trainer
    2023-10-07 support colossalai trainer
    2023-09-26 support transformers trainer
    2023-08-16 推理可选使用 Rope NtkScale , 不训练扩展推理长度
    2023-08-02 增加 muti lora infer 例子, 手动升级 aigc_zoo , pip install -U git+https://github.com/ssbuild/deep_training.zoo.git --force-reinstall --no-deps
    2023-06-13 fix llama resize_token_embeddings
    2023-06-01 support deepspeed training for lora adalora prompt,0.1.9 和 0.1.10合并
    2023-05-27 add qlora transformers>=4.30
    2023-05-24 fix p-tuning-v2 load weight bugs
    2023-05-12 fix lora int8 多卡训练 , ppo training move to https://github.com/ssbuild/rlhf_llm
    2023-05-02 增加p-tuning-v2
    2023-04-28 deep_training 0.1.3 pytorch-lightning 改名 ligntning ,旧版本 deep_training <= 0.1.2
    2023-04-23 增加lora merge权重(修改infer_lora_finetuning.py enable_merge_weight 选项)
    2023-04-11 升级 lora , 增加adalora

install

  • pip install -U -r requirements.txt
  • 如果无法安装, 可以切换官方源 pip install -i https://pypi.org/simple -U -r requirements.txt

# flash-attention对显卡算例要求算力7.5 以上 , 下面可选安装 ,如果卡不支持可以不安装。
git clone -b https://github.com/Dao-AILab/flash-attention
cd flash-attention && pip install .
pip install csrc/layer_norm
pip install csrc/rotary

weigtht select one is suitable for you

支持且不限于以下权重

data sample

单条数据示例

role one of user system function assistant
q question optional
a answer   must

{"id": 1, "paragraph": [{"q": "从南京到上海的路线", "a": ["你好,南京到上海的路线如下:", "1. 南京到上海,可以乘坐南京地铁1号线,在南京站乘坐轨道交通1号线。", "2. 南京到浦东机场,可以搭乘上海地铁1号,在陆家嘴站乘坐地铁1线,在浦东国际机场站乘坐机场快线,前往上海浦东国际机场。", "3. 上海到南京,可以换乘上海地铁2号线,从南京站换乘地铁2线,再从南京南站换乘地铁1路,然后到达上海站"]}]}
{"id": 2, "paragraph": [{"q": "写一个诗歌,关于冬天", "a": ["冬夜寂静冷,", "云在天边飘,", "冰封白雪上, ", "寒冷像一场雪。", " ", "雪花融化成冰,", "像那雪花飘洒,", "在寒冷的冬天,", "感受春天的喜悦。", " 冬日里,", "风雪渐消,", "一片寂静,", "把快乐和温暖带回家。"]}]}
{"id": 3, "paragraph": [{"q": "晚上睡不着应该怎么办", "a": ["晚上睡不着可能会让你感到焦虑或不舒服,但以下是一些可以帮助你入睡的方法:", "", "1. 制定规律的睡眠时间表:保持规律的睡眠时间表可以帮助你建立健康的睡眠习惯,使你更容易入睡。尽量在每天的相同时间上床,并在同一时间起床。", "2. 创造一个舒适的睡眠环境:确保睡眠环境舒适,安静,黑暗且温度适宜。可以使用舒适的床上用品,并保持房间通风。", "3. 放松身心:在睡前做些放松的活动,例如泡个热水澡,听些轻柔的音乐,阅读一些有趣的书籍等,有助于缓解紧张和焦虑,使你更容易入睡。", "4. 避免饮用含有咖啡因的饮料:咖啡因是一种刺激性物质,会影响你的睡眠质量。尽量避免在睡前饮用含有咖啡因的饮料,例如咖啡,茶和可乐。", "5. 避免在床上做与睡眠无关的事情:在床上做些与睡眠无关的事情,例如看电影,玩游戏或工作等,可能会干扰你的睡眠。", "6. 尝试呼吸技巧:深呼吸是一种放松技巧,可以帮助你缓解紧张和焦虑,使你更容易入睡。试着慢慢吸气,保持几秒钟,然后缓慢呼气。", "", "如果这些方法无法帮助你入睡,你可以考虑咨询医生或睡眠专家,寻求进一步的建议。"]}]}

或者

{"id": 1, "conversations": [{"from": "user", "value": "从南京到上海的路线"}, {"from": "assistant", "value": ["你好,南京到上海的路线如下:", "1. 南京到上海,可以乘坐南京地铁1号线,在南京站乘坐轨道交通1号线。", "2. 南京到浦东机场,可以搭乘上海地铁1号,在陆家嘴站乘坐地铁1线,在浦东国际机场站乘坐机场快线,前往上海浦东国际机场。", "3. 上海到南京,可以换乘上海地铁2号线,从南京站换乘地铁2线,再从南京南站换乘地铁1路,然后到达上海站"]}]}
{"id": 2, "conversations": [{"from": "user", "value": "写一个诗歌,关于冬天"}, {"from": "assistant", "value": ["冬夜寂静冷,", "云在天边飘,", "冰封白雪上, ", "寒冷像一场雪。", " ", "雪花融化成冰,", "像那雪花飘洒,", "在寒冷的冬天,", "感受春天的喜悦。", " 冬日里,", "风雪渐消,", "一片寂静,", "把快乐和温暖带回家。"]}]}
{"id": 3, "conversations": [{"from": "user", "value": "晚上睡不着应该怎么办"}, {"from": "assistant", "value": ["晚上睡不着可能会让你感到焦虑或不舒服,但以下是一些可以帮助你入睡的方法:", "", "1. 制定规律的睡眠时间表:保持规律的睡眠时间表可以帮助你建立健康的睡眠习惯,使你更容易入睡。尽量在每天的相同时间上床,并在同一时间起床。", "2. 创造一个舒适的睡眠环境:确保睡眠环境舒适,安静,黑暗且温度适宜。可以使用舒适的床上用品,并保持房间通风。", "3. 放松身心:在睡前做些放松的活动,例如泡个热水澡,听些轻柔的音乐,阅读一些有趣的书籍等,有助于缓解紧张和焦虑,使你更容易入睡。", "4. 避免饮用含有咖啡因的饮料:咖啡因是一种刺激性物质,会影响你的睡眠质量。尽量避免在睡前饮用含有咖啡因的饮料,例如咖啡,茶和可乐。", "5. 避免在床上做与睡眠无关的事情:在床上做些与睡眠无关的事情,例如看电影,玩游戏或工作等,可能会干扰你的睡眠。", "6. 尝试呼吸技巧:深呼吸是一种放松技巧,可以帮助你缓解紧张和焦虑,使你更容易入睡。试着慢慢吸气,保持几秒钟,然后缓慢呼气。", "", "如果这些方法无法帮助你入睡,你可以考虑咨询医生或睡眠专家,寻求进一步的建议。"]}]}

infer

# infer_finetuning.py 推理微调模型
# infer_lora_finetuning.py 推理微调模型
# infer_ptuning.py 推理p-tuning-v2微调模型
 python infer_finetuning.py

training

# 制作数据
cd scripts
bash train_full.sh -m dataset 
or
bash train_lora.sh -m dataset 
or
bash train_ptv2.sh -m dataset 

注: num_process_worker 为多进程制作数据 , 如果数据量较大 , 适当调大至cpu数量
dataHelper.make_dataset_with_args(data_args.train_file,mixed_data=False, shuffle=True,mode='train',num_process_worker=0)

# 全参数训练 
    bash train_full.sh -m train
    
# lora adalora ia3 
    bash train_lora.sh -m train
    
# ptv2
    bash train_ptv2.sh -m train

训练参数

训练参数

友情链接

纯粹而干净的代码

Star History

Star History Chart

llm_finetuning's People

Contributors

ssbuild avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

llm_finetuning's Issues

Error(s) in loading state_dict for MyTransformer:

python infer_finetuning.py
raise error

model_args, data_args  = parser.parse_dict(train_info_args, allow_extra_keys=True)

INFO:torch.distributed.nn.jit.instantiator:Created a temporary directory at /tmp/tmptr0d4z0k
INFO:torch.distributed.nn.jit.instantiator:Writing /tmp/tmptr0d4z0k/_remote_module_non_scriptable.py
Traceback (most recent call last):
File "/data/yrh/llm_finetuning_tizh/infer_finetuning.py", line 17, in
model_args, data_args = parser.parse_dict(train_info_args, allow_extra_keys=True)
^^^^^^^^^^^^^^^^^^^^^
ValueError: not enough values to unpack (expected 2, got 1)

modification ->

(model_args,)  = parser.parse_dict(train_info_args, allow_extra_keys=True)

raise error
Error(s) in loading state_dict for MyTransformer:
Missing key(s) in state_dict: "transformer_base.transformer.transformer.word_embeddings.weight",

Is it because I didn't do the weight conversion?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.