Giter Club home page Giter Club logo

llm_tkgc's Introduction

Knowledge-injection Assists LLM Instruction-tuning For Temporal Knowledge Graph Completion

Description

Dirs

./
	--logs 日志文件夹
	--models 基座模型存放
	--outputs 微调后模型存储
	--pretrained_emb 预训练图向量存储
	--prompts 历史挖掘等数据准备的存放,用于放入prompt template
	--templates 接收query和response生成prompt

Files

./
	--main.py 主文件,图向量pretraining和模型fine-tune
	--inference.py 推理
	--pretrain_nn.py 类GCN与KGE模型,用于完成静态拼接大图的向量训练
	--prompt.py prompter, 对应设计在template
	--model.py KGE向量注入模型
	--utils.py 一些计算和tokenize相关函数

Parameters

Fine-tune

--n-global-epoch 全局图训练轮数
--use-gnn 是否使用类GCN辅助KGE
--hidden-size 图embedding的hidden size
--do-pretrain 是否做向量预训练
--do-finetune
--batch-size fine-tune的batch_size
--sm-batch-size gradient accumulation的small batch size
--lora-rank
--lora-dropout
--n-ft-epoch 微调轮数
--base-model-path 基座模型路径
--base-model 基座模型名
--run-name wandb logging名
--output-dir 输出模型保存路径

Inference

--half kge半精
--output-dir 推理结果存放路径
——lora-weights-path ft的output-dir

Run

Fine-tune

nohup bash run_llama.sh > ./logs/llama2_7b_prefixadded_float16_adamwhf.log &

Inference

nohup bash inference.sh > ./logs/llama2_7b_prefixadded_float16_adamwhf_infer.log &

llm_tkgc's People

Contributors

lrlbbzl avatar

Stargazers

Tianle Gu avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.