训练数据中对引号的使用似乎有一些错乱？诸如

1. 整个字段被中文或英文引号包围
2. 字段中的英文引号部分没有转译成中文引号
3. 倘若字段被双引号包围，则字段内的双引号被改写成了单引号

这些现象会对模型的效果产生影响么？

pip install transformers 后，from transformers import LLaMATokenizer, LLaMAForCausalLM 报错，为什么，或者哪里错了？

<style type="text/css"></style>

Unrecognized configuration class <class 'transformers_modules.local.configuration_chatglm.ChatGLMConfig'> to build an AutoTokenizer.

按照https://github.com/LC1332/Chinese-alpaca-lora/blob/main/notebook/ChatLuotuo.ipynb 顺序执行
执行“pip install -r requirements.txt”时报错：
Error message:
ERROR: Could not find a version that satisfies the requirement gradio==3.20.0 (from versions: 0.1.0, 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5, 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.2.0, 0.2.1, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.4.0, 0.4.1, 0.4.2, 0.4.4, 0.5.0, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.7.4, 0.7.5, 0.7.6, 0.7.7, 0.7.8, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.9.2, 0.9.3, 0.9.4, 0.9.5, 0.9.6, 0.9.7, 0.9.8, 0.9.9.2, 0.9.9.3, 0.9.9.5, 0.9.9.6, 0.9.9.7, 0.9.9.8, 0.9.9.9, 0.9.9.9.2, 1.0.0a1, 1.0.0a3, 1.0.0a4, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.0.6, 1.0.7, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 1.1.8, 1.1.8.1, 1.1.9, 1.2.2, 1.2.3, 1.3.0, 1.3.1, 1.3.2, 1.4.0, 1.4.2, 1.4.3, 1.4.4, 1.5.0, 1.5.1, 1.5.3, 1.5.4, 1.6.0, 1.6.1, 1.6.2, 1.6.3, 1.6.4, 1.7.0, 1.7.1, 1.7.2, 1.7.3, 1.7.4, 1.7.5, 1.7.6, 1.7.7, 2.0.0, 2.0.1, 2.0.2, 2.0.4, 2.0.5, 2.0.6, 2.0.7, 2.0.8, 2.0.9, 2.0.10, 2.1.0, 2.1.1, 2.1.2, 2.1.4, 2.1.6, 2.1.7, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.2.4, 2.2.5, 2.2.6, 2.2.7, 2.2.8, 2.2.9a0, 2.2.9a2, 2.2.10, 2.2.11, 2.2.12, 2.2.13, 2.2.14, 2.2.15, 2.3.0a0, 2.3.0b99, 2.3.0b101, 2.3.0b102, 2.3.0, 2.3.3, 2.3.4, 2.3.5b0, 2.3.5, 2.3.6, 2.3.7b0, 2.3.7b1, 2.3.7b2, 2.3.7, 2.3.8b0, 2.3.9, 2.4.0a0, 2.4.0, 2.4.1, 2.4.2, 2.4.4, 2.4.5, 2.4.6, 2.4.7b0, 2.4.7b2, 2.4.7b3, 2.4.7b4, 2.4.7b5, 2.4.7b6, 2.4.7b7, 2.4.7b8, 2.4.7b9, 2.5.0, 2.5.1, 2.5.2, 2.5.3, 2.5.8a0, 2.6.0, 2.6.1a0, 2.6.1b0, 2.6.1b3, 2.6.1, 2.6.2, 2.6.3, 2.6.4b0, 2.6.4b2, 2.6.4b3, 2.6.4, 2.7.0a101, 2.7.0a102, 2.7.0b70, 2.7.0, 2.7.5, 2.7.5.1, 2.7.5.2b0, 2.7.5.2, 2.8.0a100, 2.8.0b0, 2.8.0b2, 2.8.0b3, 2.8.0b4, 2.8.0b5, 2.8.0b6, 2.8.0b10, 2.8.0b12, 2.8.0b20, 2.8.0b22, 2.8.0, 2.8.1, 2.8.2, 2.8.3, 2.8.4, 2.8.5, 2.8.6, 2.8.7, 2.8.8, 2.8.9, 2.8.10, 2.8.11, 2.8.12, 2.8.13, 2.8.14, 2.9.0b0, 2.9.0b1, 2.9.0b2, 2.9.0b3, 2.9.0b5, 2.9.0b6, 2.9.0b7, 2.9.0b8, 2.9.0b9, 2.9.0b10, 2.9b11, 2.9b12, 2.9b13, 2.9b14, 2.9b15, 2.9b20, 2.9b21, 2.9b22, 2.9b23, 2.9b24, 2.9b25, 2.9b26, 2.9b27, 2.9b28, 2.9b30, 2.9b31, 2.9b32, 2.9b33, 2.9b40, 2.9b48, 2.9b50, 2.9.0, 2.9.0.1, 2.9.1, 2.9.2, 2.9.3, 2.9.4, 3.0b0, 3.0b1, 3.0b2, 3.0b5, 3.0b6, 3.0b8, 3.0b9, 3.0b10, 3.0, 3.0.1, 3.0.2, 3.0.3, 3.0.4, 3.0.5, 3.0.6b1, 3.0.6b2, 3.0.6b3, 3.0.6, 3.0.7, 3.0.8b1, 3.0.8, 3.0.9b10, 3.0.9b11, 3.0.9b20, 3.0.9, 3.0.10, 3.0.11b1, 3.0.11, 3.0.12)
ERROR: No matching distribution found for gradio==3.20.0
尝试单独安装gradio，命令pip install gradio==3.20.0 -i https://mirrors.aliyun.com/pypi/simple
还尝试过以下镜像源：
https://pypi.tuna.tsinghua.edu.cn/simple/
https://pypi.org/simple
都是类似的报错，没有这个版本的安装包。

llama原生模型是没有经过中文预训练的，词表也基本没中文，直接fine-tune是不是效果没法保障。你们有试过扩展词表吗。谢谢回答。

为啥连个issues都没有？？

https://github.com/microsoft/DeepSpeed

能出个部署教程吗

我用默认的colab跑了一下骆驼说，似乎效果不佳，是需要升级到colab pro么？还是需要做别的事情？

生成链接

### Instruction:11份23加起来等于多少

### Response:11加23即24。

### Instruction:11乘以23等于多少

### Response:11乘23=243。

### Instruction:不对，是253

### Response:我错！

### Instruction:11乘以23等于多少

### Response:11乘23=243。

https://guanaco-model.github.io/
https://huggingface.co/datasets/JosephusCheung/GuanacoDataset

请教，训练目标是100此迭代，但是50次中断了，如何能在50次继续训练？t

python app.py --base_url llama7b --ft_ckpt_url luotuolora7b03 --port 6006 --share yes

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

CUDA SETUP: Required library version not found: libsbitsandbytes_cpu.so. Maybe you need to compile it from source?
CUDA SETUP: Defaulting to libbitsandbytes_cpu.so...
argument of type 'WindowsPath' is not iterable
D:\ProgramData\Anaconda3\envs\chinese_luotuo3\lib\site-packages\bitsandbytes\cextension.py:31: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.
warn("The installed version of bitsandbytes was compiled without GPU support. "
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization.
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'.
The class this function is called from is 'LlamaTokenizer'.
Overriding torch_dtype=None with torch_dtype=torch.float16 due to requirements of bitsandbytes to enable model loading in mixed int8. Either pass torch_dtype=torch.float16 or don't pass this argument at all to remove this warning.
Traceback (most recent call last):
File "D:\chinese_alpaca_lora\Chinese-alpaca-lora\notebook\Alpaca-LoRA-Serve\app.py", line 178, in
run(args)
File "D:\chinese_alpaca_lora\Chinese-alpaca-lora\notebook\Alpaca-LoRA-Serve\app.py", line 79, in run
model, tokenizer = load_model(
File "D:\chinese_alpaca_lora\Chinese-alpaca-lora\notebook\Alpaca-LoRA-Serve\model.py", line 12, in load_model
model = LlamaForCausalLM.from_pretrained(
File "D:\ProgramData\Anaconda3\envs\chinese_luotuo3\lib\site-packages\transformers\modeling_utils.py", line 2588, in from_pretrained
raise ValueError(
ValueError:
Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to fit
the quantized model. If you want to dispatch the model on the CPU or the disk while keeping
these modules in 32-bit, you need to set load_in_8bit_fp32_cpu_offload=True and pass a custom
device_map to from_pretrained. Check
https://huggingface.co/docs/transformers/main/en/main_classes/quantization#offload-between-cpu-and-gpu
for more details.

win10 + 3080ti
I type in command and get these...it seems my GPU RAM (12 G) is not enough for it?

感谢各位大佬对中文大语言模型社区的贡献！！
我在huggingface中找到一份数据集silk-road/alpaca-data-gpt4-chinese（https://huggingface.co/datasets/silk-road/alpaca-data-gpt4-chinese），它的翻译质量和格式处理很棒，我想要在我的研究中使用它，但是遗憾的是我没有找到相关的详细说明。
我想询问这份数据集的中文部分的制作方式，它看上去是由英文部分翻译而来的，如果是，我想请问翻译使用的模型是什么呢？是GPT3.5或者GPT-4吗？如果有更详细的说明，十分期待您可以帮忙指明它的位置！！

[doge]

除了readme 上的一些examples 对比，有没有在各方面能力上的对比？

Hey, I saw your license of luotuo-lora-7b-0.1 is apache2. and to my best knowledge, your model was finetuned on top of llama7b which is non-commercial, also your code base of alpaca-lora is also only for research. Are you sure you can make the license Apache2? Let me know, cause I also want to finetune llama for commercial purpose.

win10 + anaconda prompt
I use pip to install the libs,I got this...

C:\Users\j>pip install -q datasets loralib sentencepiece
ERROR: Error [WinError 225] 无法成功完成操作，因为文件包含病毒或潜在的垃圾软件。 while executing command python setup.py egg_info
ERROR: Could not install packages due to an OSError: [WinError 225] 无法成功完成操作，因为文件包含病毒或潜在的垃圾软件。

ValueError: Unrecognized model in [./model/adapter_config.json.] (Chinese-alpaca-lora/model/adapter_config.json.) Should have a model_type key in its config.json, or contain one of the following strings in its name:..

Model type is not metioned in adapter_config.json.
Should I fill model type 'llama'? But llama is not transformers structures

best regards.

运行下面单元格时：
torch.set_default_tensor_type(torch.cuda.HalfTensor)

tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True)

model = AutoModel.from_pretrained(
"THUDM/chatglm-6b",
trust_remote_code=True,
device_map=DeviceMap("ChatGLM").get()
)
出现报错：
AttributeError Traceback (most recent call last)
in <cell line: 3>()
1 torch.set_default_tensor_type(torch.cuda.HalfTensor)
2
----> 3 tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True)
4
5 model = AutoModel.from_pretrained(

7 frames
~/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b/8b7d33596d18c5e83e2da052d05ca4db02e60620/tokenization_chatglm.py in vocab_size(self)
242 def vocab_size(self):
243 """ Returns vocab size """
--> 244 return self.sp_tokenizer.num_tokens
245
246 def get_vocab(self):

AttributeError: 'ChatGLMTokenizer' object has no attribute 'sp_tokenizer'

Put your rich text here which you want to summarize it: 佛挡杀佛水电费
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/local/lib/python3.9/dist-packages/IPython/core/interactiveshell.py:3553 in run_code │
│ │
│ 3550 │ │ │ │ elif async_ : │
│ 3551 │ │ │ │ │ await eval(code_obj, self.user_global_ns, self.user_ns) │
│ 3552 │ │ │ │ else: │
│ ❱ 3553 │ │ │ │ │ exec(code_obj, self.user_global_ns, self.user_ns) │
│ 3554 │ │ │ finally: │
│ 3555 │ │ │ │ # Reset our crash handler in place │
│ 3556 │ │ │ │ sys.excepthook = old_excepthook │
│ in <cell line: 1>:1 │
│ in evaluate:10 │
│ │
│ /usr/local/lib/python3.9/dist-packages/peft/peft_model.py:716 in generate │
│ │
│ 713 │ │ self.base_model.prepare_inputs_for_generation = self.prepare_inputs_for_generati │
│ 714 │ │ try: │
│ 715 │ │ │ if not isinstance(peft_config, PromptLearningConfig): │
│ ❱ 716 │ │ │ │ outputs = self.base_model.generate(**kwargs) │
│ 717 │ │ │ else: │
│ 718 │ │ │ │ if "input_ids" not in kwargs: │
│ 719 │ │ │ │ │ raise ValueError("input_ids must be provided for Peft model generati │
│ │
│ /usr/local/lib/python3.9/dist-packages/peft/peft_model.py:716 in generate │
│ │
│ 713 │ │ self.base_model.prepare_inputs_for_generation = self.prepare_inputs_for_generati │
│ 714 │ │ try: │
│ 715 │ │ │ if not isinstance(peft_config, PromptLearningConfig): │
│ ❱ 716 │ │ │ │ outputs = self.base_model.generate(**kwargs) │
│ 717 │ │ │ else: │
│ 718 │ │ │ │ if "input_ids" not in kwargs: │
│ 719 │ │ │ │ │ raise ValueError("input_ids must be provided for Peft model generati │
│ │
│ /usr/local/lib/python3.9/dist-packages/torch/utils/_contextlib.py:115 in decorate_context │
│ │
│ 112 │ @functools.wraps(func) │
│ 113 │ def decorate_context(*args, **kwargs): │
│ 114 │ │ with ctx_factory(): │
│ ❱ 115 │ │ │ return func(*args, **kwargs) │
│ 116 │ │
│ 117 │ return decorate_context │
│ 118 │
│ │
│ /usr/local/lib/python3.9/dist-packages/transformers/generation/utils.py:1437 in generate │
│ │
│ 1434 │ │ │ │ ) │
│ 1435 │ │ │ │
│ 1436 │ │ │ # 11. run greedy search │
│ ❱ 1437 │ │ │ return self.greedy_search( │
│ 1438 │ │ │ │ input_ids, │
│ 1439 │ │ │ │ logits_processor=logits_processor, │
│ 1440 │ │ │ │ stopping_criteria=stopping_criteria, │
│ │
│ /usr/local/lib/python3.9/dist-packages/transformers/generation/utils.py:2248 in greedy_search │
│ │
│ 2245 │ │ │ model_inputs = self.prepare_inputs_for_generation(input_ids, **model_kwargs) │
│ 2246 │ │ │ │
│ 2247 │ │ │ # forward pass to get next token │
│ ❱ 2248 │ │ │ outputs = self( │
│ 2249 │ │ │ │ **model_inputs, │
│ 2250 │ │ │ │ return_dict=True, │
│ 2251 │ │ │ │ output_attentions=output_attentions, │
│ │
│ /usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py:1501 in _call_impl │
│ │
│ 1498 │ │ if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks │
│ 1499 │ │ │ │ or _global_backward_pre_hooks or _global_backward_hooks │
│ 1500 │ │ │ │ or _global_forward_hooks or _global_forward_pre_hooks): │
│ ❱ 1501 │ │ │ return forward_call(*args, **kwargs) │
│ 1502 │ │ # Do not call functions when jit is used │
│ 1503 │ │ full_backward_hooks, non_full_backward_hooks = [], [] │
│ 1504 │ │ backward_pre_hooks = [] │
│ │
│ /usr/local/lib/python3.9/dist-packages/accelerate/hooks.py:166 in new_forward │
│ │
│ 163 │ │ │ │ output = old_forward(*args, **kwargs) │
│ 164 │ │ else: │
│ 165 │ │ │ output = old_forward(*args, **kwargs) │
│ ❱ 166 │ │ return module._hf_hook.post_forward(module, output) │
│ 167 │ │
│ 168 │ module.forward = new_forward │
│ 169 │ return module │
│ │
│ /usr/local/lib/python3.9/dist-packages/accelerate/hooks.py:292 in post_forward │
│ │
│ 289 │ │ │ │ set_module_tensor_to_device(module, name, "meta") │
│ 290 │ │ │
│ 291 │ │ if self.io_same_device and self.input_device is not None: │
│ ❱ 292 │ │ │ output = send_to_device(output, self.input_device) │
│ 293 │ │ │
│ 294 │ │ return output │
│ 295 │
│ │
│ /usr/local/lib/python3.9/dist-packages/accelerate/utils/operations.py:133 in send_to_device │
│ │
│ 130 │ def _has_to_method(t): │
│ 131 │ │ return hasattr(t, "to") │
│ 132 │ │
│ ❱ 133 │ return recursively_apply(_send_to_device, tensor, device, non_blocking, test_type=_h │
│ 134 │
│ 135 │
│ 136 def get_data_structure(data): │
│ │
│ /usr/local/lib/python3.9/dist-packages/accelerate/utils/operations.py:92 in recursively_apply │
│ │
│ 89 │ │ │ ), │
│ 90 │ │ ) │
│ 91 │ elif isinstance(data, Mapping): │
│ ❱ 92 │ │ return type(data)( │
│ 93 │ │ │ { │
│ 94 │ │ │ │ k: recursively_apply( │
│ 95 │ │ │ │ │ func, v, *args, test_type=test_type, error_on_other_type=error_on_ot │
│ in init:8 │
│ │
│ /usr/local/lib/python3.9/dist-packages/transformers/utils/generic.py:246 in post_init │
│ │
│ 243 │ │ first_field = getattr(self, class_fields[0].name) │
│ 244 │ │ other_fields_are_none = all(getattr(self, field.name) is None for field in class │
│ 245 │ │ │
│ ❱ 246 │ │ if other_fields_are_none and not is_tensor(first_field): │
│ 247 │ │ │ if isinstance(first_field, dict): │
│ 248 │ │ │ │ iterator = first_field.items() │
│ 249 │ │ │ │ first_field_iterator = True │
│ │
│ /usr/local/lib/python3.9/dist-packages/transformers/utils/generic.py:86 in is_tensor │
│ │
│ 83 │ │ if isinstance(x, torch.Tensor): │
│ 84 │ │ │ return True │
│ 85 │ if is_tf_available(): │
│ ❱ 86 │ │ import tensorflow as tf │
│ 87 │ │ │
│ 88 │ │ if isinstance(x, tf.Tensor): │
│ 89 │ │ │ return True │
│ │
│ /usr/local/lib/python3.9/dist-packages/tensorflow/init.py:37 in │
│ │
│ 34 import sys as _sys │
│ 35 import typing as _typing │
│ 36 │
│ ❱ 37 from tensorflow.python.tools import module_util as _module_util │
│ 38 from tensorflow.python.util.lazy_loader import LazyLoader as _LazyLoader │
│ 39 │
│ 40 # Make sure code inside the TensorFlow codebase can use tf2.enabled() at import. │
│ │
│ /usr/local/lib/python3.9/dist-packages/tensorflow/python/init.py:37 in │
│ │
│ 34 # pylint: disable=wildcard-import,g-bad-import-order,g-import-not-at-top │
│ 35 │
│ 36 from tensorflow.python import pywrap_tensorflow as _pywrap_tensorflow │
│ ❱ 37 from tensorflow.python.eager import context │
│ 38 │
│ 39 # pylint: enable=wildcard-import │
│ 40 │
│ │
│ /usr/local/lib/python3.9/dist-packages/tensorflow/python/eager/context.py:28 in │
│ │
│ 25 from absl import logging │
│ 26 import numpy as np │
│ 27 │
│ ❱ 28 from tensorflow.core.framework import function_pb2 │
│ 29 from tensorflow.core.protobuf import config_pb2 │
│ 30 from tensorflow.core.protobuf import rewriter_config_pb2 │
│ 31 from tensorflow.python import pywrap_tfe │
│ │
│ /usr/local/lib/python3.9/dist-packages/tensorflow/core/framework/function_pb2.py:5 in │
│ │
│ 2 # Generated by the protocol buffer compiler. DO NOT EDIT! │
│ 3 # source: tensorflow/core/framework/function.proto │
│ 4 """Generated protocol buffer code.""" │
│ ❱ 5 from google.protobuf.internal import builder as _builder │
│ 6 from google.protobuf import descriptor as _descriptor │
│ 7 from google.protobuf import descriptor_pool as _descriptor_pool │
│ 8 from google.protobuf import symbol_database as _symbol_database │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ImportError: cannot import name 'builder' from 'google.protobuf.internal'
(/usr/local/lib/python3.9/dist-packages/google/protobuf/internal/init.py)

我使用win10系统 3090 24G显存运行报错

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

D:\anaconda3\envs\torch_test\lib\site-packages\bitsandbytes\cuda_setup\main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('D')}
warn(msg)
D:\anaconda3\envs\torch_test\lib\site-packages\bitsandbytes\cuda_setup\main.py:136: UserWarning: D:\anaconda3\envs\torch_test did not contain libcudart.so as expected! Searching further paths...
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
D:\anaconda3\envs\torch_test\lib\site-packages\bitsandbytes\cuda_setup\main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/usr/local/cuda/lib64')}
warn(msg)
CUDA SETUP: WARNING! libcuda.so not found! Do you have a CUDA driver installed? If you are on a cluster, make sure you are on a CUDA machine!
D:\anaconda3\envs\torch_test\lib\site-packages\bitsandbytes\cuda_setup\main.py:136: UserWarning: WARNING: No libcudart.so found! Install CUDA or the cudatoolkit package (anaconda)!
warn(msg)
D:\anaconda3\envs\torch_test\lib\site-packages\bitsandbytes\cuda_setup\main.py:136: UserWarning: WARNING: No GPU detected! Check your CUDA paths. Proceeding to load CPU-only library...
warn(msg)
CUDA SETUP: Loading binary D:\anaconda3\envs\torch_test\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.so...
argument of type 'WindowsPath' is not iterable
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
CUDA SETUP: WARNING! libcuda.so not found! Do you have a CUDA driver installed? If you are on a cluster, make sure you are on a CUDA machine!
CUDA SETUP: Loading binary D:\anaconda3\envs\torch_test\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.so...
argument of type 'WindowsPath' is not iterable
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
CUDA SETUP: WARNING! libcuda.so not found! Do you have a CUDA driver installed? If you are on a cluster, make sure you are on a CUDA machine!
CUDA SETUP: Loading binary D:\anaconda3\envs\torch_test\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.so...
argument of type 'WindowsPath' is not iterable
CUDA SETUP: Problem: The main issue seems to be that the main CUDA library was not detected.
CUDA SETUP: Solution 1): Your paths are probably not up-to-date. You can update them via: sudo ldconfig.
CUDA SETUP: Solution 2): If you do not have sudo rights, you can do the following:
CUDA SETUP: Solution 2a): Find the cuda library via: find / -name libcuda.so 2>/dev/null
CUDA SETUP: Solution 2b): Once the library is found add it to the LD_LIBRARY_PATH: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:FOUND_PATH_FROM_2a
CUDA SETUP: Solution 2c): For a permanent solution add the export from 2b into your .bashrc file, located at ~/.bashrc
Traceback (most recent call last):
File "D:\xiangmu\luotuo-silk-road-main\TuoLing\cc.py", line 17, in
from peft import get_peft_model, LoraConfig, TaskType
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 664, in _load_unlocked
File "", line 627, in load_backward_compatible
File "", line 259, in load_module
File "D:\anaconda3\envs\torch_test\lib\site-packages\peft-0.3.0.dev0-py3.9.egg\peft_init.py", line 22, in
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 664, in _load_unlocked
File "", line 627, in _load_backward_compatible
File "", line 259, in load_module
File "D:\anaconda3\envs\torch_test\lib\site-packages\peft-0.3.0.dev0-py3.9.egg\peft\mapping.py", line 16, in
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 664, in _load_unlocked
File "", line 627, in _load_backward_compatible
File "", line 259, in load_module
File "D:\anaconda3\envs\torch_test\lib\site-packages\peft-0.3.0.dev0-py3.9.egg\peft\peft_model.py", line 31, in
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 664, in _load_unlocked
File "", line 627, in load_backward_compatible
File "", line 259, in load_module
File "D:\anaconda3\envs\torch_test\lib\site-packages\peft-0.3.0.dev0-py3.9.egg\peft\tuners_init.py", line 20, in
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 664, in _load_unlocked
File "", line 627, in load_backward_compatible
File "", line 259, in load_module
File "D:\anaconda3\envs\torch_test\lib\site-packages\peft-0.3.0.dev0-py3.9.egg\peft\tuners\lora.py", line 36, in
File "D:\anaconda3\envs\torch_test\lib\site-packages\bitsandbytes_init.py", line 7, in
from .autograd.functions import (
File "D:\anaconda3\envs\torch_test\lib\site-packages\bitsandbytes\autograd_init.py", line 1, in
from ._functions import undo_layout, get_inverse_transform_indices
File "D:\anaconda3\envs\torch_test\lib\site-packages\bitsandbytes\autograd_functions.py", line 9, in
import bitsandbytes.functional as F
File "D:\anaconda3\envs\torch_test\lib\site-packages\bitsandbytes\functional.py", line 17, in
from .cextension import COMPILED_WITH_CUDA, lib
File "D:\anaconda3\envs\torch_test\lib\site-packages\bitsandbytes\cextension.py", line 22, in
raise RuntimeError('''
RuntimeError:
CUDA Setup failed despite GPU being available. Inspect the CUDA SETUP outputs above to fix your environment!
If you cannot find any issues and suspect a bug, please open an issue with detals about your environment:
https://github.com/TimDettmers/bitsandbytes/issues

我知道这是我环境问题 这是因为bitsandbytes不支持windows系统吗 libcudart.so我并没有这个文件 有办法提供一点安装思路吗 谢谢

2023-04-06 11:12:35.832309: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /usr/lib64-nvidia did not contain libcudart.so as expected! Searching further paths...
warn(msg)
/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/sys/fs/cgroup/memory.events /var/colab/cgroup/jupyter-children/memory.events')}
warn(msg)
/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('decapoda-research/llama-7b-hf')}
warn(msg)
/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('http'), PosixPath('8013'), PosixPath('//172.28.0.1')}
warn(msg)
/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('--listen_host=172.28.0.12 --target_host=172.28.0.12 --tunnel_background_save_url=https'), PosixPath('//colab.research.google.com/tun/m/cc48301118ce562b961b3c22d803539adc1e0c19/gpu-t4-s-20gy7jrxovdgv --tunnel_background_save_delay=10s --tunnel_periodic_background_save_frequency=30m0s --enable_output_coalescing=true --output_coalescing_required=true')}
warn(msg)
/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/env/python')}
warn(msg)
/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('//ipykernel.pylab.backend_inline'), PosixPath('module')}
warn(msg)
/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('qychen/luotuo-lora-7b-0.1')}
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 7.5
CUDA SETUP: Detected CUDA version 118
CUDA SETUP: Loading binary /usr/local/lib/python3.9/dist-packages/bitsandbytes/libbitsandbytes_cuda118.so...
usage: app.py
[-h]
[--base_url BASE_URL]
[--ft_ckpt_url FT_CKPT_URL]
[--port PORT]
[--batch_size BATCH_SIZE]
[--api_open]
[--share]
[--gen_config_path GEN_CONFIG_PATH]
[--gen_config_summarization_path GEN_CONFIG_SUMMARIZATION_PATH]
[--multi_gpu]
[--force_download_ckpt]
app.py: error: unrecognized arguments: yes

请问 如果我想在32g显存的服务器加载llama 30b，我能不能先把每个shard做了量化之后再载进来呀，因为直接载模型的话就out of memory了，或者其他有没有什么办法可以做到呀，谢谢

你好，我在训练过程中，模型无法学习到自定义数据，lora模型的训练也不收敛，loss一直1.0左右，请问这个遇到过吗？

不好意思，我记得在你们的项目看到有些研究论文的介绍，但是返回来找，怎么都找不到了，不知道是我记错了？还是这边有一些其他链接是包含相关的技术报告和研究论文。谢谢！

Hello，感谢您的工作，请问WizardLM的进化指令翻译质量如何，有经过过滤吗？另外回答是爬取的GPT4还是GPT3.5吗？谢谢回复～

请问用的是啥GPU，V100可以跑吗？

在隔壁看到了https://github.com/ymcui/Chinese-LLaMA-Alpaca 这样的项目
先在llama上使用中文数据集做了pretrain，然后再使用alpaca进行的微调
对比了下glm-6b-alpaca和llama-7b-zh-alpaca，感觉llama更擅长instruction任务一些

I am sorry if i have not noticed. I am just curious about it.
Are these models based on LLaMA, Alpaca or ChatGLM? The model definition and configuration of ChatGLM are in the ./PythonFiles folder of this repository.

您好，我们公司正在训练一个QA模型，请问怎么才能申请到贵公司翻译后的CoQA数据集呢

基于向量化和大型语言模型的知识库与交互系统

各位见笑。我就是一菜鸡，下文如果说得不对的地方请指正，莫要见怪。
就目前而言，各类开源大语言模型最大的使用方向就是知识库、问答系统等垂直领域。
目前的解决方案有二：

1、模型外挂知识库，比如【闻达】。

优点：技术比较简单，实用性比较强。
缺点：受模型token限制，自然语言文本信息承载量也比较低，能处理的信息有限，难以整合大量信息。

2、模型微调

优点：专业，准确，受限制小。
缺点：我看了charglm的lssues，貌似成功的是凤翎毛角。绝大部分都把模型搞爆了。要求太专业。

我有一个不成熟的想法，供大家探讨！
自然语言直接与模型对话，只适合人机交互。其实并不适合信息存储和大语言模型运算。
效率相对比较高的方案是知识图谱、或直接向量交互。但这种方案对于人类极不友好。
现在的可行的解决方案是使用Milvus作为知识库的检索引擎。

1、预处理阶段：

1.1 首先用text2vec或其他技术转换为向量，存入Milvus，作为知识库。

2、询问阶段：

2.1 预处理

   用户交互时，先用text2vec转换问题为向量，在 Milvus中查询，并将结果的文本内容转换为自然语言。

2.2 模型运算

   携带查询结果将和用户问题，覆盖特定引导词交给模型运算。

3、输出阶段

3.1 模型输出自然语言

如果是目前的知识系统。这里就结束了。如果要增强系统功能或者不是问题系统，系统要求长期记忆。还有接下来的步骤。

4、数据保存阶段

4.1 清洗数据，将没用的格式转换为需要的格式。

   每个模型都有自己的脾气，总爱带上自己的小脾气。如果系统也有要求，可能需要清洗没有用的特定格式。

4.2 存储数据

  使用先用text2vec转换问题为向量，存入Milvus，作为知识库或其他后续运算的长期记忆的一部分。

表面看上去没问题。其实问题很严重的。

目前，大语言模型最大的问题就是token长度限制造成的失忆问题。这使得大语言模型难以做大型任务（当然也可以通过反复提示解决，不过你不觉得累吗，也是遭罪）。AutoGPT在某中意义上最大的贡献就是解决了记忆问题。（当然，装上了手脚也很重要，不过不再本次的讨论范围内，所以用词偏颇了一点应该可以理解吧），然而，前边说过，自然语言对人类友好，但信息携带并不高。嗯，就是传说中的信息熵不高。虽然昨天才看到有大佬说AI的时代中，中文由于信息桑较高，所以占优，不过对于机器来说，其实还是没有任何优势。只能都只能是低劣。知识图谱，向量对于大语言模型来说才是最优解。
所以问题还是比较多的。
1、向量的频繁转换造成信息的丢失。当然了，方案中一直都在使用text2vec同一个模型做转换的话，问题也不大，效果如何取决于text2vec的能力。无非就是，准确性好坏的问题，但多少还是有的。而且完全可能是多余的。（没做测试，构建这类测试数据有些难，有些费功夫。）
2、如果是知识库问题还不大，输出知识一次性的。但是如果是增强系统，需要存储结果，多出来的那个过程，自然语言又转换为向量，使用时有转换回来，这一步损失就比较严重了。

我有个想法。如果模型支持向量的输入、输出、这一切的问题都不存在了（好不容易有一个可以任性用极限词的地方不罚款的地方，就让我放飞一下！）。用词夸张了，这种方案不能解决根本问题，但理论上可以在一定时间内大幅度提高模型能力和效率，并节约token空间。

1、预处理阶段：

【原流程】 1.1 _首先用text2vec或其他技术转换为向量，存入Milvus，作为知识库。

1.1【新】 变为将知识库的文本内容交给模型，让模型转化为向量，并存入Milvus，作为知识库。

2、询问阶段：

2.1 预处理

【原流程】用户交互时，先用text2vec转换问题为向量，在 Milvus中查询，并将结果的文本内容转换为自然语言。
【新】用户交互时，先让模型自己将问题转换为向量，并在 Milvus中查询，无需对结果做任何处理。

2.2 模型运算

  【原流程】 携带查询结果将和用户问题，覆盖特定引导词交给模型运算。
  【新】将结果的向量结果 与用户提问一起交给模型。无需特定引导词。

3、输出阶段

【原流程】3.1 模型输出自然语言

【新】 3.1 模型输出自然语言及向量。

###4、数据保存阶段
【原流程】 4.1 清洗数据，将没用的格式转换为需要的格式。
####【新】 4.1 清洗数据，将没用的格式转换为需要的格式。看实现的方案，有可能完全不需要。
每个模型都有自己的脾气，总爱带上自己的风格。如果系统也有要求，可能需要清洗没有用的特定格式。

4.2 存储数据

  【原流程】使用先用text2vec转换问题为向量，存入Milvus，作为知识库或其他后续运算的长期记忆的一部分。
    【新】无需转换，直接存入Milvus，作为知识库或其他后续运算的长期记忆的一部分。

总结：

理论上任何模型都可以改造。而且改动幅度小，只需要给任何模型增加向量输入输出接口功能。输入输出上搞搞就行。
这样一来，只需要模型自己的向量的直接表达和交互功能+任何向量数据库不需要大规模改造模型就可以大幅度提高其工作效率及能力。
那意味着，信息0损失，配合知识库后形成长期记忆，在有限token下将大幅度提高工作能力和准确性。
当然，从此思路延伸，还有其他的方案可以优化效率和准确性。不过，不在此次本文的讨论范围内。
最后，不知道是否有研究者写过此类论文，如果没有，喜欢的可以拿去随便用，我没有相关需求，也不在乎。如果用得上的话，就拿走，给我发个消息，让我也高兴高兴！！

https://huggingface.co/datasets/chengli-thu/yuebuqun/viewer/default/train?p=1&row=166

谢谢，刚开始看这个项目，还没有找到如何启动

现在用tuoling摘要，每个对话需要运行15s才能有结果，希望后续能推出多gpu加速后的推理。

使用你们提供的代码,input任何信息都会报错.

when I ask something,server having error:

RetryError[<Future at 0x7fb1a0073b50 state=finished raised TypeError>]

logs:

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

/root/anaconda3/envs/luotuo/lib/python3.11/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/luotuo did not contain libcudart.so as expected! Searching further paths...
warn(msg)
CUDA SETUP: CUDA runtime path found: /usr/local/cuda-11.0/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 3.7
CUDA SETUP: Detected CUDA version 110
/root/anaconda3/envs/luotuo/lib/python3.11/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!
warn(msg)
CUDA SETUP: Loading binary /root/anaconda3/envs/luotuo/lib/python3.11/site-packages/bitsandbytes/libbitsandbytes_cuda110_nocublaslt.so...
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization.
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'.
The class this function is called from is 'LlamaTokenizer'.
Overriding torch_dtype=None with torch_dtype=torch.float16 due to requirements of bitsandbytes to enable model loading in mixed int8. Either pass torch_dtype=torch.float16 or don't pass this argument at all to remove this warning.
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 33/33 [00:20<00:00, 1.63it/s]
Running on local URL: http://0.0.0.0:6006
Running on public URL: https://cb229e153757d1c9cd.gradio.live

This share link expires in 72 hours. For free permanent hosting and GPU upgrades (NEW!), check out Spaces: https://huggingface.co/spaces
RetryError[<Future at 0x7fb1a0319f90 state=finished raised TypeError>]

/home/zsd/.local/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('qychen/luotuo-lora-7b-0.1')} warn(msg) /home/zsd/.local/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('unix')} warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64... CUDA SETUP: WARNING! libcuda.so not found! Do you have a CUDA driver installed? If you are on a cluster, make sure you are on a CUDA machine! CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so /home/zsd/.local/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: No GPU detected! Check your CUDA paths. Proceeding to load CPU-only library... warn(msg) CUDA SETUP: Loading binary /home/zsd/.local/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so... /home/zsd/.local/lib/python3.10/site-packages/bitsandbytes/cextension.py:31: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.... from model import load_model File "/home/zsd/Chinese-alpaca-lora/notebook/Alpaca-LoRA-Serve/model.py", line 2, in <module> from transformers import LlamaTokenizer, LlamaForCausalLM ImportError: cannot import name 'LlamaTokenizer' from 'transformers' (/home/zsd/anaconda3/lib/python3.10/site-packages/transformers/__init__.py)

命令如下：wget https://github.com/LC1332/Chinese-alpaca-lora/blob/main/data/trans_chinese_alpaca_data.json

LLaMA在中文上表现挺一般的，基于LLaMA的微调项目，我感觉中文能力都很差，根本比不上glm-6b。基于glm6b去微调，中文表现可能会更好一些

这个没法安装，怎么解决