When running the notebooks out of the box, I am getting an device error when trying to train the DPO.
Somehow something is on the cpu while it expects a CUDA device. I am running on the T4 in google colab... do you know what it could be and how to solve? Thanks!

Hi,

I keep getting the following dataloader error when I run the DPO code:
Traceback (most recent call last): File "/home/ml/users/---/research/learning_from_preferences/rlhf_starter_code/rlhf/dpo_falcon.py", line 114, in <module> dpo_trainer.train() File "/home/ml/users/---/anaconda3/envs/trl/lib/python3.9/site-packages/transformers/trainer.py", line 1885, in train return inner_training_loop( File "/home/ml/users/---/anaconda3/envs/trl/lib/python3.9/site-packages/transformers/trainer.py", line 2178, in _inner_training_loop for step, inputs in enumerate(epoch_iterator): File "/home/ml/users/----/anaconda3/envs/trl/lib/python3.9/site-packages/accelerate/data_loader.py", line 454, in __iter__ current_batch = next(dataloader_iter) File "/home/ml/users/----/anaconda3/envs/trl/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 631, in __next__ data = self._next_data() File "/home/ml/users/----/anaconda3/envs/trl/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 675, in _next_data data = self._dataset_fetcher.fetch(index) # may raise StopIteration File "/home/ml/users/----/anaconda3/envs/trl/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 54, in fetch return self.collate_fn(data) File "/home/ml/users/----/anaconda3/envs/trl/lib/python3.9/site-packages/trl/trainer/utils.py", line 338, in __call__ to_pad = [torch.LongTensor(ex[k]) for ex in features] File "/home/ml/users/----/anaconda3/envs/trl/lib/python3.9/site-packages/trl/trainer/utils.py", line 338, in <listcomp> to_pad = [torch.LongTensor(ex[k]) for ex in features] TypeError: an integer is required (got type NoneType) 0%| | 0/50 [00:00<?, ?it/s]

Hi all,

I'm currently working on fine tuning Mistral using my own dataset. But I'm wondering of how to upload the data. I used dataset = load_dataset('json', data_files='/path to the dataset/.jsonl', split='train')

But this error is showing when I run AutoModelForCausalLM.from_pretrained:
ValueError: You need to pass dataset in order to quantize your model

It looks like it can't see it.

Will appreciate any help.

Hi,
First of all thanks for your contributions and sharing.
I'm having trouble to get inference stopped. Actually, after my fine-tuning, when I'm doing an inference. the model keep generating tokens until it reaches "max_tokens". What should I do (I'm newbie 😃) to get rid of this and stop at the end of the answer.

Any insight ?

thanks for your feedback.

Finetuning Zephyr:

You passed quantization_config to from_pretrained but the model you're loading already has a quantization_config attribute and has already quantized weights. However, loading attributes (e.g. use_exllama, exllama_config, use_cuda_fp16, max_input_length) will be overwritten with the one you passed to from_pretrained. The rest will be ignored.
WARNING:auto_gptq.nn_modules.qlinear.qlinear_cuda:CUDA extension not installed.
WARNING:auto_gptq.nn_modules.qlinear.qlinear_cuda_old:CUDA extension not installed.
ERROR:auto_gptq.nn_modules.qlinear.qlinear_exllama:exllama_kernels not installed.