when `per_device_eval_batch_size` > 1 and launch by deepspeed, RuntimeError: Tensors must be contiguous about chatglm-efficient-tuning HOT 5 CLOSED

我在ChatGLM和LLaMA的efficient_tuning下,用deepspeed和accelerate的多卡infer同报错
accelerate launch
./LLaMA-Efficient-Tuning/src/train_bash.py
--max_samples 50
--model_name_or_path "Llama-2-13B-fp16/"
--do_predict
--dataset alpaca_zh
--dataset_dir "LLaMA-Efficient-Tuning/data"
--finetuning_type lora
--output_dir Efficient_Tuning/llama2-13b
--per_device_eval_batch_size 4
--predict_with_generate
--fp16
蹲解决方法

from chatglm-efficient-tuning.

目前 do_predict 仅支持单卡

from chatglm-efficient-tuning.

LLaMA-Efficient-Tuning 之前是可以的，代码越更新越多bug了

from chatglm-efficient-tuning.

是这样的 6月份的版本可以accelerate launch多卡

from chatglm-efficient-tuning.

@kuailehaha pull 最新的代码试一下

from chatglm-efficient-tuning.