你好，按照第三节中，Llama3图片理解能力的微调文档进行操作，在进行测试的时候，执行如下的命令
export MKL_SERVICE_FORCE_INTEL=1 xtuner chat /root/model/Meta-Llama-3-8B-Instruct \ --visual-encoder /root/model/clip-vit-large-patch14-336 \ --llava /root/llama3_llava_pth/iter_1200_hf \ --prompt-template llama3_chat \ --image /root/tutorial/xtuner/llava/llava_data/test_img/oph.jpg
并且输入describe this image . 后，模型一直重复输出相同的内容，就是一句话输出之后，又重复生成，没有暂停。，好像是续写模式，请问出现这种情况是什么原因？谢谢

Does llama3 support inference and fine-tuning on multi-graphics card machines? Could you please add some sample code for a single machine with multiple cards?

load model begin.
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
2024-04-26 09:50:49.558 Uncaught app exception
Traceback (most recent call last):
File "/root/anaconda3/envs/llama3/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 584, in _run_script
exec(code, module.dict)
File "/root/Llama3-XTuner-CN/tools/internstudio_web_demo.py", line 274, in
main(arg1)
File "/root/Llama3-XTuner-CN/tools/internstudio_web_demo.py", line 222, in main
model, tokenizer = load_model(arg1)
File "/root/anaconda3/envs/llama3/lib/python3.10/site-packages/streamlit/runtime/caching/cache_utils.py", line 168, in wrapper
return cached_func(*args, **kwargs)
File "/root/anaconda3/envs/llama3/lib/python3.10/site-packages/streamlit/runtime/caching/cache_utils.py", line 197, in call
return self._get_or_create_cached_value(args, kwargs)
File "/root/anaconda3/envs/llama3/lib/python3.10/site-packages/streamlit/runtime/caching/cache_utils.py", line 224, in _get_or_create_cached_value
return self._handle_cache_miss(cache, value_key, func_args, func_kwargs)
File "/root/anaconda3/envs/llama3/lib/python3.10/site-packages/streamlit/runtime/caching/cache_utils.py", line 280, in _handle_cache_miss
computed_value = self._info.func(*func_args, **func_kwargs)
File "/root/Llama3-XTuner-CN/tools/internstudio_web_demo.py", line 174, in load_model
model = AutoModelForCausalLM.from_pretrained(arg1, torch_dtype=torch.float16).cuda()
File "/root/anaconda3/envs/llama3/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 563, in from_pretrained
return model_class.from_pretrained(
File "/root/anaconda3/envs/llama3/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3677, in from_pretrained
) = cls._load_pretrained_model(
File "/root/anaconda3/envs/llama3/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4084, in _load_pretrained_model
state_dict = load_state_dict(shard_file, is_quantized=is_quantized)
File "/root/anaconda3/envs/llama3/lib/python3.10/site-packages/transformers/modeling_utils.py", line 507, in load_state_dict
with safe_open(checkpoint_file, framework="pt") as f:
safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge
^C Stopping...

when deploying web-demo with streamlit,
OSError: Incorrect path_or_model_id: '/root/model/Llama-3-8B-Instruct'. Please provide either the path to a local folder or the repo_id of a model on the Hub will prompt.
according to context, the model name is incorrect and should be /root/model/Meta-Llama-3-8B-Instruct.
However, it's a very tiny error and I belive most people will correct it themselves. just in case I submit this issue.

Excellent work!!! There is a error I face T T.

When I used this web-ui framework on Qwen1.5-14B pre-trained model, the agent cannot stop generating content like below:

I will very appriciated if you can help me solve this problem!

Excellent work!!! There is a error I face T T.

When I used this web-ui framework on Qwen1.5-14B pre-trained model, the agent cannot stop generating content like below:

I will very appriciated if you can help me solve this problem!

非internstudio，使用3090显卡我使用的是脚本, 下面是我的part1

#######################################################################
#                          PART 1  Settings                           #
#######################################################################
# Model
llm_name_or_path = '/app/InternLM_cqy/a_weights/LLM-Research/Meta-Llama-3-8B-Instruct'
visual_encoder_name_or_path = '/app/InternLM_cqy/a_weights/clip-vit-large-patch14-336'
# Specify the pretrained pth
pretrained_pth = '/app/InternLM_cqy/a_weights/iter_2181.pth'  # noqa: E501

# Data
data_root = '/app/InternLM_cqy/Tutorial-camp2/xtuner/llava/llava_data/'
data_path = data_root + 'repeated_data.json'
image_folder = data_root
prompt_template = PROMPT_TEMPLATE.llama3_chat
max_length = int(2048 - (336 / 14)**2)

# Scheduler & Optimizer
batch_size = 1  # per_device
accumulative_counts = 1
dataloader_num_workers = 0
max_epochs = 1
optim_type = AdamW
lr = 2e-4
betas = (0.9, 0.999)
weight_decay = 0
max_norm = 1  # grad clip
warmup_ratio = 0.03

# Save
save_steps = 500
save_total_limit = 2  # Maximum checkpoints to keep (-1 means unlimited)

# Evaluate the generation performance during the training
evaluation_freq = 500
SYSTEM = ''
evaluation_images = 'https://llava-vl.github.io/static/images/view.jpg'
evaluation_inputs = ['Please describe this picture','What is the equipment in the image?']

然后我使用zero2出现oom适应zero3出现

Traceback (most recent call last):
  File "/app/InternLM_cqy/XTuner/xtuner/tools/train.py", line 342, in <module>
    main()
  File "/app/InternLM_cqy/XTuner/xtuner/tools/train.py", line 338, in main
    runner.train()
  File "/usr/local/lib/python3.10/dist-packages/mmengine/runner/_flexible_runner.py", line 1200, in train
    model = self.train_loop.run()  # type: ignore
  File "/usr/local/lib/python3.10/dist-packages/mmengine/runner/loops.py", line 271, in run
    self.runner.call_hook('before_train')
  File "/usr/local/lib/python3.10/dist-packages/mmengine/runner/_flexible_runner.py", line 1273, in call_hook
    raise TypeError(f'{e} in {hook}') from e
TypeError: All input tensors need to be on the same GPU, but found some tensors to not be on a GPU:
 [(torch.Size([1, 8388608]), device(type='cuda', index=0)), (torch.Size([262144]), device(type='cpu')), (torch.Size([4096, 4096]), device(type='cuda', index=0))] in <xtuner.engine.hooks.evaluate_chat_hook.EvaluateChatHook object at 0x7fb7cbb85390>

可以帮忙看一下吗？