linksoul-ai / llasm Goto Github PK

命令：python infer.py --input_audio_file d:/tmp/output.wav --llasm_model F:/models/LLaSM-Cllama2 --llasm_audio_tower F:/models/whisper-medium --llm_type "Chinese_llama2"
报错：RuntimeError: mat1 and mat2 shapes cannot be multiplied (64x1024 and 1280x4096)

是哪里的问题？求解答，谢谢！~

'(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /openai/whisper-large-v2/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7fd9399957b0>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 7b68228e-e6a6-4ecd-bc7e-3affc92e3ac3)')' thrown while requesting HEAD https://huggingface.co/openai/whisper-large-v2/resolve/main/config.json
Traceback (most recent call last):
File "/root/anaconda3/envs/llasm/lib/python3.10/site-packages/transformers/utils/hub.py", line 417, in cached_file
resolved_file = hf_hub_download(
File "/root/anaconda3/envs/llasm/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "/root/anaconda3/envs/llasm/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1291, in hf_hub_download
raise LocalEntryNotFoundError(
huggingface_hub.utils._errors.LocalEntryNotFoundError: Connection error, and we cannot find the requested files in the disk cache. Please try again or make sure your Internet connection is on.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/work/liukai/LLaSM/infer.py", line 122, in
main(args)
File "/work/liukai/LLaSM/infer.py", line 46, in main
model = LlaaaLlamaForCausalLM.from_pretrained(
File "/root/anaconda3/envs/llasm/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2700, in from_pretrained
model = cls(config, *model_args, **model_kwargs)
File "/work/liukai/LLaSM/llasm.py", line 160, in init
self.model = LlaaaLlamaModel(config)
File "/work/liukai/LLaSM/llasm.py", line 41, in init
self.audio_tower = [load_whisper(config.mm_audio_tower)]
File "/work/liukai/LLaSM/llasm.py", line 28, in load_whisper
model = WhisperModel.from_pretrained(audio_tower_name)
File "/root/anaconda3/envs/llasm/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2325, in from_pretrained
config, model_kwargs = cls.config_class.from_pretrained(
File "/root/anaconda3/envs/llasm/lib/python3.10/site-packages/transformers/configuration_utils.py", line 590, in from_pretrained
config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, **kwargs)
File "/root/anaconda3/envs/llasm/lib/python3.10/site-packages/transformers/configuration_utils.py", line 617, in get_config_dict
config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
File "/root/anaconda3/envs/llasm/lib/python3.10/site-packages/transformers/configuration_utils.py", line 672, in _get_config_dict
resolved_config_file = cached_file(
File "/root/anaconda3/envs/llasm/lib/python3.10/site-packages/transformers/utils/hub.py", line 452, in cached_file
raise EnvironmentError(
OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like openai/whisper-large-v2 is not the path to a directory containing a file named config.json.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.

请问百度网盘下载地址的是本文方法的预训练模型吗？还是原生的开源llama？谢谢。