Comments (2)
- I tried to download the model without using
cache_dir
, and it succeeds to download the model.
mask_model = transformers.AutoModelForSeq2SeqLM.from_pretrained(mask_filling_model_name)
- The tree of
~/.cache/huggingface
is listed as below:
$ tree .
.
└── hub
├── models--Salesforce--codet5p-770m
│ ├── blobs
│ │ ├── 05564ee645140947482e12fa785eab3aa5f789a8
│ │ └── 61b6f30de1356fbc8e91f663454f01f2b7a635d93ccb255b88b07896dde679f1
│ ├── refs
│ │ └── main
│ └── snapshots
│ └── 627d346e49cb68e600895ab8744e0e1efd5c4536
│ ├── config.json -> ../../blobs/05564ee645140947482e12fa785eab3aa5f789a8
│ └── pytorch_model.bin -> ../../blobs/61b6f30de1356fbc8e91f663454f01f2b7a635d93ccb255b88b07896dde679f1
└── version.txt
7 directories, 6 files
- Then, I tried
transformers.AutoModelForSeq2SeqLM.from_pretrained(mask_filling_model_name, cache_dir=cache_dir)
again, but it threw another FileNotFoundError. However, the file '~/.cache/huggingface/hub/models--Salesforce--codet5p-770m/snapshots/627d346e49cb68e600895ab8744e0e1efd5c4536/config.json' does exist on my computer.
File "/Users/xiaowen/Documents/GitHub/DetectCodeGPT/code-detection/baselines/utils/loadmodel.py", line 58, in load_mask_filling_model
mask_model = transformers.AutoModelForSeq2SeqLM.from_pretrained(mask_filling_model_name, cache_dir=model_config['cache_dir'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Caskroom/miniconda/base/envs/sklearn/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 524, in from_pretrained
config, kwargs = AutoConfig.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Caskroom/miniconda/base/envs/sklearn/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py", line 976, in from_pretrained
config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Caskroom/miniconda/base/envs/sklearn/lib/python3.12/site-packages/transformers/configuration_utils.py", line 632, in get_config_dict
config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Caskroom/miniconda/base/envs/sklearn/lib/python3.12/site-packages/transformers/configuration_utils.py", line 722, in _get_config_dict
config_dict = cls._dict_from_json_file(resolved_config_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Caskroom/miniconda/base/envs/sklearn/lib/python3.12/site-packages/transformers/configuration_utils.py", line 824, in _dict_from_json_file
with open(json_file, "r", encoding="utf-8") as reader:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '~/.cache/huggingface/hub/models--Salesforce--codet5p-770m/snapshots/627d346e49cb68e600895ab8744e0e1efd5c4536/config.json'
from transformers.
I found the cause! The issue is that cache_dir = "~/.cache/huggingface/hub"
, where ~
should be replaced with /Users/username
.
from transformers.
Related Issues (20)
- ValueError: Cannot use apply_chat_template() because tokenizer.chat_template is not set and no template argument was passed! For information about writing templates and setting the tokenizer.chat_template attribute, please see the documentation HOT 1
- TypeError: '<' not supported between instances of 'NoneType' and 'int' HOT 2
- Can’t train Mamba2 with FP16 (Mamba(/2)ForCausalLM) HOT 1
- batch inference scales linearly with batch size when input is long HOT 1
- Exception raised with trainer + `accelerate launch` FSDP + large gradient accumulation steps + small dataset
- Do Transformers onnx export support the input of the Llama is the input_embeds? HOT 1
- Cannot batch them ({'num_frames', 'input_features', 'is_last'} != {'input_features', 'is_last'}) HOT 1
- The examples in the examples directory are mostly for fine-tuning pre-trained models?how to trian from scratch
- `gpt2` with `output_attentions=True` has different attentions shape between flash and eager
- Try to convert LlamaForCausalLM to ONNX with input_embeds as input
- eval_loss not found when training a peft model using trainer.py HOT 2
- documents not being applied in apply_chat_tempplate HOT 4
- Default setting of GenerationConfig
- `Zero-shot object detection` documentation sentence rephrase
- Usage of deprecated task will break CI after datasets-3.0
- Character sets not supported on windows11 HOT 10
- Support Pixtral
- [Distilbert] Torch jit trace failed with `load_in_8bit=True`.
- RuntimeError when performing Mask2Former traced model inference on a different device
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from transformers.