Giter Club home page Giter Club logo

Comments (2)

Xiaoven avatar Xiaoven commented on September 12, 2024
  1. I tried to download the model without using cache_dir, and it succeeds to download the model.
mask_model = transformers.AutoModelForSeq2SeqLM.from_pretrained(mask_filling_model_name)
  1. The tree of ~/.cache/huggingface is listed as below:
$ tree .
.
└── hub
    ├── models--Salesforce--codet5p-770m
    │   ├── blobs
    │   │   ├── 05564ee645140947482e12fa785eab3aa5f789a8
    │   │   └── 61b6f30de1356fbc8e91f663454f01f2b7a635d93ccb255b88b07896dde679f1
    │   ├── refs
    │   │   └── main
    │   └── snapshots
    │       └── 627d346e49cb68e600895ab8744e0e1efd5c4536
    │           ├── config.json -> ../../blobs/05564ee645140947482e12fa785eab3aa5f789a8
    │           └── pytorch_model.bin -> ../../blobs/61b6f30de1356fbc8e91f663454f01f2b7a635d93ccb255b88b07896dde679f1
    └── version.txt

7 directories, 6 files
  1. Then, I tried transformers.AutoModelForSeq2SeqLM.from_pretrained(mask_filling_model_name, cache_dir=cache_dir) again, but it threw another FileNotFoundError. However, the file '~/.cache/huggingface/hub/models--Salesforce--codet5p-770m/snapshots/627d346e49cb68e600895ab8744e0e1efd5c4536/config.json' does exist on my computer.
  File "/Users/xiaowen/Documents/GitHub/DetectCodeGPT/code-detection/baselines/utils/loadmodel.py", line 58, in load_mask_filling_model
    mask_model = transformers.AutoModelForSeq2SeqLM.from_pretrained(mask_filling_model_name, cache_dir=model_config['cache_dir'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/sklearn/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 524, in from_pretrained
    config, kwargs = AutoConfig.from_pretrained(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/sklearn/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py", line 976, in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/sklearn/lib/python3.12/site-packages/transformers/configuration_utils.py", line 632, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/sklearn/lib/python3.12/site-packages/transformers/configuration_utils.py", line 722, in _get_config_dict
    config_dict = cls._dict_from_json_file(resolved_config_file)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/sklearn/lib/python3.12/site-packages/transformers/configuration_utils.py", line 824, in _dict_from_json_file
    with open(json_file, "r", encoding="utf-8") as reader:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '~/.cache/huggingface/hub/models--Salesforce--codet5p-770m/snapshots/627d346e49cb68e600895ab8744e0e1efd5c4536/config.json'

from transformers.

Xiaoven avatar Xiaoven commented on September 12, 2024

I found the cause! The issue is that cache_dir = "~/.cache/huggingface/hub", where ~ should be replaced with /Users/username.

from transformers.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.