fairseq about av_hubert HOT 7 CLOSED

Hi, I'm meeting the same problem. Did you solve your problem? How did you solve it? Thanks!

from av_hubert.

然后我单独下载fairseq 遇到了这样的问题：
root@62835adb13396ed678a8bb16:/user-data/bert/av_hubert/avhubert# HYDRA_FULL_ERROR=1 fairseq-hydra-train --config-dir conf/finetune --config-name base_lrs3_30h.yaml task.data=/user-data/bert/av_hubert/medium/30h_data task.label_dir=/30h_data task.tokenizer_bpe_model=/user-data/bert/av_hubert/medium/spm499/spm_unigram499.model model.w2v_path=/user-data/bert/av_hubert/base_lrs3_iter5.pt hydra.run.dir=/user-data/bert/av_hubert/medium common.user_dir=pwd
Traceback (most recent call last):
File "/opt/conda/bin/fairseq-hydra-train", line 33, in
sys.exit(load_entry_point('fairseq', 'console_scripts', 'fairseq-hydra-train')())
File "/opt/conda/bin/fairseq-hydra-train", line 25, in importlib_load_entry_point
return next(matches).load()
File "/opt/conda/lib/python3.8/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/opt/conda/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 848, in exec_module
File "", line 219, in _call_with_frames_removed
File "/user-data/bert/av_hubert/fairseq/fairseq_cli/hydra_train.py", line 10, in
from fairseq.dataclass.initialize import add_defaults, hydra_init
File "/user-data/bert/av_hubert/fairseq/fairseq/init.py", line 20, in
from fairseq.distributed import utils as distributed_utils
File "/user-data/bert/av_hubert/fairseq/fairseq/distributed/init.py", line 7, in
from .fully_sharded_data_parallel import fsdp_enable_wrap, fsdp_wrap, FullyShardedDataParallel
File "/user-data/bert/av_hubert/fairseq/fairseq/distributed/fully_sharded_data_parallel.py", line 10, in
from fairseq.dataclass.configs import DistributedTrainingConfig
File "/user-data/bert/av_hubert/fairseq/fairseq/dataclass/init.py", line 6, in
from .configs import FairseqDataclass
File "/user-data/bert/av_hubert/fairseq/fairseq/dataclass/configs.py", line 24, in
from omegaconf import II, MISSING
ImportError: cannot import name 'II' from 'omegaconf' (/opt/conda/lib/python3.8/site-packages/omegaconf/init.py)
Segmentation fault (core dumped)
root@62835adb13396ed678a8bb16:/user-data/bert/av_hubert/avhubert#

麻烦解决一下啦，表示感谢！！

from av_hubert.

Hi,

You should install fairseq of our version and also do git submodule init and git submodule update. See here

from av_hubert.

hi!
我按照你的建议做了 还是不行，这是否正确？Submodule path 'fairseq': checked out 'afc77bdf4bb51453ce76f1572ef2ee6ddcda8eeb'？？

root@62835adb13396ed678a8bb16:/user-data/av_hubert# git submodule init
root@62835adb13396ed678a8bb16:/user-data/av_hubert# git submodule update
root@62835adb13396ed678a8bb16:/user-data/av_hubert# cd fairseq/
root@62835adb13396ed678a8bb16:/user-data/av_hubert/fairseq# cd ..
root@62835adb13396ed678a8bb16:/user-data/av_hubert# rm -rf fairseq/
root@62835adb13396ed678a8bb16:/user-data/av_hubert# git submodule init
root@62835adb13396ed678a8bb16:/user-data/av_hubert# git submodule update
Submodule path 'fairseq': checked out 'afc77bdf4bb51453ce76f1572ef2ee6ddcda8eeb'
root@62835adb13396ed678a8bb16:/user-data/av_hubert# cd fairseq/
root@62835adb13396ed678a8bb16:/user-data/av_hubert/fairseq# pip install --editable ./
Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Obtaining file:///user-data/av_hubert/fairseq
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing wheel metadata ... done
Requirement already satisfied: torch in /opt/conda/lib/python3.8/site-packages (from fairseq==1.0.0a0+afc77bd) (1.8.0)
Requirement already satisfied: cython in /opt/conda/lib/python3.8/site-packages (from fairseq==1.0.0a0+afc77bd) (0.29.23)
Requirement already satisfied: cffi in /opt/conda/lib/python3.8/site-packages (from fairseq==1.0.0a0+afc77bd) (1.14.5)
Requirement already satisfied: hydra-core<1.1 in /opt/conda/lib/python3.8/site-packages (from fairseq==1.0.0a0+afc77bd) (0.11.3)
Requirement already satisfied: omegaconf<2.1 in /opt/conda/lib/python3.8/site-packages (from fairseq==1.0.0a0+afc77bd) (1.4.0)
Requirement already satisfied: regex in /opt/conda/lib/python3.8/site-packages (from fairseq==1.0.0a0+afc77bd) (2021.4.4)
Requirement already satisfied: sacrebleu>=1.4.12 in /opt/conda/lib/python3.8/site-packages (from fairseq==1.0.0a0+afc77bd) (2.0.0)
Requirement already satisfied: tqdm in /opt/conda/lib/python3.8/site-packages (from fairseq==1.0.0a0+afc77bd) (4.59.0)
Requirement already satisfied: numpy in /opt/conda/lib/python3.8/site-packages (from fairseq==1.0.0a0+afc77bd) (1.20.2)
Requirement already satisfied: six in /opt/conda/lib/python3.8/site-packages (from omegaconf<2.1->fairseq==1.0.0a0+afc77bd) (1.15.0)
Requirement already satisfied: PyYAML in /opt/conda/lib/python3.8/site-packages (from omegaconf<2.1->fairseq==1.0.0a0+afc77bd) (5.4.1)
Requirement already satisfied: colorama in /opt/conda/lib/python3.8/site-packages (from sacrebleu>=1.4.12->fairseq==1.0.0a0+afc77bd) (0.4.4)
Requirement already satisfied: portalocker in /opt/conda/lib/python3.8/site-packages (from sacrebleu>=1.4.12->fairseq==1.0.0a0+afc77bd) (2.4.0)
Requirement already satisfied: tabulate>=0.8.9 in /opt/conda/lib/python3.8/site-packages (from sacrebleu>=1.4.12->fairseq==1.0.0a0+afc77bd) (0.8.9)
Requirement already satisfied: pycparser in /opt/conda/lib/python3.8/site-packages (from cffi->fairseq==1.0.0a0+afc77bd) (2.20)
Requirement already satisfied: typing_extensions in /opt/conda/lib/python3.8/site-packages (from torch->fairseq==1.0.0a0+afc77bd) (3.7.4.3)
Installing collected packages: fairseq
Running setup.py develop for fairseq
Successfully installed fairseq-1.0.0a0+afc77bd
WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv
root@62835adb13396ed678a8bb16:/user-data/av_hubert/fairseq# cd ..
root@62835adb13396ed678a8bb16:/user-data/av_hubert# cd avhubert/
root@62835adb13396ed678a8bb16:/user-data/av_hubert/avhubert# HYDRA_FULL_ERROR=1 fairseq-hydra-train --config-dir conf/finetune --config-name base_lrs3_30h.yaml task.data=/user-data/bert/av_hubert/medium/30h_data task.label_dir=/30h_data task.tokenizer_bpe_model=/user-data/bert/av_hubert/medium/spm499/spm_unigram499.model model.w2v_path=/user-data/bert/av_hubert/base_lrs3_iter5.pt hydra.run.dir=/user-data/bert/av_hubert/medium common.user_dir=pwd
Traceback (most recent call last):
File "/opt/conda/bin/fairseq-hydra-train", line 33, in
sys.exit(load_entry_point('fairseq', 'console_scripts', 'fairseq-hydra-train')())
File "/opt/conda/bin/fairseq-hydra-train", line 25, in importlib_load_entry_point
return next(matches).load()
File "/opt/conda/lib/python3.8/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/opt/conda/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 848, in exec_module
File "", line 219, in _call_with_frames_removed
File "/user-data/av_hubert/fairseq/fairseq_cli/hydra_train.py", line 10, in
from fairseq.dataclass.initialize import add_defaults, hydra_init
File "/user-data/av_hubert/fairseq/fairseq/init.py", line 20, in
from fairseq.distributed import utils as distributed_utils
File "/user-data/av_hubert/fairseq/fairseq/distributed/init.py", line 7, in
from .fully_sharded_data_parallel import fsdp_enable_wrap, fsdp_wrap, FullyShardedDataParallel
File "/user-data/av_hubert/fairseq/fairseq/distributed/fully_sharded_data_parallel.py", line 10, in
from fairseq.dataclass.configs import DistributedTrainingConfig
File "/user-data/av_hubert/fairseq/fairseq/dataclass/init.py", line 6, in
from .configs import FairseqDataclass
File "/user-data/av_hubert/fairseq/fairseq/dataclass/configs.py", line 24, in
from omegaconf import II, MISSING
ImportError: cannot import name 'II' from 'omegaconf' (/opt/conda/lib/python3.8/site-packages/omegaconf/init.py)
Segmentation fault (core dumped)
root@62835adb13396ed678a8bb16:/user-data/av_hubert/avhubert#

from av_hubert.

It seems some dependent packages of fairseq are still of the old one. Can you start a new environment and install it from scratch according to the instructions?

from av_hubert.

OK，我重新试一试，谢谢～

from av_hubert.

Hi, I'm meeting the same problem. Did you solve your problem? How did you solve it? Thanks!

Hi~ you should try deleting and re-downloading~

from av_hubert.