happycolor / speechformer Goto Github PK

File "train_model_new.py", line 173, in test
pred_logits = initail_model(features)
File "/home/kaldi/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/data4/rj/SpeechFormer/model/speechformer.py", line 122, in forward
x = self.layers(x).squeeze(dim=1)
File "/home/kaldi/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/kaldi/.local/lib/python3.7/site-packages/torch/nn/modules/container.py", line 117, in forward
input = module(input)
File "/home/kaldi/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/data4/rj/SpeechFormer/model/speechformer.py", line 83, in forward
output = self.input_norm(x)
File "/home/kaldi/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(input, **kwargs)
File "/home/kaldi/.local/lib/python3.7/site-packages/torch/nn/modules/normalization.py", line 170, in forward
input, self.normalized_shape, self.weight, self.bias, self.eps)
File "/home/kaldi/.local/lib/python3.7/site-packages/torch/nn/functional.py", line 2095, in layer_norm
torch.backends.cudnn.enabled)
RuntimeError: Given normalized_shape=[96], expected input with shape [, 96], but got input of size[1, 101, 76]

因为代码里使用model_json['input_dim'] = (self.feat_dim // model_json['num_heads']) * model_json['num_heads']
在推理的时候会报上述错误，这个您有遇到吗？谢谢

I ran your code on the DAIC data set, the model is SpeechFormer-S, the audio features are logmel, and there is overfitting. Specifically, the loss in the training phase can be reduced, but the loss in the testing phase is getting higher and higher. Could you please give me some suggestions?

Hi. Can you upload the code that generates the .csv?

Note that you should create a metadata file (.csv format) for each dataset to record the name and label (and state, e.g. train or dev or test) of the samples. Then modify the argument: meta_csv_file in ./config/xxx_feature_config.json according to the absolute path of the corresponding .csv file.

after i did everything you told me to do
i got this message after running train_model.py:
Use arguments from command line: []
Use arguments from train json file: ['train.EPOCH', 60, 'train.batch_size', 16, 'train.lr', 0.0005]
Modified mark: L2244_expa111
Use GPU: 0
world_size=0 Using dist_url=tcp://127.0.0.1:1869

and i dont have any .mat file and model is not working
what should i do to solve this?

你好，不同数据集的结构是不一样的，你能提供怎么把不同的数据集的音频文件整合到对应的音频文件夹wav_file中以及其音频文件所对应的label的代码吗？

Thank you for your research, I am trying to reproduce your results. I cut the DAIC-WOZ database according to your standards, then extracted the wav2vec features, and used the meta-feature file you provided for training, but the final F1 was only 0.603, Where do you think the problem may occur? In addition, when preprocessing the data, did you eliminate the voice of the virtual agent and only retain the voice of the subject?

Hi,
Thanks for sharing the codebase.

For DAIC-WOZ, the example metadata csv file has a mid-tag for each audio name which I assume they are different segments. Is it possible to share the audio segmentation scripts / procedures? Since for similar reproduction purpose, statistics of segment feed into the model is really crucial.

Hi,
I want to extract features by extract_wav2vec.py code and I've selected a lot of pre-trained wav2vec models but all of them had problem. For example while I am using https://dl.fbaipublicfiles.com/fairseq/wav2vec/vq-wav2vec_kmeans.pt model from https://github.com/facebookresearch/fairseq/tree/main/examples/wav2vec site. I have this problem:
if cfg.activation == "relu":
AttributeError: 'Namespace' object has no attribute 'activation'

And for other models I give similar error about args.

How can I solve it ?
Thanks.