rishikksh20 / hifigan-denoiser Goto Github PK

HiFi-GAN: High Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks

License: Apache License 2.0

Python 100.00%

denoiser denoising wavenet hifigan speech speech-processing waveform pytorch python gan

hifigan-denoiser's Introduction

HiFiGAN Denoiser

This is a Unofficial Pytorch implementation of the paper HiFi-GAN: High Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks.

Citations

@misc{su2020hifigan,
      title={HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks}, 
      author={Jiaqi Su and Zeyu Jin and Adam Finkelstein},
      year={2020},
      eprint={2006.05694},
      archivePrefix={arXiv},
      primaryClass={eess.AS}
}

Requirement

Tested on Python 3.6

pip install -r requirements.txt

Train & Tensorboard

python train.py -c [config yaml file]
tensorboard --logdir log_dir

Inference

python inference.py -p [checkpoint path] -i [input wav path]

Checkpoint :

References

hifigan-denoiser's People

Contributors

Stargazers

Watchers

hifigan-denoiser's Issues

Loss

Hello Rishi,

I am experimenting Speech-Bandwdith-Extension(NarrwoBand - SuperWIdeBand) using this network without Post-Net. I could observe that Generator loss going high-value and that to fluctuating, But evaluating with unseen signal , I could able to recunstruct SuperWideBand from NarrowBand signal.

I am having confusion on model convergence .. Can you plz give some insights on model convergence?

Did you ever tried to evaluate your model?

I wonder whether tried to test the score for all the checkpoints you trained. Your model juest output the same waveshape you input .Your model can output the input waveshape even you don't train the model. What's more, if anyone who want to find the code for paper Hifi-GAN and you fortunately see this issue, please go to the correct repository: https://github.com/jik876/hifi-gan

Data simulation and augmentation

Can you detail the way you are using to make the noise audio for training?

Does it the same with described in the paper?

Are you using kaldi or any tool for this, and can you share your noise dataset !

Thank rishikksh !

Data structure for training Hifigan model

hi
thank you for valuable repo
can you provide an explicit data preparation procedure to train the model?
best regards
@rishikksh20
@DinoTheDinosaur
@karan-deepsync

Pretrained checkpoint

Hi.
Thank's for your research!
Could you please guide me where download the weights (pretrained checkpoint)?

Anyone tried adapting this repo to HIFIGAN-2 paper?

As per https://pixl.cs.princeton.edu/pubs/Su_2021_HSS/Su-HiFi-GAN-2-WASPAA-2021.pdf

postnet parameters

I noticed that the postnet filter size is 32, which makes the output have different shapes than the input. Also, the dropout rate is so high that it's not learning anything meaningful. Is that supposed to be like this?

Tensorboard after adding Postnet

Can you share your log after you adding Postnet at 500k iters?

In my experiment, I add Postnet at 250k iters, does it look right?

KeyError: 'getstate'

Hi, thanks for opensourcing your code! During the training process, I met an error with the command bellow.

COMMAND
python train.py -c config.yaml

ERROR
Initializing Training Process..
Batch size per GPU : 0
Traceback (most recent call last):
File "train.py", line 304, in
main()
File "train.py", line 298, in main
mp.spawn(train, nprocs=hp.train.num_gpus, args=(args, hp, hp_str,))
File "/home/lian/.conda/envs/hifi-GAN-denoise/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 199, in spawn
return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
File "/home/lian/.conda/envs/hifi-GAN-denoise/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 148, in start_processes
process.start()
File "/home/lian/.conda/envs/hifi-GAN-denoise/lib/python3.6/multiprocessing/process.py", line 105, in start
self._popen = self._Popen(self)
File "/home/lian/.conda/envs/hifi-GAN-denoise/lib/python3.6/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/home/lian/.conda/envs/hifi-GAN-denoise/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 32, in init
super().init(process_obj)
File "/home/lian/.conda/envs/hifi-GAN-denoise/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init
self._launch(process_obj)
File "/home/lian/.conda/envs/hifi-GAN-denoise/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/home/lian/.conda/envs/hifi-GAN-denoise/lib/python3.6/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
KeyError: 'getstate'

Would you like to tell me why it happened and how to solve it? Thank you! Have a nice day.

Tensorshape mismatch error when Postnet starts

Hello, I've been trying to train a model and when postnet starts I run into the following issue

Traceback (most recent call last):
  File "train.py", line 300, in <module>
    main()
  File "train.py", line 296, in main
    train(0, args, hp, hp_str)
  File "train.py", line 169, in train
    sc_loss_, mag_loss_ = stft_loss(y_g_hat[:, :, :y.size(2)].squeeze(1), y.squeeze(1))
  File "/home/guest/Supreeth/hifigan-denoiser/hifigan/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/guest/Supreeth/hifigan-denoiser/stft_loss.py", line 130, in forward
    sc_l, mag_l = f(x, y)
  File "/home/guest/Supreeth/hifigan-denoiser/hifigan/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/guest/Supreeth/hifigan-denoiser/stft_loss.py", line 91, in forward
    sc_loss = self.spectral_convergenge_loss(x_mag, y_mag)
  File "/home/guest/Supreeth/hifigan-denoiser/hifigan/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/guest/Supreeth/hifigan-denoiser/stft_loss.py", line 46, in forward
    return torch.norm(y_mag - x_mag, p="fro") / torch.norm(y_mag, p="fro")
RuntimeError: The size of tensor a (641) must match the size of tensor b (640) at non-singleton dimension 1

Is there a fix for this? thank you!