Giter Club home page Giter Club logo

diff-sv's Introduction

Diff-SV

Pytorch code for following paper:

  • Title : Diff-SV: A Unified Hierarchical Framework for Noise-Robust Speaker Verification Using Score-Based Diffusion Probabilistic Models (Accepted for ICASSP 2024, available here)
  • Autor : Ju-ho Kim, Jungwoo Heo, Hyun-seo Shin, Chan-yeong Lim and Ha-Jin Yu

Abstract

Background noise considerably reduces the accuracy and reliability of speaker verification (SV) systems. These challenges can be addressed using a speech enhancement system as a front-end module. Recently, diffusion probabilistic models (DPMs) have exhibited remarkable noise-compensation capabilities in the speech enhancement domain. Building on this success, we propose Diff-SV, a noise-robust SV framework that leverages DPM. Diff-SV unifies a DPM-based speech enhancement system with a speaker embedding extractor, and yields a discriminative and noise-tolerable speaker representation through a hierarchical structure. The proposed model was evaluated under both in-domain and out-of-domain noisy conditions using the VoxCeleb1 test set, an external noise source, and the VOiCES corpus. The obtained experimental results demonstrate that Diff-SV achieves state-of-the-art performance, outperforming recently proposed noise-robust SV systems.

Prerequisites

Environment Setting

  • We used 'nvcr.io/nvidia/pytorch:21.04-py3' image of Nvidia GPU Cloud for conducting our experiments.
  • Run 'build.sh' file to make docker image
./docker/build.sh
  • Run 'interactive.sh' file to activate docker container
  • Note that you must modify the mapping path before running the 'interactive.sh' file
./docker/interactive.sh

Datasets

  • We used VoxCeleb1 dataset for training and test.
  • For noisy test, we used the MUSAN, Nonspeech100, and VOiCES datasets.
  • Each downloaded dataset should be mapped to the 'data' folder in docker environment.

Train and test

python3 code/diff_sv/main.py 

Citation

Please cite this paper if you make use of the code.

@article{kim2023diff,
  title={Diff-SV: A Unified Hierarchical Framework for Noise-Robust Speaker Verification Using Score-Based Diffusion Probabilistic Models},
  author={Kim, Ju-ho and Heo, Jungwoo and Shin, Hyun-seo and Lim, Chan-yeong and Yu, Ha-Jin},
  journal={arXiv preprint arXiv:2309.08320},
  year={2023}
}

diff-sv's People

Contributors

wngh1187 avatar

Stargazers

Taein KIM avatar  avatar MichaelChen avatar Rishabh avatar Tianhao Wang avatar  avatar  avatar  avatar MaxMax avatar  avatar  avatar DS.Xu avatar  avatar  avatar  avatar Hyun-seo Shin avatar  avatar

Watchers

Kostas Georgiou avatar  avatar

diff-sv's Issues

some training problems

Hello, I tried to retrain to reproduce, but the following errors were always reported, no matter which server I was training on, but I did not modify the code, I do not know if you have come across it, could you give me some advice? Thank you very much!
device_issue

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.