CAT provides a complete workflow for CRF-based data-efficient end-to-end speech recognition.

• Overview
• Features
• Installation
• Getting started
• ASR results
• Further reading

CAT aims at combining the advantages of both the hybrid and the E2E ASR approches to achieve data-efficiency, by judiciously examining the pros and cons of modularity versus unified neural network, separate optimization versus joint optimization. CAT advocates global normalization modeling and discriminative training in the framework of Conditional Random Field (CRF), currently with Connectionist Temporal Classification (CTC) inspired state topology.

1. CAT contains a full-fledged CUDA/C/C++ implementation of CTC-CRF loss function binding to PyTorch.

2. One-stop CTC/CTC-CRF/RNN-T/LM training & inference. See the templates.

3. Flexible configuration with JSON. Check the guideline for configuration.

4. Scalable and extensible. It is easy to be extended to train tens of thousands of speech data and add new models and tasks.

1. Dependencies

   • CUDA compatible device, NVIDIA driver installed and CUDA lib available.

   • PyTorch: >=1.9.0 is required. Installation guide from PyTorch

   • Kaldi [optional, but recommended]: used for speech data preparation and some FST-related operations. This is optional for most of the basic functions. Required only when you want to conduct CTC-CRF training.

     Besides Kaldi, you could use torchaudio for feature extraction. Take a look at data.sh for how to prepare data with torchaudio.

2. Clone and install CAT

   git clone https://github.com/thu-spmi/CAT.git && cd CAT
   # Get installation helping message
   ./install.sh -h
   # Install with default configurations
   #./install.sh

To get started with this project, please refer to TEMPLATE for tutorial.

• What's New
• Guideline for configuring settings
• Tutorial for CUSIDE based streaming ASR: English | 中文
• Some tips about the usage of third party tools
• Guide to train models on more than 1500 hours of speech data: English | 中文

@inproceedings{xiang2019crf,
  title={CRF-based single-stage acoustic modeling with CTC topology},
  author={Xiang, Hongyu and Ou, Zhijian},
  booktitle={IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages={5676--5680},
  year={2019},
  organization={IEEE}
}

@inproceedings{an2020cat,
  title={CAT: A CTC-CRF based ASR toolkit bridging the hybrid and the end-to-end approaches towards data efficiency and low latency},
  author={An, Keyu and Xiang, Hongyu and Ou, Zhijian},
  booktitle={INTERSPEECH},
  pages={566--570},
  year={2020}
}