vimalmanohar / pychain-old Goto Github PK

Create a new class (maybe derived from Function) with the arguments:

[in] chain_training_options

A struct containing various options relevant to LF-MMI training:

• leaky_hmm_coefficient [0.1]

[in] transition_matrix

SparseTensor of size N x N x D, where
N = num states in denominator graph
D = num pdfs
T = chunk size
B = minibatch size
A sparse tensor representation of the transition probabilities of the arcs in the denominator graph. Depending on the implementation, we might require this to be stored as real numbers or in log format.

[in] initial_probs

Tensor of size N x 1
A tensor containing the initial probabilities for the states in the denominator graph. This is also needed for leaky-hmm computation.

[in] nnet_outputs

Tensor of size D x T x B
A tensor of neural network activations.

[out] nnet_gradients

Tensor of size D x T x B
Gradients computed by the network. In a python implementation, this will be computed using autograd backward() function.

DataLoader class to load MFCC features and e2e-format FSTs from disk into a pair of features Tensor and FST and create minibatches.

We can follow the way Deepspeech does this i.e. to use an SCP-like file that has the filepaths so that the features and FSTs can be read at the time of creating the minibatch. This should be implemented such that it can return a single pair (features, fst) read from disk when used with python iterator.

• This probably needs as input an SCP-like file that has all the paths for features as well as FSTs including additional auxiliary information such as the duration of the utterance, which will be required by the Sampler.
• We assume that the SCP-like file is sorted by length so we can create minibatches of same chunk size. A Sampler will use this information to create minibatches. #3

Sampler to create minibatches based on the length of the utterances so that similar length utterances are together.
This needs to be moved to a separate module outside the loss function module.

It has access to a list of utterances sorted by length so that same-size batches can be created easily.

• Create batches of appropriate size.
• Return randomly sampled batches.
• Random sampling within a batch.
• Different length utterances should have different minibatch sizes (This might be different from the usual approach which uses same minibatch size. We have to use the kaldi approach for this).

[Note from Dan: we could consider padding with silence or speed-perturbing (like Hossein does) to make sure the utterance lengths are not all distinct. Also, I’d like to be able to support chunks of utterances, but this is not a hurry right now. You may have to have a mechanism to use different minibatch sizes for different utterance lengths, if you have very variable utterance lengths.]

Implement the denominator computation in C++ instead of python as in #1.
This will involve writing CUDA kernels such as the one already in kaldi (https://github.com/kaldi-asr/kaldi/blob/master/src/chain/chain-denominator.cc and https://github.com/kaldi-asr/kaldi/blob/master/src/chain/chain-kernels.cu). However this might not actually be required if the python version is already fast.