• Bayesian search
  • Automated Machine Learning Hyperparameter Tuning in Python
• mfcc vs logMelFilterbanks

Implement and test residual neural network architecture.

Resources:

• DEEP RESIDUAL LEARNING FOR SMALL-FOOTPRINT KEYWORD SPOTTING
• A PyTorch Reimplementation of Convolutional Neural Networks for Keyword Spotting

Hello Matteo,

I am looking for a very low processor load "wake-word/phrase" mechanism for my Raspberry Pi3B based home robot (similar to PicoVoice Porcupine.)

In my search I came upon your "Small-footprint Keyword Spotting" that appears to use the full TensorFlow package. I have experimented with tflite_runtime.interpreter on my robot with an off-robot trained camera object recognition model.

Is this how I would be able to utilize your keyword spotter?

• Install TensorFlow (or TFLite?) to my Mac
• Build a wake-word/phrase model (I have no prior experience in this step)
• Bring down the custom model to robot
• Modify your TensorFlow demo.py to use TFLite (also no experience here)

PicoVoice Porcupine uses about 10% of one core of my robot's Pi3B and is phenomenally good at far-field recognition and false rejection, (but does not allow custom wake-words for personal projects).

I have experimented with Vosk-api (the successor to Kaldi successor to PocketSphinx) but the processor load averages four times that of Porcupine (40% 15min ave. 30-100% 1min average of one core), so I continue to look for a "Small-footprint keyword spotter"

Your thoughts and suggestions?

Regards,

Alan McDonley

We can use denoising autoencoders to learn robust features.

First test seems to be working:

Long Short Term Memory Network

LSTM Networks are useful in processing two-dimensional processions of words

• Evaluate if LSTM are interesting and decide wether to implement it or not

Useful documentation:

• A Beginner's Guide to LSTMs and Recurrent Neural Networks

Analyze different audio features

• Find new audio features to implement
• Implement audio features
  • Implement rhythm features from librosa
• Test and compare results

Test different training method (e.g. optimisers, lr schedulers, ... ) for the CNN model.

Useful resources:

• Convolutional Neural Networks for Small-footprint Keyword Spotting

Use a CNN / RNN to extract features from audio files followed by a One Class classifier trained to to recognise the keyword (marvin in our case).

As classifier we can use either a OC-SVM or Isolation Forest.

• Study
• Implement RNN
  • LSTM
  • GRU
  • Autoencoders
• Test
  • activate/deactivate Attention
  • dropout
  • double vs single RNN layer
  • visualize attention scores

Useful references:

• A Beginner's Guide to Attention Mechanisms and Memory Networks
• Intuitive Understanding of Attention Mechanism in Deep Learning
• Tutorial on Attention-based Models (Part 1)
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention