The goal of this project is to provide a voice assistant to the Data Observatory able to execute various commands and to support users during their presentation.

This project offers a modular framework for a voice-enabled chat service. The voicebot folder contains the components of the voice assistant. You will find there the following services:

• Dialog manager service
• Speech-to-Text service
• Voice assistant service
• Text-to-Speech service
• Emotion recognition service
• Hotword detection service
• Grammar correction service
• (Draft) Speech Filler service

There's also a web client to help you with the interaction.

The evolution of the system will be best seen after looking at the system diagrams for different versions:

• Version 1
  • Author(s): Aurélie Beaugeard
  • Project type: UROP - 10 weeks
  • Objective: Setup the whole infrastructure for the system, creation of the initial dialogue model and first interactive model.
• Version 2
  • Author(s): Mifu Suzuki
  • Project type: MSc Individual Project - 5 months
  • Objective: Recognition of emotion from speech, translation of continuous emotions into a discrete space and adaptation of the dialogue model based on the recognized emotion.
• Version 3, Galileo Team
  • Author(s): Bianca Ganescu, Izabella Kacprzak, Una Miralles, Vlad Nicolaescu, Nicole Obretincheva, Alex Stocks
  • Project type: Undergraduate Group Project - 8 weeks
  • Objective: Hotword detection. Grammar correction to improve transcription quality. A draft speech filler model.

These models will help you to better understand how the project works:

• Front-end perpective
• Back-end perspective
• Dialogue Use Case

The various components of the project can be configured using the docker compose file. Several components require a bit of tweaking and configuration. All details should be available in the component readme file.

All the components can be run independently and the instruction should be found in the readme file of each service.