noboxio / tj-python Goto Github PK

Couple of fixes:

• Maybe add debug levels. Right now bunches of scary alsa messages are getting output into console, and somewhere hidden in there is what the user actually says. A better user interface would be just to print what the user says while streaming transcription. But also be able to re-enable those messages for debug purposes.
• Automatically catch ErrNo 9981: Input Overflowed errors. It seems like most of the kids except like one or two got this error, and we fixed all of the issues by simply changing the RATE from 48000 to 44100. How hard can it be to automatically catch the error and retry with a different RATE?
• Don't have a hardcoded record time. Right now, streaming.py records for 3 seconds and then stops. So the kids would have to squeeze whatever they have to say into three seconds. streaming.py needs to be able to record until the kid is done talking. This may be harder, because it's streaming. I'll see. I can't simply take the code from speechRecorder.py because the way speechRecorder.py works is it stores all of the audio data in these massive deques and then writes the entire deque to a file once its done. Not very streaming-ish.
• Figure out how to use custom names. Some of the kids had to change their robot's names to common words like "table" because streaming transcription did not support custom terms. Information about customization is found here.

Something that could be nice is to have the ability to enter commands wether they be friendly text or literal python code into the code window while it is running the program. This way we could show the kids something like led on and the led turn on. Or we could do ledP.setColor("red") or whatever we wanted. Basically create an interactive console.

This could also combine the use of some other terminal techniques I have seen that allow for the screen to be divided into multiple "windows" so you could have the program running at the top and then have a command area below. This is advanced I know but I know people have taken the time to program some frameworks that allow us to do this.

PROMPT AREA

This is just a thought that we can explore.
The first version of adding this feature would be to just have 3 threads that output text onto a screen and then use screen to view them simultaneously and then later on it is just combining the output from the 3 different threads or processes or whatever that we end up using to do this.

Need to work on the audio controls in raspbian and figure out how to disable the GUI from overwriting the system changes. Also need to add the configuration settings to use the analog audio out as the default and use the usb mic as the default input.

Need to document all of the code that conforms with the lint standards and so it works well with pydoc

Need to work on the syntax that pulls commands from the returned conversation text. Currently using ~ACTION but I am thinking we need to do something with the tag syntax
<tj action="LED">
or
<tj action="LED" color="RED"><tj action="SPEAK">TURN LIGHT ON, SPEAK THIS, AND THEN TURN LIGHT OFF></tj></tj>

That way actions can be triggered in the code that last a duration but the programmer doesn't actually have to know the duration of the inner action. And if another action is called it always trumps, basically how we have the treads working right now.

need to add a license file with the rules or whatever for this repository

Right now we get zombie processes that seem to be a result of all of the different threads and processes that are being run. We need to program in an interpreter for keys like Ctrl+C and Ctrl+X so that they can actually kill the entire program and then there will be no need for the script that kills all of the existing python processes before running it again.

Need to create a setup script that works on the Pi and installs all of the dependencies required.