Realtime streaming example? about whispercpp.py HOT 3 OPEN

After having to work with stream.cpp, I see the microphone data is retrieved from SDL iteratively, in parts which are then transcribed by whisper.cpp.

The current implementation will need a way to grab so many seconds of microphone audio data. So it can be transcribed, then all the microphone data needs to be grabbed since the last time, before transcribing again etc.

from whispercpp.py.

I would like to get update for this topic

from whispercpp.py.

I haven't really explored the streaming functionality. However, from a brief look achieving this doesn't seem very straight forward. You'd need to look at whisper.cpp/stream/stream.cpp and attempt to reimplement in Cython some of the functionality which interfaces with SDL for the audio capture (could be attempted in pure Python, but it would be easier to deal with the low level data structures). Alternativley you could keep most of stream.cpp and make and expose a new single extern (i.e. transcribe_stream()) a black box, which cython can pick up on and expose to Python, where everything else is handled in c++. But you'd need to figure out a nice way to return the text back to Python.

I'm not quite sure how the example works, unfortunately I can't delve particularly deep right now, but a particular issue will be having the data sent back continuously.

A simple yet inelegant method is simply launching a subprocess instead comprising a modified stream.cpp and capturing stdout in Python.

from whispercpp.py.