Medium models missing? about whisperkit HOT 4 CLOSED

Thanks for the note @coder543! medium models were hitting an edge case with the Neural Engine that we triaged away for now. Technically, you can still use https://github.com/argmaxinc/whisperkittools to prepare the medium and medium.en model assets and use them with cpuAndGPU compute units without issues. We decided not to fork availability of models across different compute units and preserve a non-leaky abstraction for seamless switching.

from whisperkit.

Gotcha, that is unfortunate, since in my extensive testing of other Whisper apps on iPhone, the Medium model is the best one that can realistically run in real time over long durations. But, small is pretty good too, I guess!

from whisperkit.

@coder543 Have you noticed large models being too slow? Would be great to get an example audio/video where it falls back in streaming mode on iPhone 12+. We are always looking to improve based on feedback and we can follow up when we improve performance.

from whisperkit.

On the 15 Pro Max that I have, the large models run at an RTF of slightly greater than 1, and they’re just slow in general. The medium models are half the size, so they are just about perfect. When I’ve tested things more in Hello Transcribe over the months, the large models are tolerable and seem to barely keep up with real time… but I prefer the balance that medium provides, if I’m running a model on my phone. (On a powerful desktop, the large models are great.)

I didn’t spend too much time trying the distil models, but I’ve had mixed feeling about the accuracy of those models in past testing.

from whisperkit.