whisper_timestamped blocks from an URL in CLI into subprocess module about whisper-timestamped HOT 6 CLOSED

Hi,
Thank you for your help,
I hotfix my code.

I was waiting for stdout and stderr and p.Poll() was waiting for something from stdout because I was only reading from stderr.
To fix, I redirected the stderr to stdout.

BEFORE :

p = subprocess.Popen(['whisper_timestamped', url, '--model', 'tiny.en', '--language', 'en', '--device', device, '--fp16', str(settings.GPU), '--no_speech_threshold', str(no_speech_threshold), '--detect_disfluencies', str(True), '--vad', str(True), '--verbose', str(False)], text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
while p.poll() is None:
        line = p.stderr.readline()

AFTER :

p = subprocess.Popen(['whisper_timestamped', url, '--model', 'tiny.en', '--language', 'en', '--device', device, '--fp16', str(settings.GPU), '--no_speech_threshold', str(no_speech_threshold), '--detect_disfluencies', str(True), '--vad', str(True), '--verbose', str(False)], text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while p.poll() is None:
        line = p.stdout.readline()

Now everything works perfectly.
Best regards,
David

from whisper-timestamped.

Hmm I think relying on result.stderr.readline() to decide when to stop waiting is awkward.
I recommend you should use subprocess in another way.
Something like result.poll() / result.communicate() / result.wait()

from whisper-timestamped.

Hi,

I tested with poll() but the whisper timestamped process always gets blocked if it is a URL.
Here are 2 tests (a file and a URL)

result` = subprocess.Popen(['whisper_timestamped', url, '--model', 'tiny.en', '--language', 'en', '--device', device, '--fp16', str(settings.GPU), '--no_speech_threshold', str(no_speech_threshold), '--detect_disfluencies', str(True), '--vad', str(True), '--verbose', str(False)], text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    while result.poll() is None:
        line = result.stderr.readline()
        print(line)
        print('result.poll(): {}'.format(result.poll()))
        print('result.returncode: {}'.format(result.returncode))
                
    if result.returncode == 0:
        print('The process completed successfully.')
    else:
        print('The process returned an error with the code: ', `process.returncode)

### Test with url = '/mnt/bob/sintel-2048-stereo [00.01.51.541-00.02.24.041].mp4' => it works !!!

openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:14,224: WARNING/MainProcess] result.poll(): None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:14,225: WARNING/MainProcess] result.returncode: None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:14,225: WARNING/MainProcess] warnings.warn(
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:14,225: WARNING/MainProcess] result.poll(): None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:14,225: WARNING/MainProcess] result.returncode: None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:15,068: WARNING/MainProcess] Downloading: "https://github.com/snakers4/silero-vad/zipball/master" to /root/.cache/torch/hub/master.zip
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:15,068: WARNING/MainProcess] result.poll(): None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:15,068: WARNING/MainProcess] result.returncode: None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:19,529: WARNING/MainProcess] result.poll(): None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:19,529: WARNING/MainProcess] result.returncode: None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:22,205: WARNING/MainProcess] 0%| | 0/2700 [00:00<?, ?frames/s]
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:22,205: WARNING/MainProcess] result.poll(): None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:22,205: WARNING/MainProcess] result.returncode: None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:22,208: WARNING/MainProcess] 100%|██████████| 2700/2700 [00:02<00:00, 1009.27frames/s]
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:22,208: WARNING/MainProcess] result.poll(): None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:22,208: WARNING/MainProcess] result.returncode: None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:22,210: WARNING/MainProcess] 100%|██████████| 2700/2700 [00:02<00:00, 1009.13frames/s]
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:22,211: WARNING/MainProcess] result.poll(): None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:22,211: WARNING/MainProcess] result.returncode: None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:22,908: WARNING/MainProcess] result.poll(): 0
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:22,908: WARNING/MainProcess] result.returncode: 0
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:22,910: WARNING/MainProcess] The process completed successfully.
openai-whisper-api-celery_worker-1 | [2023-11-19 11:24:22,914: INFO/MainProcess] Task app.tasks.transcribe_v2[3c9b96b1-a043-4037-8a0b-c5d2d6a57560] succeeded in 31.483451699999932s: {'text': ' in a shed much more innocent blood. You're a fool for traveling alone so completely unprepared. You're lucky you're blessed to a flooring. Thank you. So, what brings you to the land of the gatekeepers? I'm searching for someone. Someone very dear. A kindred spirit. A dragon.', 'segments': [{'start': 0.07, 'end': 2.47, 'text': ' in a shed much more innocent blood.', 'confidence': 0.59, 'words': [...]}, {'start': 6.21, 'end': 8.75, 'text': ' You're a fool for traveling alone so completely unprepared.', 'confidence': 0.83, 'words': [...]}, {'start': 9.87, 'end': 11.53, 'text': ' You're lucky you're blessed to a flooring.', 'confidence': 0.72, 'words': [...]}, {'start': 13.51, 'end': 14.07, 'text': ' Thank you.', 'confidence': 0.99, 'words': [...]}, {'start': 17.45, 'end': 19.77, 'text': ' So, what brings you to the land of the gatekeepers?', 'confidence': 0.88, 'words': [...]}, {'start': 23.53, 'end': 24.73, 'text': ' I'm searching for someone.', 'confidence': 0.89, 'words': [...]}, {'start': , ...}]}

### Test with url = 'http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/WhatCarCanYouGetForAGrand.mp4' => it doesn't work because the processus is blocked

openai-whisper-api-celery_worker-1 | [2023-11-19 11:28:53,667: WARNING/MainProcess] result.poll(): None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:28:53,667: WARNING/MainProcess] result.returncode: None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:28:59,615: WARNING/MainProcess] 67%|██████▋ | 34000/50969 [01:16<00:39, 427.43frames/s]
openai-whisper-api-celery_worker-1 | [2023-11-19 11:28:59,615: WARNING/MainProcess] result.poll(): None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:28:59,616: WARNING/MainProcess] result.returncode: None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:05,608: WARNING/MainProcess] 72%|███████▏ | 36900/50969 [01:22<00:31, 444.16frames/s]
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:05,609: WARNING/MainProcess] result.poll(): None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:05,609: WARNING/MainProcess] result.returncode: None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:10,793: WARNING/MainProcess] 78%|███████▊ | 39800/50969 [01:28<00:24, 455.51frames/s]
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:10,793: WARNING/MainProcess] result.poll(): None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:10,793: WARNING/MainProcess] result.returncode: None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:17,639: WARNING/MainProcess] 84%|████████▍ | 42700/50969 [01:33<00:17, 482.64frames/s]
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:17,639: WARNING/MainProcess] result.poll(): None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:17,639: WARNING/MainProcess] result.returncode: None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:22,685: WARNING/MainProcess] 90%|████████▉ | 45696/50969 [01:40<00:11, 467.79frames/s]
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:22,685: WARNING/MainProcess] result.poll(): None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:22,685: WARNING/MainProcess] result.returncode: None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:26,791: WARNING/MainProcess] 95%|█████████▍| 48384/50969 [01:45<00:05, 484.52frames/s]
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:26,791: WARNING/MainProcess] result.poll(): None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:26,791: WARNING/MainProcess] result.returncode: None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:26,793: WARNING/MainProcess] 100%|██████████| 50969/50969 [01:49<00:00, 517.98frames/s]
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:26,794: WARNING/MainProcess] result.poll(): None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:26,794: WARNING/MainProcess] result.returncode: None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:26,797: WARNING/MainProcess] 100%|██████████| 50969/50969 [01:49<00:00, 463.56frames/s]
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:26,797: WARNING/MainProcess] result.poll(): None
openai-whisper-api-celery_worker-1 | [2023-11-19 11:29:26,797: WARNING/MainProcess] result.returncode: None

Thank you very much for your help.
Best regards,
David

from whisper-timestamped.

What is hanging is this:

line = result.stderr.readline()

I don't understand the reason.
Maybe have a look at https://stackoverflow.com/questions/12419198/python-subprocess-readlines-hangs

from whisper-timestamped.

Hi,
To remove doubt, I tested with whisper instead of whisper_timestamped and it works with the URL below.

It works with whisper module !!!

    url = 'http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/WhatCarCanYouGetForAGrand.mp4'
    result = subprocess.Popen(['whisper', url, '--model', 'tiny.en', '--language', 'en', '--device', device, '--fp16', str(settings.GPU), '--no_speech_threshold', str(no_speech_threshold), '--verbose', str(False)], text=True)
result.wait()
    
    if result.returncode == 0:
        print('The process completed successfully.')
    else:
        print('The process returned an error with the code: ', process.returncode)

It doesn't work with whisper_timestamped module

    url = 'http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/WhatCarCanYouGetForAGrand.mp4'
    result = subprocess.Popen(['whisper_timestamped', url, '--model', 'tiny.en', '--language', 'en', '--device', device, '--fp16', str(settings.GPU), '--no_speech_threshold', str(no_speech_threshold), '--verbose', str(False)], text=True)
result.wait()
    
    if result.returncode == 0:
        print('The process completed successfully.')
    else:
        print('The process returned an error with the code: ', process.returncode)

I suspect something is not closed properly

Than you very much for your help.
Best regards,
David

from whisper-timestamped.

Perfect. Well done, and thanks for the feedback.

from whisper-timestamped.