A chat app that transcribes audio in real-time, streams back a response from a language model, and synthesizes this response as natural-sounding speech.
Home Page: https://modal.com/docs/guide/llm-voice-chat
License: MIT License
Hello,
I get the error "task exited with failure, status = exit status: 1" when I try to run modal serve src.app.
What could be the reason for this?
Hi there,
I am experiencing an error running the code with modal serve src.app. Here is the error message:
Building image im-uapgkrskiwyzvdznvrsjdwytfles
=> Step 0: FROM base
=> Step 1: RUN cd /FastChat && python download-model.py anon8231489123/vicuna-13b-GPTQ-4bit-128g
Traceback (most recent call last):
File "download-model.py", line 196, in <module>
links, sha256, is_lora = get_download_links_from_huggingface(model, branch)
File "download-model.py", line 130, in get_download_links_from_huggingface
fname = dict[i]['path']
KeyError: 0
Terminating task due to error: command "cd /FastChat && python download-model.py anon8231489123/vicuna-13b-GPTQ-4bit-128g" had exit status: 1
Could you please help me identify the root cause of this error and provide guidance on how to fix it?
Thank you
modal serve src.app
The cache for model files in Transformers v4.22.0 has been updated. Migrating your old cache. This is a one-time only operation. You can interrupt this and resume the migration later on by calling `transformers.utils.move_cache()`.
0it [00:00, ?it/s]0it [00:00, ?it/s]
Downloading (…)okenizer_config.json: 0%| | 0.00/727 [00:00<?, ?B/s]Downloading (…)okenizer_config.json: 100%|██████████| 727/727 [00:00<00:00, 196kB/s]
Downloading tokenizer.model: 0%| | 0.00/500k [00:00<?, ?B/s]Downloading tokenizer.model: 100%|██████████| 500k/500k [00:00<00:00, 94.7MB/s]
Downloading (…)cial_tokens_map.json: 0%| | 0.00/411 [00:00<?, ?B/s]Downloading (…)cial_tokens_map.json: 100%|██████████| 411/411 [00:00<00:00, 511kB/s]
Loading GPTQ quantized model...
Could not find the quantized model in .pt or .safetensors format, exiting...
Traceback (most recent call last):
File "/pkg/modal/_container_entrypoint.py", line 301, in handle_user_exception
yield
File "/pkg/modal/_container_entrypoint.py", line 443, in call_function_async
obj.__enter__()
File "/root/src/llm_vicuna.py", line 79, in __enter__
model = load_quantized(MODEL_NAME)
File "/FastChat/fastchat/serve/load_gptq_model.py", line 60, in load_quantized
exit()
File "/usr/lib/python3.8/_sitebuiltins.py", line 26, in __call__
raise SystemExit(code)
SystemExit: None
Runner failed with exception: SystemExit(None)
2023-05-30T15:45:37+0000 User exception caught, exiting
downloaded zip, ran pip install modal-client, then modal serve src.app, and got the following error:
modal : The term 'modal' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ modal serve src.app
+ ~~~~~
+ CategoryInfo : ObjectNotFound: (modal:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
This sounds like a cool app, but unfortunately the demo doesn't seem to be working: https://modal-labs--quillman-web.modal.run/
Thanks!
For those who don't have a microphone.
And would that give any cost savings with Modal?
https://towardsdatascience.com/speech-to-text-with-openais-whisper-53d5cea9005e
Loading the example image provided in llm_vicuna.py throws a huge warning:
==========
== CUDA ==
==========
CUDA Version 11.7.0
Container image Copyright (c) 2016-2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license
A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.
*************************
** DEPRECATION NOTICE! **
*************************
THIS IMAGE IS DEPRECATED and is scheduled for DELETION.
https://gitlab.com/nvidia/container-images/cuda/blob/master/doc/support-policy.md
I've tried updating it to later version and I found myself in the unfortunate rabbit hole of "pytorch has been compiled with the Cuda 11.7 version".
I raise this issue hoping someone at their discretion could update providing an example of how it should be done. Thank you. :)
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.