Your current environment <div class="snippet-clipboard-content notranslate posit

Normally there should be log similar to <div class="snippet-clipboard-content notr

dear <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

[Bug]: Unable to process request about vllm HOT 4 CLOSED

JPonsa commented on May 27, 2024

[Bug]: Unable to process request

from vllm.

Comments (4)

simon-mo commented on May 27, 2024

Normally there should be log similar to

INFO ] Started server process [1754022]
INFO ] Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

Please check whether the port is open?

from vllm.

JPonsa commented on May 27, 2024

dear @simon-mo . It is it just appears in another server log

--2024-04-17 21:17:33--  https://raw.githubusercontent.com/vllm-project/vllm/main/collect_env.py
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.110.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 24853 (24K) [text/plain]
Saving to: ‘collect_env.py’

     0K .......... .......... ....                            100% 19.6M=0.001s

2024-04-17 21:17:33 (19.6 MB/s) - ‘collect_env.py’ saved [24853/24853]

  File "collect_env.py", line 715
    print(msg, file=sys.stderr)
                   ^
SyntaxError: invalid syntax
INFO:     Started server process [83214]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

from vllm.

simon-mo commented on May 27, 2024

I see. The API server is supposed to be running forever because it is a live server. This is the intended behavior.

For your use case, consider running it in a background process?

https://stackoverflow.com/questions/44222883/run-a-shell-script-and-immediately-background-it-however-keep-the-ability-to-in

from vllm.

JPonsa commented on May 27, 2024

@simon-mo Thanks a lot! It seems to be working. Fingers crossed. I also had to include a sleep steps to give time to the server to spin up on the background.

from vllm.

Recommend Projects