ValueError: Could not fetch config for http://0.0.0.0:40000 - Using LLaVa-v1.6 about h2ogpt HOT 5 CLOSED

Seems you didn't do the "Run server:" step? i.e.

Run server:
```bash
pip install gradio==4.17.0
python -m llava.serve.gradio_web_server --controller http://localhost:$server_port --model-list-mode once

i.e. h2oGPT talks to custom llava web server with API exposed that takes to controller that talks to workers.

Will close for now, feel free to ask new questions.

from h2ogpt.

Oh, ok, I will try that! Thank you for such a fast response!

from h2ogpt.

This absolutely worked! If anybody runs into this issue in the future, in a new terminal, I ran:

pip3 install gradio==4.17.0
python -m llava.serve.gradio_web_server --controller http://localhost:8080 --model-list-mode once

The gradio_web_server ran on local URL http://0.0.0.0:7860, so the final command became:

python generate.py --score_model=None --llava_model= http://0.0.0.0:7860 --base_model=liuhaotian/llava-v1.6-vicuna-13b --inference_server=http://0.0.0.0:7860 --prompt_type=plain

from h2ogpt.

Take note of other things in that FAQ section, e.g.:

When launching LLaVa, if you want the server and worker to work with a remote gradio, then replace `localhost` with the IP of the server.

Also yes, by default the llava gradio runs on 7860. If that's an issue because h2oGPT runs on 7860, then just change one of them. I'll add note to FAQ

from h2ogpt.

I changed the h2oGPT port to 8000 with:

export GRADIO_SERVER_PORT=8000

from h2ogpt.