Can't change model, httpx module not found about h2ogpt HOT 17 CLOSED

What does pip freeze show? Seems like install is incomplete.

I updated requirements.txt to ensure it is present in base dependencies.

from h2ogpt.

I go the same problem, exexcuting
python3.10 ./generate.py --base_model=TheBloke/Mixtral-8x7B-Instruct-v0.1-GPTQ --load_gptq=model --load_4bit=True --use_safetensors=True --prompt_type=mistral --save_dir='/workspace/save/' --use_gpu_id=False --score_model=None --max_max_new_tokens=2048 --max_new_tokens=1024

I got

Traceback (most recent call last):
File "/workspace/./generate.py", line 7, in
from src.gen import main
File "/workspace/src/gen.py", line 15, in
import httpx
ModuleNotFoundError: No module named 'httpx'

pip freeze show

anyio==4.2.0
cachetools==5.3.2
certifi==2019.11.28
chardet==3.0.4
dbus-python==1.2.16
distro-info==0.23+ubuntu1.1
exceptiongroup==1.2.0
h11==0.14.0
httpcore==0.17.3
httpx==0.24.1
idna==2.8
nvidia-ml-py==12.535.133
nvitop==1.3.2
pipx==0.12.3.1
psutil==5.9.7
PyGObject==3.36.0
python-apt==2.0.1+ubuntu0.20.4.1
requests==2.22.0
requests-unixsocket==0.2.0
six==1.14.0
sniffio==1.3.0
termcolor==2.4.0
typing-extensions==4.9.0
unattended-upgrades==0.1
urllib3==1.25.8

from h2ogpt.

It's there

https://github.com/h2oai/h2ogpt/blob/main/requirements.txt#L71

from h2ogpt.

Ad you can see from the pip freeze I have listed, it's installed in the exact required version.

from h2ogpt.

From your pip freeze it seems like you are showing inside of docker?

/workspace/./generate.py

Or not?

Can you even do:

python

then do:

import httpx

?

I suspect your pip ran in different environment than the generate.py uses.

from h2ogpt.

Yes it's running in a cloud docker image on an A6000. I have only access to it inside a docker image.
It's not clear to me what env generate.py is running on.
I was able to run it (not with Mixtral) a few days ago in this infrastructure.

from h2ogpt.

Yeah, I couldn't get this to work on either Windows or a Linux PC through docker-compose. Kept getting the same error listed above. I might try to just run it without docker. It's unfortunate, because I'm running like ~30 containers so I'm not super new to docker or anything. :/

from h2ogpt.

FYI we run docker compose with h2ogpt docker all the time. @achraf-mer any idea what could be wrong?

from h2ogpt.

I have overcomed this problem working as root inside the container, when I created a new user and used sudo I got always this problem. On the contrary of someguitarist I'm not a Docker expert so no idea why, maybe there is a way to make it work also with an user without root privileges.

from h2ogpt.

I use docker all the time myself, and the way to set the user is shown in the instructions.

https://github.com/h2oai/h2ogpt/blob/main/docs/README_DOCKER.md#run-h2ogpt-using-docker

@achraf-mer I presume these compose instructions are still valid:

https://github.com/h2oai/h2ogpt/blob/main/docs/README_DOCKER.md#docker-compose-setup--inference

from h2ogpt.

Sorry, just to be clear, I'm definitely no expert at all! I just run a lot of them!

Yeah, just to follow up I'm using the docker-compose instructions mentioned at the link below, not the regular docker setup instructions, and I'm getting the same error on Linux and Windows.

https://github.com/h2oai/h2ogpt/blob/main/docs/README_DOCKER.md#docker-compose-setup--inference

from h2ogpt.

it is possible the image is outdated, and not at the latest hash as we overwrote the tag 0.1.0 a while ago.
My suggestion would be:

1. set the docker runner to always pull, or if testing locally can docker pull gcr.io/vorvan/h2oai/h2ogpt-runtime:0.1.0-275, notice I used the tag with -275 to fore pulling the latest from a couple of hours ago.
2. use this PR: #1273 which will make sure docker compose always builds a new image when needed.

I have tried the latest image and I can't see the inconsistencies with said package, see below:

:~$ docker run --rm --entrypoint python3.10 gcr.io/vorvan/h2oai/h2ogpt-runtime:0.1.0-275 -c "import httpx;print(httpx.__file__)"
Unable to find image 'gcr.io/vorvan/h2oai/h2ogpt-runtime:0.1.0-275' locally
0.1.0-275: Pulling from vorvan/h2oai/h2ogpt-runtime
Digest: sha256:b81d30fca1d761bb2be3305624b3f6723ea7d8d8b9259fda08506964d4591d56
Status: Downloaded newer image for gcr.io/vorvan/h2oai/h2ogpt-runtime:0.1.0-275
/h2ogpt_conda/lib/python3.10/site-packages/httpx/__init__.py

If the issue persist after pulling the new image, or after building with new docker compose way, then likley related to using a different environment through some docker mount or PYTHONPATH setup perhaps, @davide445 @Someguitarist can you please confirm the above is good for you? if not then if you could share how you setup the docker runner I can help. Thanks!

from h2ogpt.

Hey, just to update, changing the docker-compose to gcr.io/vorvan/h2oai/h2ogpt-runtime:0.1.0-275 worked for me! Thanks for ya'lls help.

from h2ogpt.

Need to clarify: I'm running h2ogpt on vast.ai cloud instances, so I didn't have access to a server but to a pre-made docker image, I just pass the instance to run on this:
gcr.io/vorvan/h2oai/h2ogpt-runtime:latest
--gpus all
--shm-size=2g
-p 7860:7860
--rm --init
--network host
-v /etc/passwd:/etc/passwd:ro
-v /etc/group:/etc/group:ro
-u id -u:id -g
-v "${HOME}"/.cache:/workspace/.cache
-v "${HOME}"/save:/workspace/save
-v "${HOME}"/user_path:/workspace/user_path
-v "${HOME}"/db_dir_UserData:/workspace/db_dir_UserData
-v "${HOME}"/users:/workspace/users
-v "${HOME}"/db_nonusers:/workspace/db_nonusers
-v "${HOME}"/llamacpp_path:/workspace/llamacpp_path
Mybe one of these options is the origin of my problems in having the non-root user using a different env than root.

from h2ogpt.

@achraf-mer can confirm, but when you compose with :latest I think it doesn't by default download any new image. You have to docker pull first. :latest is just a tag, and otherwise refers to just the local tag.

from h2ogpt.

@achraf-mer can confirm, but when you compose with :latest I think it doesn't by default download any new image. You have to docker pull first. :latest is just a tag, and otherwise refers to just the local tag.

correct, @davide445 you'd need to pull first, or use the latest code which will build from the Dockerfile instead of pulling.

from h2ogpt.

let's assume that is the issue.

from h2ogpt.