Windows manual install, some error and strange behaviour about h2ogpt HOT 6 CLOSED

Hi, the AWQ thing looks like its specific to AWQ. I don't have any good ideas w.r.t. that.

Note that GGUF and AWQ are independent. If you are finding AWQ sometimes works, probably some race/bug in AWQ. Please check their repo: https://github.com/casper-hansen/AutoAWQ and post an issue if you can just follow their basic instructions.

If you find their basic instructions work but h2oGPT does not, I can look again.

from h2ogpt.

I don't think set "PIP_EXTRA_INDEX_URL=https://download.pytorch.org/whl/cu118" is correct, it probably makes it use non-cuda torch and then since AWQ only supports cuda then it would fail, might explain your situation.

Closing for now until get evidence that not AWQ issue.

from h2ogpt.

I'm sure about:

F:\h2oGPT\installer_files\env\lib\site-packages\gradio\components\dropdown.py:231: UserWarning: The value passed into gr.Dropdown() is not in the list of choices. Please update the list of choices to include: or set allow_custom_value=True.
  warnings.warn(
Got prompt_type 1:

Can you explain exactly what you did to hit this?

from h2ogpt.

I'm sure about:

F:\h2oGPT\installer_files\env\lib\site-packages\gradio\components\dropdown.py:231: UserWarning: The value passed into gr.Dropdown() is not in the list of choices. Please update the list of choices to include: or set allow_custom_value=True.
  warnings.warn(
Got prompt_type 1:

Can you explain exactly what you did to hit this?

If I remember correctly I loaded and then unloaded, with the relative buttons, the same model

from h2ogpt.

I don't think set "PIP_EXTRA_INDEX_URL=https://download.pytorch.org/whl/cu118" is correct, it probably makes it use non-cuda torch and then since AWQ only supports cuda then it would fail, might explain your situation.

Closing for now until get evidence that not AWQ issue.

I mean in the readme for Windows it said:

from h2ogpt.

I'm sure about:

F:\h2oGPT\installer_files\env\lib\site-packages\gradio\components\dropdown.py:231: UserWarning: The value passed into gr.Dropdown() is not in the list of choices. Please update the list of choices to include: or set allow_custom_value=True.
  warnings.warn(
Got prompt_type 1:

Can you explain exactly what you did to hit this?

If I remember correctly I loaded and then unloaded, with the relative buttons, the same model

I noticed now that in the list of models the same one appears twice, but I don't see it downloaded in any folder.

from h2ogpt.