Comments (6)
I can import upper level thing but I don't have the other thing:
(h2ogpt) jon@pseudotensor:~/h2ogpt$ python
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import awq_inference_engine
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'awq_inference_engine'
>>> from awq import AutoAWQForCausalLM
>>>
Probably the awq install instructions are out of date for windows, will check on it.
from h2ogpt.
I'm not an expert at .bat files. I used chatgpt to help me with .bat file stuff for h2oGPT, so I recommend doing same with your errors or seeing if someone else will respond.
Also, you didn't say what the issue is.
from h2ogpt.
I don't believe the issue lies with the .bat file itself. The .bat file functions similarly to the python generate.py command with parameters. The problem arises when attempting to run the AWQ model through the .bat file. Interestingly, the AWQ model runs perfectly fine when loaded from the UI, but H2OGPT doesn't start when using the .bat file.
Furthermore, if I use --base_model=TheBloke/zephyr-7B-beta-GGUF instead of --base_model=TheBloke/zephyr-7B-beta-AWQ in the .bat file, H2OGPT starts without any issues with the GGUF model loaded.
There appears to be a discrepancy in behavior between loading the model from the UI and using the --base_model input in the .bat file.
from h2ogpt.
Hi, please say what the issue actually is. You haven't said what the error or stack trace is etc.
from h2ogpt.
Sorry, I didn't know where to find the error.
After consulting GPT, I found the error in the h2ogpt_exception.log file:
Traceback (most recent call last):
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\win_run_app.py", line 150, in main
_main()
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\win_run_app.py", line 136, in _main
main_h2ogpt()
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\generate.py", line 12, in entrypoint_main
H2O_Fire(main)
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\src\utils.py", line 65, in H2O_Fire
fire.Fire(component=component, command=args)
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\fire\core.py", line 141, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\fire\core.py", line 475, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\fire\core.py", line 691, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\src\gen.py", line 1977, in main
model0, tokenizer0, device = get_model_retry(reward_type=False,
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\src\gen.py", line 2305, in get_model_retry
model1, tokenizer1, device1 = get_model(**kwargs)
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\src\gen.py", line 2455, in get_model
model_loader, tokenizer_loader, conditional_type = get_loaders(**loader_kwargs)
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\..\src\loaders.py", line 100, in get_loaders
from awq import AutoAWQForCausalLM
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\awq\__init__.py", line 2, in <module>
from awq.models.auto import AutoAWQForCausalLM
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\awq\models\__init__.py", line 1, in <module>
from .mpt import MptAWQForCausalLM
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\awq\models\mpt.py", line 1, in <module>
from .base import BaseAWQForCausalLM
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\awq\models\base.py", line 16, in <module>
from awq.modules.linear import WQLinear_GEMM, WQLinear_GEMV
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\awq\modules\linear.py", line 4, in <module>
import awq_inference_engine # with CUDA kernels
ImportError: DLL load failed while importing awq_inference_engine: Le module sp cifi est introuvable.
from h2ogpt.
I got this:
Traceback (most recent call last):
File "C:\Users\balas\AppData\Local\Programs\h2oGPT\Python\test.py", line 1, in <module>
import awq_inference_engine
ImportError: DLL load failed while importing awq_inference_engine: Le module spécifié est introuvable.
I don't understand why it works when I load the model from UI and not in this way
from h2ogpt.
Related Issues (20)
- Mac OS automatic installation runs errors HOT 2
- about Add Doc to Chat HOT 1
- One Click Installers for MacOS not working on MacMini M2 HOT 3
- Attention sink error with h2oai/h2ogpt-4096-llama2-13b-chat HOT 1
- ImportError: libcudnn.so.8: cannot open shared object file: No such file or directory HOT 6
- Size of Tensor A must match size of Tensor B HOT 6
- auth related feature HOT 9
- Loading a Large model on Multiples GPU system HOT 12
- Permissions in VectorDB HOT 6
- Support for AWS Bedrock HOT 1
- vLLM GROQ issue HOT 1
- Mac OS auto installer doesn't work after manual uninstallation
- RuntimeError: An error occurred while downloading using `hf_transfer`. HOT 1
- python dependency module version tweaks HOT 1
- Rest API for inference locally HOT 5
- HuggingFaceM4/idefics2-8b as vision model
- How to delete content in user_paste HOT 2
- Can you make_db from documents stored on another (for example, PostgreSQL) HOT 2
- No way to save prompt/response pairs in a database?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from h2ogpt.