Giter Club home page Giter Club logo

Comments (6)

pseudotensor avatar pseudotensor commented on August 16, 2024 1

I can import upper level thing but I don't have the other thing:

(h2ogpt) jon@pseudotensor:~/h2ogpt$ python
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import awq_inference_engine
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'awq_inference_engine'
>>> from awq import AutoAWQForCausalLM
>>> 

Probably the awq install instructions are out of date for windows, will check on it.

from h2ogpt.

pseudotensor avatar pseudotensor commented on August 16, 2024

I'm not an expert at .bat files. I used chatgpt to help me with .bat file stuff for h2oGPT, so I recommend doing same with your errors or seeing if someone else will respond.

Also, you didn't say what the issue is.

from h2ogpt.

Yusuf-Demiryurek avatar Yusuf-Demiryurek commented on August 16, 2024

I don't believe the issue lies with the .bat file itself. The .bat file functions similarly to the python generate.py command with parameters. The problem arises when attempting to run the AWQ model through the .bat file. Interestingly, the AWQ model runs perfectly fine when loaded from the UI, but H2OGPT doesn't start when using the .bat file.

Furthermore, if I use --base_model=TheBloke/zephyr-7B-beta-GGUF instead of --base_model=TheBloke/zephyr-7B-beta-AWQ in the .bat file, H2OGPT starts without any issues with the GGUF model loaded.

There appears to be a discrepancy in behavior between loading the model from the UI and using the --base_model input in the .bat file.

from h2ogpt.

pseudotensor avatar pseudotensor commented on August 16, 2024

Hi, please say what the issue actually is. You haven't said what the error or stack trace is etc.

from h2ogpt.

Yusuf-Demiryurek avatar Yusuf-Demiryurek commented on August 16, 2024

Sorry, I didn't know where to find the error.
After consulting GPT, I found the error in the h2ogpt_exception.log file:

Traceback (most recent call last):
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\win_run_app.py", line 150, in main
    _main()
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\win_run_app.py", line 136, in _main
    main_h2ogpt()
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\generate.py", line 12, in entrypoint_main
    H2O_Fire(main)
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\src\utils.py", line 65, in H2O_Fire
    fire.Fire(component=component, command=args)
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\fire\core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\fire\core.py", line 475, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\fire\core.py", line 691, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\src\gen.py", line 1977, in main
    model0, tokenizer0, device = get_model_retry(reward_type=False,
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\src\gen.py", line 2305, in get_model_retry
    model1, tokenizer1, device1 = get_model(**kwargs)
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\src\gen.py", line 2455, in get_model
    model_loader, tokenizer_loader, conditional_type = get_loaders(**loader_kwargs)
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\..\src\loaders.py", line 100, in get_loaders
    from awq import AutoAWQForCausalLM
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\awq\__init__.py", line 2, in <module>
    from awq.models.auto import AutoAWQForCausalLM
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\awq\models\__init__.py", line 1, in <module>
    from .mpt import MptAWQForCausalLM
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\awq\models\mpt.py", line 1, in <module>
    from .base import BaseAWQForCausalLM
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\awq\models\base.py", line 16, in <module>
    from awq.modules.linear import WQLinear_GEMM, WQLinear_GEMV
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\pkgs\awq\modules\linear.py", line 4, in <module>
    import awq_inference_engine  # with CUDA kernels
ImportError: DLL load failed while importing awq_inference_engine: Le module sp cifi  est introuvable.

from h2ogpt.

Yusuf-Demiryurek avatar Yusuf-Demiryurek commented on August 16, 2024

I got this:

Traceback (most recent call last):
  File "C:\Users\balas\AppData\Local\Programs\h2oGPT\Python\test.py", line 1, in <module>
    import awq_inference_engine
ImportError: DLL load failed while importing awq_inference_engine: Le module spécifié est introuvable.

I don't understand why it works when I load the model from UI and not in this way

from h2ogpt.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.