LangChain-Chatchat shows RuntimeError: could not create a primitive about ipex-llm HOT 9 OPEN

Hi @MYaoBQ,

This may due to a bug from IPEX when iGPU and dGPU are both available, and especially when dGPU is not recognized as 'xpu:0'.

To solve this problem and run on ARC A770, you may try to disable UHD Graphics 770 on your machine first and have a try again :)

You could do that through

Device manager (设备管理器) -> Display adapter (显示适配器) -> UHD Graphics 770 - right click (右击) -> Disable device (禁用设备)

Please let us know for any further problems :)

from ipex-llm.

from ipex-llm.

Hi @MYaoBQ,

There are several things you could have a try:

1. Restart the machine after disabling iGPU
2. No need to set ONEAPI_DEVICE_SELECTOR any more as only one GPU is available :)

from ipex-llm.

from ipex-llm.

Hi @MYaoBQ ,

We have recently fixed this 通信错误 error, please kindly update to our latest code and have a try again :)

Besides, for Arc A770, here are the recommended configuations:

set USE_XETLA=OFF
set SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
set SYCL_CACHE_PERSISTENT=1
set BIGDL_QUANTIZE_KV_CACHE=1
set BIGDL_LLM_XMX_DISABLED=1

set no_proxy=localhost,127.0.0.1
set BIGDL_IMPORT_IPEX=0

python startup.py -a

Please let us know for any further problems :)

from ipex-llm.

Hi @MYaoBQ ,

We have recently fixed this 通信错误 error, please kindly update to our latest code and have a try again :)

Besides, for Arc A770, here are the recommended configuations:

set USE_XETLA=OFF
set SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
set SYCL_CACHE_PERSISTENT=1
set BIGDL_QUANTIZE_KV_CACHE=1
set BIGDL_LLM_XMX_DISABLED=1

set no_proxy=localhost,127.0.0.1
set BIGDL_IMPORT_IPEX=0

python startup.py -a

Please let us know for any further problems :)

Base on your reply, I try it on Arc dGPU, but get error.
set BIGDL_IMPORT_IPEX=0 means that IPEX won't be imported, right?

from ipex-llm.

Hi @violet17,

We have currently updated our Windows guide on Arc A-Series, it is recommended to follow our guide for Langchain-chatchat deployment on WIndows Arc A-Series.

Besides, set BIGDL_IMPORT_IPEX=0 means not automatically importing ipex, but we manually did that in our Langchain-chatchat support code.

Please let us know for any further problems :)

from ipex-llm.

Hi @Oscilloscope98 , thank you for your reply. I fix could not create a primitive with disable iGPU when using dGPU for inference instead of using set BIGDL_IMPORT_IPEX=0.

from ipex-llm.

Hi @Oscilloscope98 , thank you for your reply. I fix could not create a primitive with disable iGPU when using dGPU for inference instead of using set BIGDL_IMPORT_IPEX=0.

Hi @violet17,

I'm glad you solved the problem :) Just some clarification that set BIGDL_IMPORT_IPEX=0 is not for resolving the could not create a primitive issue. It is related to this issue.

Please let me know for any further problems :)

from ipex-llm.