无法使用“加载知识库”功能。 about chinese-langchain HOT 4 OPEN

已修复

config = LangChainCFG()
application = LangChainApplication(config)

application.source_service.init_source_vector()

from chinese-langchain.

https://huggingface.co/spaces/ChallengeHub/Chinese-LangChain
在线demo也是加载失败

from chinese-langchain.

我也是

from chinese-langchain.

python app.py
No compiled kernel found.
Compiling kernels : /root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b-int4-qe/977d9df4cfae6b7a756e07698483872c5c070eee/quantization_kernels_parallel.c
Compiling gcc -O3 -fPIC -pthread -fopenmp -std=c99 /root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b-int4-qe/977d9df4cfae6b7a756e07698483872c5c070eee/quantization_kernels_parallel.c -shared -o /root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b-int4-qe/977d9df4cfae6b7a756e07698483872c5c070eee/quantization_kernels_parallel.so
Kernels compiled : /root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b-int4-qe/977d9df4cfae6b7a756e07698483872c5c070eee/quantization_kernels_parallel.so
Load kernel : /root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b-int4-qe/977d9df4cfae6b7a756e07698483872c5c070eee/quantization_kernels_parallel.so
Setting CPU quantization kernel threads to 16
Using quantization cache
Applying quantization to glm layers
No sentence-transformers model found with name /root/.cache/torch/sentence_transformers/GanymedeNil_text2vec-large-chinese. Creating a new one with MEAN pooling.
姚明.txt
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /root/Chinese-LangChain/app.py:25 in │
│ │
│ 22 │
│ 23 config = LangChainCFG() │
│ 24 application = LangChainApplication(config) │
│ ❱ 25 application.source_service.init_source_vector() │
│ 26 │
│ 27 │
│ 28 def get_file_list(): │
│ │
│ /root/Chinese-LangChain/clc/source_service.py:39 in init_source_vector │
│ │
│ 36 │ │ │ if doc.endswith('.txt'): │
│ 37 │ │ │ │ print(doc) │
│ 38 │ │ │ │ loader = UnstructuredFileLoader(f'{self.docs_path}/{doc}', mode="element │
│ ❱ 39 │ │ │ │ doc = loader.load() │
│ 40 │ │ │ │ docs.extend(doc) │
│ 41 │ │ self.vector_store = FAISS.from_documents(docs, self.embeddings) │
│ 42 │ │ self.vector_store.save_local(self.vector_store_path) │
│ │
│ /root/miniconda3/lib/python3.8/site-packages/langchain/document_loaders/unstructured.py:61 in │
│ load │
│ │
│ 58 │ │
│ 59 │ def load(self) -> List[Document]: │
│ 60 │ │ """Load file.""" │
│ ❱ 61 │ │ elements = self._get_elements() │
│ 62 │ │ if self.mode == "elements": │
│ 63 │ │ │ docs: List[Document] = list() │
│ 64 │ │ │ for element in elements: │
│ │
│ /root/miniconda3/lib/python3.8/site-packages/langchain/document_loaders/unstructured.py:93 in │
│ _get_elements │
│ │
│ 90 │ │ super().init(mode=mode, **unstructured_kwargs) │
│ 91 │ │
│ 92 │ def _get_elements(self) -> List: │
│ ❱ 93 │ │ from unstructured.partition.auto import partition │
│ 94 │ │ │
│ 95 │ │ return partition(filename=self.file_path, **self.unstructured_kwargs) │
│ 96 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ModuleNotFoundError: No module named 'unstructured.partition.auto'
root@autodl-container-ebe411a150-49ec68c6:/Chinese-LangChain# python app.py
No compiled kernel found.
Compiling kernels : /root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b-int4-qe/977d9df4cfae6b7a756e07698483872c5c070eee/quantization_kernels_parallel.c
Compiling gcc -O3 -fPIC -pthread -fopenmp -std=c99 /root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b-int4-qe/977d9df4cfae6b7a756e07698483872c5c070eee/quantization_kernels_parallel.c -shared -o /root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b-int4-qe/977d9df4cfae6b7a756e07698483872c5c070eee/quantization_kernels_parallel.so
Kernels compiled : /root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b-int4-qe/977d9df4cfae6b7a756e07698483872c5c070eee/quantization_kernels_parallel.so
Load kernel : /root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b-int4-qe/977d9df4cfae6b7a756e07698483872c5c070eee/quantization_kernels_parallel.so
Setting CPU quantization kernel threads to 16
Using quantization cache
Applying quantization to glm layers
No sentence-transformers model found with name /root/.cache/torch/sentence_transformers/GanymedeNil_text2vec-large-chinese. Creating a new one with MEAN pooling.
王治郅.txt
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /root/Chinese-LangChain/app.py:36 in │
│ │
│ 33 │
│ 34 │
│ 35 file_list = get_file_list() │
│ ❱ 36 application.source_service.init_source_vector() │
│ 37 │
│ 38 def upload_file(file): │
│ 39 │ if not os.path.exists("docs"): │
│ │
│ /root/Chinese-LangChain/clc/source_service.py:39 in init_source_vector │
│ │
│ 36 │ │ │ if doc.endswith('.txt'): │
│ 37 │ │ │ │ print(doc) │
│ 38 │ │ │ │ loader = UnstructuredFileLoader(f'{self.docs_path}/{doc}', mode="element │
│ ❱ 39 │ │ │ │ doc = loader.load() │
│ 40 │ │ │ │ docs.extend(doc) │
│ 41 │ │ self.vector_store = FAISS.from_documents(docs, self.embeddings) │
│ 42 │ │ self.vector_store.save_local(self.vector_store_path) │
│ │
│ /root/miniconda3/lib/python3.8/site-packages/langchain/document_loaders/unstructured.py:61 in │
│ load │
│ │
│ 58 │ │
│ 59 │ def load(self) -> List[Document]: │
│ 60 │ │ """Load file.""" │
│ ❱ 61 │ │ elements = self._get_elements() │
│ 62 │ │ if self.mode == "elements": │
│ 63 │ │ │ docs: List[Document] = list() │
│ 64 │ │ │ for element in elements: │
│ │
│ /root/miniconda3/lib/python3.8/site-packages/langchain/document_loaders/unstructured.py:93 in │
│ _get_elements │
│ │
│ 90 │ │ super().init(mode=mode, **unstructured_kwargs) │
│ 91 │ │
│ 92 │ def _get_elements(self) -> List: │
│ ❱ 93 │ │ from unstructured.partition.auto import partition │
│ 94 │ │ │
│ 95 │ │ return partition(filename=self.file_path, **self.unstructured_kwargs) │
│ 96 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ModuleNotFoundError: No module named 'unstructured.partition.auto'
root@autodl-container-ebe411a150-49ec68c6:/Chinese-LangChain#

from chinese-langchain.