imartinez / privategpt Goto Github PK

View Code? Open in Web Editor NEW

52.1K 448.0 7.0K 1.99 MB

Interact with your documents using the power of GPT, 100% privately, no data leaks

Home Page: https://docs.privategpt.dev

License: Apache License 2.0

Python 76.82% Makefile 1.11% MDX 22.08%

privategpt's People

Contributors

Stargazers

Watchers

Forkers

xsn-cloud xjaroo mkinney isichan0501 stipot jeremypeters dumpmemory 0xking-sol xayaraj zkl94 apollohuang1 patentgpt alejandrofdzllorente williamdeve techthiyanes hbyido dhivehi youmuyou homer-hq muranava allthingsllm wakamex herrera-luis fanjingang christophebe yathishram ukaserge williamtran29 hang-9 markedmondson1234 santoshdahale aocsa hackerhoursacad vicktor hijaz garrethpxy tuantranf vonwooding miscarriage87 gwd666 anservat mvreri aiyenigbacharles the-question saik0s pokapro unbanksytv thinkifyai celerio-sg alberto-codes petercao ahmed-ali cedric-anamorphik evdcush edskamor kelkalot simbaninja917 anojangra ai-ld game-challenge ccaiccie nwajdee karnauskas cr0kz soon14 asdlei99 hhy5277 ziqian2023 lpstart koolob buttercannfly goswamig tngamemo longjingaa meetpateltech drgonzalomora ozgurkk maria1991sh byblonomikon felixdollack mishuirestik kola92 bitelchux beanmakerturnipfarmer roywikan sayrex173 julian618 xiaolie188 cheraphp gmh5225 showjim stivenhacker ashtrindade deepsea-l antdke 0xken wolf688688 tyldev ai-imitation jaedukseo

privategpt's Issues

volume of text?

What kind of corpus of text is reasonable to bring in, do you think? A few megabytes, a few gigabytes or perhaps more?

Longer output issue??

When I type a question, I get a lot of context output (based on the custom document I trained) and very short responses. I added return_source_documents=False to privateGPT.py:

qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever, return_source_documents=False)

but that still prints the source documents. I also increased the tokens to 4096 but the question I ask gets trimmed down.

Do you have any suggestions on settings I can apply to print longer responses? I don't know if the source documents being printed uses up Tokens?

Not sure what's happening here after the latest update!

~/dev/privateGPT$ python ingest.py

Traceback (most recent call last):
  File "/home/sharan/dev/privateGPT/ingest.py", line 7, in <module>
    from constants import CHROMA_SETTINGS
  File "/home/sharan/dev/privateGPT/constants.py", line 11, in <module>
    CHROMA_SETTINGS = Settings(
  File "pydantic/env_settings.py", line 39, in pydantic.env_settings.BaseSettings.__init__
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for Settings
persist_directory
  none is not an allowed value (type=type_error.none.not_allowed)

Cannot ingest Chinese text file

Hi,

I've tried the following command:
python ingest someTChinese.txt

However, it will return some error as shown below, any suggestion?
Tried to modify the parameters (chunk_size=500, chunk_overlap=50) and nothing worked here.

llama.cpp: loading model from ./models/ggml-model-q4_0.bin
llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this
llama_model_load_internal: format     = 'ggml' (old version with low tokenizer quality and no mmap support)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 512
llama_model_load_internal: n_embd     = 4096
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 32
llama_model_load_internal: n_layer    = 32
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: ftype      = 2 (mostly Q4_0)
llama_model_load_internal: n_ff       = 11008
llama_model_load_internal: n_parts    = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 4113748.20 KB
llama_model_load_internal: mem required  = 5809.33 MB (+ 2052.00 MB per state)
...................................................................................................
.
llama_init_from_file: kv self size  =  512.00 MB
AVX = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 1 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | VSX = 0 | 
Using embedded DuckDB with persistence: data will be stored in: db
llama_tokenize: too many tokens
Traceback (most recent call last):
  File "/Users/jmosx/devp/allAboutGPT/privateGPT/ingest.py", line 22, in <module>
    main()
  File "/Users/jmosx/devp/allAboutGPT/privateGPT/ingest.py", line 17, in main
    db = Chroma.from_documents(texts, llama, persist_directory=persist_directory)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jmosx/miniconda3/envs/privateGPT/lib/python3.11/site-packages/langchain/vectorstores/chroma.py", line 412, in from_documents
    return cls.from_texts(
           ^^^^^^^^^^^^^^^
  File "/Users/jmosx/miniconda3/envs/privateGPT/lib/python3.11/site-packages/langchain/vectorstores/chroma.py", line 380, in from_texts
    chroma_collection.add_texts(texts=texts, metadatas=metadatas, ids=ids)
  File "/Users/jmosx/miniconda3/envs/privateGPT/lib/python3.11/site-packages/langchain/vectorstores/chroma.py", line 158, in add_texts
    embeddings = self._embedding_function.embed_documents(list(texts))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jmosx/miniconda3/envs/privateGPT/lib/python3.11/site-packages/langchain/embeddings/llamacpp.py", line 111, in embed_documents
    embeddings = [self.client.embed(text) for text in texts]
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jmosx/miniconda3/envs/privateGPT/lib/python3.11/site-packages/langchain/embeddings/llamacpp.py", line 111, in <listcomp>
    embeddings = [self.client.embed(text) for text in texts]
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jmosx/miniconda3/envs/privateGPT/lib/python3.11/site-packages/llama_cpp/llama.py", line 514, in embed
    return list(map(float, self.create_embedding(input)["data"][0]["embedding"]))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jmosx/miniconda3/envs/privateGPT/lib/python3.11/site-packages/llama_cpp/llama.py", line 478, in create_embedding
    tokens = self.tokenize(input.encode("utf-8"))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jmosx/miniconda3/envs/privateGPT/lib/python3.11/site-packages/llama_cpp/llama.py", line 200, in tokenize
    raise RuntimeError(f'Failed to tokenize: text="{text}" n_tokens={n_tokens}')
RuntimeError: Failed to tokenize: text="b'\xe7\x8b\x82\xe5\xaf\x86\xe8\x88\x87\xe7\x9c\x9f\xe5\xaf\x86\xe7\xac\xac\xe4\xb8\x80\xe8\xbc\xaf\n\n\xe3\x80\x80\n\n\xe5\xb9\xb3\xe5\xaf\xa6\xe5\xb1\x85\xe5\xa3\xab\xe8\x91\x97\n\n \xe6\x9c\x89\xe5\xb8\xab\xe4\xba\x91\xef\xbc\x9a\n\n   \xe3\x80\x8c\xe5\xaf\x86\xe5\xae\x97\xe6\x98\xaf\xe4\xb8\x80\xe5\x80\x8b\xe9\x87\x91\xe5\x89\x9b\xe9\x91\xbd\xe5\xa4\x96\xe5\x9c\x8d\xe6\x93\xba\xe6\xbb\xbf\xe4\xba\x86\xe9\x8d\x8d\xe9\x87\x91\xe5\x9e\x83\xe5\x9c\xbe\xe7\x9a\x84\xe5\xae\x97\xe6\x95\x99\xe3\x80\x82\xe3\x80\x8d\n\n\xe3\x80\x80\n\n \xe5\xb9\xb3\xe5\xaf\xa6\xe6\x96\xbc\xe6\xad\xa4\xe8\xa8\x80\xe5\xbe\x8c\xe6\x9b\xb4\xe5\x8a\xa0\xe8\xa8\xbb\xe8\x85\xb3\xef\xbc\x9a\n\n   \xe3\x80\x8c\xe9\x82\xa3\xe9\xa1\x86\xe9\x87\x91\xe5\x85\x89\xe9\x96\x83\xe9\x96\x83\xe7\x9a\x84\xe9\x91\xbd\xe7\x9f\xb3\xef\xbc\x8c\xe5\x8d\xbb\xe6\x98\xaf\xe7\x8e\xbb\xe7\x92\x83\xe6\x89\x93\xe7\xa3\xa8\xe8\x80\x8c\xe6\x88\x90\xef\xbc\x8c\xe4\xb8\x8d\xe5\xa0\xaa\xe6\xaa\xa2\xe9\xa9\x97\xe3\x80\x82\xe3\x80\x8d\n\n\xe3\x80\x80\n\n\xe8\xb7\x8b\n\n\xe6\x95\xb8\xe5\x8d\x81\xe5\xb9\xb4\xe4\xbe\x86\xef\xbc\x8c\xe8\xa5\xbf\xe8\x97\x8f\xe5\xaf\x86\xe5\xae\x97\xe7\xb3\xbb\xe7\xb5\xb1\xe8\xab\xb8\xe6\xb3\x95\xe7\x8e\x8b\xe8\x88\x87\xe8\xab\xb8\xe4\xb8\x8a\xe5\xb8\xab\xef\xbc\x8c\xe6\x82\x89\xe7\x9a\x86\xe5\x92\x90\xe5\x9b\x91\xe8\xab\xb8\xe5\xbc\x9f\xe5\xad\x90\xef\xbc\x9a\xe3\x80\x8c\xe8\x8b\xa5\xe6\x9c\x89\xe5\xa4\x96\xe4\xba\xba\xe8\xa9\xa2\xe5\x95\x8f\xe5\xaf\x86\xe5\xae\x97\xe6\x98\xaf\xe5\x90\xa6\xe6\x9c\x89\xe9\x9b\x99\xe8\xba\xab\xe4\xbf\xae\xe6\xb3\x95\xef\xbc\x9f\xe6\x87\x89\xe4\xb8\x80\xe5\xbe\x8b\xe7\xad\x94\xe5\xbe\xa9\xef\xbc\x9a\xe3\x80\x8e\xe5\x8f\xa4\xe6\x99\x82\xe5\xaf\x86\xe5\xae\x97\xe6\x9c\x89\xe9\x9b\x99\xe8\xba\xab\xe6\xb3\x95\xef\xbc\x8c\xe4\xbd\x86\xe7\x8f\xbe\xe5\x9c\xa8\xe5\xb7\xb2\xe7\xb6\x93\xe5\xbb\xa2\xe6\xa3\x84\xe4\xb8\x8d\xe7\x94\xa8\xe3\x80\x82\xe7\x8f\xbe\xe5\x9c\xa8\xe4\xb8\x8a\xe5\xb8\xab\xe9\x83\xbd\xe5\x9a\xb4\xe7\xa6\x81\xe5\xbc\x9f\xe5\xad\x90\xe5\x80\x91\xe4\xbf\xae\xe5\xad\xb8\xe9\x9b\x99\xe8\xba\xab\xe6\xb3\x95\xef\xbc\x8c\xe6\x89\x80\xe4\xbb\xa5\xe7\x8f\xbe\xe5\x9c\xa8\xe7\x9a\x84\xe5\xaf\x86\xe5\xae\x97\xe5\xb7\xb2\xe7\xb6\x93\xe6\xb2\x92\xe6\x9c\x89\xe4\xba\xba\xe5\xbc\x98\xe5\x82\xb3\xe5\x8f\x8a\xe4\xbf\xae\xe5\xad\xb8\xe9\x9b\x99\xe8\xba\xab\xe6\xb3\x95\xe4\xba\x86\xe3\x80\x82\xe3\x80\x8f\xe3\x80\x8d\n\n\xe7\x84\xb6\xe8\x80\x8c\xe5\xaf\xa6\xe9\x9a\x9b\xe4\xb8\x8a\xef\xbc\x8c\xe8\xa5\xbf\xe8\x97\x8f\xe5\xaf\x86\xe5\xae\x97\xe4\xbb\x8d\xe6\x9c\x89\xe7\x94\x9a\xe5\xa4\x9a\xe4\xb8\x8a\xe5\xb8\xab\xe7\xb9\xbc\xe7\xba\x8c\xe5\xbc\x98\xe5\x82\xb3\xe9\x9b\x99\xe8\xba\xab\xe6\xb3\x95\xef\xbc\x8c\xe4\xb8\xa6\xe9\x9d\x9e\xe6\x9c\xaa\xe5\x82\xb3\xef\xbc\x8c\xe4\xba\xa6\xe9\x9d\x9e\xe7\xa6\x81\xe5\x82\xb3\xef\xbc\x8c\xe8\x80\x8c\xe6\x98\xaf\xe7\xb9\xbc\xe7\xba\x8c\xe5\x9c\xa8\xe7\x89\xa9\xe8\x89\xb2\xe9\x81\xa9\xe5\x90\x88\xe4\xbf\xae\xe9\x9b\x99\xe8\xba\xab\xe6\xb3\x95\xe4\xb9\x8b\xe5\xbc\x9f\xe5\xad\x90\xef\xbc\x8c\xe7\x82\xba\xe5\xbd\xbc\xe7\xad\x89\xe8\xab\xb8\xe4\xba\xba\xe6\x9a\x97\xe4\xb8\xad\xe4\xbd\x9c\xe7\xa7\x98\xe5\xaf\x86\xe7\x81\x8c\xe9\xa0\x82\xef\xbc\x8c\xe4\xbb\xa4\xe5\x85\xb6\xe6\x88\x90\xe7\x82\xba\xe5\x8b\x87\xe7\x88\xb6\xe8\x88\x87\xe7\xa9\xba\xe8\xa1\x8c\xe6\xaf\x8d\xef\xbc\x8c\xe7\x84\xb6\xe5\xbe\x8c\xe5\x90\x88\xe4\xbf\xae\xe9\x9b\x99\xe8\xba\xab\xe6\xb3\x95\xe3\x80\x82\xe4\xbb\x8a\xe6\x99\x82\xe5\xaf\x86\xe5\xae\x97\xe5\xa6\x82\xe6\x98\xaf\xe7\xb9\xbc\xe7\xba\x8c\xe6\x9a\x97\xe4\xb8\xad\xe5\xbc\x98\xe5\x82\xb3\xe9\x9b\x99\xe8\xba\xab\xe6\xb3\x95\xef\xbc\x8c\xe4\xbb\xa5\xe5\xbb\xb6\xe7\xba\x8c\xe5\x85\xb6\xe5\xaf\x86\xe6\xb3\x95\xe4\xb9\x8b\xe5\x91\xbd\xe8\x84\x88\xef\xbc\x9b\xe7\x9d\xbd\xe4\xba\x8e\xe9\x9b\xbb\xe8\xa6\x96\xe6\x96\xb0\xe8\x81\x9e\xe4\xb9\x8b\xe5\xa0\xb1\xe5\xb0\x8e\xe9\x99\xb3\xe5\xb1\xa5\xe5\xae\x89\xe5\x85\xac\xe5\xad\x90\xe9\x99\xb3\xe5\xae\x87\xe5\xbb\xb7\xef\xbc\x8c\xe6\xb1\x82\xe8\x97\x8f\xe5\xa5\xb3\xe7\x82\xba\xe5\x85\xb6\xe7\xa9\xba\xe8\xa1\x8c\xe6\xaf\x8d\xe7\xad\x89\xe8\xa8\x80\xef\xbc\x8c\xe4\xba\xa6\xe5\x8f\xaf\xe7\x9f\xa5\xe7\x9f\xa3\xe3\x80\x82\xe6\x98\xaf\xe6\x95\x85\xe5\xaf\x86\xe5\xae\x97\xe5\xbc\x9f\xe5\xad\x90\xe8\x88\x87\xe4\xb8\x8a\xe5\xb8\xab\xe7\xad\x89\xe4\xba\xba\xef\xbc\x8c\xe9\x9b\x96\xe7\x84\xb6\xe5\xb0\x8d\xe5\xa4\x96\xe5\x8f\xa3\xe5\xbe\x91\xe4\xb8\x80\xe8\x87\xb4\xef\xbc\x8c\xe6\x82\x89\xe7\x9a\x86\xe5\x80\xa1\xe8\xa8\x80\xef\xbc\x9a\xe3\x80\x8c\xe5\xaf\x86\xe5\xae\x97\xe7\x8f\xbe\xe5\x9c\xa8\xe5\xb7\xb2\xe6\x8d\xa8\xe6\xa3\x84\xe9\x9b\x99\xe8\xba\xab\xe6\xb3\x95\xef\xbc\x8c\xe4\xbb\x8a\xe5\xb7\xb2\xe7\x84\xa1\xe4\xba\xba\xe5\xbc\x98\xe5\x82\xb3\xe6\x88\x96\xe4\xbf\xae\xe5\xad\xb8\xe9\x9b\x99\xe8\xba\xab\xe6\xb3\x95\xe3\x80\x82\xe3\x80\x8d\xe5\x85\xb6\xe5\xaf\xa6\xe6\x98\xaf\xe6\xac\xba\xe7\x9e\x9e\xe7\xa4\xbe\xe6\x9c\x83\xe4\xb9\x8b\xe8\xa8\x80\xef\xbc\x8c\xe4\xbb\xa5\xe6\xad\xa4\xe4\xbb\xa4\xe7\xa4\xbe\xe6\x9c\x83\xe5\xa4\xa7\xe7\x9c\xbe\xe5\xb0\x8d\xe5\xaf\x86\xe5\xae\x97\xe4\xb8\x8d\xe8\x87\xb4\xe5\x8a\xa0\xe4\xbb\xa5\xe5\xa4\xaa\xe5\xa4\x9a\xe4\xb9\x8b\xe6\xb3\xa8\xe6\x84\x8f\xef\xbc\x8c\xe4\xbb\xa5\xe4\xbe\xbf\xe7\xb9\xbc\xe7\xba\x8c\xe6\x9a\x97\xe4\xb8\xad\xe5\xbc\x98\xe5\x82\xb3\xe9\x9b\x99\xe8\xba\xab\xe6\xb3\x95\xe3\x80\x82'" n_tokens=-723

llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this

feature request: ingestion .md files and automation for ingestion of all files in a folder and its subfolders

gpt_tokenize: unknown token '?'

gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
gpt_tokenize: unknown token '?'
[1] 32658 killed python3 privateGPT.py

TypeError: Model.generate() got an unexpected keyword argument 'new_text_callback'

Every time I run a query I get this output.

Any way can get GPU work?

Can anyone suggest how to make GPU work with this project?

Sending data to posthog when trying to run offline

When running this in an offline container i noticed that it's trying to send requests to app.posthog.com

Enter a query: [2023-05-11 09:04:09,252] {_common.py:105} INFO - Backing off send_request(...) for 0.8s (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='app.posthog.com', port=443): Max retries exceeded with url: /batch/ (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f8150e6b190>: Failed to resolve 'app.posthog.com' ([Errno -3] Temporary failure in name resolution)")))

Whats going on here?

spend too much time waiting for the answer?any method can speed up?

each time I ask the program.it will take me nearly 5-8minites to give me the simple answer.and the text I use is just the demo.So a simple demo need so much time to give me the answer??? Any method can provide to speed up the program?

pip requirement install step fails on macOS

It looks like ggml is not transitively pulled by pip. see extract of log :

  × Building wheel for pygptj (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [115 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.macosx-13-arm64-cpython-311
      creating build/lib.macosx-13-arm64-cpython-311/pygptj
      copying ./pygptj/constants.py -> build/lib.macosx-13-arm64-cpython-311/pygptj
      copying ./pygptj/__init__.py -> build/lib.macosx-13-arm64-cpython-311/pygptj
      copying ./pygptj/model.py -> build/lib.macosx-13-arm64-cpython-311/pygptj
      copying ./pygptj/_logger.py -> build/lib.macosx-13-arm64-cpython-311/pygptj
      copying ./pygptj/cli.py -> build/lib.macosx-13-arm64-cpython-311/pygptj
      running build_ext
      -- The C compiler identification is AppleClang 14.0.0.14000029
      -- The CXX compiler identification is AppleClang 14.0.0.14000029
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- pybind11 v2.9.2
      -- Found PythonInterp: /opt/homebrew/Cellar/[email protected]/3.11.3/bin/python3.11 (found version "3.11.3")
      -- Found PythonLibs: /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib
      -- Performing Test HAS_FLTO
      -- Performing Test HAS_FLTO - Success
      -- Performing Test HAS_FLTO_THIN
      -- Performing Test HAS_FLTO_THIN - Success
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
      -- Found Threads: TRUE
      -- CMAKE_SYSTEM_PROCESSOR: arm64
      -- ARM detected
      -- Accelerate framework found
      -- Found Git: /opt/homebrew/bin/git (found version "2.40.0")
      -- CMAKE_SYSTEM_PROCESSOR: arm64
      -- ARM detected
      -- Accelerate framework found
      -- Configuring done (0.8s)
      CMake Error at ggml/src/CMakeLists.txt:188 (add_library):
        Cannot find source file:

          ggml.c

        Tried extensions .c .C .c++ .cc .cpp .cxx .cu .mpp .m .M .mm .ixx .cppm .h
        .hh .h++ .hm .hpp .hxx .in .txx .f .F .for .f77 .f90 .f95 .f03 .hip .ispc


      CMake Error at ggml/src/CMakeLists.txt:188 (add_library):
        No SOURCES given to target: ggml


      CMake Generate step failed.  Build files cannot be regenerated correctly.
      Traceback (most recent call last):
        File "/opt/homebrew/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/opt/homebrew/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/homebrew/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/build_meta.py", line 413, in build_wheel
          return self._build_with_temp_dir(['bdist_wheel'], '.whl',
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/build_meta.py", line 398, in _build_with_temp_dir
          self.run_setup()
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/build_meta.py", line 335, in run_setup
          exec(code, locals())
        File "<string>", line 132, in <module>
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/__init__.py", line 108, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/dist.py", line 1221, in run_command
          super().run_command(command)
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/wheel/bdist_wheel.py", line 343, in run
          self.run_command("build")
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/dist.py", line 1221, in run_command
          super().run_command(command)
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/dist.py", line 1221, in run_command
          super().run_command(command)
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 84, in run
          _build_ext.run(self)
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
          self.build_extensions()
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "<string>", line 118, in build_extension
        File "/opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 571, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['cmake', '/private/var/folders/qh/5q_cgpnd3fv05hd_9bmn7m180000gp/T/pip-install-3gt7amzx/pygptj_0e9d7cd0354c4618b6d84cd8f7260c7f', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/private/var/folders/qh/5q_cgpnd3fv05hd_9bmn7m180000gp/T/pip-install-3gt7amzx/pygptj_0e9d7cd0354c4618b6d84cd8f7260c7f/build/lib.macosx-13-arm64-cpython-311/', '-DPYTHON_EXECUTABLE=/opt/homebrew/Cellar/[email protected]/3.11.3/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DEXAMPLE_VERSION_INFO=2.0.0', '-GNinja', '-DCMAKE_MAKE_PROGRAM:FILEPATH=/private/var/folders/qh/5q_cgpnd3fv05hd_9bmn7m180000gp/T/pip-build-env-y9b39355/overlay/lib/python3.11/site-packages/ninja/data/bin/ninja']' returned non-zero exit status 1.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pygptj
  Building wheel for validators (setup.py) ... done
  Created wheel for validators: filename=validators-0.20.0-py3-none-any.whl size=19579 sha256=f3d96d28aa5cd28ebb09ed84d06e27d105c40b1873ed04f8a08d1e121f395f86
  Stored in directory: /Users/mtt/Library/Caches/pip/wheels/82/35/dc/f88ec71edf2a5596bd72a8fa1b697277e0fcd3cde83048b8bf
Successfully built pygpt4all llama-cpp-python hnswlib sentence-transformers validators
Failed to build pygptj
ERROR: Could not build wheels for pygptj, which is required to install pyproject.toml-based projects

Why NATO in the example?

Many people in the global south would not agree that NATO is the peaceful organization. Can we use less controversial example? I am happy to suggest better one?

Other embedding models?

Has anybody tried other embedding models such as those from Hugging Face?

Also, it can be finicky about certain things-- some unicode characters, size of chunks. I've had it crash on me on the same file (not example file)

Unexpected Termination due to std::runtime_error during model load in llama.cpp

python ingest.py sample.txt

Issue Description:
I encountered an unexpected termination error when trying to run ingest.py on sample.txt in the privateGPT repository. The specific error is a std::runtime_error stating that the end of the file was unexpectedly reached. The error appears to occur during the model load process in llama.cpp.

Expected Behavior:
The script should successfully load the model from ggml-gpt4all-j-v1.3-groovy.bin and process the sample.txt file without any errors.

Actual Behavior:
The script abruptly terminates and throws the following error:

llama.cpp: loading model from ggml-gpt4all-j-v1.3-groovy.bin
libc++abi: terminating due to uncaught exception of type std::runtime_error: unexpectedly reached end of file
[1]    69096 abort      python3 ingest.py sample.txt

Environment:
Python: 3.11.3
OS: MacOS Ventura 13.3.1 (a)
Hardware: Apple M1 Pro

Additional Context:
I downloaded the model from the link you provided, and also used your sample text.

Which languages can be used?

Hello, Does this app only works with English? Which languages does it support? Thanks.

Wrong part of document choosen as context, yields incorrect answer to simple question

Hi,
Thanks for providing such tool,

using this diff

diff --git a/ingest.py b/ingest.py
index e5d27b3..acae658 100644
--- a/ingest.py
+++ b/ingest.py
@@ -2,20 +2,25 @@ from langchain.document_loaders import TextLoader
 from langchain.text_splitter import RecursiveCharacterTextSplitter
 from langchain.vectorstores import Chroma
 from langchain.embeddings import LlamaCppEmbeddings
+from sys import argv

 def main():
     # Load document and split in chunks
-    loader = TextLoader('./source_documents/state_of_the_union.txt', encoding='utf8')
+    print(f"ingest.py: ingesting {argv[1]}")
+    loader = TextLoader(argv[1], encoding="utf8")
     documents = loader.load()
     text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
     texts = text_splitter.split_documents(documents)

I tried to feed a private git wiki, consisting of many md files with

find source_documents | grep "md$" | parallel -j1 python3 ingest.py {}

One of those md file contains a section called Setting up a nightly build.

Then, when querying the model with simple queries like How to setup a nightly build?, I see that the step Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer. chooses a document that has nothing to do with the requested topic, then yields a hallucinated and wrong answer.

Index list out of range with too many PDFs in source_documents?

First, thanks for your work. It's amazing!

Running on a Mac M1, when I upload more than 7-8 PDFs in the source_documents folder, I get this error:

% python ingest.py

llama.cpp: loading model from models/ggml-model-q4_0.bin
llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this
llama_model_load_internal: format     = 'ggml' (old version with low tokenizer quality and no mmap support)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 1000
llama_model_load_internal: n_embd     = 4096
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 32
llama_model_load_internal: n_layer    = 32
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: ftype      = 2 (mostly Q4_0)
llama_model_load_internal: n_ff       = 11008
llama_model_load_internal: n_parts    = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 4113748.20 KB
llama_model_load_internal: mem required  = 5809.33 MB (+ 2052.00 MB per state)
...................................................................................................
.
llama_init_from_file: kv self size  = 1000.00 MB
AVX = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 1 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | VSX = 0 |
Using embedded DuckDB with persistence: data will be stored in: db
Traceback (most recent call last):
  File "/Users/pierrick.lozach/Documents/privateGPT/ingest.py", line 36, in <module>
    main()
  File "/Users/pierrick.lozach/Documents/privateGPT/ingest.py", line 31, in main
    db = Chroma.from_documents(texts, llama, persist_directory=PERSIST_DIRECTORY, client_settings=CHROMA_SETTINGS)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 413, in from_documents
    return cls.from_texts(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 381, in from_texts
    chroma_collection.add_texts(texts=texts, metadatas=metadatas, ids=ids)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 159, in add_texts
    self._collection.add(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/chromadb/api/models/Collection.py", line 97, in add
    ids, embeddings, metadatas, documents = self._validate_embedding_set(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/chromadb/api/models/Collection.py", line 340, in _validate_embedding_set
    ids = validate_ids(maybe_cast_one_to_many(ids))
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/chromadb/api/types.py", line 75, in maybe_cast_one_to_many
    if isinstance(target[0], (int, float)):
IndexError: list index out of range

Is this due to a memory/file limit?

ChromaDB with Python version 3.10.6

Hi all,
just tried to install privateGPT in a t3.small Ubuntu 22.04.2 x64.
The installation fails at this point:
/home/ubuntu/.local/bin/pip install chromadb==0.3.22 -v
giving the following output

Using pip 23.1.2 from /home/ubuntu/.local/lib/python3.10/site-packages/pip (python 3.10)
Defaulting to user installation because normal site-packages is not writeable
Collecting chromadb==0.3.22
Using cached chromadb-0.3.22-py3-none-any.whl (69 kB)
Collecting pandas>=1.3 (from chromadb==0.3.22)
Using cached pandas-2.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.3 MB)
Collecting requests>=2.28 (from chromadb==0.3.22)
Using cached requests-2.30.0-py3-none-any.whl (62 kB)
Requirement already satisfied: pydantic>=1.9 in /home/ubuntu/.local/lib/python3.10/site-packages (from chromadb==0.3.22) (1.10.7)
Collecting hnswlib>=0.7 (from chromadb==0.3.22)
Using cached hnswlib-0.7.0.tar.gz (33 kB)
Running command pip subprocess to install build dependencies
Collecting setuptools>=42
Using cached setuptools-67.7.2-py3-none-any.whl (1.1 MB)
Collecting wheel
Using cached wheel-0.40.0-py3-none-any.whl (64 kB)
Collecting numpy>=1.10.0
Using cached numpy-1.24.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Collecting pybind11>=2.0
Using cached pybind11-2.10.4-py3-none-any.whl (222 kB)
Installing collected packages: wheel, setuptools, pybind11, numpy
Successfully installed numpy-1.24.3 pybind11-2.10.4 setuptools-67.7.2 wheel-0.40.0
Installing build dependencies ... done
Running command Getting requirements to build wheel
running egg_info
writing hnswlib.egg-info/PKG-INFO
writing dependency_links to hnswlib.egg-info/dependency_links.txt
writing requirements to hnswlib.egg-info/requires.txt
writing top-level names to hnswlib.egg-info/top_level.txt
reading manifest file 'hnswlib.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'hnswlib.egg-info/SOURCES.txt'
Getting requirements to build wheel ... done
Running command Preparing metadata (pyproject.toml)
running dist_info
creating /tmp/pip-modern-metadata-jkexsokp/hnswlib.egg-info
writing /tmp/pip-modern-metadata-jkexsokp/hnswlib.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-modern-metadata-jkexsokp/hnswlib.egg-info/dependency_links.txt
writing requirements to /tmp/pip-modern-metadata-jkexsokp/hnswlib.egg-info/requires.txt
writing top-level names to /tmp/pip-modern-metadata-jkexsokp/hnswlib.egg-info/top_level.txt
writing manifest file '/tmp/pip-modern-metadata-jkexsokp/hnswlib.egg-info/SOURCES.txt'
reading manifest file '/tmp/pip-modern-metadata-jkexsokp/hnswlib.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file '/tmp/pip-modern-metadata-jkexsokp/hnswlib.egg-info/SOURCES.txt'
creating '/tmp/pip-modern-metadata-jkexsokp/hnswlib-0.7.0.dist-info'
Preparing metadata (pyproject.toml) ... done
Collecting clickhouse-connect>=0.5.7 (from chromadb==0.3.22)
Using cached clickhouse_connect-0.5.24-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (922 kB)
Collecting sentence-transformers>=2.2.2 (from chromadb==0.3.22)
Using cached sentence_transformers-2.2.2-py3-none-any.whl
Collecting duckdb>=0.7.1 (from chromadb==0.3.22)
Using cached duckdb-0.7.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (15.2 MB)
Collecting fastapi>=0.85.1 (from chromadb==0.3.22)
Using cached fastapi-0.95.1-py3-none-any.whl (56 kB)
Collecting uvicorn[standard]>=0.18.3 (from chromadb==0.3.22)
Using cached uvicorn-0.22.0-py3-none-any.whl (58 kB)
Requirement already satisfied: numpy>=1.21.6 in /home/ubuntu/.local/lib/python3.10/site-packages (from chromadb==0.3.22) (1.24.3)
Collecting posthog>=2.4.0 (from chromadb==0.3.22)
Using cached posthog-3.0.1-py2.py3-none-any.whl (37 kB)
Requirement already satisfied: typing-extensions>=4.5.0 in /home/ubuntu/.local/lib/python3.10/site-packages (from chromadb==0.3.22) (4.5.0)
Requirement already satisfied: certifi in /usr/lib/python3/dist-packages (from clickhouse-connect>=0.5.7->chromadb==0.3.22) (2020.6.20)
Requirement already satisfied: urllib3>=1.26 in /usr/lib/python3/dist-packages (from clickhouse-connect>=0.5.7->chromadb==0.3.22) (1.26.5)
Requirement already satisfied: pytz in /usr/lib/python3/dist-packages (from clickhouse-connect>=0.5.7->chromadb==0.3.22) (2022.1)
Collecting zstandard (from clickhouse-connect>=0.5.7->chromadb==0.3.22)
Using cached zstandard-0.21.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting lz4 (from clickhouse-connect>=0.5.7->chromadb==0.3.22)
Using cached lz4-4.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
Collecting starlette<0.27.0,>=0.26.1 (from fastapi>=0.85.1->chromadb==0.3.22)
Using cached starlette-0.26.1-py3-none-any.whl (66 kB)
Collecting python-dateutil>=2.8.2 (from pandas>=1.3->chromadb==0.3.22)
Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting tzdata>=2022.1 (from pandas>=1.3->chromadb==0.3.22)
Using cached tzdata-2023.3-py2.py3-none-any.whl (341 kB)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from posthog>=2.4.0->chromadb==0.3.22) (1.16.0)
Collecting monotonic>=1.5 (from posthog>=2.4.0->chromadb==0.3.22)
Using cached monotonic-1.6-py2.py3-none-any.whl (8.2 kB)
Collecting backoff>=1.10.0 (from posthog>=2.4.0->chromadb==0.3.22)
Using cached backoff-2.2.1-py3-none-any.whl (15 kB)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/ubuntu/.local/lib/python3.10/site-packages (from requests>=2.28->chromadb==0.3.22) (3.1.0)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests>=2.28->chromadb==0.3.22) (3.3)
Collecting huggingface-hub>=0.4.0 (from sentence-transformers>=2.2.2->chromadb==0.3.22)
Using cached huggingface_hub-0.14.1-py3-none-any.whl (224 kB)
Collecting nltk (from sentence-transformers>=2.2.2->chromadb==0.3.22)
Using cached nltk-3.8.1-py3-none-any.whl (1.5 MB)
Collecting scikit-learn (from sentence-transformers>=2.2.2->chromadb==0.3.22)
Using cached scikit_learn-1.2.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.6 MB)
Link requires a different Python (3.10.6 not in: '>=3.7,<3.10'): https://files.pythonhosted.org/packages/99/f1/c00d6be56e1a718a3068079e3ec8ce044d7179345280f6a3f5066068af0d/scipy-1.6.2.tar.gz (from https://pypi.org/simple/scipy/) (requires-python:>=3.7,<3.10)
Link requires a different Python (3.10.6 not in: '>=3.7,<3.10'): https://files.pythonhosted.org/packages/fe/fd/8704c7b7b34cdac850485e638346025ca57c5a859934b9aa1be5399b33b7/scipy-1.6.3.tar.gz (from https://pypi.org/simple/scipy/) (requires-python:>=3.7,<3.10)
Link requires a different Python (3.10.6 not in: '>=3.7,<3.10'): https://files.pythonhosted.org/packages/8e/e1/ba481fb2154410f9c5a2d875695873597a7b1bea8c7f87cf444479c3260c/scipy-1.7.0rc1.tar.gz (from https://pypi.org/simple/scipy/) (requires-python:>=3.7,<3.10)
Link requires a different Python (3.10.6 not in: '>=3.7,<3.10'): https://files.pythonhosted.org/packages/a2/ae/ef8b23f5383403a597f2f79df44124a9b8f011a8beb1bca4c6b663a61d15/scipy-1.7.0rc2.tar.gz (from https://pypi.org/simple/scipy/) (requires-python:>=3.7,<3.10)
Link requires a different Python (3.10.6 not in: '>=3.7,<3.10'): https://files.pythonhosted.org/packages/bb/bb/944f559d554df6c9adf037aa9fc982a9706ee0e96c0d5beac701cb158900/scipy-1.7.0.tar.gz (from https://pypi.org/simple/scipy/) (requires-python:>=3.7,<3.10)
Link requires a different Python (3.10.6 not in: '>=3.7,<3.10'): https://files.pythonhosted.org/packages/47/33/a24aec22b7be7fdb10ec117a95e1e4099890d8bbc6646902f443fc7719d1/scipy-1.7.1.tar.gz (from https://pypi.org/simple/scipy/) (requires-python:>=3.7,<3.10)
Collecting scipy (from sentence-transformers>=2.2.2->chromadb==0.3.22)
Using cached scipy-1.10.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.4 MB)
Collecting sentencepiece (from sentence-transformers>=2.2.2->chromadb==0.3.22)
Using cached sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
Collecting torch>=1.6.0 (from sentence-transformers>=2.2.2->chromadb==0.3.22)
Killed

Do I need mandatory python 3.9.x?
3.10.6 is installed by default and out since Aug. 2, 2022.

Error building llama-cpp-python hnswlib

Would you help me to fix it? Thanks a lot, Iim tring to install the package using
pip install -r requirements.txt

All is going OK untill this point:

Building wheels for collected packages: llama-cpp-python, hnswlib
Building wheel for llama-cpp-python (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [308 lines of output]

  --------------------------------------------------------------------------------
  -- Trying 'Ninja (Visual Studio 17 2022 x64 v143)' generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  Not searching for unused variables given on the command line.
  -- The C compiler identification is unknown
  CMake Error at CMakeLists.txt:3 (ENABLE_LANGUAGE):
    No CMAKE_C_COMPILER could be found.

    Tell CMake where to find the compiler by setting either the environment
    variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
    the compiler, or to the compiler name if it is in the PATH.


  -- Configuring incomplete, errors occurred!
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying 'Ninja (Visual Studio 17 2022 x64 v143)' generator - failure
  --------------------------------------------------------------------------------



  --------------------------------------------------------------------------------
  -- Trying 'Visual Studio 17 2022 x64 v143' generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  Not searching for unused variables given on the command line.
  CMake Error at CMakeLists.txt:2 (PROJECT):
    Generator

      Visual Studio 17 2022

    could not find any instance of Visual Studio.



  -- Configuring incomplete, errors occurred!
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying 'Visual Studio 17 2022 x64 v143' generator - failure
  --------------------------------------------------------------------------------



  --------------------------------------------------------------------------------
  -- Trying 'Ninja (Visual Studio 16 2019 x64 v142)' generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  Not searching for unused variables given on the command line.
  -- The C compiler identification is unknown
  CMake Error at CMakeLists.txt:3 (ENABLE_LANGUAGE):
    No CMAKE_C_COMPILER could be found.

    Tell CMake where to find the compiler by setting either the environment
    variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
    the compiler, or to the compiler name if it is in the PATH.


  -- Configuring incomplete, errors occurred!
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying 'Ninja (Visual Studio 16 2019 x64 v142)' generator - failure
  --------------------------------------------------------------------------------



  --------------------------------------------------------------------------------
  -- Trying 'Visual Studio 16 2019 x64 v142' generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  Not searching for unused variables given on the command line.
  CMake Error at CMakeLists.txt:2 (PROJECT):
    Generator

      Visual Studio 16 2019

    could not find any instance of Visual Studio.



  -- Configuring incomplete, errors occurred!
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying 'Visual Studio 16 2019 x64 v142' generator - failure
  --------------------------------------------------------------------------------



  --------------------------------------------------------------------------------
  -- Trying 'Ninja (Visual Studio 15 2017 x64 v141)' generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  Not searching for unused variables given on the command line.
  -- The C compiler identification is unknown
  CMake Error at CMakeLists.txt:3 (ENABLE_LANGUAGE):
    No CMAKE_C_COMPILER could be found.

    Tell CMake where to find the compiler by setting either the environment
    variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
    the compiler, or to the compiler name if it is in the PATH.


  -- Configuring incomplete, errors occurred!
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying 'Ninja (Visual Studio 15 2017 x64 v141)' generator - failure
  --------------------------------------------------------------------------------



  --------------------------------------------------------------------------------
  -- Trying 'Visual Studio 15 2017 x64 v141' generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  Not searching for unused variables given on the command line.
  CMake Error at CMakeLists.txt:2 (PROJECT):
    Generator

      Visual Studio 15 2017

    could not find any instance of Visual Studio.



  -- Configuring incomplete, errors occurred!
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying 'Visual Studio 15 2017 x64 v141' generator - failure
  --------------------------------------------------------------------------------



  --------------------------------------------------------------------------------
  -- Trying 'NMake Makefiles (Visual Studio 17 2022 x64 v143)' generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  Not searching for unused variables given on the command line.
  CMake Error at CMakeLists.txt:2 (PROJECT):
    Running

     'nmake' '-?'

    failed with:

     El sistema no puede encontrar el archivo especificado


  -- Configuring incomplete, errors occurred!
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying 'NMake Makefiles (Visual Studio 17 2022 x64 v143)' generator - failure
  --------------------------------------------------------------------------------



  --------------------------------------------------------------------------------
  -- Trying 'NMake Makefiles (Visual Studio 16 2019 x64 v142)' generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  Not searching for unused variables given on the command line.
  CMake Error at CMakeLists.txt:2 (PROJECT):
    Running

     'nmake' '-?'

    failed with:

     El sistema no puede encontrar el archivo especificado


  -- Configuring incomplete, errors occurred!
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying 'NMake Makefiles (Visual Studio 16 2019 x64 v142)' generator - failure
  --------------------------------------------------------------------------------



  --------------------------------------------------------------------------------
  -- Trying 'NMake Makefiles (Visual Studio 15 2017 x64 v141)' generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  Not searching for unused variables given on the command line.
  CMake Error at CMakeLists.txt:2 (PROJECT):
    Running

     'nmake' '-?'

    failed with:

     El sistema no puede encontrar el archivo especificado


  -- Configuring incomplete, errors occurred!
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying 'NMake Makefiles (Visual Studio 15 2017 x64 v141)' generator - failure
  --------------------------------------------------------------------------------

                  ********************************************************************************
                  scikit-build could not get a working generator for your system. Aborting build.

                  Building windows wheels for Python 3.10 requires Microsoft Visual Studio 2022.
  Get it with "Visual Studio 2017":

    https://visualstudio.microsoft.com/vs/

  Or with "Visual Studio 2019":

      https://visualstudio.microsoft.com/vs/

  Or with "Visual Studio 2022":

      https://visualstudio.microsoft.com/vs/

                  ********************************************************************************
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for llama-cpp-python
Building wheel for hnswlib (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for hnswlib (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [5 lines of output]
running bdist_wheel
running build
running build_ext
building 'hnswlib' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for hnswlib
Failed to build llama-cpp-python hnswlib
ERROR: Could not build wheels for llama-cpp-python, hnswlib, which is required to install pyproject.toml-based projects

Support for PDF files

Most of the documents are in the form of pdf so it would be better to have a PDF support. If you guys want I can help with it

Error while querying

Hi, nice project!!

I am getting the following error while querying:

Enter a query: what is 2 + 2?

llama_print_timings: load time = 11190.14 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 11185.03 ms / 8 tokens ( 1398.13 ms per token)
llama_print_timings: eval time = 1541.14 ms / 1 runs ( 1541.14 ms per run)
llama_print_timings: total time = 12747.00 ms
Traceback (most recent call last):
File "/Users/robsonliebke/projects/python/privateGPT/privateGPT.py", line 39, in
main()
File "/Users/robsonliebke/projects/python/privateGPT/privateGPT.py", line 24, in main
res = qa(query)
^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/langchain/chains/base.py", line 140, in call
raise e
File "/opt/homebrew/lib/python3.11/site-packages/langchain/chains/base.py", line 134, in call
self._call(inputs, run_manager=run_manager)
File "/opt/homebrew/lib/python3.11/site-packages/langchain/chains/retrieval_qa/base.py", line 119, in _call
docs = self._get_docs(question)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/langchain/chains/retrieval_qa/base.py", line 181, in _get_docs
return self.retriever.get_relevant_documents(question)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/langchain/vectorstores/base.py", line 333, in get_relevant_documents
docs = self.vectorstore.similarity_search(query, **self.search_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/langchain/vectorstores/chroma.py", line 181, in similarity_search
docs_and_scores = self.similarity_search_with_score(query, k, filter=filter)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/langchain/vectorstores/chroma.py", line 227, in similarity_search_with_score
results = self.__query_collection(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/langchain/utils.py", line 50, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/langchain/vectorstores/chroma.py", line 120, in __query_collection
return self._collection.query(
^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/chromadb/api/models/Collection.py", line 219, in query
return self._client._query(
^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/chromadb/api/local.py", line 408, in _query
uuids, distances = self._db.get_nearest_neighbors(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/chromadb/db/clickhouse.py", line 583, in get_nearest_neighbors
uuids, distances = index.get_nearest_neighbors(embeddings, n_results, ids)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/chromadb/db/index/hnswlib.py", line 230, in get_nearest_neighbors
raise NoIndexException(
chromadb.errors.NoIndexException: Index not found, please create an instance before querying

chromadb error

Hey, I love the idea behind this repo, keep up the good work.

I am getting the following error:

Traceback (most recent call last):
  File "/Users/tjones/Projects/privateGPT/privatechatenv/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 111, in __query_collection
    return self._collection.query(
  File "/Users/tjones/Projects/privateGPT/privatechatenv/lib/python3.10/site-packages/chromadb/api/models/Collection.py", line 202, in query
    return self._client._query(
  File "/Users/tjones/Projects/privateGPT/privatechatenv/lib/python3.10/site-packages/chromadb/api/local.py", line 247, in _query
    uuids, distances = self._db.get_nearest_neighbors(
  File "/Users/tjones/Projects/privateGPT/privatechatenv/lib/python3.10/site-packages/chromadb/db/clickhouse.py", line 520, in get_nearest_neighbors
    uuids, distances = index.get_nearest_neighbors(embeddings, n_results, ids)
  File "/Users/tjones/Projects/privateGPT/privatechatenv/lib/python3.10/site-packages/chromadb/db/index/hnswlib.py", line 223, in get_nearest_neighbors
    raise NoIndexException("Index not found, please create an instance before querying")
chromadb.errors.NoIndexException: Index not found, please create an instance before querying

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/tjones/Projects/privateGPT/privateGPT.py", line 27, in <module>
    main()
  File "/Users/tjones/Projects/privateGPT/privateGPT.py", line 24, in main
    qa.run(query)    
  File "/Users/tjones/Projects/privateGPT/privatechatenv/lib/python3.10/site-packages/langchain/chains/base.py", line 238, in run
    return self(args[0], callbacks=callbacks)[self.output_keys[0]]
  File "/Users/tjones/Projects/privateGPT/privatechatenv/lib/python3.10/site-packages/langchain/chains/base.py", line 142, in __call__
    raise e
  File "/Users/tjones/Projects/privateGPT/privatechatenv/lib/python3.10/site-packages/langchain/chains/base.py", line 136, in __call__
    self._call(inputs, run_manager=run_manager)
  File "/Users/tjones/Projects/privateGPT/privatechatenv/lib/python3.10/site-packages/langchain/chains/retrieval_qa/base.py", line 119, in _call
    docs = self._get_docs(question)
  File "/Users/tjones/Projects/privateGPT/privatechatenv/lib/python3.10/site-packages/langchain/chains/retrieval_qa/base.py", line 181, in _get_docs
    return self.retriever.get_relevant_documents(question)
  File "/Users/tjones/Projects/privateGPT/privatechatenv/lib/python3.10/site-packages/langchain/vectorstores/base.py", line 333, in get_relevant_documents
    docs = self.vectorstore.similarity_search(query, **self.search_kwargs)
  File "/Users/tjones/Projects/privateGPT/privatechatenv/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 171, in similarity_search
    docs_and_scores = self.similarity_search_with_score(query, k, filter=filter)
  File "/Users/tjones/Projects/privateGPT/privatechatenv/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 217, in similarity_search_with_score
    results = self.__query_collection(
  File "/Users/tjones/Projects/privateGPT/privatechatenv/lib/python3.10/site-packages/langchain/utils.py", line 42, in wrapper
    return func(*args, **kwargs)
  File "/Users/tjones/Projects/privateGPT/privatechatenv/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 117, in __query_collection
    except chromadb.errors.NotEnoughElementsException:
NameError: name 'chromadb' is not defined. Did you mean: 'Chroma'?
[2023-05-08 15:38:15,087] {duckdb.py:445} INFO - PersistentDuckDB del, about to run persist
[2023-05-08 15:38:15,087] {duckdb.py:388} INFO - Persisting DB to disk, putting it in the save folder: db

chromadb is installed in the right directory, I checked the chromadb version, and I also tried importing it into privateGPT.py. I also uninstalled and reinstalled all of the dependencies in requirements.txt, same issue.

Error when running "python privateGPT.py": cannot pickle '_pygptj.gptj_model' object (type=type_error)

Hi,

I am trying to run the command "python privateGPT.py" to use the privateGPT tool, but I am encountering an error that prevents me from doing so. The error message I receive is as follows:

llama.cpp: loading model from ./models/ggml-model-q4_0.bin
llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this
llama_model_load_internal: format     = 'ggml' (old version with low tokenizer quality and no mmap support)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 512
llama_model_load_internal: n_embd     = 4096
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 32
llama_model_load_internal: n_layer    = 32
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: ftype      = 2 (mostly Q4_0)
llama_model_load_internal: n_ff       = 11008
llama_model_load_internal: n_parts    = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 4113748.20 KB
llama_model_load_internal: mem required  = 5809.33 MB (+ 2052.00 MB per state)
...................................................................................................
.
llama_init_from_file: kv self size  =  512.00 MB
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 | 
Using embedded DuckDB with persistence: data will be stored in: db
gptj_model_load: loading model from './models/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ...
gptj_model_load: n_vocab = 50400
gptj_model_load: n_ctx   = 2048
gptj_model_load: n_embd  = 4096
gptj_model_load: n_head  = 16
gptj_model_load: n_layer = 28
gptj_model_load: n_rot   = 64
gptj_model_load: f16     = 2
gptj_model_load: ggml ctx size = 4505.45 MB
gptj_model_load: memory_size =   896.00 MB, n_mem = 57344
gptj_model_load: ................................... done
gptj_model_load: model size =  3609.38 MB / num tensors = 285
Traceback (most recent call last):
  File "privateGPT.py", line 39, in <module>
    main()
  File "privateGPT.py", line 16, in main
    qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever, return_source_documents=True)
  File "/home/lzt/.local/lib/python3.8/site-packages/langchain/chains/retrieval_qa/base.py", line 91, in from_chain_type
    combine_documents_chain = load_qa_chain(
  File "/home/lzt/.local/lib/python3.8/site-packages/langchain/chains/question_answering/__init__.py", line 218, in load_qa_chain
    return loader_mapping[chain_type](
  File "/home/lzt/.local/lib/python3.8/site-packages/langchain/chains/question_answering/__init__.py", line 63, in _load_stuff_chain
    llm_chain = LLMChain(
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for LLMChain
llm
  cannot pickle '_pygptj.gptj_model' object (type=type_error)

I am not sure what is causing this error or how to fix it. I have tried searching for solutions online, but have not been able to find anything that works for me.

Here are some additional details that might be helpful:

I am running the command on a Ubuntu 20.04 machine.
I have installed all the necessary dependencies as instructed in the documentation.
I am using the "ggml-gpt4all-j-v1.3-groovy.bin" model.
I have successfully run the ingest command.

If anyone has any ideas on how to fix this error, I would greatly appreciate your help. Thank you in advance!

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe0 in position 64350: invalid continuation byte

Import error when ingesting

ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with LibreSSL 2.8.3. See: https://github.com/urllib3/urllib3/issues/2168

Unable to ingest any document due to this.

Future ideas

Introduction

I think this is a wonderful little project! I think we can use this issue to discuss several ways we can turn this from a promising demo into a full polished tool.

Most of these points are heavily inspired by Tailwind demo presented at Google IO. Perhaps if we're fast enough, we might even outpace Google, given that they seem to be quite partial to overdue betas and interminable wait-lists.

Suggestions:

Web, desktop, and mobile interfaces: The overall idea can be quite similar to Tailwind. Users could write or upload text files, with perhaps a possibility of auto-indexing arbitrary directories on the server, including virtual file systems, or cloud storage, sort them into folders or tags (perhaps with an option of automatic LLM-based grouping), and ask questions within these folders or for the entire account. Other features, such as quizes, todo lists, and so on, can too be considered.
Client, Server architecture: The app can be split into two parts, the server running on a cloud instance, powerful computer, server, or the same device as the interface, and a client communicating to it over a REST API or a WebSocket. Clients can be platform specific, web, or command-line, being quite small and easy to maintain, while the server can be free to perform background tasks and focus on performance.
Multi-user: Perhaps a simple user account system could be used to let multiple users work on the same server, perhaps with support for shared folders. Additionally, admin tools to monitor the amount of space used, user disk and rate-limit/token quotas, indexing jobs, and other such things would be welcome. Such features can also allow this app to act as an enterprise knowledge-base or just make it easier to share the same server with friends.
Arbitrary document ingestion: Perhaps adding plugins to capture more types of information could be promising. Transcribing and paragraphizing audio, parsing link lists into well-formatted summarized documents, etc.
Topic detection, clustering: Perhaps a visual representation of each document in a vector space could be an intuitive way to communicate how they are searched. It can also be a useful alternative to manual tagging or links that regular note-taking apps use to organize large knowage-bases.
Performance: I think for this type of project, performance remains critical, so there's quite a few considerations in this area. Perhaps the server can use C++ to avoid dealing with the bindings and allow the server to be one neat binary. LLaMa embedding are horribly inefficient and poor in general, so a fine-tuned BERT or instructor-xl can be used to search for text. And maybe there is some model smaller than LLaMA that can do the job of summarizing the found documents. RWKV-LM looks quite promising and maybe can be fine-tuned for this task in specific. And also, it should be looked into the actual text being embedded, perhaps splitting it by paragraphs or sentences to throw less data into the prompt of a performance-heavy LLM.

line 223, in get_nearest_neighbors raise NoIndexException("Index not found, please create an instance before querying")

Hi, thanks for creating this incredible tool.

I have been able to parse the .txt file correctly, and python privateGPT.py runs fine, outputting the "Enter a query".

But when this tries to run i get the following error message:

line 223, in get_nearest_neighbors
raise NoIndexException("Index not found, please create an instance before querying")

i have added two inputs of different lengths using the python ingest.py <path_to_your_txt_file> command.

Am i missing something simple?

Modern model formats

When running, I see the warning:

llama.cpp: loading model from ./models/ggml-model-q4_0.bin
llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this
llama_model_load_internal: format     = 'ggml' (old version with low tokenizer quality and no mmap support)

Do you have recommendations for a better model choice?

env

May I have an installation environment form? I keep having problems installing it.

pip requirement install step fails on WIndows [SOLVED]

The requirements installation fails when installing llama-cpp-python==0.1.41

Building wheel for pygptj (pyproject.toml) did not run successfully)

This is caused by a corrupted / missing VS installation

Steps to fix:

Install VS Studio (if not already installed)

In Visual Studio Installer modify Visual Studio Community nnnn
Install or repair Universal Windows Platform Developments. Make sure to install C++(v142) (right panel, select the checkbox)

You will now be able to install the requirements normally.

Any models that support Chinese?

I've download the two models. Can I exchange anyone of them to make it support Chinese?
or you can add the thrid model that can translate all the language

TypeError: generate() got an unexpected keyword argument 'new_text_callback'

/privateGPT/gpt4all_j.py", line 152, in _call
text = self.client.generate(
TypeError: generate() got an unexpected keyword argument 'new_text_callback'

Is this normal?

I asked one question that was very simple to test the digital thinking meat.

Question:
How many days are in a year?

Answer:
There are 365 days in a year.

------------------------------- This part it just spit out on it's own.

source_documents/state_of_the_union.txt:
Please rise if you are able and show that, Yes, we the United States of America stand with the Ukrainian people.

Throughout our history we’ve learned this lesson when dictators do not pay a price for their aggression they cause more chaos.

They keep moving.

And the costs and the threats to America and the world keep rising.

That’s why the NATO Alliance was created to secure peace and stability in Europe after World War 2.

source_documents/state_of_the_union.txt:
And we remain clear-eyed. The Ukrainians are fighting back with pure courage. But the next few days weeks, months, will be hard on them.

Putin has unleashed violence and chaos. But while he may make gains on the battlefield – he will pay a continuing high price over the long run.

And a proud Ukrainian people, who have known 30 years of independence, have repeatedly shown that they will not tolerate anyone who tries to take their country backwards.

source_documents/state_of_the_union.txt:
But in my administration, the watchdogs have been welcomed back.

We’re going after the criminals who stole billions in relief money meant for small businesses and millions of Americans.

And tonight, I’m announcing that the Justice Department will name a chief prosecutor for pandemic fraud.

By the end of this year, the deficit will be down to less than half what it was before I took office.

The only president ever to cut the deficit by more than one trillion dollars in a single year.

source_documents/state_of_the_union.txt:
We can do both. At our border, we’ve installed new technology like cutting-edge scanners to better detect drug smuggling.

We’ve set up joint patrols with Mexico and Guatemala to catch more human traffickers.

We’re putting in place dedicated immigration judges so families fleeing persecution and violence can have their cases heard faster.

We’re securing commitments and supporting partners in South and Central America to host more refugees and secure their own borders.

FR: Can docker deployment be provided?

after yesterday update it get error , couldn't run it

llama.cpp: loading model from ./models/ggml-model-q4_0.bin
llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this
llama_model_load_internal: format = 'ggml' (old version with low tokenizer quality and no mmap support)
llama_model_load_internal: n_vocab = 32000
llama_model_load_internal: n_ctx = 512
llama_model_load_internal: n_embd = 4096
llama_model_load_internal: n_mult = 256
llama_model_load_internal: n_head = 32
llama_model_load_internal: n_layer = 32
llama_model_load_internal: n_rot = 128
llama_model_load_internal: ftype = 2 (mostly Q4_0)
llama_model_load_internal: n_ff = 11008
llama_model_load_internal: n_parts = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 4113748.20 KB
llama_model_load_internal: mem required = 5809.33 MB (+ 2052.00 MB per state)
...................................................................................................
.
llama_init_from_file: kv self size = 512.00 MB
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
Using embedded DuckDB with persistence: data will be stored in: db
Traceback (most recent call last):
File "/home/rex/privateGPT/privateGPT.py", line 39, in
main()
File "/home/rex/privateGPT/privateGPT.py", line 15, in main
llm = GPT4All(model='./models/ggml-gpt4all-j-v1.3-groovy.bin', backend='gptj', callbacks=callbacks, verbose=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pydantic/main.py", line 341, in pydantic.main.BaseModel.init
pydantic.error_wrappers.ValidationError: 1 validation error for GPT4All
root
Could not import pygpt4all python package. Please install it with pip install pygpt4all. (type=value_error)
[2023-05-10 15:44:11,424] {duckdb.py:414} INFO - Persisting DB to disk, putting it in the save folder: db

try install pygpt4all already , but same errors.

FIXED : Model path related error.

File "/home/USER/.local/lib/python3.10/site-packages/langchain/embeddings/llamacpp.py", line 64, in validate_environment
   model_path = values["model_path"]
KeyError: 'model_path'

Presumably, it does not implement the proper path for the 'model', and I assume that it does not even implement any 'model', until I somehow tell it to use one (But I don't think that's how it is suppoesed to work).

I receive this error when I type in 'python ingest.py'

Full error below:

Python-dotenv could not parse statement starting at line 1
Python-dotenv could not parse statement starting at line 2
Python-dotenv could not parse statement starting at line 3
Python-dotenv could not parse statement starting at line 4
Python-dotenv could not parse statement starting at line 5
Traceback (most recent call last):
  File "/home/USER/Downloads/git/GPT_project/ingest.py", line 36, in <module>
    main()
  File "/home/USER/Downloads/git/GPT_project/ingest.py", line 29, in main
    llama = LlamaCppEmbeddings(model_path=llama_embeddings_model, n_ctx=model_n_ctx)
  File "pydantic/main.py", line 339, in pydantic.main.BaseModel.__init__
  File "pydantic/main.py", line 1102, in pydantic.main.validate_model
  File "/home/jotarokujo/.local/lib/python3.10/site-packages/langchain/embeddings/llamacpp.py", line 64, in validate_environment
    model_path = values["model_path"]
KeyError: 'model_path'

STEPS TO REPRODUCE:

1- pip install -r requirements.txt

2- Rename example.env to .env and edit the variables appropriately.

MODEL_TYPE: GPT4All
PERSIST_DIRECTORY: is the folder you want your vectorstore in

LLAMA_EMBEDDINGS_MODEL: ~/Downloads/git/GPT_project/EMBEDDING_bin //<- this contains ggml-model-q4_0.bin

MODEL_PATH: ~/Downloads/git/GPT_project/GPT4ALL_bin //<- this  contains ggml-gpt4all-j-v1.3-groovy.bin).
MODEL_N_CTX: 1024

3- pip install pdfminer.six

4- run python ingest.py

Suggestions for speeding up ingestion?

I presume I must be doing something wrong, as it is taking hours to ingest a 500kbyte text on an i9-12900 with 128GB. In fact it's not even done yet. Using models are recommended.

Help?

Thanks

Some output:

llama_print_timings: load time = 674.34 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 12526.78 ms / 152 tokens ( 82.41 ms per token)
llama_print_timings: eval time = 157.46 ms / 1 runs ( 157.46 ms per run)
llama_print_timings: total time = 12715.48 ms

chromadb.errors.NoIndexException: Index not found, please create an instance before querying

···
$ python privateGPT.py
llama.cpp: loading model from models/ggml-model-q4_0.bin
llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this
llama_model_load_internal: format = 'ggml' (old version with low tokenizer quality and no mmap support)
llama_model_load_internal: n_vocab = 32000
llama_model_load_internal: n_ctx = 1000
llama_model_load_internal: n_embd = 4096
llama_model_load_internal: n_mult = 256
llama_model_load_internal: n_head = 32
llama_model_load_internal: n_layer = 32
llama_model_load_internal: n_rot = 128
llama_model_load_internal: ftype = 2 (mostly Q4_0)
llama_model_load_internal: n_ff = 11008
llama_model_load_internal: n_parts = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 4113748.20 KB
llama_model_load_internal: mem required = 5809.33 MB (+ 2052.00 MB per state)
...................................................................................................
.
llama_init_from_file: kv self size = 1000.00 MB
AVX = 1 | AVX2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | VSX = 0 |
Using embedded DuckDB with persistence: data will be stored in: db
gptj_model_load: loading model from 'models/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ...
gptj_model_load: n_vocab = 50400
gptj_model_load: n_ctx = 2048
gptj_model_load: n_embd = 4096
gptj_model_load: n_head = 16
gptj_model_load: n_layer = 28
gptj_model_load: n_rot = 64
gptj_model_load: f16 = 2
gptj_model_load: ggml ctx size = 4505.45 MB
gptj_model_load: memory_size = 896.00 MB, n_mem = 57344
gptj_model_load: ................................... done
gptj_model_load: model size = 3609.38 MB / num tensors = 285

Enter a query:
The RTCPeerConnection.connectionState state in webrtc should be 'connected' or 'connected' in order to start creating a createDataChannel
···

error
···
lama_print_timings: load time = 1395.38 ms
llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per run)
llama_print_timings: prompt eval time = 4648.07 ms / 32 tokens ( 145.25 ms per token)
llama_print_timings: eval time = 170.89 ms / 1 runs ( 170.89 ms per run)
llama_print_timings: total time = 4834.98 ms
Traceback (most recent call last):
File "/Users/51pwn/MyWork/privateGPT/privateGPT.py", line 57, in
main()
File "/Users/51pwn/MyWork/privateGPT/privateGPT.py", line 42, in main
res = qa(query)
File "/Users/51pwn/anaconda3/envs/privateGPT/lib/python3.10/site-packages/langchain/chains/base.py", line 140, in call
raise e
File "/Users/51pwn/anaconda3/envs/privateGPT/lib/python3.10/site-packages/langchain/chains/base.py", line 134, in call
self._call(inputs, run_manager=run_manager)
File "/Users/51pwn/anaconda3/envs/privateGPT/lib/python3.10/site-packages/langchain/chains/retrieval_qa/base.py", line 119, in _call
docs = self._get_docs(question)
File "/Users/51pwn/anaconda3/envs/privateGPT/lib/python3.10/site-packages/langchain/chains/retrieval_qa/base.py", line 181, in _get_docs
return self.retriever.get_relevant_documents(question)
File "/Users/51pwn/anaconda3/envs/privateGPT/lib/python3.10/site-packages/langchain/vectorstores/base.py", line 366, in get_relevant_documents
docs = self.vectorstore.similarity_search(query, **self.search_kwargs)
File "/Users/51pwn/anaconda3/envs/privateGPT/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 181, in similarity_search
docs_and_scores = self.similarity_search_with_score(query, k, filter=filter)
File "/Users/51pwn/anaconda3/envs/privateGPT/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 228, in similarity_search_with_score
results = self.__query_collection(
File "/Users/51pwn/anaconda3/envs/privateGPT/lib/python3.10/site-packages/langchain/utils.py", line 50, in wrapper
return func(*args, **kwargs)
File "/Users/51pwn/anaconda3/envs/privateGPT/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 120, in __query_collection
return self._collection.query(
File "/Users/51pwn/anaconda3/envs/privateGPT/lib/python3.10/site-packages/chromadb/api/models/Collection.py", line 219, in query
return self._client._query(
File "/Users/51pwn/anaconda3/envs/privateGPT/lib/python3.10/site-packages/chromadb/api/local.py", line 408, in _query
uuids, distances = self._db.get_nearest_neighbors(
File "/Users/51pwn/anaconda3/envs/privateGPT/lib/python3.10/site-packages/chromadb/db/clickhouse.py", line 583, in get_nearest_neighbors
uuids, distances = index.get_nearest_neighbors(embeddings, n_results, ids)
File "/Users/51pwn/anaconda3/envs/privateGPT/lib/python3.10/site-packages/chromadb/db/index/hnswlib.py", line 230, in get_nearest_neighbors
raise NoIndexException(
chromadb.errors.NoIndexException: Index not found, please create an instance before querying
···

Any chance of PDF ingestion?

after upgrade still get errors pygpt4all error

(privategpt) root@alienware17B:/home/rex/privateGPT# python privateGPT.py
llama.cpp: loading model from models/ggml-model-q4_0.bin
llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this
llama_model_load_internal: format = 'ggml' (old version with low tokenizer quality and no mmap support)
llama_model_load_internal: n_vocab = 32000
llama_model_load_internal: n_ctx = 1000
llama_model_load_internal: n_embd = 4096
llama_model_load_internal: n_mult = 256
llama_model_load_internal: n_head = 32
llama_model_load_internal: n_layer = 32
llama_model_load_internal: n_rot = 128
llama_model_load_internal: ftype = 2 (mostly Q4_0)
llama_model_load_internal: n_ff = 11008
llama_model_load_internal: n_parts = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 4113748.20 KB
llama_model_load_internal: mem required = 5809.33 MB (+ 2052.00 MB per state)
...................................................................................................
.
llama_init_from_file: kv self size = 1000.00 MB
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
Using embedded DuckDB with persistence: data will be stored in: db
Traceback (most recent call last):
File "/home/rex/privateGPT/privateGPT.py", line 57, in
main()
File "/home/rex/privateGPT/privateGPT.py", line 30, in main
llm = GPT4All(model=model_path, n_ctx=model_n_ctx, backend='gptj', callbacks=callbacks, verbose=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pydantic/main.py", line 341, in pydantic.main.BaseModel.init
pydantic.error_wrappers.ValidationError: 1 validation error for GPT4All
root
Could not import pygpt4all python package. Please install it with pip install pygpt4all. (type=value_error)
[2023-05-12 09:20:17,126] {duckdb.py:414} INFO - Persisting DB to disk, putting it in the save folder: db

Index not found, please create an instance before querying

Got this error when enter a query:

Traceback (most recent call last):
File "privateGPT.py", line 39, in
main()
File "privateGPT.py", line 24, in main
res = qa(query)
File "D:\software\code_tools\Anaconda\envs\py3.8\lib\site-packages\langchain\chains\base.py", line 140, in call
raise e
File "D:\software\code_tools\Anaconda\envs\py3.8\lib\site-packages\langchain\chains\base.py", line 134, in call
self._call(inputs, run_manager=run_manager)
File "D:\software\code_tools\Anaconda\envs\py3.8\lib\site-packages\langchain\chains\retrieval_qa\base.py", line 119, in _call
return func(*args, **kwargs)
File "D:\software\code_tools\Anaconda\envs\py3.8\lib\site-packages\langchain\vectorstores\chroma.py", line 120, in __query_collection
return self._collection.query(
File "D:\software\code_tools\Anaconda\envs\py3.8\lib\site-packages\chromadb\api\models\Collection.py", line 219, in query
return self._client._query(
File "D:\software\code_tools\Anaconda\envs\py3.8\lib\site-packages\chromadb\api\local.py", line 408, in _query
uuids, distances = self._db.get_nearest_neighbors(
File "D:\software\code_tools\Anaconda\envs\py3.8\lib\site-packages\chromadb\db\clickhouse.py", line 583, in get_nearest_neighbors
uuids, distances = index.get_nearest_neighbors(embeddings, n_results, ids)
File "D:\software\code_tools\Anaconda\envs\py3.8\lib\site-packages\chromadb\db\index\hnswlib.py", line 230, in get_nearest_neighbors
raise NoIndexException(
chromadb.errors.NoIndexException: Index not found, please create an instance before querying

python privateGPT.py : core dumping

draeician@vivy:~/git/privateGPT$ python3 privateGPT.py
llama.cpp: loading model from ./models/ggml-model-q4_0.bin
llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this
llama_model_load_internal: format = 'ggml' (old version with low tokenizer quality and no mmap support)
llama_model_load_internal: n_vocab = 32000
llama_model_load_internal: n_ctx = 512
llama_model_load_internal: n_embd = 4096
llama_model_load_internal: n_mult = 256
llama_model_load_internal: n_head = 32
llama_model_load_internal: n_layer = 32
llama_model_load_internal: n_rot = 128
llama_model_load_internal: ftype = 2 (mostly Q4_0)
llama_model_load_internal: n_ff = 11008
llama_model_load_internal: n_parts = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 4113748.20 KB
llama_model_load_internal: mem required = 5809.33 MB (+ 2052.00 MB per state)
...................................................................................................
.
llama_init_from_file: kv self size = 512.00 MB
AVX = 1 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
Using embedded DuckDB with persistence: data will be stored in: db
Illegal instruction (core dumped)

This machine has no gpu, but 200gb of ram. So it's not a ram issue, and 24 cores. There's 9tb of space free, it's got resources for days, just not sure why it's core dumping.

I tokenized the state of the union sample, and this is my first run at asking it questions.

Any way to use this with any custom model, outside the gpt4all ecosystem?

If yes, can you please document the process of creating compatible model files, and incorporate in the codebase?

How to fix this error after tried to embed my text file? Traceback (most recent call last): File "privateGPT\ingest.py", line 22, in <module>

I received this error after trying to embed my text data which is [here](url)
Error here https://pastebin.com/BjaW6XpN

How can I fix this error?

how to utilize GPU where I am using WSL with Ubuntu2204

I have follow ubuntu website to install CUDA with my WSL distro. When I run privategpt, seems it do NOT use GPU at all. is there any support for that?
thanks
Rex

In the background, what happens if I ask for a summary?

If I ask for a summary of the document, how are the right parts of the embeddings being selected in order to summarise?

If I understand correctly, an embedding of the question is compared with the embeddings in the document to pull out relevant phrases.

What are some ways I could improve summarisation? Thanks

README.md Typo

@imartinez Hi, there is a typo in line 35:

If you want to start from scracth, ...

.env file not loaded automatically

With the latest update, when running python .\ingest.py , it reported

Traceback (most recent call last):
  File "F:\code\privateGPT\ingest.py", line 34, in <module>
    main()
  File "F:\code\privateGPT\ingest.py", line 27, in main
    llama = LlamaCppEmbeddings(model_path=llama_embeddings_model, n_ctx=model_n_ctx)
  File "pydantic\main.py", line 339, in pydantic.main.BaseModel.__init__
  File "pydantic\main.py", line 1102, in pydantic.main.validate_model
  File "C:\Users\teristam\Anaconda3\envs\private_gpt\lib\site-packages\langchain\embeddings\llamacpp.py", line 64, in validate_environment
    model_path = values["model_path"]
KeyError: 'model_path'

I notice the .env file is not loaded anywhere in the Python code?

Illegal hardware instruction

Hello,

Thanks again for this nice project! After the ingest process, when I try to run python3 privateGPT.py I've an error:

llama.cpp: loading model from ./models/ggml-model-q4_0.bin
llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this
llama_model_load_internal: format     = 'ggml' (old version with low tokenizer quality and no mmap support)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 512
llama_model_load_internal: n_embd     = 4096
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 32
llama_model_load_internal: n_layer    = 32
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: ftype      = 2 (mostly Q4_0)
llama_model_load_internal: n_ff       = 11008
llama_model_load_internal: n_parts    = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 4113748.20 KB
llama_model_load_internal: mem required  = 5809.33 MB (+ 2052.00 MB per state)
....................................................................................................
llama_init_from_file: kv self size  =  512.00 MB
AVX = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | VSX = 0 |
Using embedded DuckDB with persistence: data will be stored in: db
[1]    18314 illegal hardware instruction  python3 privateGPT.py

Hardware: MacBook Pro M1
Software: macOS Ventura

Regards,
Hisxo