zilliztech / akcio Goto Github PK

https://github.com/zilliztech/akcio/blob/a48121d8fb93765ed7cc2de28de29ad5504ae117/src_langchain/store/vector_store/milvus.py

What would you like to be added?

Akcio offers two options to build the system: langchain or towhee.
The option using Towhee has already supported Llama-2 as LLM.
To support Llama-2 for LangChain, we need to add a llama_2_chat.py under https://github.com/zilliztech/akcio/tree/main/src_langchain/llm.

With llama-2 supported in LangChain mode, the following steps should start service successfully:

1. Set up

Change to your own Milvus & Postgres connection details (modify config.py if needed.)

$ export LLM_OPTION=llama_2
$ export MILVUS_URI=https://localhost:19530
$ export SQL_URI=postgresql://postgres:postgres@localhost/chat_history

3. Start service

$ python main.py --langchain

OR start gradio demo:

$ python gradio_demo.py --langchain

Why is this needed?

No response

Anything else?

No response

What would you like to be added?

• Dockerfile to build docker image
• Docker image to start FastAPI service
• Update deployment method in docs

Why is this needed?

The deployment option of Docker image offers greater flexibility, consistency, scalability, and manageability compared to starting a service directly from source code.

Anything else?

No response

Current Behavior

This is the procedure I followed:

• I have set the environment variable OPENAI_API_KEY
• I created an account on Zilliz and set up a standard cluster using the free trial. in order to have URI and user and password
• I tryied the connection of the Zilliz account using the following script and seems to be connected, then I setted the enviroment variables

import os
from pymilvus import connections, utility

connections.connect(uri=os.getenv('MILVUS_URI', 'https://the_link_given:19530'), user=os.getenv('MILVUS_USER', 'db_admin'), password=os.getenv('MILVUS_PASSWORD', 'tmy_password'), secure=True)
print(utility.list_collections())

• I installed postgresql and setted a std database named chathistory and checked the connection to the database with the code below and it was working. Then setted the enviroment variable

import psycopg2
import os

connect_str = os.getenv('SQL_URI', 'postgresql://postgres:My_password@localhost/chathistory')

try:
    conn = psycopg2.connect(connect_str)
    print("its working!")
except Exception as e:
    print(f"not working: {e}")

• Then I tryied to run the code using:

py main.py --towhee

I've seen that there's a similar problem that has already been solved #43
so I tryied both Zilliztech/akcio and Jaelgu/akcio but i have exactly the same error.
Any advice how to fix it?

The error shown is:

Traceback (most recent call last):
  File "C:\Users\erric\Desktop\Corso python\Notizie internazionale 2\internazionale\Lib\site-packages\pymilvus\client\grpc_handler.py", line 131, in _wait_for_channel_ready
    grpc.channel_ready_future(self._channel).result(timeout=timeout)
  File "C:\Users\erric\Desktop\Corso python\Notizie internazionale 2\internazionale\Lib\site-packages\grpc\_utilities.py", line 151, in result
    self._block(timeout)
  File "C:\Users\erric\Desktop\Corso python\Notizie internazionale 2\internazionale\Lib\site-packages\grpc\_utilities.py", line 97, in _block
    raise grpc.FutureTimeoutError()
grpc.FutureTimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\erric\Desktop\Corso python\Notizie internazionale 2\03 - Secondo Tentativo Akcio\akcio-main\main.py", line 40, in <module>
    from src_towhee.operations import chat, insert, drop, check, get_history, clear_history, count  # pylint: disable=C0413
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\erric\Desktop\Corso python\Notizie internazionale 2\03 - Secondo Tentativo Akcio\akcio-main\src_towhee\operations.py", line 14, in <module>
    towhee_pipelines = TowheePipelines()
                       ^^^^^^^^^^^^^^^^^
  File "C:\Users\erric\Desktop\Corso python\Notizie internazionale 2\03 - Secondo Tentativo Akcio\akcio-main\src_towhee\pipelines\__init__.py", line 58, in __init__
    connections.connect(
  File "C:\Users\erric\Desktop\Corso python\Notizie internazionale 2\internazionale\Lib\site-packages\pymilvus\orm\connections.py", line 355, in connect
    connect_milvus(**kwargs, user=user, password=password, token=token, db_name=db_name)
  File "C:\Users\erric\Desktop\Corso python\Notizie internazionale 2\internazionale\Lib\site-packages\pymilvus\orm\connections.py", line 302, in connect_milvus
    gh._wait_for_channel_ready(timeout=timeout)
  File "C:\Users\erric\Desktop\Corso python\Notizie internazionale 2\internazionale\Lib\site-packages\pymilvus\client\grpc_handler.py", line 134, in _wait_for_channel_ready
    raise MilvusException(
pymilvus.exceptions.MilvusException: <MilvusException: (code=2, message=Fail connecting to server on in01-54ec8f16cac25a2.aws-us-west-2.vectordb.zillizcloud.com:19530. Timeout)>

Expected Behavior

No response

Steps To Reproduce

No response

Environment

No response

Anything else?

No response

Hello,
So I am running main.py with langchain and after I inserted an md file I was able to ask it a question. However, when I try to ask a second question, I get this response:

  {
    "status": true,
    "msg": "Something went wrong:\ntuple indices must be integers or slices, not str",
    "debug": {
      "original question": "Which tech does Lagrange use?",
      "modified question": "Which tech does Lagrange use?",
      "answer": "Something went wrong:\ntuple indices must be integers or slices, not str"
    }
  },
  200
]

P.S Every time I use a new session id, my first question is instantly answered but the second one returns this response.

What would you like to be added?

It would be nice to tweak this project a bit to include GPTCache.

Thanks.

Why is this needed?

Well GPTCache reduces the API bill and also improves speed. It's also in the same Zilliz family so it is beneficial to be integrated.

Anything else?

No response

Current Behavior

it don't match with a args result

Expected Behavior

should delete token_count

Steps To Reproduce

No response

Environment

No response

Anything else?

No response

What would you like to be added?

is there other way to download llm model in advance, and place it in the appropriate directory？

Why is this needed?

No response

Anything else?

No response

What would you like to be added?

python start_gradio.py --towhee should change to python gradio_demo.py --towhee

Why is this needed?

No response

Anything else?

No response

Each time I try running python main.py --towhee it gives me this error:

Traceback (most recent call last):
File "C:\Users\victo\AppData\Local\Programs\Python\Python311\Lib\site-packages\pymilvus\client\grpc_handler.py", line 119, in _wait_for_channel_ready
grpc.channel_ready_future(self._channel).result(timeout=timeout)
File "C:\Users\victo\AppData\Local\Programs\Python\Python311\Lib\site-packages\grpc_utilities.py", line 151, in result
self._block(timeout)
File "C:\Users\victo\AppData\Local\Programs\Python\Python311\Lib\site-packages\grpc_utilities.py", line 97, in _block
raise grpc.FutureTimeoutError()
grpc.FutureTimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\Users\victo\Documents\CODE FOLDER\Protected Akcio\akcio\main.py", line 24, in
from src_towhee.operations import chat, insert, drop # pylint: disable=C0413
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\victo\Documents\CODE FOLDER\Protected Akcio\akcio\src_towhee\operations.py", line 14, in
towhee_pipelines = TowheePipelines()
^^^^^^^^^^^^^^^^^
File "C:\Users\victo\Documents\CODE FOLDER\Protected Akcio\akcio\src_towhee\pipelines_init_.py", line 51, in init
connections.connect(
File "C:\Users\victo\AppData\Local\Programs\Python\Python311\Lib\site-packages\pymilvus\orm\connections.py", line 349, in connect
connect_milvus(**kwargs, user=user, password=password, token=token, db_name=db_name)
File "C:\Users\victo\AppData\Local\Programs\Python\Python311\Lib\site-packages\pymilvus\orm\connections.py", line 282, in connect_milvus
gh._wait_for_channel_ready(timeout=timeout)
File "C:\Users\victo\AppData\Local\Programs\Python\Python311\Lib\site-packages\pymilvus\client\grpc_handler.py", line 123, in _wait_for_channel_ready
raise MilvusException(Status.CONNECT_FAILED,
pymilvus.exceptions.MilvusException: <MilvusException: (code=2, message=Fail connecting to server on in01-9434ec54c12222e.gcp-us-west1.vectordb.zillizcloud.com:443. Timeout)>

Reproduction steps:
I cloned the git repo and ran the dependency installing script. Then I modified the config.py file with my OPENAI API key like so:
'openai': { 'openai_model': 'gpt-3.5-turbo', 'openai_api_key': 'my_key_here', # will use environment value 'OPENAI_API_KEY' if None 'llm_kwargs': { 'temperature': 0.8, # 'max_tokens': 200, } },

I left the embedding code untouched but I downloaded sentence-transformers model multi-qa-mpnet-base-cos-v1 using pip install -U sentence-transformers.

For the vector db, I set up a Zilliz cloud cluster on a standard plan so that it would let me input a user and password. inside the cluster I intialized a collection with IP metrics and 768 dimensions. To set up a connection to this cluster I modified the config.py file with the cluster URI, my username and the corresponding password:
'connection_args': { 'uri': os.getenv('MILVUS_URI', 'my_uri_here;), 'user': os.getenv('MILVUS_USER', 'my_user_here'), 'password': os.getenv('MILVUS_PASSWORD', 'my_password_here'), 'secure': True if os.getenv('MILVUS_SECURE', 'False').lower() == 'true' else False },

The memory db was set up using postgresql through pgAdmin 4. I simply set up a database with default owner and all other properties, then connected to it by modifying the config.py file with the instructed string {database_type}://{user}:{password}@{host}/{database_name}, my string looked like: postgresql://{user}:{password}@localhost:5432/{database_name}.

Everything else was left as is. Running the LangChain option gives me a link to localhost:8900, but the link produces an empty page.

Hello, so I am using a unique session ID and I am using PostgreSQL for memory. The thing is I don't see how I should set it up so that the bot uses the chat_history for context. For example, I will ask the bot to call me Lydacious and in my next message it will say "Unfortunately, I couldn't find any information about what you asked me to call you earlier.". I don't know if it is relative but in my prompt, I've made sure so the bot never comes up with new answers, only takes them from the doc chunks that have been provided.

Hi team,

I tried to run this project and had an error as follows with the token, whereas I have set the tokens and it can be seen in the config object.

Appreciate for any tip to run this. Thanks.

python3 gradio_demo.py --towhee

2023-12-08 10:15:15,782 - 140703044237120 - decorators.py-decorators:88 - WARNING: [__internal_register] retry:4, cost: 0.27s, reason: <_InactiveRpcError: StatusCode.UNKNOWN, token not found>

grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNKNOWN
details = "token not found"
debug_error_string = "UNKNOWN:Error received from peer ipv4:34.160.220.160:443 {grpc_message:"token not found", grpc_status:2, created_time:"2023-12-08T10:15:29.907598478+07:00"}"

I printed out the config object it looks like this:

config: type='RecursiveCharacter' chunk_size=300 splitter_kwargs={} embedding_model='BAAI/bge-base-en' openai_api_key='sk-tdnNgbL----------lkQudP2XntZ' embedding_device=-1 embedding_normalize=True milvus_uri='https://in03-f2-------cf.api.gcp-us-west1.zillizcloud.com' milvus_token='ec22ee0b9cef0daffc401b049--------c8a8061f7f41f27a76276bde1fb469b5' milvus_host=None milvus_port=None milvus_user=None milvus_password=None es_enable=False es_connection_kwargs={'hosts': ['https://127.0.0.1:9200'], 'basic_auth': ('elastic', 'my_password')} token_model='gpt-3.5-turbo' llm_kwargs={'temperature': 0.2, 'max_tokens': 20} openai_model='gpt-3.5-turbo' llm_src='openai'

Current Behavior

curl -X 'POST' \
  'http://localhost:8900/project/add?project=test' \
  -H 'accept: application/json' \
  -H 'Content-Type: multipart/form-data' \
  -F '[email protected];type=text/plain'

Response body

[
  {
    "status": false,
    "msg": "Failed to load data:\nNode-token-counter-1 runs failed, error msg: name 'enc' is not defined, Traceback (most recent call last):\n  File \"/export/xxx/anaconda3/envs/akcio/lib/python3.10/site-packages/towhee/runtime/nodes/node.py\", line 158, in _call\n    return True, self._op(*inputs), None\n  File \"/home/xxx/.towhee/operators/towhee/token-counter/versions/main/count_token.py\", line 13, in __call__\n    num_tokens = len(enc.encode(str(data)))\nNameError: name 'enc' is not defined\n, Traceback (most recent call last):\n  File \"/export/xxx/anaconda3/envs/akcio/lib/python3.10/site-packages/towhee/runtime/nodes/node.py\", line 171, in process\n    self.process_step()\n  File \"/export/xxx/anaconda3/envs/akcio/lib/python3.10/site-packages/towhee/runtime/nodes/_map.py\", line 63, in process_step\n    assert succ, msg\nAssertionError: name 'enc' is not defined, Traceback (most recent call last):\n  File \"/export/xxx/anaconda3/envs/akcio/lib/python3.10/site-packages/towhee/runtime/nodes/node.py\", line 158, in _call\n    return True, self._op(*inputs), None\n  File \"/home/xxx/.towhee/operators/towhee/token-counter/versions/main/count_token.py\", line 13, in __call__\n    num_tokens = len(enc.encode(str(data)))\nNameError: name 'enc' is not defined\n\n\n"
  },
  400
]

towhee version: 1.1.1

Expected Behavior

No response

Steps To Reproduce

No response

Environment

No response

Anything else?

No response