[BUG] AttributeError: 'coroutine' object has no attribute 'get' about core HOT 15 OPEN

#783 Will gonna resolve most problem with ollama, wait some time

from core.

@enrichicco thank you for the effort you are putting! We already resolve the problem with the merge of #783 #813 in develop branch.
To summarize the problem:

1. Langchain patched all problems with Ollama (before that we used the ollama_utils.py to patch some issues)
2. Ollama update the body of the APIs (this is why we have the error)

With the new changes, we also resolve the tool selection issue when a local model tries to select a tool (now even Phi-3 can launch tools)

from core.

ok... I am back... I was able to obtain at least one answer.

It seems that the problem is in the file (inside the docker backend container):

/app/cat/looking_glass/agent_manager.py

I changed the line 170, from

return await memory_chain.ainvoke({**agent_input, "stop": "Human:"}, config=RunnableConfig(callbacks=[NewTokenHandler(stray)]))

to

return await memory_chain.ainvoke({**agent_input, "stop":["Human:"]}, config=RunnableConfig(callbacks=[NewTokenHandler(stray)]))

and the call apparently worked... again, take this with caution: smally tested (just one shot), from a beginner on both sides (python api and ollama api interface)

I solved it like this too

from core.

@Pingdred do you know if this fix needs to apply also for the new chain?

Not for now because we no longer use the stop sequence (branch develop), however, there is to test and see if it is necessary to keep it

from core.

Confirm possible bug with Ollama 1.33 and Qdrant 1.9.1, same problem.

from core.

First of all, really great job, ceshire-cat is great.

Then...

Please, take my observation with some caution, since I am fairly new to this kind of environment.
Maybe, I am writing a lot of rubbish...

It seems that here there are two problems:

1. the first is in the ceshire-cat, core application, code file:

[prjroot_core]/cat/factory/ollama_utils.py

where, at line 122, the handling of the exception passes through an async object, which "immediately" tries to call a get method on something which is not there yet:

            optional_detail = await response.json().get("error")

so the exception (I am inside the docker environment created through your compose file)

cheshire_cat_core           |   File "/app/cat/factory/ollama_utils.py", line 121, in _acreate_stream_patch
cheshire_cat_core           |     optional_detail = await response.json().get("error")
cheshire_cat_core           | AttributeError: 'coroutine' object has no attribute 'get'

My pesonal fix here, inspired by this stackoverflow post 'coroutine' object has no attribute get || pyppeteer

                    asy_optional_detail = await response.json()
                    optional_detail = asy_optional_detail.get("error")

2. Apparently, with my fix, the exception catcher does catch the exception, which at the end is, on the web UI:

Ollama call failed with status code 500. Details: option "stop" must be of type array

here is an extract of the log inside the container:

cheshire_cat_core           |   File "/app/cat/factory/ollama_utils.py", line 123, in _acreate_stream_patch
cheshire_cat_core           |     raise ValueError(
cheshire_cat_core           | ValueError: Ollama call failed with status code 500. Details: option "stop" must be of type array

Which is the ollama exception.
On this one I am still lost.
I googled a little, giving the exact exception nothing was there.
I'll keep searching. however, still looking at the log, there are these two lines more or less immediately above the catcher:

cheshire_cat_core           |   File "/usr/local/lib/python3.10/site-packages/langchain_community/llms/ollama.py", line 343, in _astream_with_aggregation
cheshire_cat_core           |     async for stream_resp in self._acreate_generate_stream(prompt, stop, **kwargs):

which maybe could be interesting in order to solve this...

thank you again for this framework, really...

from core.

ok... I am back... I was able to obtain at least one answer.

It seems that the problem is in the file (inside the docker backend container):

/app/cat/looking_glass/agent_manager.py

I changed the line 170, from

return await memory_chain.ainvoke({**agent_input, "stop": "Human:"}, config=RunnableConfig(callbacks=[NewTokenHandler(stray)]))

to

return await memory_chain.ainvoke({**agent_input, "stop":["Human:"]}, config=RunnableConfig(callbacks=[NewTokenHandler(stray)]))

and the call apparently worked... again, take this with caution: smally tested (just one shot), from a beginner on both sides (python api and ollama api interface)

from core.

Appreciate it, @valentimarco! Any idea when we can expect this to be part of the main branch?

from core.

Appreciate it, @valentimarco! Any idea when we can expect this to be part of the main branch?

Appreciate it, @valentimarco! Any idea when we can expect this to be part of the main branch?

not soon, is already in the develop branch but need some testing. If you want you can try it and give a feedback

from core.

Hi @valentimarco,

I checked the commits and the development branch, and it seems to me that the coroutine object error is still there, i.e, this one:

         optional_detail = await response.json().get("error")

According to me, this should be something like this:

                    asy_optional_detail = await response.json()
                    optional_detail = asy_optional_detail.get("error")

it is not so bad, but in case of response codes != 200 and 404, it masks the (manually) raised value error (lets say, the real request exception) with the "coroutine object has no attribute get error" .

from core.

Hi @valentimarco,

I checked the commits and the development branch, and it seems to me that the coroutine object error is still there, i.e, this one:

         optional_detail = await response.json().get("error")

According to me, this should be something like this:

                    asy_optional_detail = await response.json()
                    optional_detail = asy_optional_detail.get("error")

it is not so bad, but in case of response codes != 200 and 404, it masks the (manually) raised value error (lets say, the real request exception) with the "coroutine object has no attribute get error" .

Langchain changed the code on that parts, infact i delete ollama_utils.py and from my test with ollama 1.33 and 1.34 i don't have any problem. If you want we can see it in discord, maybe i am forgetting something

from core.

@Pingdred do you know if this fix needs to apply also for the new chain?

from core.

Appreciate it, @valentimarco! Any idea when we can expect this to be part of the main branch?

Appreciate it, @valentimarco! Any idea when we can expect this to be part of the main branch?

not soon, is already in the develop branch but need some testing. If you want you can try it and give a feedback

I'm gonna apologize in advance for a dumb question: how do we switch to the develop branch? I followed the instructions here to install, but I can't figure out what the "image" line needs to be in order to use the develop branch (if that's even what you need to do lol)

If it isn't obvious, I am really new to docker and containerization. Thank you in advance!

from core.

Appreciate it, @valentimarco! Any idea when we can expect this to be part of the main branch?

Appreciate it, @valentimarco! Any idea when we can expect this to be part of the main branch?

not soon, is already in the develop branch but need some testing. If you want you can try it and give a feedback

I'm gonna apologize in advance for a dumb question: how do we switch to the develop branch? I followed the instructions here to install, but I can't figure out what the "image" line needs to be in order to use the develop branch (if that's even what you need to do lol)

If it isn't obvious, I am really new to docker and containerization. Thank you in advance!

1. git clone https://github.com/cheshire-cat-ai/core.git
2. git switch develop
3. docker compose build
4. docker compose up

from core.

Appreciate it, @valentimarco! Any idea when we can expect this to be part of the main branch?

Appreciate it, @valentimarco! Any idea when we can expect this to be part of the main branch?

not soon, is already in the develop branch but need some testing. If you want you can try it and give a feedback

I'm gonna apologize in advance for a dumb question: how do we switch to the develop branch? I followed the instructions here to install, but I can't figure out what the "image" line needs to be in order to use the develop branch (if that's even what you need to do lol)
If it isn't obvious, I am really new to docker and containerization. Thank you in advance!

1. git clone https://github.com/cheshire-cat-ai/core.git
2. git switch develop
3. docker compose build
4. docker compose up

Thank you so much!!! I cannot even begin to describe how excited I am to start experimenting with this!!!

from core.