[Bug]: Gemini missing message in history about autogen HOT 5 CLOSED

@yiranwu0 this has been accidentally introduced by the last changes to the Gemini client, which were not yet released.
I have created a PR for this issue: #2906

The example from the comment above will have everything included in the input message, because the system role is mapped later to the user role, and consecutive messages with identical roles are concatenated. This way the behavior is identical to that of the latest release 🙂

Bonus question: currently, messages of identical roles are concatenated without any separator between them. Would perhaps the addition of a newline character be desired between concatenated messages to act as a separator?

from autogen.

@BeibinLi @davorrunje Can you maybe take a look?

from autogen.

Thanks @luxzoli! Yeah I think a newline character would be good for now. Is there a specific gemini document saying that there cannot be same consecutive roles?

from autogen.

Good point @yiranwu0! I will look into it then if the concatenation can be avoided, and update my PR accordingly

from autogen.

I am getting the following exception from the Google API using the Vertex AI client, when consecutive messages of the same type are not merged (I have removed some lines from the trace):

grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.INVALID_ARGUMENT
        details = "Please ensure that multiturn requests alternate between user and model."
        debug_error_string = "UNKNOWN:Error received from peer ipv4:172.217.218.95:443 {grpc_message:"Please ensure that multiturn requests alternate between user and model.", grpc_status:3, created_time:"2024-06-10T21:03:18.368206063+00:00"}"
>

The above exception was the direct cause of the following exception:
...
google.api_core.exceptions.InvalidArgument: 400 Please ensure that multiturn requests alternate between user and model.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
...
File "/home/lux_zoltan_andras/.local/lib/python3.10/site-packages/autogen/oai/gemini.py", line 208, in create
    raise RuntimeError(f"Google GenAI exception occurred while calling Gemini API: {e}")
RuntimeError: Google GenAI exception occurred while calling Gemini API: 400 Please ensure that multiturn requests alternate between user and model.

Interestingly, I was not getting this error with the google.generativeai.
For the VertexAI client this official notebook also writes that messages can be added with roles user and model alternately. I have modified the notebook to break this rule in the chat history, and got the following exception.
So maybe it is safer to keep merging consecutive messages with the same role?
However, it seems like we can avoid concatenating the actual text by adding a list of parts to the Content object, which is what is utilized in the latest version of the PR

What do you think @yiranwu0 ?

from autogen.