Giter Club home page Giter Club logo

gemini-openai-proxy's Introduction

Gemini-OpenAI-Proxy

Gemini-OpenAI-Proxy is a proxy designed to convert the OpenAI API protocol to the Google Gemini Pro protocol. This enables seamless integration of OpenAI-powered functionalities into applications using the Gemini Pro protocol.


Table of Contents


Build

To build the Gemini-OpenAI-Proxy, follow these steps:

go build -o gemini main.go

Deploy

We recommend deploying Gemini-OpenAI-Proxy using Docker for a straightforward setup. Follow these steps to deploy with Docker:

docker run --restart=always -it -d -p 8080:8080 --name gemini zhu327/gemini-openai-proxy:latest

Adjust the port mapping (e.g., -p 8080:8080) as needed, and ensure that the Docker image version (zhu327/gemini-openai-proxy:latest) aligns with your requirements.


Usage

Gemini-OpenAI-Proxy offers a straightforward way to integrate OpenAI functionalities into any application that supports custom OpenAI API endpoints. Follow these steps to leverage the capabilities of this proxy:

  1. Set Up OpenAI Endpoint: Ensure your application is configured to use a custom OpenAI API endpoint. Gemini-OpenAI-Proxy seamlessly works with any OpenAI-compatible endpoint.

  2. Get Google AI Studio API Key: Before using the proxy, you'll need to obtain an API key from ai.google.dev. Treat this API key as your OpenAI API key when interacting with Gemini-OpenAI-Proxy.

  3. Integrate the Proxy into Your Application: Modify your application's API requests to target the Gemini-OpenAI-Proxy, providing the acquired Google AI Studio API key as if it were your OpenAI API key.

    Example API Request (Assuming the proxy is hosted at http://localhost:8080):

    curl http://localhost:8080/v1/chat/completions \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer $YOUR_GOOGLE_AI_STUDIO_API_KEY" \
     -d '{
         "model": "gpt-3.5-turbo",
         "messages": [{"role": "user", "content": "Say this is a test!"}],
         "temperature": 0.7
     }'

    Alternatively, use Gemini Pro Vision:

    curl http://localhost:8080/v1/chat/completions \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer $YOUR_GOOGLE_AI_STUDIO_API_KEY" \
     -d '{
         "model": "gpt-4-vision-preview",
         "messages": [{"role": "user", "content": [
            {"type": "text", "text": "What’s in this image?"},
            {
              "type": "image_url",
              "image_url": {
                "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
              }
            }
         ]}],
         "temperature": 0.7
     }'

    If you already have access to the Gemini 1.5 Pro api, you can use:

    curl http://localhost:8080/v1/chat/completions \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer $YOUR_GOOGLE_AI_STUDIO_API_KEY" \
     -d '{
         "model": "gpt-4-turbo-preview",
         "messages": [{"role": "user", "content": "Say this is a test!"}],
         "temperature": 0.7
     }'

    Model Mapping:

    GPT Model Gemini Model
    gpt-3.5-turbo gemini-1.0-pro-latest
    gpt-4 gemini-1.5-flash-latest
    gpt-4-turbo-preview gemini-1.5-pro-latest
    gpt-4-vision-preview gemini-1.0-pro-vision-latest

    If you wish to map gpt-4-vision-preview to gemini-1.5-pro-latest, you can configure the environment variable GPT_4_VISION_PREVIEW = gemini-1.5-pro-latest. This is because gemini-1.5-pro-latest now also supports multi-modal data.

  4. Handle Responses: Process the responses from the Gemini-OpenAI-Proxy in the same way you would handle responses from OpenAI.

Now, your application is equipped to leverage OpenAI functionality through the Gemini-OpenAI-Proxy, bridging the gap between OpenAI and applications using the Google Gemini Pro protocol.

Compatibility


License

Gemini-OpenAI-Proxy is licensed under the MIT License - see the LICENSE file for details.

gemini-openai-proxy's People

Contributors

zhu327 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gemini-openai-proxy's Issues

to become the best proxy for gemini

could you add embbending endpoint compatible to openai there is a proxy for that in go https://github.com/cheahjs/gemini-to-openai-proxy and a option to send history

example js when using langchain or redis chat memory

const {
  GoogleGenerativeAI,
  HarmCategory,
  HarmBlockThreshold,
} = require("@google/generative-ai");

const apiKey = process.env.GEMINI_API_KEY;
const genAI = new GoogleGenerativeAI(apiKey);

const model = genAI.getGenerativeModel({
  model: "gemini-1.5-flash",
  systemInstruction: ""
  });

const generationConfig = {
  temperature: 1,
  topP: 0.95,
  topK: 64,
  maxOutputTokens: 8192,
  responseMimeType: "text/plain",
};

async function run() {
  const chatSession = model.startChat({
    generationConfig,
 // safetySettings: Adjust safety settings
 // See https://ai.google.dev/gemini-api/docs/safety-settings
    history: [
      {
        role: "user",
        parts: [
          {text: "user history"},
        ],
      },
      {
        role: "model",
        parts: [
          {text: "model history "},
        ],
      },
    ],
  });

  const result = await chatSession.sendMessage("INSERT_INPUT_HERE");
  console.log(result.response.text());
}

run();```

pro vision cannot use text

I just found out that the update supports vision, but after using it, I found that using vision with plain text is not supported and must be accompanied by pictures.
image

Function support?

Not sure if gemini in itself even has function support, but it would be nice as I'd like to use this for my Home Assistant pipeline

Pythagora open ai API: 'choices' issue

Pythagora

Implementing task #1: Set up a Node.js project with package.json, install all necessary dependencies, and set up an express server.

I have successfully using for GPT pilot. during code generation, There was a problem with request to openai API:
'choices'

kindly help in this reg.

最新docker latest镜像有bug

首次对话正常,包含上下文再次对话,返回:"code":400,"message":"message.stringContent: json.Unmarshal: json: cannot unmarshal array into Go value of type string","type":""

openai api : InternalServerError: Error code: 502

its work for curl

OPENAI_API_KEY="xxx" #google ai api
curl http://localhost:8080/v1/chat/completions   -H "Content-Type: application/json"   -H "Authorization: Bearer $OPENAI_API_KEY"   -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {
        "role": "system",
        "content": "You are a poetic assistant, skilled in explaining complex programming concepts with creative flair."
      },
      {
        "role": "user",
        "content": "Compose a poem that explains the concept of recursion in programming."
      }
    ]
  }'

but failed for python

from openai import OpenAI
client = OpenAI(api_key='xxx'
                , base_url='http://localhost:8080/v1/')

completion = client.chat.completions.create(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "system", "content": "You are a poetic assistant, skilled in explaining complex programming concepts with creative flair."},
    {"role": "user", "content": "Compose a poem that explains the concept of recursion in programming."}
  ]
)

print(completion.choices[0].message)

the logs

File [~/anaconda3/envs/agents/lib/python3.9/site-packages/openai/_base_client.py:930](https://file+.vscode-resource.vscode-cdn.net/Users/jade_mayer/projects/agents/llamaindex/~/anaconda3/envs/agents/lib/python3.9/site-packages/openai/_base_client.py:930), in SyncAPIClient._request(self, cast_to, options, remaining_retries, stream, stream_cls)
    [927](https://file+.vscode-resource.vscode-cdn.net/Users/jade_mayer/projects/agents/llamaindex/~/anaconda3/envs/agents/lib/python3.9/site-packages/openai/_base_client.py:927)     if not err.response.is_closed:
    [928](https://file+.vscode-resource.vscode-cdn.net/Users/jade_mayer/projects/agents/llamaindex/~/anaconda3/envs/agents/lib/python3.9/site-packages/openai/_base_client.py:928)         err.response.read()
--> [930](https://file+.vscode-resource.vscode-cdn.net/Users/jade_mayer/projects/agents/llamaindex/~/anaconda3/envs/agents/lib/python3.9/site-packages/openai/_base_client.py:930)     raise self._make_status_error_from_response(err.response) from None
    [932](https://file+.vscode-resource.vscode-cdn.net/Users/jade_mayer/projects/agents/llamaindex/~/anaconda3/envs/agents/lib/python3.9/site-packages/openai/_base_client.py:932) return self._process_response(
    [933](https://file+.vscode-resource.vscode-cdn.net/Users/jade_mayer/projects/agents/llamaindex/~/anaconda3/envs/agents/lib/python3.9/site-packages/openai/_base_client.py:933)     cast_to=cast_to,
    [934](https://file+.vscode-resource.vscode-cdn.net/Users/jade_mayer/projects/agents/llamaindex/~/anaconda3/envs/agents/lib/python3.9/site-packages/openai/_base_client.py:934)     options=options,
   (...)
    [937](https://file+.vscode-resource.vscode-cdn.net/Users/jade_mayer/projects/agents/llamaindex/~/anaconda3/envs/agents/lib/python3.9/site-packages/openai/_base_client.py:937)     stream_cls=stream_cls,
    [938](https://file+.vscode-resource.vscode-cdn.net/Users/jade_mayer/projects/agents/llamaindex/~/anaconda3/envs/agents/lib/python3.9/site-packages/openai/_base_client.py:938) )

InternalServerError: Error code: 502

Error 429

Hello, I found that any call returns error 400 without making any modifications to the program, and the message is 429.
But when I directly call the Gemini Pro API with the Gemini Pro data format, it works fine.
I originally thought it was an incorrect key, but intentionally changed the key to the wrong one, and got the message Error 400.

The following is the data for when I called.

{
  "stream":false,
  "model":"gpt-3.5-turbo",
  "messages":[
  {
    "role":"user",
    "content":"HELLO"
  }]
}
{
  "code": 400,
  "message": "genai send message error: googleapi: Error 429:",
  "type": ""
}

Please modify to streaming reply mode: model.GenerateContentStream

ctx := context.Background()
client, err := genai.NewClient(ctx, option.WithAPIKey(os.Getenv("API_KEY")))
if err != nil {
log.Fatal(err)
}
defer client.Close()

model := client.GenerativeModel("gemini-pro")

iter := model.GenerateContentStream(ctx, genai.Text("Write a story about a magic backpack."))
for {
resp, err := iter.Next()
if err == iterator.Done {
break
}
if err != nil {
log.Fatal(err)
}

// print resp
}

400 error code

curl http://localhost:8080/v1/chat/completions \
 -H "Content-Type: application/json" \
 -H "Authorization: Bearer xxxxx" \
 -d '{
     "model": "gpt-4-vision-preview",
     "messages": [{"role": "user", "content": [
        {"type": "text", "text": "You are a zoological expert who knows what animal it is and what it is thinking"},
        {
          "type": "image_url",
          "image_url": {
            "url": "https://www.wikiwand.com/zh-hans/File:Cat_poster_2.jpg"
          }
        }
     ]}],
     "temperature": 0.7
 }'

{"code":400,"message":"genai send message error: googleapi: Error 400: Request contains an invalid argument.","type":""}%

gpt-engineer

Trying to use gemini-openai-proxy with the gpt-engineer
I invoke gpt-engineer as following OPENAI_API_KEY=1234 OPENAI_API_BASE=http://localhost:8080/v1 gpt-engineer Tic-tac-toe gpt-3.5-turbo and get the following error on proxy side

2023/12/31 07:50:03 genai get stream message error googleapi: Error 400:
[GIN] 2023/12/31 - 07:50:03 | 200 |  740.287576ms |             ::1 | POST     "/v1/chat/completions"

Not sure if the problem is on proxy side or gpt-engineer side tho

[Note] Client Compatibility List and Known Issues

Perfectly Compatible:

Known Issues:

  • There is a bug in the Google Gemini Pro SDK, which may lead to unexpected interruptions during streaming output.
  • If your prompt contains keywords like "openai," "chatgpt," "gemini," etc., it might be rejected by Gemini Pro for a response.

Feel free to contribute to the compatibility list, allowing us to continuously enhance the user experience of our project. Thank you for your support and feedback!

Replied sometimes contain /n

The gemini pro responses will randomly contain, /n sometimes instead of actually going to the next line. I suspect it’s similar to the previous issue of quotation marks getting printed.

proxy error L curl: (52) Empty reply from server

$ curl http://127.0.0.1:8081/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer $API_KEY" -d '{
"model": "gpt-4",
"messages": [{"role": "user", "content": "Say this is a test!"}],
"temperature": 0.7
}'
curl: (52) Empty reply from server

Here's the docker:
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9a49c5c0b3e4 zhu327/gemini-openai-proxy:latest "/app/gemini" 3 minutes ago Up 3 minutes 0.0.0.0:8081->8081/tcp gemini

The return value is not as expected

  1. When making a call using curl, I receive an empty object as the return value.
image
  1. Using the “openai” npm package results in the same issue as with the curl call.
    npm package openai

  2. However, using other clients (such as OpenCat / ChatX) allows for successful proxying.

image image

Chinese Location

Hello,

I am reaching out on behalf of several users from the China region who have deployed the gemini-openai-proxy Docker image but are facing consistent access timeout issues. The error being encountered frequently is a connection reset during API interactions.

Given that there might be unique considerations for accessing services like these from China, could you provide guidance or recommendations on how to ensure reliable access? Are there specific configurations, additional proxy setups, or any other steps that we should consider to address this timeout problem?

Any help or suggestions you could offer would be greatly appreciated.

Thank you for your assistance.

Best regards,
Jack

autogen getting 400 from proxy

Chat completions work fine from PostMan so it's working perfectly.

Autogen is getting a 400 error.

from autogen import AssistantAgent, UserProxyAgent

llm_config_gemini = {
    "config_list": [
        {
            "api_key": "***"
            "base_url": "http://localhost:8080/v1",
        }
    ]
}

assistant = AssistantAgent("assistant", llm_config_gemini)
user_proxy = UserProxyAgent("user_proxy", human_input_mode="TERMINATE", code_execution_config={"work_dir": "coding", "use_docker": False})

user_proxy.initiate_chat(assistant, message="Plot a chart of top performingi  blue chip stock price change YTD use dark mode")

Error:

Traceback (most recent call last):
  File "c:\Dev\ai\autogen\gemini\main..py", line 21, in <module>
    user_proxy.initiate_chat(assistant, message="Plot a chart of top performingi  blue chip stock price change YTD use dark mode")
  File "C:\Users\david\miniconda3\envs\pytorch\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 550, in initiate_chat
    self.send(self.generate_init_message(**context), recipient, silent=silent)
  File "C:\Users\david\miniconda3\envs\pytorch\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 348, in send
    recipient.receive(message, self, request_reply, silent)
  File "C:\Users\david\miniconda3\envs\pytorch\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 481, in receive
    reply = self.generate_reply(messages=self.chat_messages[sender], sender=sender)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\david\miniconda3\envs\pytorch\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 906, in generate_reply
    final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\david\miniconda3\envs\pytorch\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 625, in generate_oai_reply
    response = client.create(
               ^^^^^^^^^^^^^^
  File "C:\Users\david\miniconda3\envs\pytorch\Lib\site-packages\autogen\oai\client.py", line 247, in create
    response = self._completions_create(client, params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\david\miniconda3\envs\pytorch\Lib\site-packages\autogen\oai\client.py", line 327, in _completions_create
    response = completions.create(**params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\david\miniconda3\envs\pytorch\Lib\site-packages\openai\_utils\_utils.py", line 272, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\david\miniconda3\envs\pytorch\Lib\site-packages\openai\resources\chat\completions.py", line 645, in create
    return self._post(
           ^^^^^^^^^^^
  File "C:\Users\david\miniconda3\envs\pytorch\Lib\site-packages\openai\_base_client.py", line 1088, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\david\miniconda3\envs\pytorch\Lib\site-packages\openai\_base_client.py", line 853, in request
    return self._request(
           ^^^^^^^^^^^^^^
  File "C:\Users\david\miniconda3\envs\pytorch\Lib\site-packages\openai\_base_client.py", line 930, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "'$.messages[0].content' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}

Vercel

Create for vercel script pliss

[疑似BUG] 请求gpt-4-vision-preview时报错

{
  "error": {
    "headers": {
      "alt-svc": "h3=\":443\"; ma=86400",
      "cf-cache-status": "DYNAMIC",
      "cf-ray": "8759f2923a5d369e-YYZ",
      "connection": "keep-alive",
      "content-length": "131",
      "content-type": "application/json; charset=utf-8",
      "date": "Wed, 17 Apr 2024 05:17:14 GMT",
      "nel": "{\"success_fraction\":0,\"report_to\":\"cf-nel\",\"max_age\":604800}",
      "report-to": "{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v4?s=fV7mNNt1AvY1ZIWrL3ayQ0vCV7tjzAs0nA4pJMJCpR0wDIFcL%2BmQ7N2M4CLsOzXZy%2BRa3duFEtAgXdQYsH0uRE04%2BOM883x86W6Q2A325UVwvHc6mH150tmI7fFyl%2BNvDw8W2U0%3D\"}],\"group\":\"cf-nel\",\"max_age\":604800}",
      "server": "cloudflare"
    },
    "stack": "Error: 400 status code (no body)\n    at eP.generate (/app/.next/server/edge-chunks/316.js:4:1718)\n    at s_.makeStatusError (/app/.next/server/edge-chunks/316.js:4:14205)\n    at s_.makeRequest (/app/.next/server/edge-chunks/316.js:4:15128)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async Object.chat (/app/.next/server/edge-chunks/708.js:1:2328)\n    at async /app/.next/server/app/api/chat/[provider]/route.js:1:1500\n    at async /app/.next/server/edge-chunks/369.js:6:64203\n    at async O.execute (/app/.next/server/edge-chunks/369.js:6:61096)\n    at async O.handle (/app/.next/server/edge-chunks/369.js:6:65470)\n    at async ey.handler (/app/.next/server/edge-chunks/369.js:7:31644)",
    "status": 400
  },
  "endpoint": "https://gemini.nvoid.***.ua/v1",
  "provider": "openai"
}

请求应用为LobeChat
image
服务搭建完测试没问题
已增加环境变量使vision指向gemini1.5-pro
image
正常请求1.5-pro没问题

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.