Ollama provider in IDEA not working as expected about continue HOT 11 OPEN

Hi ,

i have the same issue with the ollama provider. My config.json:

{ "title": "CodeLlama-7b", "model": "codellama-7b", "contextLength": 2048.0, "completionOptions": {}, "apiBase": "http://localhost:11434", "request_options": { "timeout": 1000 }, "provider": "ollama" },

GPT 3.5 is working as expected, but "/edit" with intellij is not working. Normal chat is working.

Kind Regards,
Jörg

from continue.

This is my complete config, only the mistral object in the models was added by me, everything else was auto generated. Note that I've also tried with other models and in VSCode it does work.

{
  "models": [
    {
      "title": "GPT-4",
      "provider": "openai-free-trial",
      "model": "gpt-4"
    },
    {
      "title": "GPT-3.5-Turbo",
      "provider": "openai-free-trial",
      "model": "gpt-3.5-turbo"
    },
    {
      "title": "mistral",
      "provider": "ollama",
      "model": "mistral-7b"
    }
  ],
  "slashCommands": [
    {
      "name": "edit",
      "description": "Edit highlighted code",
      "step": "EditHighlightedCodeStep"
    },
    {
      "name": "comment",
      "description": "Write comments for the highlighted code",
      "step": "CommentCodeStep"
    },
    {
      "name": "share",
      "description": "Download and share this session",
      "step": "ShareSessionStep"
    },
    {
      "name": "cmd",
      "description": "Generate a shell command",
      "step": "GenerateShellCommandStep"
    }
  ],
  "customCommands": [
    {
      "name": "test",
      "prompt": "Write a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",
      "description": "Write unit tests for highlighted code"
    }
  ],
  "contextProviders": [
    { "name": "diff", "params": {} },
    {
      "name": "open",
      "params": {}
    },
    { "name": "terminal", "params": {} }
  ]
}

from continue.

@sestinj : Thanks, for me the 0.0.27 does work again, installed from disk.

from continue.

@azthec I'm having trouble reproducing, would you mind sharing your exact config.json (any API keys excluded)?

from continue.

I have the same problem using LM Studio as provider and the CodeLlama models. No matter which setting I use. Only once did the Continue plugin manage to output the response stream. Every time I tried again, I got the error shown in the screenshot above. However, I can see that LM Studio is generating the responses.

This happens using the default configuration, i.e.:

{
      "title": "CodeLlama-7b",
      "model": "codellama-7b",
      "contextLength": 2048.0,
      "apiBase": "http://localhost:1234",
      "completionOptions": {},
      "provider": "lmstudio"
    }

Same for the 13b model 🤷‍♂️.

Edit I: I also tried to increase the timeout:

"request_options": { "timeout": 1000 },

But it seems to make no difference.

Edit II:

- OS: macOS Sonoma 14.2.1 and Windows 10 (latest updates)
- Continue: 0.0.26
- IDE: JetBrains Rider 2023.3.2
- LM Studio 0.2.10

from continue.

@SommerEngineering your config looks good, except contextLength should be an integer instead of a float (I understand that Continue is responsible for this, it's something we're fixing for the next version). I don't expect this to break things though.

I'm wondering if the proxy server might have somehow not started or died. Can you lsof -i :65433 to check whether there is a running process?

We use this so that requests can be made directly from the webview, otherwise they may be blocked depending on CORS policy. If it's not up, then this would certainly cause these errors.

from continue.

@SommerEngineering @azthec @jkritzen I found the problem with the timeout and have solved it in version 0.0.27. Unfortunately JetBrains tends not to accept plugin updates on weekends, so it won't be available through the store until Monday, but in the meantime you can install manually this .zip file by clicking the gear icon at the top of the plugin settings page and then "Install from Disk".

Would also be useful to get confirmation (or the opposite) that this actually solves the issue for you all

continue-intellij-extension-0.0.27.zip

from continue.

@sestinj I can confirm that it works properly now, you can close this issue

On a side note I've noticed its very hard to search the IntelliJ plugin list and find this project, at least in the built in plugin installer, which is what I assume most people use.

from continue.

Wonderful, it now works flawlessly. Thank you very much.

from continue.