text-generation-webui label:question about butterfish HOT 4 CLOSED

Hey @txirrindulari, I've just pushed a change that enables you to set a Base URL for the model API. In other words, this allows you to point butterfish at a different server than the OpenAI api. In theory text-generation-webui supports the OpenAI API but I haven't been able to get models working on text-generation-webui to test this out.

If you want to try this, you can install butterfish from the latest commit (I haven't yet released), and then set the Base URL. Something like:

go install github.com/bakks/butterfish/cmd/butterfish@latest
$(go env GOPATH)/bin/butterfish shell -u http://localhost:1234/v1 -A

Assuming that you're running text-generation-webui at port 1234, etc.

from butterfish.

You have to run text-generation-webui (server.py) with --extensions openai then point your butterfish to http://127.0.0.1:5000/v1 by default
Here's the docs for the openai extension. It says to use --api, but that's old. https://github.com/oobabooga/text-generation-webui/wiki/12-%E2%80%90-OpenAI-API

Example: go/bin/butterfish summarize .bashrc -u "http://127.0.0.1:5000/v1" -vv
Don't forget the quotes around the address.

Anyhow.. This thing IS working! I'm so glad. THIS is AI, to me, anyways. It's so useful!

from butterfish.

I just tried it with $(go env GOPATH)/bin/butterfish shell -u http://127.0.0.1:7860 -A
It instantly 404'd - "Error prompting LLM: error, status code: 404, message: %!s()"

from butterfish.

That's awesome @JeremyBickel, thanks for testing and adding your commands! Will make a note of it in the readme.

from butterfish.