Comments (4)
Hey @txirrindulari, I've just pushed a change that enables you to set a Base URL for the model API. In other words, this allows you to point butterfish at a different server than the OpenAI api. In theory text-generation-webui supports the OpenAI API but I haven't been able to get models working on text-generation-webui to test this out.
If you want to try this, you can install butterfish from the latest commit (I haven't yet released), and then set the Base URL. Something like:
go install github.com/bakks/butterfish/cmd/butterfish@latest
$(go env GOPATH)/bin/butterfish shell -u http://localhost:1234/v1 -A
Assuming that you're running text-generation-webui at port 1234
, etc.
from butterfish.
You have to run text-generation-webui (server.py) with --extensions openai then point your butterfish to http://127.0.0.1:5000/v1 by default
Here's the docs for the openai extension. It says to use --api, but that's old. https://github.com/oobabooga/text-generation-webui/wiki/12-%E2%80%90-OpenAI-API
Example: go/bin/butterfish summarize .bashrc -u "http://127.0.0.1:5000/v1" -vv
Don't forget the quotes around the address.
Anyhow.. This thing IS working! I'm so glad. THIS is AI, to me, anyways. It's so useful!
from butterfish.
I just tried it with $(go env GOPATH)/bin/butterfish shell -u http://127.0.0.1:7860 -A
It instantly 404'd - "Error prompting LLM: error, status code: 404, message: %!s()"
from butterfish.
That's awesome @JeremyBickel, thanks for testing and adding your commands! Will make a note of it in the readme.
from butterfish.
Related Issues (20)
- Feature Request: expanded embedding capabilities HOT 2
- Feature Request: Safe Retrieval Plugin
- Error: [404:invalid_request_error] The model: `gpt-4` does not exist HOT 7
- Error: system message is required for full chat completion on summarize request HOT 3
- Feature Request: Enhanced Multiline and Clipboard Input in `butterfish prompt` HOT 13
- Feature Request:Azure OpenAI support HOT 5
- Feature Request: Shell Specific Shortcuts HOT 1
- LLM prompts get interpreted as commands with zsh/prezto HOT 3
- Bugs with text in bash/gnome terminal HOT 5
- Question: Is this compatible with atuin HOT 4
- Feature Request: fish shell support HOT 3
- Prompt mode context HOT 1
- Feature request: offer ti runa command after `gencmd` suggestion
- Feature request: option to remove all timeouts/deadlines for slow local LLMs
- please close
- [nit] When specifying a locally hosted api endpoint, butterfish still demands an API token for OpenAI
- [Feature Request] Enable using other AI sources as a first class citizen
- [Bug] panic on interface conversion when using the verbose log option HOT 4
- using a local llm HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from butterfish.