Feature Request: expanded embedding capabilities about butterfish HOT 2 OPEN

Also for this and #13, if we can figure out a solution approach that you're happy with (and that I'm capable of implementing), I'm happy to contribute.

from butterfish.

Heyo @bhazzard - these are good thoughts and I'm bullish on this. I've been most focused on shell dev and for personal reasons (just had a baby) I haven't had as much time lately for development, if you want to contribute these I'm all for it! Tag me on PRs.

Some specific notes:

• I haven't gone down the external vector store road myself because I haven't used it myself heavily and my understanding is that the threshold at which it makes sense to use a non-bruteforce vector search is actually sort of high (I've read like 100k vectors is where it starts to matter, which matches my intuition). BUT I totally buy that if you're already using an external vector store that it makes sense.
• In general the local vector management code is fairly primitive so don't be afraid of modifying it.
• If butterfish does ship with external vector store capabilities I would want to avoid installing shared libs by default, i.e. that install should be optional.
• The idea of allowing external querying similar to plugin access to your local machine is something I haven't thought of and a cool idea. If you want to implement I would copy how the plugin setup currently works.
• The plugin goes through a separate private repo of mine that I use to manage https://butterfi.sh. The pathway is OpenAI plugin -> butterfi.sh server -> local gRPC client. I will share you in to the private repo. If you implement this I'm happy to host it, up to you.
• Also if you contribute, I don't have a contributor agreement, I think it's fine to just comment with "I'll assign copyright to the butterfish project under the MIT License" and let's call it done 😃 .

Anyway I'll add you to the separate repo and reach out separately. HAVE AT IT!

from butterfish.