Comments (19)
Hey y'all, this is a real easy patch ~
Just add this block between the models listed in Master/types/ai-models.ts
'gpt-3.5-turbo-16k': {
id: 'gpt-3.5-turbo-16k',
maxLength: 48000,
tokenLimit: 16000,
requestLimit: 12000,
vendor: 'OpenAI',
},
The model will show up in the dropdown menu :)
from unsaged.
I just removed the PossibleAiModels types integration and returning every possible AiModel now, but I dont know how to get the correct information for maxLength, tokenLimit, requestLimit from the OpenAi API, any idea?
https://github.com/sebiweise/unSAGED/tree/feature/aimodel_vendor_update
Ollama isn´t working either but there is no endpoint for now - like you said.
from unsaged.
Hi @jorge-menjivar, just wondering if there was any update on adding these models?
from unsaged.
Several useful OpenAI models are missing from the types. For example gpt-3.5-turbo-16k as well as the dated models.
gpt-3.5-turbo-16k has been added
from unsaged.
@jorge-menjivar I think this issue can be closed
from unsaged.
@sebiweise I would like to make it possible to have the option to see all models including the versioned ones, but most importantly models with dynamic names like fine-tuned models. Maybe as an advanced feature that is disabled by default. This would require some moderate changes to our models structure though, which is why I'm leaving for later.
from unsaged.
You mean like a database table just for available models instead of the types integration?
from unsaged.
Yeah something like that would do it. The other issue is getting the proper max token length for arbitrary models. I know ollama at least, doesn’t have a way to do this in their endpoints API so I will need to investigate and do a pr to them (I asked on discord, and no response so I’m assuming there is no way). If we were to do this, there would not be a need to implement new models manually anymore, unless something substantial is different about the model.
from unsaged.
We might need to wait for support for this to come from OpenAI or we might have to come up with a new solution to the problem. For OpenAI models we can parse the model id and assume their base model from it.
Fine-tuned models also include the name in the id. They are all named something like ft:gpt-3.5-turbo-0613:xxxxxxxxxxxx
. Maybe letting the user manually set the max tokens per model in the UI could be another solution in the worst of cases.
from unsaged.
@jorge-menjivar What do you think of a new service that will be hosted by me or us, that will have a database of every possible AI model and that will offer a submit form to let users post new found AI models and/or vendors that they found. Maybe we could create a cron job that will gather new known AI models from known vendors automatically. When the models are saved we can have something like a status "draft" so someone can complete the corresponding model settings.
And then we have an endpoint that will return the models including the settings and also offer some filters so we can decide which vendors or model types (text/image/...) I want to get via the API call.
from unsaged.
I like this idea. So basically it's a community database/index for models?
from unsaged.
Also check the pre-release app v0.1.1. I made it so that it detects all models and let's you put the correct token window size in the settings
from unsaged.
Yes, so basically just the types we have now in a database that can be contributed to easily, I would create a new service for that so that we don´t have a problem with another endpoint that isn´t available in the desktop app.
I would just try to build a simple version to test, just a quick Nextjs application with Prisma and a free Planetscale database, Clerk auth for some admin pages and a public available submit form that will create models that will need to be check by a admin.
So then you can add another api call to that service (maybe later on including a api key) and get the "possible ai models" including all settings, you still need to get all models that the supported vendors in your app support but you could get the settings for max_tokens and so on from another endpoint.
from unsaged.
Just created a basic Nextjs App, for now it will just display the data from the Planetscale db but I´m currently working on the submit form and a little admin dashboard: https://ai-services-web.vercel.app/
https://ai-services-web.vercel.app/vendor
https://ai-services-web.vercel.app/model
The submit forms aren´t working at the moment:
https://ai-services-web.vercel.app/form/model
https://ai-services-web.vercel.app/form/vendor
from unsaged.
@sebiweise Thats really awesome. If vercel gets too slow to handle this let me know. I have over a dozen dedicated servers that I can get you a VPS to run this on at no cost.
from unsaged.
I'd be happy to add missing params if you need help
from unsaged.
Thank you for your help. I think we can just try to use Vercel for now and we will just need to have a look on the insights/usage of the endpoint. I think the database will need an upgrade later on but we will see. The data will be cached so I think for now no problem.
I will finish the database scheme and the management dashboard tomorrow so I can invite you to be able to edit/add ai models.
from unsaged.
@sebiweise Looking great! Do we need to keep the max length and request limit? I removed them from the desktop app because they seemed redundant. Max length is just a guess for most models, and request limit could be set to token_limit - 100.
- Max length is used for the max length of input text so if you go over a certain number of characters while typing, it will show you a warning. This is of course a guess because we cannot compute ahead of time the number of tokens a certain number of characters will use. The rule of thumb for now for this value has been token_limit * 3 or something like that.
- Request limit is the max number of tokens to send in the request, so if we use token_limit - 100 for its value, we guarantee that the model can return at least 100 tokens in the response.
from unsaged.
Yes I think we will have to take a look if we can remove some of the params and the params are need or not. I just pushed a working example that uses the "AI-Services" to get the possible ai models and the correct params/limits.
#163
Main implementation (maybe we can change some code in the future to reduce the http calls that are done):
https://github.com/jorge-menjivar/unsaged/pull/163/files#diff-b2b8156aafdf2a2697ee5d5d733f9e17a1d4e89734f8b97ab410121180b3baf0
from unsaged.
Related Issues (20)
- Unsupported URL Type "workspace:": workspace:* HOT 13
- [BUG] Models not loading HOT 1
- [Feature] - Allow document upload HOT 2
- [Docs] Bad link for generation script on Getting Started page HOT 2
- [Feature] Add Ollama URL input in the settings tab
- [Feature] Add AWS Bedrock
- [Feature] Update Image generation
- [Feature] Add all previous supported languages
- Blank Page Issue with Latest Build in Docker Environment HOT 4
- [BUG] npm dependency issue HOT 2
- [Feature] Add GPT Vision Models HOT 1
- [Feature] Gemini Pro Support HOT 4
- [BUG] TypeError: Cannot read properties of undefined (reading 'inTable')
- [BUG] Docker image does not start: `next: command not found` HOT 1
- [FEATURE] Support for LaTeX
- No Logs in Cloudflare AI Gateway [BUG]
- [Help Needed] Fix support for other languages HOT 1
- [Help Needed] - Is a on-prem supabase install supported ? HOT 5
- [Help Needed] Transition from NextAuth to Supabase Auth for User Authentication HOT 1
- [Question] - 401 on access to conversation table in Supabase HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from unsaged.