Comments (5)
@klxu03 This is a good idea. I've been trying to improve the vision prompt by telling GPT to reference its previous action. But also giving it a stack of previous images would allow it to reference even farther back. You would need to experiment with the downsampling and size of the previous image stack to not hit token limits as @shubhexists mentioned. Feel free to open a PR/draft demonstrating this change!
from self-operating-computer.
Out of curiosity, Doesn't the Vision model has any sort of "Token Limit" like it has in GPT4?
from self-operating-computer.
Got it. I'll go draft up a PR for the previous image stacks! I'm planning on just making it an immediate previous message in pseudo_messages.
Also, regarding the token limit, correct me if I'm wrong, is the GPT4V context window up to 128k tokens?
If so, would there be a real concern about hitting the token limit? With some calculations, where x = one dimension of maximum image size you get x = 14044, meaning you can pass up to a 14044px x 14044px image. To hit the token limit, you would need to pass a lot of images to hit 2 * 10^8 total pixels.
from self-operating-computer.
@klxu03 Can this be closed since #57 was merged?
from self-operating-computer.
closing since old and less relevant now
from self-operating-computer.
Related Issues (20)
- [Question] About the Third-party API
- [FEATURE] No update instructions?
- [BUG] WINDOWS install not finding gpt-4-with-ocr HOT 5
- [BUG] Unable to activate the virtual environment
- [BUG] Not running on Ubuntu 22.04.4 LTS HOT 3
- CogVLM Support - A better LLaVa
- [BUG] -m gemini-pro-vision asking for OPENAI_API_KEY HOT 2
- [FEATURE] Add Remote Ollama Capability
- [BUG] Cannot seem to select the right emails to delete.
- [FEATURE] Learning Process HOT 1
- [FEATURE] GUI Interface and further connectivity
- [BUG] operate -m llava return error local variable 'content' referenced before assignment
- [BUG] ModuleNotFoundError: No module named 'pkg_resources' HOT 4
- [BUG] Need GPT-4 ? HOT 1
- [FEATURE] Azure open AI support HOT 2
- OpenSource free Vision model use Instead of openAI HOT 5
- Github
- [Linux]: X get_image failed: error 8 (73, 0, 1316) [Error] --> cannot access local variable 'content' where it is not associated with a value HOT 2
- [For be deleted]
- [BUG] No such file or directory Xauthority
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from self-operating-computer.