Comments (6)
This may be caused by the size of your image. Actually the limit size of images supported by gemini-pro-vision is 4MB.
from generative-ai-android.
I'm running into this as well when inputting images that are less than 4Mb. I'm using the gemini-pro-vision model, which throws the stacktrace on inputs less than 4Mb:
com.google.ai.client.generativeai.type.ServerException: Request payload size exceeds the limit: 4194304 bytes.
I noticed the sample app uses images of 768x768, which does work.
from generative-ai-android.
I'm running into this as well when inputting images that are less than 4Mb. I'm using the gemini-pro-vision model, which throws the stacktrace on inputs less than 4Mb:
com.google.ai.client.generativeai.type.ServerException: Request payload size exceeds the limit: 4194304 bytes.
I noticed the sample app uses images of 768x768, which does work.
Hmm that's interesting. Would you be able to provide an example image that causes this?
from generative-ai-android.
So all I'm doing is capturing an image using MediaStore's ACTION_IMAGE_CAPTURE intent. This creates a ~6Mb picture for my test device. I'm also focusing on popular 4:3 aspect ratios, so I scale the image down to meet the 4Mb limit, while keeping the ratio.
2048x1536 does not work, even though it is 2.5Mb in the file system
The largest image I have got to work is 1152x864, which is 1.8Mb.
This is all with only one image in the content builder, no text prompting. I have tested with .jpg and .bmp, jpeg compression was:
bmp.compress(Bitmap.CompressFormat.JPEG, 80, stream)
Interestingly, if I take a completely black picture, it does process at the higher resolutions, and does correctly limit payloads larger than 4Mb.
from generative-ai-android.
The ImagePart
is provided as a convenience to make sending images easier, it automatically converts the images to a PNG formatting of max quality. If you are custom-compressing the image to fit a size limit, it may be better for you to use BlobPart
so you can manually specify and control the encoding and compression.
Also of note, is that images get converted to Base64 while being sent to the server, increasing their size by a further 35%
However, your example of a file that is 2.5mb on the file system should work if sent as a BlobPart (since this would be approximately 3.4MB).
Your code example can be changed like this
val baos = ByteArrayOutputStream()
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, baos)
val inputContent = content {
blob("image/jpeg", boas.toByteArray())
text("Read text from image")
}
from generative-ai-android.
Thanks for the response, that does answer some questions. I think in that case the Android documentation is a bit deceptive:
https://ai.google.dev/docs/gemini_api_overview#image_requirements
-
"Maximum of 4MB for the entire prompt, including images and text" should be maximum of 4MB post-encoding
-
"Images must be in one of the following image data MIME types:
PNG - image/png
JPEG - image/jpeg
WEBP - image/webp
HEIC - image/heic
HEIF - image/heif"
From what I can find, the documentation doesn't describe the BlobPart, and the code documentation doesn't differentiate between BlobPart and ImagePart, it says 'represents binary' or 'represents image' data. Is there more documentation on this?
from generative-ai-android.
Related Issues (18)
- Migrate to Kotlin Multiplatform HOT 3
- As a programming enthusiast, I had some problems deploying projects HOT 1
- Something went wrong while trying to deserialize a response from the server HOT 9
- Use the same version of gradle-wrapper and agp HOT 1
- I am getting issue in Release build HOT 3
- kotlin version HOT 1
- Expose the HttpClient to allow proxies etc HOT 1
- Request for Function Calling Feature HOT 1
- Using the sdk on my flutter app through MethodChannel HOT 6
- API available to more countries HOT 1
- add generative-ai-java on springboot
- not available in Europe HOT 1
- Failed to send message HOT 6
- [v0.2.1] API Error 400: please use a valid role: user, model
- Sample
- New release HOT 10
- Obs project contribute
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from generative-ai-android.