Talk is cheap, show me your code. Code is cheap, show me your proof.
rorical / clip-as-service-rs Goto Github PK
View Code? Open in Web Editor NEWA blazing fast CLIP gRPC service in rust.
A blazing fast CLIP gRPC service in rust.
Hey there! ๐
I tried running this project using ViT-B-32-laion2b-s34b-b79k
and ViT-L-14@336px
. However, for both models I received the following error when attempting to generate text embeddings:
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value:
SessionRun(Msg("
Non-zero status code returned while running Add node.
Name:'Add_2'
Status Message: /Users/runner/work/1/s/onnxruntime/core/providers/cpu/math/element_wise_ops.h:560
void onnxruntime::BroadcastIterator::Append(ptrdiff_t, ptrdiff_t) axis == 1 || axis == largest was false.
Attempting to broadcast an axis by a dimension other than 1. 4 by 77
"))
', src/main.rs
The error occurs at the following location where the textual model is executed by the ONNX runtime:
clip-as-service-rs/src/server.rs
Line 103 in 1f79631
After digging a little and comparing it to the example code found in the OpenCLIP repo, I found that the input tensor should always have a length of 77 (padded by 0
).
I am not sure if or why the code works for you as-is but for me applying the following changes solved the issue:
77
tokenizer.with_padding(Some(PaddingParams {
strategy: PaddingStrategy::BatchLongest,
direction: PaddingDirection::Right,
pad_to_multiple_of: Some(77), // <-- this right here
pad_id: 0,
pad_type_id: 0,
pad_token: "[PAD]".to_string(),
}));
v1
and v2
arrays so that the iterator takes at most 77
elements (as the tokenizer might produce any multiple of it)let v1: Vec<i32> = preprocessed
.iter()
.map(|i|
i.get_ids()
.iter()
.take(77) // <-- this right here
.map(|b| *b as i32)
.collect()
)
.concat();
let v2: Vec<i32> = preprocessed
.iter()
.map(|i| {
i.get_attention_mask()
.iter()
.take(77) // <-- this right here
.map(|b| *b as i32)
.collect()
})
.concat();
No idea if this is the best approach but I thought I'd just leave my findings here in case anyone else stumbles across this error (or if you @Rorical encountered this as well) ๐
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.