dhg-wei / decap Goto Github PK
View Code? Open in Web Editor NEWICLR 2023 DeCap: Decoding CLIP Latents for Zero-shot Captioning
ICLR 2023 DeCap: Decoding CLIP Latents for Zero-shot Captioning
I use t-SNE but can't get the visualization in the paper, can you share the code for embedding feature visualization?
在inference的Decoding函数中您的next_token是通过decoder解码出的logits计算得来,而后面进行文本解码的时候是使用了CLIP的tokenizer,但是这里是否存在一个问题:logits得到的张量有50257维,而CLIP的SimpleTokenizer仅能解码最大index为49407的token,会不会出现out-of-range的问题呢?我暂时的理解是你们使用了CLIP的token index来训练Decoder,在概率上避免了这个问题,但是没有在维度上面解决。刚进入该领域学习,如有理解错误请求指正。
Hi, I'd like to express my gratitude for sharing your wonderful work.
And congratulation on your paper being accepted to ICLR 2023!
To evaluate the model performance, I made an inference code and got scores using pycocoevalcap repo.
But it seems the results I got were slightly different from what you reported in paper.
So I want to refer to your code for reproducing. Can you share your inference code?
Thanks :)
Hello! Thank you for your contribution to the field.
Where can I find the model mentioned in the inference notebook ./coco_model/coco_prefix-009.pt?
Thanks very much for your excellent work and congratulations on your paper being accepted by ICLR 2023! I want to reproduce the work, and I am wondering if you could make public the code for calculating the metrics in the paper, such as BLEU@4, METEOR, CIDEr, and SPICE. I would appreciate it if you could take the time to reply to me.
COCO train2017 or COCO train2014?
Hi, thanks for your wonderful work and congratulations on your paper being accepted by ICLR 2023.
But I have some questions about the expression (i.e., "A sentence with a large norm is usually not visual-related.") in the Experiments.
Could you provide some explanation or citations?
Thanks.
Thanks for uploading the trained model.
I install the clip using the following command:
!pip install openai-clip
I receive this error when I try to run inference.
AttributeError: 'ClipGpt2Model' object has no attribute 'encode_image'
How can I solve this?
Congrats! It's a nice work for zero-shot captioning.
In the paper, zero-shot video captioning results on MSR-VTT, Activity-Net, etc. have been reported. But from the this repo, I couldn't find codes and pretraine models to perform such repreductions. I'd like to know whether these models and instructions on video caption will be relelased.
Thanks a lot!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.