dhg-wei / decap Goto Github PK

Hi, I'd like to express my gratitude for sharing your wonderful work.
And congratulation on your paper being accepted to ICLR 2023!

To evaluate the model performance, I made an inference code and got scores using pycocoevalcap repo.
But it seems the results I got were slightly different from what you reported in paper.
So I want to refer to your code for reproducing. Can you share your inference code?

Thanks :)

Thanks for uploading the trained model.

I install the clip using the following command:
!pip install openai-clip

I receive this error when I try to run inference.
AttributeError: 'ClipGpt2Model' object has no attribute 'encode_image'

How can I solve this?

COCO train2017 or COCO train2014？

Hello! Thank you for your contribution to the field.

Where can I find the model mentioned in the inference notebook ./coco_model/coco_prefix-009.pt?

Hi, thanks for your wonderful work and congratulations on your paper being accepted by ICLR 2023.
But I have some questions about the expression (i.e., "A sentence with a large norm is usually not visual-related.") in the Experiments.
Could you provide some explanation or citations?
Thanks.

在inference的Decoding函数中您的next_token是通过decoder解码出的logits计算得来，而后面进行文本解码的时候是使用了CLIP的tokenizer，但是这里是否存在一个问题：logits得到的张量有50257维，而CLIP的SimpleTokenizer仅能解码最大index为49407的token，会不会出现out-of-range的问题呢？我暂时的理解是你们使用了CLIP的token index来训练Decoder，在概率上避免了这个问题，但是没有在维度上面解决。刚进入该领域学习，如有理解错误请求指正。

Congrats! It's a nice work for zero-shot captioning.
In the paper, zero-shot video captioning results on MSR-VTT, Activity-Net, etc. have been reported. But from the this repo, I couldn't find codes and pretraine models to perform such repreductions. I'd like to know whether these models and instructions on video caption will be relelased.

Thanks a lot!

Thanks very much for your excellent work and congratulations on your paper being accepted by ICLR 2023! I want to reproduce the work, and I am wondering if you could make public the code for calculating the metrics in the paper, such as BLEU@4, METEOR, CIDEr, and SPICE. I would appreciate it if you could take the time to reply to me.

I use t-SNE but can't get the visualization in the paper, can you share the code for embedding feature visualization?