yzhuoning / awesome-clip Goto Github PK

Hi!
Thanks for this great repository.
I'm searching for different papers that used CLIP for image captioning. I read image captioning papers in this repository but I think some papers can be added to this section:

• Distinctive Image Captioning via CLIP Guided Group Optimization
  paper link: link
• The Unreasonable Effectiveness of CLIP Features for Image Captioning:
  An Experimental Analysis
  paper link: link

CLIMS: Cross Language Image Matching for Weakly Supervised Semantic Segmentation
Paper: https://arxiv.org/abs/2203.02668
Code: https://github.com/CVI-SZU/CLIMS

• Learning to Prompt for Vision-Language Models [paper][code]
• Conditional Prompt Learning for Vision-Language Models [paper][code]
• Prompt-aligned Gradient for Prompt Tuning [paper][code]
• CLIP-Adapter: Better Vision-Language Models with Feature Adapters [paper][code]

CLIP-as-service is a low-latency high-scalability service for embedding images and text. It can be easily integrated as a microservice into neural search solutions.

Code: https://github.com/jina-ai/clip-as-service
Docs: https://clip-as-service.jina.ai/

Would be awesome to add to this awesome list! Thanks in advance!

Hi!

This is an excellent collection of CLIP-related works. We recently put out a preprint on prompt learning. It would be awesome if you could include our work under the Prompt Learning section. Below are the details:

• Learning to Compose Soft Prompts for Compositional Zero-Shot Learning [paper] [code]

Please let me know if you want me to send a PR instead.

Thank you!

Hello!

Thanks for creating this repository, it is super useful!

My colleagues and I recently finalized the work on using CLIP for information retrieval in e-commerce domain. The paper is called 'Extending CLIP for Category-to-image Retrieval in E-commerce', we presented it on ECIR 2022 a couple of month ago.
I would really appreciate it if you could add it to the Information Retrieval subsection. Here is the markdown code in case it is helpful:

• Extending CLIP for Category-to-image Retrieval in E-commerce [paper]

Please let me know if you'd rather me send a pull request.

Thank you!

Thanks for creating this repository! It's a very comprehensive source of information.

Could you please add our ICML 2023 paper, POUF: Prompt-oriented unsupervised fine-tuning for large pre-trained models
? The code is provided in this link?

We appreciate your help. Thanks!

In the Representation Learning part, the code link following DetCLIP is DeCLIP. I can't find the code of DetCLIP. Maybe it is close source.

Text2Mesh : Their approach can modify a given mesh with given text/image information via CLIP text/image encoder.

Detecting Twenty-thousand Classes using Image-level Supervision : Which is a object detection research by facebook, they use CLIP text embedding as classifier weight.

The above papers are I want to add.

I think Crop-CLIP should be put at Object Detection

For more image manipuliation / generation applications, I summarized in my medium
You can add them if you think they are valueable.
Text-Driven Image Manipulation/Generation with CLIP

The list of above image is allocated at this google sheet

Hi Zhuoning,

Thanks for your contribution for this nice repo!

Just wanted to give a update that our work RegionCLIP (CVPR 2022) is now public (https://github.com/microsoft/RegionCLIP). Feel free to give it a try!

PS: the name of RegionCLIP was misspelled.

Best,
Yiwu

This repository is very useful to learn about the works bootstrapping off CLIP, thank you for curating it!

We have just published on arXiv a work that investigates how to best use pseudolabels generated by CLIP to enhance CLIP itself. We believe this work to have good applicability for practitioners that want to adapt CLIP to novel tasks efficiently and with limited, or no, labeled data.

You can find the paper here and the code here

I'm happy to submit a pull request if needed :)