OpenAI CLIP model for photo search based on OpenVINO acceleration

In this project, we use OpenVINO for OpenVINO CLip model acceleration, and construct one photo search demo with converted model.

• A description text, encoded once
• The entire collection of images, encoded only once and will be indexed and resued later

• The K most similar images

Follow Below guide to set up the virtual environment

conda create -n openvino_env python=3.8
conda activate openvino_env
# Upgrade pip to the latest version to ensure compatibility with all dependencies
python -m pip install --upgrade pip==21.3.*
pip install -r requirements.txt

You can follow below guide to get converted CLIP image model and text model, and put them under ir folder, or even convert it to lower precision with nncf OpenVINO notebook for clip model conversion

In current code the images folder under data will be searched recursively, pls put your images under data folder

python clip-search.py

Below please find a video show the demo