Giter Club home page Giter Club logo

cnn-for-image-retrieval's Introduction

CNN for Image Retrieval

License

博文:Image retrieval using MatconvNet and pre-trained imageNet,对应web演示主页picSearch

2017/10/08: 构建CBIR检索对比框架cnn-cbir-benchmark,包括Fisher Vector, VLAD, FC, RMAC, CROW.

2017/08/15更新:增加Python版本,Caffe版本Keras版本

2015/12/31更新:添加对MatConvNet最新版version 1.0-beta17的支持,预训练的模型请到Matconvnet官网下载最新的模型。

2015/12/31更新:添加对MatConvNet最新版version 1.0-beta17的支持,预训练的模型请到Matconvnet官网下载最新的模型。

2015/10/20更新:Web演示部分代码公开CNN-Web-Demo-for-Image-Retrieval

2015/09/24更新:添加对MatConvNet最新版version 1.0-beta14的支持。

2015/12/31更新:添加对MatConvNet最新版version 1.0-beta17的支持,删掉原来的版本(预训练的模型请到matconvnet官网下载最新的模型)。

2015/06/29更新:添加对MatConvNet最新版version 1.0-beta12的支持。

注意:其中文件夹matconvnet-1.0-beta17是已经编译好了的,鉴于MatConvNet只能在Matlab 2014及其以上以及系统必须是64位,所以在使用此工具箱之前得满足这两个条件。如果是Pythoner,推荐使用flask-keras-cnn-image-retrieval,纯Python,非常易于写成在线图像搜索应用。

Caltech-256图像数据库上搜索结果

search result

运行步骤

1). 如果不需要计算mAP的话,那就直接把你的图像库文件夹名字命名为database,并将图片全部放在放在database文件夹下即可。如果你要在后面计算MAP(平均检索精度)的话,要确保图像数据库做成文件夹databaseClassified中的形式,然后执行下面命令:

python movefiles.py

2). 接着便可以抽取特征。运行extractCNN.m,要用parfor并行的话,直接修改注释部分即可。

3). 检索可视化。这一步运行queryInDatabaseDemo.m即可。

4). 计算mAP。不需要计算MAP的这步略过。运行compute_MAP.m,关于mAP的计算,请参阅我画的mAP计算过程示意图:信息检索评价指标,这个计算mAP的脚本是按照那个流程中定义的mAP计算方式来写的。

降维

非常的amazing, 除了验证降维到128D后损失不减外,惊奇地发现4096D的CNN降维到128D后精度还有提高,一种可能的解释:CNN特征也有一定的信息冗余,信息冗余所带来的影响比降维所带来的损失的影响要更大。结论:You should reduce the dimension of CNN when you use if.

PCA降维对CNN特征的影响

search result

上面实验使用的是本项目代码,图像数据集使用的是Caltech101。

关于PCA对PCA降维的影响,Neural Codes for Image Retrieval中也有探讨,以及曾跟Adrian Rosebrock也有过这方面的交流:

ANN is really fantastic, it makes such much easier. You could also try something like PCA on your 4096-d vector and try to get it down to 128-d. It would save some space and (ideally) not hurt accuracy.

所以,如果采用了CNN特征的话,推荐将其降维到128D。

CNN资源列表

C++

conv-net-version-3,对应博客Convolutional Neural Networks III

Python

Keras,强力推荐

Keras资源列表:

DeepLearning tutorial(6)易用的深度学习框架Keras简介

DeepLearning tutorial(7)深度学习框架Keras的使用-进阶

Keras VGG-16模型 VGG16 model for Keras

PDNN,对应主页PDNN: A Python Toolkit for Deep Learning

Matlab

GoogLeNet, A GPU Implementation of GoogLeNet.

cnn-for-image-retrieval's People

Contributors

willard-yuan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cnn-for-image-retrieval's Issues

blog打开不了

你好,
我打开你readme里面的link想看看对应的blog,但是打开不了

faiss

大神有没有评估过faiss在处理咱这一系列绕着image retrieval的工作中有多大实际的提升吗

ESP GAME

Do you have the ESP Game dataset? If you have, would you give me the dataset?
Thank you.

PicSearch demo?

楼主您好,请问您博客中的Pic Search 的demo- “CNN-Web-Demo-for-Image-Retrieval”. 还开放吗? 链接好像失效了.....也没有找到,不知道可否发一下链接呢? 谢谢。

cnn提取的特征是否还有必要用hash

首先感谢大佬的cnn case和各种hash benchmark!致敬!想请教下大佬,我用vgg提取了2048d的特征,在用hash索引的方法是不是不太好,我看到你用的是矩阵相乘排序的方式,因为我的图片大概有500多G 抽取下来的特征大概也有几个G,加载到内存有点太伤了,另外看到你关于CNN 抽取特征做PCA的实验非常感谢指点,这个也是个思路(有点好奇cnn输出的特征做dense输出和pca的效果),但是最终 有必要用hash搜索吗,是否会有很大的损失。目前我试过2048D 做LSH 效果 很不太好。

Mistake in computing average precision

Hi, I think there is mistake in computing average precision as ap(i,1) = sum(precision)/queryClassNum. According to the formula, the denominator should be the non-zero items in precision vector rather than the number of retrieval categories, right?

error in extractCNN

hi ,
my error is :

res = vl_simplenn(net, im_) ;

"Attempt to execute SCRIPT vl_nnconv as a function:
C:\Users\Mehrdad\Source\Repos\CNN-for-Image-Retrieval\matconvnet-1.0-beta17\matlab\vl_nnconv.m"
how can i solve it?

question for video retrieval

hi, i am very interested in your work on image retrieval, and my question is how can i apply it to the video retrieval domains. there mainly two pints make me confused:
firstly, image retrieval take CNN as the feature extractor , and the CNN mainly trained with the classification loss on such cifar-10 or landmark dataset(Oxf5k), can it generalize to the actual video scenarios(usually are not landmark)? fine-tuning needed? and how?
secondly, how can i calculate the similarity between the query video and reference video, my method is dividing video into key frames and match the similar frames by extracting the frame features
can you give me some opinion. thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.