A generalized gradient-based CNN visualization technique code for the paper:
To be presented at WACV 2018,
Authors:
Aditya Chattopadhyay*,
Anirban Sarkar*,
Prantik Howlader* and
Vineeth N Balasubramanian,
(* equal contribution)
Above we do a comparision of the performance of gradcam++ with respect to grad-cam. Gradcam++ does a better image localization that Gradcam, not only in scenarios where there is more than one object of same class in an image, but also in cases where there is a single object in an image.
Implementation in python using tensorflow 1.3.
Kindly download the pretrained weights of the vgg16 network (vgg16.npy) from the following link, and copy the file to the models/
subdirectory.
https://drive.google.com/drive/folders/0BzS5KZjihEdyUjBHcGFNRnk4bFU?usp=sharing
python classify.py -f desc_image/water-bird.JPEG -gpu 3 -o output.jpeg
f
: path to input imagegpu
: the gpu id to use, 0-indexedl
: class label, default is -1 (chooses the class predicted by the model)o
: Specify output file name for Grad-CAM++ visualization, default isoutput.jpeg
. All results would be saved in theoutput/
subdirectory.
python classify.py -h
The above code is for the vgg16 network, pre-trained on imagenet.
We tested our code on tensorflow 1.3, compatibility with other versions is not guaranteed.
Parts of the code have been borrowed and modified from:
https://github.com/Ankush96/grad-cam.tensorflow
https://github.com/insikk/Grad-CAM-tensorflow
https://github.com/ry/tensorflow-vgg16
@article{chattopadhyay2017grad,
title={Grad-CAM++: Generalized Gradient-based Visual Explanations for Deep Convolutional Networks},
author={Chattopadhyay, Aditya and Sarkar, Anirban and Howlader, Prantik and Balasubramanian, Vineeth N},
journal={arXiv preprint arXiv:1710.11063},
year={2017}
}
P.S. Currently our work only supports classification probems with CNN, we plan to soon include support for Image Captionin, VQA and other such CNN based architectures.
- Windows 10
- python 3.5
- tensorflow 1.4
- https://github.com/tensorflow/models/tree/master/research/slim
- Download ckpt file and move the file to "checkpoint" folder