Giter Club home page Giter Club logo

generative_mlzsl's Introduction

PWC

Generative Multi-Label Zero-Shot Learning

(* denotes equal contribution)

Webpage: https://akshitac8.github.io/GAN_MLZSL/

Overview

This repository contains the implementation of Generative Multi-Label Zero-Shot Learning.

In this work, we tackle the problem of synthesizing multi-label features in the context of zero-shot setting for recognition all (un)seen labels with a novel training mechanism.

Installation

The codebase is built on PyTorch 1.1.0 and tested on Ubuntu 16.04 environment (Python3.6, CUDA9.0, cuDNN7.5).

For installing, follow these intructions

conda create -n mlzsl python=3.6
conda activate mlzsl
conda install pytorch=1.1 torchvision=0.3 cudatoolkit=9.0 -c pytorch
pip install matplotlib scikit-image scikit-learn opencv-python yacs joblib natsort tqdm pandas h5py==2.10.0

Data Preparation

Training using NUS-WIDE dataset:

Download the NUS-WIDE features, tags and other required training files from the drive link shared below.

link: https://drive.google.com/drive/folders/1tCo-xawWrnGQGaWYJEKQOQ31ts__rAse?usp=sharing

Extract them in the ./datasets folder.

Training using custom dataset:

Download the custom dataset in the same data folder. Please make sure to convert your custom dataset in the same format as NUS-WIDE.

python preprocess.py --image_dir data/custom_data/ --output_dir data/custom_data_jsons/ --train_json custom_data_train  --test_json custom_data_test

Above preprocessing step will create train and test jsons with ZSL and GZSL requirements.

The train and test jsons are used as an input when running feature extraction code.

python extract_4096_features.py --train_json custom_data_train  --test_json custom_data_test --gpu

Above feature extraction will save features in .h5 format which is used for training for our CLF model.

Training and Evaluation

NUS-WIDE

To train and evaluate zero-shot learning model on full NUS-WIDE dataset, run:

sh ./scripts/train_nus_wide.sh

Model Checkpoint

We also include the checkpoint of the zero-shot generative model on NUS-WIDE for fast evaluation in weights folder. Please download the pretrained weights according to the intructions within the folder. To reproduce results, run:

sh ./scripts/eval_nus_wide.sh

Citation

If this code is helpful for your research, we would appreciate if you cite the work:

@article{gupta2021generative,
  title={Generative Multi-Label Zero-Shot Learning},
  author={Gupta, Akshita and Narayan, Sanath and Khan, Salman and Khan, Fahad Shahbaz and Shao, Ling and van de Weijer, Joost},
  journal={arXiv preprint arXiv:2101.11606},
  year={2021}

Acknowledgments

I thank Dat Huynh for discussions and feedback regarding the evaluation protocol and sharing details for the baseline zero-shot methods. I thank Aditya Arora for suggestions on the figure aesthetics.

generative_mlzsl's People

Contributors

akshitac8 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.