zhmiao / opencompounddomainadaptation-ocda Goto Github PK

View Code? Open in Web Editor NEW

139.0 7.0 15.0 1.57 MB

Pytorch implementation for "Open Compound Domain Adaptation" (CVPR 2020 ORAL)

Home Page: https://liuziwei7.github.io/projects/CompoundDomain.html

License: BSD 3-Clause "New" or "Revised" License

Python 100.00%

open-compound-domain-adaptation pytorch-implementation computer-vision domain-adaptation ocda cvpr2020 deep-learning

opencompounddomainadaptation-ocda's Introduction

Open Compound Domain Adaptation

[Project] [Paper] [Demo] [Blog]

Overview

Open Compound Domain Adaptation (OCDA) is the author's re-implementation of the compound domain adaptator described in:
"Open Compound Domain Adaptation"
Ziwei Liu^*, Zhongqi Miao^*, Xingang Pan, Xiaohang Zhan, Dahua Lin, Stella X. Yu, Boqing Gong (CUHK & Berkeley & Google) in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020, Oral Presentation

Further information please contact Zhongqi Miao and Ziwei Liu.

Requirements

PyTorch (version >= 0.4.1)
scikit-learn

Updates:

11/09/2020: We have uploaded C-Faces dataset. Corresponding codes will be updated shortly. Please be patient. Thank you very much!
06/16/2020: We have released C-Digits dataset and corresponding weights.

Data Preparation

[OCDA Datasets]

First, please download C-Digits, save it to a directory, and change the dataset root in the config file accordingly. The file contains MNIST, MNIST-M, SVHN, SVHN-bal, and SynNum.

For C-Faces, please download Multi-PIE first. Since it is a proprietary dataset, we can only privide the data list we used during training here. We will update the dataset function accordingly.

Getting Started (Training & Testing)

C-Digits

To run experiments for both training and evaluation on the C-Digits datasets (SVHN -> Multi):

python main.py --config ./config svhn_bal_to_multi.yaml

After training is completed, the same command will automatically evaluate the trained models.

C-Faces

We will be releasing code for C-Faces experiements very soon.

C-Driving

Please refer to: https://github.com/XingangPan/OCDA-Driving-Example .

Reproduced Benchmarks and Model Zoo

NOTE: All reproduced weights need to be decompressed into results directory:

OpenCompoundedDomainAdaptation-OCDA
    |--results

C-Digits (Results may currently have variations.)

Source	MNIST (C)	MNIST-M (C)	USPS (C)	SymNum (O)	Avg. Acc	Download
SVHN	89.62	64.53	81.17	87.86	80.80	model

License and Citation

The use of this software is released under BSD-3.

@inproceedings{compounddomainadaptation,
  title={Open Compound Domain Adaptation},
  author={Liu, Ziwei and Miao, Zhongqi and Pan, Xingang and Zhan, Xiaohang and Lin, Dahua and Yu, Stella X. and Gong, Boqing},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2020}
}

opencompounddomainadaptation-ocda's People

Contributors

Stargazers

Watchers

Forkers

killsking peterzs trantorrepository yuhonghong95721 deeplearning2012 aaron2bin sizhewei alva-2020 prannoy-noel bruinxiong ankushpanwar19 sorrowyn eunjuyang langyijun degraded-ai-vision-lab

opencompounddomainadaptation-ocda's Issues

Motivation behined Concept and Domain Factor Selector Linear Networks

As I understand from the code, instead if using the Domain Factor network for weighting the memory value as explained in the paper, you train a "Concept Selector" and a "Domain Factor Selector" linear networks to output weights for both the memory and the domain factor net output to get the final v_transfer value when combing with the direct encoder output.

Can you please explain the motivation behind this change?
Can you explain the motivation behind training Man network before the domain factor network and not training both concept and domain factor selectors at once? (and in short, why not remove step 2)
Can you explain how these changes are applied in the semantic segmentation task training?

Thanks.

Missing dataset "C-Faces" in https://liuziwei7.github.io/projects/CompoundDomain.html

Hi, I have recently started working on Domain Adaption. I am trying to reproduce the results from your code. I followed the gdrive link from github site and found that the C-Faces was missing, Can you please add this dataset.
Thank you

Semantic Segmentation Differences

Hi,
First of all thanks for this repository. The approach in this article is very interesting.
I have a few questions about the implementation:

The training session in main.py consists of training MANN net without scheduling before training the Disentangle Domain Factor Net. Is there any reason to train it before the Disentangle Domain Factor Net? is there any reason to train the MANN network without scheduling prior to training it with scheduling?
What were the architectures used in order to train the domain adaptation task on semantic segmentation data. in particular:

What layer of the DeepLab (I assume V3+) network was used as the last encoder layer for the DA task?
What was the architecture of the decoder/domain encoder for the reconstruction loss used in the domain disentanglement train?

Thanks.

C-Digits Dataset

Where can I download the C-Digits dataset?

There is no semantic segmentation label in C-Driving's train dataset.

Thanks for the interesting paper.

However, there is no semantic segmentation label for C-Driving's training dataset.

I can't even find it in the BDD100K dataset.

C-Driving has only images, but can I know where to get the semantic segmentation map..?

C-Faces experiments

Hi,

I would like to reproduce the results of C-Faces dataset but I noticed that you shared only the code to reproduce the experiments on Digits. Can you also share the code for C-Faces?

Thank you!

Memory module for Semantic Segmentation

Hi,
from my understanding the memory module is a matrix which holds for each row the mean latent vector for each class (or - its centroid) over the source data. Later these centroids are also used to order the target data for a curriculum learning step.

What I don't understand is how the memory module is implemented for semantic segmentation. when there is a single class for each image the division to centroids is clear but what is your method when the labels are segmentation maps?

I did not find an explanation for this in the paper, or the code.

Thanks,
Nadav

C-Driving segmentation code

Hi, I am trying to reproduce the results from your paper using your code. I have noticed that the code which you have shared is for classification of digits. Can you also share the architecture and code for C-Driving segmentation as well.
Thank you

Questions regarding the decoder

Thanks for releasing the code. It seems that the paper forgets to mention how to train the decoder while in the code the decoder is updated jointly with the domain encoder.

Pre-trained model links are broken

Hi, thanks for sharing this project - can you kindly update the links to the pre-trained models?

Questions about training details.

Hello, thanks for your impressive work.
I am trying to reproduce the results of source only, AdaptSeg, and proposed method on C-Driving benchmarks.
I checked the appendices (C.2.Training details), but there are some points unclear to me.
I’d really appreciate your kind reply.

Which initial weights did you use for training each methods?
Random initialization or vgg16_bn provided by torchvision or any other?
Did you use similar training process on C-Driving benchmarks as in the OCDA of classification tasks?
Specifically, is the overall process as follows?
(1) Train source net
(2) Compute class centroids from trained source net
(3) Fine-tune the model, which is initialized from source model(1), with fixed centroids and curriculum learning.
When you construct visual memory, did you average all the features belonging to the same category at once or firstly average the features of same category in each image?