Giter Club home page Giter Club logo

tgrs_disoptnet's Introduction

DisOptNet: Distilling Semantic Knowledge from Optical Images for Weather-independent Building Segmentation


Jian Kang, Zhirui Wang, Zhirui Wang, Junshi Xia, Xian Sun, Ruben Fernandez-Beltran, Antonio Plaza

This repo contains the codes for the TGRS paper: DisOptNet: Distilling Semantic Knowledge from Optical Images for Weather-independent Building Segmentation. Compared to optical images, the semantics inherent to SAR images are less rich and interpretable due to factors like speckle noise and imaging geometry. In this scenario, most state-of-the-art methods are focused on designing advanced network architectures or loss functions for building footprint extraction. However, few works have been oriented towards improving segmentation performance through knowledge transfer from optical images. In this paper, we propose a novel method based on the DisOptNet network which can distill the useful semantic knowledge from optical images into a network only trained with SAR data.

drawing

Data

We use the SpaceNet6 dataset in the paper. The dataset preparation is based on the SpaceNet6 baseline repo.

Usage

# first stage training (RGB)
./run_st1.sh
# second stage training (SAR+RGB)
./run_st2.sh

Model weights

DisOptNet-B4

DisOptNet-B3

Citation

@article{kang2021RiDe,
  title={DisOptNet: Distilling Semantic Knowledge from Optical Images for Weather-independent Building Segmentation},
  author={Kang, Jian and Wang, Zhirui and Zhu, Ruoxin and Xia, Junshi and Sun, Xian and Fernandez-Beltran, Ruben and Plaza, Antonio},
  journal={IEEE Transactions on Geoscience and Remote Sensing},
  year={2022},
  note={DOI:10.1109/TGRS.2022.3165209}
  publisher={IEEE}
}

tgrs_disoptnet's People

Contributors

jiankang1991 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

tgrs_disoptnet's Issues

Paper Model Weights

I really appreciate you providing the code. It has been very helpful for my research. @jiankang1991

I was wondering if it would be possible to get the weight that produce the scores mentioned in the paper.
I have tried several times using the provided code, but I have not been able to reach the scores mentioned in the paper.
If it is not a problem, could I receive the weight?

Thank you.

Question about Model Loss Flow during second train (Distillation Loss)

I have read your paper and would like to respectfully inquire whether my understanding of it is correct.
I have some questions that I would like to ask for clarification about Distillation Loss

I hope you don't mind me asking, but from my understanding of the formula and code presented in the paper, it seems that only the SAR-Optical Shared Encoder is trained using the distillation loss during the second training. Additionally, it appears that the distillation loss is calculated solely based on the results obtained from the Optical Stage(pre-trained Optical network and SAR train network) and not from the SAR Stage. Could you please confirm if my understanding is correct?

If my understanding is correct, would this mean that the SAR Stage is not affected by the Distillation Loss during backpropagation?
(And I assume you did this because the features of SAR and Optical networks become increasingly dissimilar at higher levels.)

I have attached the code for your reference as well in your code.
dist_rgb_loss = sum([self.dist_f_loss(sar_f, rgb_f) for sar_f, rgb_f in zip(sar_features, rgb_features)])
sar_features : Optical Stage features from second Training Process (SAR+RGB training process)
rgb_featuers : Optical Stage features from pre-trained Optical Network

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.