Giter Club home page Giter Club logo

rfnet's Introduction

RFNet: Real-time Fusion Network for RGB-D Semantic Segmentation Incorporating Unexpected Obstacle Detection of Road-driving Images

This repository is a Pytorch implementation for

Sun, Lei, et al. "Real-time Fusion Network for RGB-D Semantic Segmentation Incorporating Unexpected Obstacle Detection for Road-driving Images." arXiv preprint arXiv:2002.10570 (2020).

If you want to use this code in your research, please cite the paper.

Requirement

Python 3.6  
Pytorch 1.1  
Torchvision 0.3  
Opencv 3.3.1

Datasets

Get dataset from Cityscapes, and from Lost and Found.

If you want to use multi-dataset training, mix two datasets and the directory structure should be like this:

├─disparity
│  ├─test
│  │  ├─berlin
│  │  ├─bielefeld
│  │  ├─bonn
│  │  ├─...
│  │  └─munich
│  ├─train
│  │  ├─01_Hanns_Klemm_Str_45
│  │  ├─03_Hanns_Klemm_Str_19
│  │  ├─...
│  │  └─zurich
│  └─val
│      ├─02_Hanns_Klemm_Str_44
│      ├─04_Maurener_Weg_8
│      ├─05_Schafgasse_1
│      ├─...
│      └─munster
├─gtFine
│  ├─train
│  │  ├─01_Hanns_Klemm_Str_45
│  │  ├─03_Hanns_Klemm_Str_19
│  │  ├─...
│  │  └─zurich
│  └─val
│      ├─02_Hanns_Klemm_Str_44
│      ├─04_Maurener_Weg_8
│      ├─05_Schafgasse_1
│      ├─...
│      └─munster
└─leftImg8bit
    ├─test
    │  ├─berlin
    │  ├─bielefeld
    │  ├─bonn
    │  ├─...
    │  └─munich
    ├─train
    │  ├─01_Hanns_Klemm_Str_45
    │  ├─03_Hanns_Klemm_Str_19
    │  ├─...
    │  └─zurich
    └─val
        ├─02_Hanns_Klemm_Str_44
        ├─04_Maurener_Weg_8
        ├─05_Schafgasse_1
        ├─...
        └─munster

Pretrained weights

BaiduNetdisk

Download weights for Cityscapes, password: 4lts
Download weights for Multi-dataset: Cityscapes and Lost and Found, password: t1mh

Google Drive

Download weights for Cityscapes
Download weights for Multi-dataset: Cityscapes and Lost and Found

Training

Edit path to your dataset in mypath.py.

python train.py --depth --lr 1e-4 --weight-decay 2.5e-5 --workers 4 --epochs 200 --batch-size 8 --val-batch-size 3 --gpu-ids 0 --checkname test --eval-interval 2 --dataset citylostfound --loss-type ce --use-balanced-weights

Evaluation

python eval.py --dataset citylostfound --weight-path your/path/to/weight/ --depth

Note: The code is partially based Swiftnet and DeepLab v3+.

rfnet's People

Contributors

ahupujr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

rfnet's Issues

mIOU of CityScapes val subset

I run RFNet on RTX 3090 by using your settings.The dataset is CityScapes.But I got mIoU 68.17%.
I want to know where is my question and the key to get mIoU 72%.
Looking forward to your reply.

dataset

thank you for your opening code!
How can I get to the depth of cityscape?from the cityscape website,I download it but without disparity ,can you share with me ? Thank you very much

IndexError: list index out of range

I am training with only cityscapes dataset
But I have some error below:
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "C:\Users\parke.conda\envs\untitled\lib\site-packages\torch\utils\data_utils\worker.py", line 198, in _worker_loop
data = fetcher.fetch(index)
File "C:\Users\parke.conda\envs\untitled\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\Users\parke.conda\envs\untitled\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "E:\RFNet-master\dataloaders\datasets\cityscapes.py", line 54, in getitem
lbl_path = self.labels[self.split][index].rstrip()
IndexError: list index out of range

Pretrained weights not available anymore

Hello,

thank you for this very interesting work. I would like to try one of your pretrained model but I can't download the weights. It seems there are not available anymore on Google Drive or Baidu... Could you check ?

Running code on custom segmentation dataset

Hi, I am trying to run this code on my custom image segmentation dataset which has 13 label classes but I am getting the following error :

RuntimeError: 1only batches of spatial targets supported (non-empty 3D tensors) but got targets of size: : [4, 768, 768, 3]

I checked for the output and target tensor sizes and they are

-------------Output------------ torch.Size([4, 13, 768, 768])
-------------Target------------ torch.Size([4, 768, 768, 3])

I am not getting why the code is predicting an image with 13 channels. Can you please let me know what changes do I need to make in order to run this code.

Problem with loading weights

Hi, first of all thank you for sharing the code!

I have a problem with loading weight's, when i start the eval.py i get error that model SemsegModel dosen't exist in file models.rfnet.py, so i changed SemsegModel to RFnet model, and when I try load weight i get an error that some layers do not appear in model.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.