mc-cnn-chainer's Introduction

MC-CNN-Chainer

This repository provides a Chainer implementation and pre-trained models of MC-CNN proposed in

Jure Zbontar and Yann LeCun:
"Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches",
Journal of Machine Learning Research, vol. 17, pages 1-32, 2016.

Our pre-trained models are produced by importing network weights from their original pre-trained models in https://github.com/jzbontar/mc-cnn. Our imlementation uses much less GPU memories for computing matching costs.

Remarks

We only provide code for inference. Training code is not included.
We only provide code of convolutional neural networks. Code for subsequent cross-based and SGM-based cost aggregation parts is not included.
We do not guarantee the exactly same results with the original implementations. (There might be slight numerical differences)
If you are not satisfied with the above remarks, please use the original code by the authors.

Required Environments

Python 3
Chainer (v2 or v3)
OpenCV-Python
CUDA + CuDNN (for the GPU mode)

Install

pip install chainer
pip install opencv-contrib-python==3.2.0.8

Usage

After installing the required environments, do either of the following commands.

% For CPU mode
python demo.py

% For GPU mode
python demo.py -g 1

% For saving cost volume data
python demo.py -g 1 -v 1

Reference

This code is intended to facilitate generation of cost volume data for input of our stereo method below. If you also find this work useful for your research, please cite our TPAMI paper.

@article{Taniai18,
  author    = {Tatsunori Taniai and
               Yasuyuki Matsushita and
               Yoichi Sato and
               Takeshi Naemura},
  title     = {{Continuous 3D Label Stereo Matching using Local Expansion Moves}},
  journal   = {{IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)}},
  year      = {2018},
  volume    = {40},
  number    = {11},
  pages     = {2725--2739},
  doi       = {10.1109/TPAMI.2017.2766072},
}

Contributors

Stargazers

Watchers

mc-cnn-chainer's Issues

run on middlebury dataset

How long dose it take to run demo.py on middlebury dataset.
It's been processing for 1more hours and it's not done yet. I don't know if that makes sense.

.acrt

When I run demo.py, I get the file named .bin that stores the cost volume. But what is the difference between the .bin and the .acrt ? How to convert the .bin to the .acrt?

the size of .acrt file is much smaller than you provided

The size of .acrt file is much smaller than you provided. What's the difference between a 300M .acrt file and a 800M .acrt file?
Detailed:
I used your mccnn model and get a .bin file. The original images are the following images:

But the file size is 300M, where the size of .acrt file you provided is 800M. How can I get the .acrt file you provided?
The following image is the disparity map generated by my .acrt file and the algorithm you provided, localExpansionStereo:

lack of memory on single GPU

We have used the model mb_slow to get the match cost of middlebury training test, trainingF, but it threw the error, out of memory. As we are not familiar with chainer, we tried to look up how chainer uses multiple GPUs on the Web, but failed. Could you please tell me how to use multiple Gpus？
The resolution of left and right images are 2872*1984.

Recommend Projects