This is the official implementation of paper title "Beyond Joint Demosaicking and Denoising: An Image Processing Pipeline for a Pixel-bin Image Sensor". The paper has been accepted and expected to be published in the proceedings of CVPRW21. To download full paper [Click Here].

Please consider to cite this paper as follows:

@inproceedings{a2021beyond,
  title={Beyond joint demosaicking and denoising: An image processing pipeline for a pixel-bin image sensor},
  author={Sharif, SMA and Naqvi, Rizwan Ali and Biswas, Mithun},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={233--242},
  year={2021}
}

BJDD tackle the challenges of joint demosaicing and denoising (JDD) of artefact prone non-Bayer Quad-Bayer CFA and Bayer CFA pattern. Notably, such CFA pair is substantially leveraging in the recent smartphone cameras (denoted as Pixel-bin image sensor).

Figure: Commonly used CFA patterns of pixel-bin image 086 sensors. Left : Quad-Bayer CFA. Right: Bayer-CFA.

Figure: Overview of the proposed method, including network architecture and submodules.

Figure: Quantitative comparison between proposed method and existing JDD methods for Quad-Bayer reconstruction.

Python 3.8
CUDA 10.1 + CuDNN
pip
Virtual environment (optional)

Please consider using a virtual environment to continue the installation process.

git clone https://github.com/sharif-apu/BJDD_CVPR21.git
cd BJDD_CVPR21
pip install -r requirement.txt

** [Click Here]** to download pretrained weights and save it to weights/ directory for inferencing with Quad-bayer CFA
python main.py -i

A few testing images are provided in a sub-directory under testingImages (i.e., testingImages/sampleImages/)
In such occasion, reconstructed image(s) will be available in modelOutput/sampleImages/.

To inference with custom setting execute the following command:
python main.py -i -s path/to/inputImages -d path/to/outputImages -ns=sigma(s)
Here,-ns specifies the standard deviation of a Gaussian distribution (i.e., -ns=5, 10, 15),-s specifies the root directory of the source images (i.e., testingImages/), and -d specifies the destination root (i.e., modelOutput/).

To start training we need to sampling the images according to the CFA pattern and have to pair with coresponding ground-truth images. To sample images for pair training please execute the following command:

python main.py -ds -s /path/to/GTimages/ -d /path/to/saveSamples/ -g 2 -n 10000
Here -s flag defines your root directory of GT images, -d flag defines the directory where sampled images should be saved, and -g flag defines the binnig factr (i.e., 1 for bayer CFA, 2 for Quad-bayer), -n defines the number of images have to sample (optional)

After extracting samples, please execute the following commands to start training:

python main.py -ts -e X -b Y To specify your trining images path, go to mainModule/config.json and update "gtPath" and "targetPath" entity.
You can specify the number of epoch with -e flag (i.e., -e 5) and number of images per batch with -b flag (i.e., -b 12).

For transfer learning execute:
python main.py -tr -e -b

We also trained our model with Bayer CFA. To download pretrained Bayer weights [Click Here]. In such occasion, please update binning factor entity in mainModule/config.json file.

Check model configuration:
python main.py -ms
Create new configuration file:
python main.py -c
Update configuration file:
python main.py -u
Overfitting testing
python main.py -to

For any further query, feel free to contact us through the following emails: [email protected], [email protected], or [email protected]