To run the test script, open Jupyter and use the notebook FlatNet-separable.ipynb to evaluate flatnet-separable on captured measurements.

Pretrained models can be found at : [Dropbox]

Full dataset used for the paper is available at: [Dropbox] or [G-Drive]

Example data is provided in the directory example_data. It contains some measurements along with their Tikhonov reconstructions. You can use these measurements to test the reconstruction as well without having to download the whole dataset. fc_x.png refers to the measurement while rec_x.png refers to the corresponding Tikhonov reconstruction.

Please run main.py to train from scratch

Alternatively, run the shell script flatnet.sh found in execs directory with desired arguments.

Please make sure your path is set properly for the dataset and saving models. For saving model, make sure the variable 'data' in main.py and for dataset, make sure the variable 'temp' in dataloader.py are changed appropriately.

• Transpose Initializations: flatcam_prototype2_calibdata.mat found in the data folder contains the calibration matrices : Phi_L and Phi_R. They are named as P1 and Q1 respectively once you load the mat file. Please note that there are separate P1 and Q1 for each channel (b,gr,gb,r). For the paper, we use only one of them (P1b and Q1b) for initializing the weights (W_1 and W_2) of trainable inversion layer.

• Random Toeplitz Initializations: phil_toep_slope22.mat and phir_toep_slope22 found in the data folder contain the random toeplitz matrices corresponding to W_1 and W_2 of the trainable inversion layer.

• Download data as imagenet_caps_384_12bit_Feb_19 and place under data (or symlink it).
• Download Point Spread Function(s) and Mask(s) as phase_psf and place under data (or symlink it).
• Download checkpoints from ckpts_phase_mask_Feb_2020_size_384 and place as ckpts_phase_mask_Feb_2020_size_384.

You should then have the following directory structure:

.
|-- ckpts_phase_mask_Feb_2020_size_384
|   |-- ours-fft-1280-1408-learn-1280-1408-meas-1280-1408
|   `-- le-admm-fft-1280-1408-learn-1280-1408-meas-1280-1408
|-- data
|   |-- imagenet_caps_384_12bit_Feb_19
|   `-- phase_psf

Run as: streamlit run test_streamlit.py

Streamlit is an actively developed package, and while we install the latest version in this project, please note that backward compatibility may break in upcoming months. Nevertheless, we shall try to keep test_streamlit.py updated to reflect these changes.

Run as:

python train.py with ours_meas_1280_1408 -p

Here, ours_meas_1280_1408 is a config function, defined in config.py, where you can also find an exhaustive list of other configs available. For a multi-gpu version (we use pytorch's distdataparallel):

python -m torch.distributed.launch --nproc_per_node=3 --use_env train.py with ours_meas_1280_1408 distdataparallel=True -p

python val.py with ours_meas_1280_1408 -p

Metrics and Outputs are writen at output_dir/exp_name/.

See config.py for exhaustive set of config options. Create a new function to add a configuration.

Here, (sensor_height,sensor_width) can be (1280, 1408), (990, 1254), (864, 1120), (608, 864), (512, 640), (400, 400).

Finetuned refers to finetuning with contextual loss on indoor measurements (see our paper for more details).