paper has sRGB input comparision, I modify the last two level conv10(12 channel ) to output 3channel for the rgb input image generated by rawpy.
the training loss is down to 0.025, the result is not bad, but some image is not real. is this correct way?

It's hard to ignore that in your paper Table 3, the psnr of Sony rgb input is 17.4 while another psnr is 25.11. I couldn't understand why they differ so much when both are rgb images. I mean，if input rgb images, they must have gone through the same process before put into the network.
And would you mind to explain how you convert your raw datasets to rgb images, and how to understand this difference?

A question in the test_Sony.py is that, the scipy.io is really slow and cv2.imwrite is really fast. In the 8-bit saving way, do this two methods inference the finnaly PSNR & SSIM?

If we only have rgb input images, how should I retrain the model and test it?
Thanks

The model is nice, but how can I use the model to convert my own single jpg(or png) img ? Is there any Introduction?

after download and extract raw Sony data in dir dataset/Sony.
run test_Sony.py
got error at line 151:
im = raw.postprocess(use_camera_wb=True, half_size=False, no_auto_bright=True, output_bps=16)

rawpy._rawpy.LibRawFatalError: b'Unsufficient memory'
could you have met this?
Appreciate for answer.

1. Can I use a Sony network model for a 16bit bayer filter?
2. Is it possible to share iphone 6s raw images(dng format) in paper?

I don't have enough RAM，can you give me some suggestions to revise the code. Thanks for a lot.

I have try to test the Sony Image Data:

/home/ljz/桌面/DB3065DB.png

why black level equal 512 and why is 16383 ,What is meaning of the number 16383?

Hi @cchen156 ,

This work is very exciting. But I have a question about the quantitative results reported in the paper Table3.

I downloaded the source code and trained model, and tested Sony dataset. I generate GT by Rawpy and apply Gamma transfer after it, since you apply Gamma transfer on the output of the network in your test code. I found that the results looks fine, but the average PSNR/SSIM in the testset is 25.44/0.713, which is quite lower than 28.88/0.787 reported in Table3 in your paper.

If I have made some mistake? How can I get the reported results.

Thanks,
Wenbo

download_models.py

Amazing results, congrats!

Is it possible to download from somewhere the pre-trained model?

Thanks a lot!

What`s this probably mean?Here is the log:

Traceback (most recent call last):
File "test_Sony.py", line 117, in
out_image = network(in_image)
File "test_Sony.py", line 76, in network
up6 = upsample_and_concat(conv5, conv4, 256, 512)
File "test_Sony.py", line 50, in upsample_and_concat
deconv_output = tf.concat([deconv, x2], 3)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_ops.py", line 1075, in concat
dtype=dtypes.int32).get_shape(
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 669, in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 176, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 165, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 367, in make_tensor_proto
_AssertCompatible(values, dtype)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 302, in _AssertCompatible
(dtype.name, repr(mismatch), type(mismatch).name))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

I think that train_Sony.py at line 129

allfolders = glob.glob('./result/*0')

and train_Sony.py at line 136

if os.path.isdir("result/%04d" % epoch):

are wrong.

The correct programs are

allfolders = glob.glob(result_dir + '*0')
if os.path.isdir(result_dir + "%04d" % epoch):

, right?

The official website contains a low light image and its result when run on SID model (http://web.engr.illinois.edu/~cchen156/SID/examples/6s.html). I could not find the weights for this in the repository. Is it possible to upload the weights for when trained on iPhone 6s?

Hello,@cchen156,I used the cpkt file you provided to test PNSR and SSIM, but the measured value is still lower than the original text, PNSR=28.639, SSIM=0.7682, is it different from the method I used to test pnsr and ssim? Can you provide the method or code for testing pnsr and ssim?

Hi,
Thanks for sharing this work and the result is awesome ! @cchen156
I find comparison with hdrnet in the project page ， so are you using HDRnet prettrained model from @mgharbi 's repo or you trained the comparison model youself？

I want to reproduce the comparison but seems no night related pretrained model is found for hdrnet， would you like to share the comparison pretrained model ? Thanks

I don't understand why you set the batch_size=1, i.e., only 1 picture will be trained in one iteration. Have you ever tried other batch_sizes?

I tried to compare the output generated by SID (O-sid) and photoshop (O-ps, with all auto from DNG to PNG), and according to what I saw:
1/ when the light is extremely low, O-sid is better than O-ps.
2/ but when the light is low and not extremely, O-ps is better than O-sid.

Also, O-sid loses quite a few texture compared to O-ps.
I wonder is this as expected?

I downloaded the sony data of 22 GB, the training and testing programs were executed, the folder named 'result_sony' is created, with some files in it, but those files are not executable and there were no images to be seen, can you please tell me that was result was expected once the program is executed completely.

I noticed that the README suggests it needs Python 2.7 but from looking at the libraries, nothing jumps out that would be dependent on 2.7, so I'd have expected it to run with 3.

Is it simply that 2.7 is what you happened to develop / test with, or is there something in your code that specifically needs 2.7?

Can you get the inference freeze.pb file for your model?
what is your input node name ? output node name?

when I run the download_models.py
there is no response. can i download the models in other ways?

I reimplemented the paper with pytorch,but the result is very bad.I am not sure whether pytorch has changed the effect of tensorflow in this paper.

Hi!
I was able to run the examples for Sony raw data and the results are amazing.
I would like to experiment with Canon and Nikon raws but with Canon's raws the model seems to never converge. I am able to run it but i don't get any output. I modified the test_Sony file changing the ARW extension in CR2 (Canon's raw). Any advice?

Thank you!

Hi:
I found I cannot download the dataset, could you pls give another URL?
https://docs.google.com/uc?export=download

Bad Request
Error 400

@kkruups @prabodhw96 @cchen156 Hi, congrats for your work. In your paper and code, during training, input and corresponding ground truth are first scaled between [0-1], and then input are scaled by ratio but the ground truth is still between [0-1]. Have you tried not to scale input by ratio and keep input and gt both in range [0-1] to train the network? Will it get better result and what's your opinion? THX!

@cchen156 Hi, congrats first. In your training settings now, you aim to learn white balance within the network. Have you tried to apply white balance on input(those short exposure images) before feeding them into networks? If so, how does the result look like? THX!

Hello,thank you for your help.Because my computer configuration is too low, RAM is only 4G, gpu memory 2G, although you provide sonny(12G) data set, but still a bit big for me and I can't download it .so could you provide training test data set within 2G. I just want to use a small amount of data to test the model.
Thank you !

The gpu memory is occupied when I run the train_Sony.py, so I set

tfconfig = tf.ConfigProto(allow_soft_placement=True) tfconfig.gpu_options.allow_growth=True sess = tf.Session(config=tfconfig)

But there is still the question...?

Hi @cchen156 i test psnr of ur model on test samples 1-49 in ur test_list.txt , and the psnr is ~32dB ,is that the same with urs ? i didnt change any of ur codes
My PSNR Calculation ：
PSNR（img1,img2）:
mse = np.mean((img1 - img2)^2)
return 10*np.log10(255^2 / mse);

why can not i download the data?
The website always return:
Bad Request
Error 400

@cchen156 Do you have the "Croplayer" crop operation in this fuctions,
It only have 'DECONVOLUTION' ans 'CONCAT'，

I just tried to train Sony dataset after converting the raw data into jpg image and found the result quite terrible. So are there any tricks when training 8bit data?
P.S.:
The channel number of input is 12, which results from packing each channel of rgb into four.

Hi,

Thanks for your novel work. I have some questions. I noticed that you restrict the value of ground truth image around 0 to 1, then force the model to generate output image which is also around 0 to 1. I am confused that why not just use the original ground truth image as the label and produce the output image directly without multiplying 255. Is there any difference between these two ways? Could you please help answer this question?

Thanks,

I use your sony pretrainned model, Execute test_sony.py
I wonder why green dots are spreading in the output image.

GREAT WORK!

There's a certain app you mentioned in your paper, which can set parameters and export raw data.
Could you please tell me what's the name of the app?

I only know we can access YUV data through Apple's API.

Thanks~!

Hi, thanks for your great work.
This issue is just a suggestion of possible enhancement.
When testing the model, currently the output images are ground truth, the output from the neural network, and the input scaled up to the same mean as ground truth.

My suggestion is to add

scipy.misc.toimage(input_full[0,:,:,:]*255,  high=255, low=0, cmin=0, cmax=255).save(result_dir + 'final/%5d_00_%d_in.png'%(test_id,ratio))

to output the initial input image, to show how unusable the original photo is.

I download the pretrained model and run 'test_Sony.py'.But the output is very dark!

Is there any evaluation network?

The download_models.py always got a HTTP connection error, could you please provide a available download links? Thanks.

I downloaded the two zip files for myself, and with my host (wasabi) I pay for storage but not bandwidth, so I'm willing to share the links to the datasets here:

https://s3.wasabisys.com/datasets/Learning-to-See-in-the-Dark/Fuji.zip
https://s3.wasabisys.com/datasets/Learning-to-See-in-the-Dark/Sony.zip

Feel free to use them, or get back in touch if you want me to stop making them public. I'll be keeping them up for the foreseeable future.

How long did the training on Titan X take? What was the final loss?

Hi, I am work on the project which can detect and track pedestrian in low light or night. Then how can I use your model for my dataset?

Hi Chen,

Thanks for your code, it is very useful!

I just wonder could you please provide the script for downloading Sony 12GB? It will help me a lot!

Thanks,

@cchen156 Hi, nice work and congratulations. I am wondering why you use np.maximum(xx,0) in pack_raw() function to ignore negative values. Will it affect the result?