coldog2333 / pytoflow Goto Github PK

Hi @Coldog2333

Thank you for the great code! I noticed that the code is for 4x super-resolution. Is it possible to train a 2x super-resolution model? Thanks.

Best,
Yongcheng

Facing below error while attempting to use 'run.py'.
command-
python run.py --f1 example/im1.png --f2 example/im3.png --o example/out.png --gpuID 0

Error-
Loading TOFlow Net... Traceback (most recent call last):
File "run.py", line 112, in
net = TOFlow(intPreprocessedHeight, intPreprocessedWidth, cuda_flag=CUDA)
TypeError: init() missing 1 required positional argument: 'task'

Any help. Thanks!!

1、TOFlow(intPreprocessedHeight, intPreprocessedWidth,'interp', cuda_flag=CUDA)
need to add interp
2、at function Estimate ，should add the following respect
inTensor = torch.cat((tensorPreprocessedFirst.unsqueeze(0),tensorPreprocessedSecond.unsqueeze(0)), 1)
tensorFlow = torch.nn.functional.interpolate(
input=net(inTensor),
size=(intHeight, intWidth), mode='bilinear', align_corners=False)

and i test some pics ，especially at the edge ，the artifact is obvious ，this flow based method is not well enough

Hi, thank you for sharing your nice reimplementation.

I was wondering whether you have any idea where the performance gap between this version and the authors' original implementation comes from?

Did you have any idea how to improve your implementation?
Thank you!

Thank you for porting this work to pytorch. Could you provide more information about the trained models given in the folder toflow_models?

In particular, for the video denoising model (denoise.pkl):

1. In the original work there are 3 types of noise considered (Gaussian 15, Gaussian 25 and mixed noise). Your denoise.pkl, on which type of noise was it trained?
2. Is it the one trained by the original authors or the one you retrained?
3. If it is the one you retrained, which trainset did you use, and for how many epochs.

I've found the Pytoflow's temporal frame interpolation useful. Could you please add a license to the repository? Thanks

when I use the code on Viemo90K, I have encountered a problem that the value range of processed
images of the model is out of (0, 1), have you encountered this similar problem? and could you
give me some help? thanks a lot!

Hi @Coldog2333

Thank you for the great code! I want to ask some questions when reproduce the sr/super-resolutionexperimental results. Something went wrong when I run the "train.py".
" framex = np.transpose(frames[0:N, :, :, :], (0, 3, 1, 2))
framey = np.transpose(frames[-1, :, :, :], (2, 0, 1))
IndexError:too many indices for array
"
Have you ever had this problem? How to deal with it？

Dear friend,
First all, we hope express our thanks for your hard work.
Second, we have an question: Why super resolution output size is same as input images ?
Best Regards,

Can you reupload it?

Hi, thank you for your great reimplementation work. As we know, toflow can achieve video SR tasks. I want to know how to generate consecutive HR video frames instead of one frame from 7 consecutive LR video frames.

Since the results on video denoising of pytoflow is very different from original results of toflow, so could you put how you evaluate your models? on matlab or using skimage of python?@Coldog2333

how can I test my dataset rather than vimeo 90K？Thanks！！

Many thanks for your work. It seems that the mask network isn't implemented in this verson and there is a mask network as discribed in the paper, right?

Not an issue :)
Just letting you and your users know I've created a small training set from the vimeo dataset to try and denoise analog VHS noise.

Process was basically taking the images from the vimeo-set, playing them back via DV cam-corder and recoding on to VHS tape, then capturing them back to the PC from tape.

The training-set and models can be found on.
https://drive.google.com/drive/folders/16DaZLdTsNIvVx7IND1g2j7qtiQMzYF0D?usp=sharing

Many thanks for your work.When I run the code the following error occurs：“ValueError: Floating point image RGB values must be in the 0..1 range.”How to solve it?Thank you!

Hi,
Trying to reproduce the SR results but it the output of the image appears to be the same size as the input. Is there a way to validate SR?

Hey, you have amazing work. I am facing an issue with SSIM score you have calculated. it's like reshaping the image and passing it from SSIM function is not consistent with the Original Implementation of authors of SSIM. See the code for reproducing please.

% LR image available here 'https://raw.githubusercontent.com/mugheesahmad/Fun_testing/master/LR0000001.jpg' 
% HR image available here 'https://raw.githubusercontent.com/mugheesahmad/Fun_testing/master/HR0000001.jpg' 
lr = imread('LR0000001.jpg');
hr = imread('HR0000001.jpg');
ssim(hr, lr) %colored image
% ans = 0.8433
ssim(rgb2gray(hr), rgb2gray(lr)) %builtin MATLAB function
% ans =    0.7570
original_ssim(rgb2gray(hr), rgb2gray(lr)) %author implementation available here https://ece.uwaterloo.ca/~z70wang/research/ssim/
% original implementation doesnot accept the RGB image
% ans =    0.7574 
lru = reshape(lr, [380*672,3]);  %your way of doing
hru = reshape(hr, [380*672,3]);
ssim(lru, hru)
%ans = 86.70

As per the docs of Matlab SSIM, only gray images can be passed. Your way of using it does not consistent with the original and also with the SSIM implementation with skimage and pytorch version. see this and this colab file.

I'm attempting to use run.py - but getting the below error. Any ideas?

Loading TOFlow Net... Done.
Processing...
Traceback (most recent call last):
  File "run.py", line 131, in <module>
    predict = Estimate(net, Firstfilename=frameFirstName, Secondfilename=frameSecondName, cuda_flag=CUDA)
  File "run.py", line 88, in Estimate
    input=net(tensorPreprocessedFirst, tensorPreprocessedSecond),
  File "C:\Users\pete\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
TypeError: forward() takes 2 positional arguments but 3 were given