wyhuai / ddnm Goto Github PK

If I have a batch of data, which contains different types of data, I want to generate different mask images according to the pixel values of different images, and then restore the images with corresponding masks, how can I achieve this in your method.

I followed the instructions in README.md and tried the 8x super resolution application on several test images using DDNM, here are my python commands.

python main.py --ni --simplified --config celeba_hq.yml --path_y celeba_hq --eta 0.85 --deg "sr_averagepooling" --deg_scale 8.0 --sigma_y 0 -i demo

The content of "celeba_hq.yml" is：

data:
    dataset: "CelebA_HQ"
    category: ""
    image_size: 256
    channels: 3
    logit_transform: false
    uniform_dequantization: false
    gaussian_dequantization: false
    random_flip: true
    rescaled: true
    num_workers: 32
    out_of_dist: false

model:
    type: "simple"
    in_channels: 3
    out_ch: 3
    ch: 128
    ch_mult: [1, 1, 2, 2, 4, 4]
    num_res_blocks: 2
    attn_resolutions: [16, ]
    dropout: 0.0
    var_type: fixedsmall
    ema_rate: 0.999
    ema: True
    resamp_with_conv: True

diffusion:
    beta_schedule: linear
    beta_start: 0.0001
    beta_end: 0.02
    num_diffusion_timesteps: 1000

sampling:
    batch_size: 1
    
time_travel:
    T_sampling: 100
    travel_length: 1
    travel_repeat: 1

My inputs are:

The DDNM outputs are:

where blurring effect appears in the results. Is there any way to eliminate this effect?

Thanks for your wonderful work, I have a question about the testset:
The pre-trained model was trained on CelebA-HQ dataset. The images in testset is also from CelebA-HQ dataset. As far as I know, there seem to be no validation images provided officially. I would like to know how you make the testset. Because I want to use your testset as validation dataset in my experiment. And I don't want to mix the images in the testset during training.
Looking forward to your reply.

Trying to run the first example after installing and downloading the model files...

  File "D:\Python\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'Diffusion.simplified_ddnm_plus.<locals>.seed_worker'

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\Python\lib\multiprocessing\spawn.py", line 107, in spawn_main
    new_handle = reduction.duplicate(pipe_handle,
  File "D:\Python\lib\multiprocessing\reduction.py", line 79, in duplicate
    return _winapi.DuplicateHandle(
OSError: [WinError 6] The handle is invalid

Your work is great! What kind of GPU is required for model training? Can your model be trained on 4 12GB GPUs?

I am running the following command:
python main.py --ni --config celeba_hq.yml --path_y celeba_hq --eta 0.85 --deg "denoising" --sigma_y 0.3 -i demo --add_noise --simplified
However there is no noise added to the input signal.
I changed the code and added noise after line 306 in guided_diffusion/diffusion.py with:
if self.args.add_noise: y = get_gaussian_noisy_img(y, sigma_y)
However the results are unsatisfying. Here is one output example:

When removing --simplified from the command everything works well (with the SVD version). Is this an expected result from the simplified model? Or am I doing something wrong?
Thanks

I'd like to do inpainting inference on a dataset of images where the mask is different for each image. I'd like to use a batch size > 1 to increase inference speed. Is there any way to do this? If not, how can I perform inference more efficiently?

Let's say an Image Generation model is meant to generate image from random noise in 50 steps:

1. What if I use Image Generation model for the initial 20 steps to generate a half-way noisy image.
2. Input that noisy image into a Zero-Shot Image Restoration model to generate a clean image.

Has anyone tried this approach? Does it have any advantage in classical image generation quality, few-shot image generation?

I test ddnm on arbitrary imagenet image but did not get as good results as the demo.
The command I use is :
CUDA_VISIBLE_DEVICES=3 python main.py --resize_y --config confs/inet256.yml --path_y ./ILSVRC2012_val_00046632.JPEG --class 22 --deg "sr_averagepooling" --scale 4 -i imagenet_46632_cc_256x96

Apy:

final:

Is there any parameter I need to adjust?

Why am I unable to achieve the same high-quality results as the paper when performing the inpainting task on the ImageNet dataset?

Hi, bro, a nice work! But I am not very understanding of a problem.
In the section "2.2 RANGE-NULL SPACE DECOMPOSITION", I have seen this equation

And in "3.1 DENOISING DIFFUSION NULL-SPACE MODEL", in this context,

I don't understand the difference between $\hat{x}$ and $\bar{x}$

and here the $y=Ax$?

Hi authors, thanks for releasing this great work!

I am trying to use mask-shift technique on my image-to-image translation task in medical images. I trained my diffusion model with crop data augmentation. But I can’t get the smooth results like yours. Is there any way to improve it?

The following are the results of my task.

This is the generated result of the first patch.

Then, I shift the patch right and leave a half of overlapped regions with the previous patch.

This is the result of the step 2.

Hello, I hope this message finds you well.
First of all, I wanted to express my admiration for the outstanding work you have done on your repository.

I'm currently working on a project that requires the use of the DIV2K 0810.png image shown in Figure 1(a) in the paper. However, I couldn't find any information about the sampling configuration used to generate this image. If it's not too much trouble, I would like to kindly request information on the sampling configuration used to generate this image.

Thank you very much for your help!

Thanks for your great work!

I tried to reproduce the results in the bottom right of fig3 in the paper based on the “imagenet_256.yml” and “256*256_duffusion_uncond.pt” you provided, but my reconstructed image was poor.

May I ask if I didn't notice anything that caused this poor result?

Hi,

Thanks for you interesting and great work!

The link to download the SDEdit weights seems to have expired: ermongroup/SDEdit#18. It appears that the SDEdit repository has not been updated for quite some time. Could you please provide a new link to download the weights?

Hi,

Could you please tell how to set class number, when the class cannot be found here?

Thanks a lot for your nice work!

How do i train celeba_hq.ckpt with my own dataset？I would appreciate it if you could provide me with more information on how to train the model or point me towards any resources that could be helpful.

When I run the unconditional ImageNet model on sample ImageNet validation images, the inpainted portion is often very blurry. Is there a way to fix this?

Hi, I came across the paper and was impressed by its performance. However, I wanted to bring to your attention that the figure which removes the Islamic veils may potentially pose ethical concerns. To prevent any possible issues, I would suggest removing the figure from the paper..

I'm curious to understand why, when I comment out the projection step in the code, it fails to function as DDIM sampling and generate any output. It's puzzling why the sanity check isn't working. I am not getting any errors, but rather I am getting the following images for inpainting task where I removed the line related to the projection, i.e., converting x0_t_hat = x0_t - lambda_t*Ap(A(x0_t) - y) to x0_t_hat = x0_t.

I'm curious to understand why, when I comment out the projection step in the code, it fails to function as DDIM sampling and generate any output. It's puzzling why the sanity check isn't working.

Hi
I'm trying to run this code for colorization. I'm loading the "celeba_hq.ckpt" model on ddnm to convert my grayscale to color. It produces brownish images. How should I train for colorization? I really appreciate any help you can provide.

Hi, first thanks for your great work!

I was wondering if it is possible to apply your method to a more sophisticated kernel, a random 21x21 kernel for example.
I don't understand how you link the blur kernel to the SVD decomposition (basically how you define A_small1 and A_small2 in Deblurring2D). Could you point to some literature that explain the derivation? and maybe how you think the random kernel case can be handled?

Thanks.

Hello Author, I wangt to train myself model, and i dont find train.py

First, thanks a lot for your nice work!

Now i need restore my own images , The document metions that we can use some tools to crap the images or draw a mask, but the link is broken, how can i find them?

• If your are using CelebA pretrained models, try this tool to crop and align your photo.
• If there are local artifacts on your photo, try this tool to draw a mask to cover them, and save this mask to DDNM/exp/inp_masks/mask.png. Then run DDNM/exp/inp_masks/get_mask.py to generate mask.npy.

Dear authors,
Thank you very much for the interesting solution and publication.
Do you plan to put your solution on https://replicate.com/
Thank you

Dear authors,

when i run python main.py --ni --simplified --config celeba_hq.yml --path_y celeba_hq --eta 0.85 --deg "sr_averagepooling" --deg_scale 4.0 --sigma_y 0 -i demo
it stacks!
er message :
File "X:\game\DDNM-main\main.py", line 166, in main
runner.sample(args.simplified)
File "X:\game\DDNM-main\guided_diffusion\diffusion.py", line 200, in sample
self.simplified_ddnm_plus(model, cls_fn)
File "X:\game\DDNM-main\guided_diffusion\diffusion.py", line 302, in simplified_ddnm_plus
for x_orig, classes in pbar:
File "X:\python\Anaconda\envs\game\lib\site-packages\tqdm\std.py", line 1195, in iter
for obj in iterable:
File "X:\python\Anaconda\envs\game\lib\site-packages\torch\utils\data\dataloader.py", line 435, in iter
return self._get_iterator()
File "X:\python\Anaconda\envs\game\lib\site-packages\torch\utils\data\dataloader.py", line 381, in _get_iterator
return _MultiProcessingDataLoaderIter(self)
File "X:\python\Anaconda\envs\game\lib\site-packages\torch\utils\data\dataloader.py", line 1034, in init
w.start()
File "X:\python\Anaconda\envs\game\lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
File "X:\python\Anaconda\envs\game\lib\multiprocessing\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "X:\python\Anaconda\envs\game\lib\multiprocessing\context.py", line 336, in _Popen
return Popen(process_obj)
File "X:\python\Anaconda\envs\game\lib\multiprocessing\popen_spawn_win32.py", line 93, in init
reduction.dump(process_obj, to_child)
File "X:\python\Anaconda\envs\game\lib\multiprocessing\reduction.py", line 61, in dump
pickle.dump(obj, file, protocol)
AttributeError: Can't pickle local object 'Diffusion.simplified_ddnm_plus..seed_worker'

Traceback (most recent call last):
File "", line 1, in
File "X:\python\Anaconda\envs\game\lib\multiprocessing\spawn.py", line 107, in spawn_main
new_handle = reduction.duplicate(pipe_handle,
File "X:\python\Anaconda\envs\game\lib\multiprocessing\reduction.py", line 79, in duplicate
return _winapi.DuplicateHandle(
OSError: [WinError 6] 句柄无效。

(game) X:\game\DDNM-main>
(game) X:\game\DDNM-main>
(game) X:\game\DDNM-main>python main.py --ni --config celeba_hq.yml --path_y celeba_hq --eta 0.85 --deg "sr_averagepooling" --deg_scale 4.0 --sigma_y 0 -i demo
INFO - main.py - 2022-12-06 12:28:28,418 - Using device: cuda
Run SVD-based DDNM. 100 sampling steps. travel_length = 1, travel_repeat = 1. Task: sr_averagepooling.
Dataset has size 8
X:\python\Anaconda\envs\game\lib\site-packages\torch\utils\data\dataloader.py:554: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 12 (cpuset is not taken into account), which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
warnings.warn(_create_warning_msg(
Start from 0
0%| | 0/8 [00:00<?, ?it/s]
ERROR - main.py - 2022-12-06 12:28:36,459 - Traceback (most recent call last):
File "X:\game\DDNM-main\main.py", line 166, in main
runner.sample(args.simplified)
File "X:\game\DDNM-main\guided_diffusion\diffusion.py", line 208, in sample
self.svd_based_ddnm_plus(model, cls_fn)
File "X:\game\DDNM-main\guided_diffusion\diffusion.py", line 537, in svd_based_ddnm_plus
for x_orig, classes in pbar:
File "X:\python\Anaconda\envs\game\lib\site-packages\tqdm\std.py", line 1195, in iter
for obj in iterable:
File "X:\python\Anaconda\envs\game\lib\site-packages\torch\utils\data\dataloader.py", line 435, in iter
return self._get_iterator()
File "X:\python\Anaconda\envs\game\lib\site-packages\torch\utils\data\dataloader.py", line 381, in _get_iterator
return _MultiProcessingDataLoaderIter(self)
File "X:\python\Anaconda\envs\game\lib\site-packages\torch\utils\data\dataloader.py", line 1034, in init
w.start()
File "X:\python\Anaconda\envs\game\lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
File "X:\python\Anaconda\envs\game\lib\multiprocessing\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "X:\python\Anaconda\envs\game\lib\multiprocessing\context.py", line 336, in _Popen
return Popen(process_obj)
File "X:\python\Anaconda\envs\game\lib\multiprocessing\popen_spawn_win32.py", line 93, in init
reduction.dump(process_obj, to_child)
File "X:\python\Anaconda\envs\game\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'Diffusion.svd_based_ddnm_plus..seed_worker'

Traceback (most recent call last):
File "", line 1, in
File "X:\python\Anaconda\envs\game\lib\multiprocessing\spawn.py", line 107, in spawn_main
new_handle = reduction.duplicate(pipe_handle,
File "X:\python\Anaconda\envs\game\lib\multiprocessing\reduction.py", line 79, in duplicate
return _winapi.DuplicateHandle(
OSError: [WinError 6] 句柄无效。

i used the wrong python version???i have installed the required environment

Thanks for the great work !
I can not see codes example using HiR method .
the solution of MSR and HiR, only work in pixel space, doesn't apply to latent space ,because the consistency can't be satisfied in latent space ?
Consistency : Axˆ ≡ y, Realness : xˆ ∼ q(x),

Dear authors, thank you for putting up the code for study, I am currently working on image translation from grey scale ultrasound images to colour elastography as shown below.
Therefore, what I have to do is to train two separate guided-diffusion models on the grey-scale and colour images, and later use your DDNM method?
As this issue mentioned, is the image colorization script now ready to run?
#7

How to apply to my own task? The A matrix is hard to represent

For example, if the task is demoire or derain, how to do that with ddnm?

报下面的错误，是不是预训练的模型有问题？
ERROR - main.py - 2022-12-10 22:02:41,698 - Traceback (most recent call last):
File "E:\project\py\DDNM\main.py", line 166, in main
runner.sample(args.simplified)
File "E:\project\py\DDNM\guided_diffusion\diffusion.py", line 138, in sample
model.load_state_dict(torch.load(ckpt, map_location=self.device))
File "D:\anaconda3\lib\site-packages\torch\serialization.py", line 795, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "D:\anaconda3\lib\site-packages\torch\serialization.py", line 1020, in _legacy_load
typed_storage._storage._set_from_file(
RuntimeError: unexpected EOF, expected 551739 more bytes. The file might be corrupted.

There is a way to apply this method on Tiny-ImageNet (64 X 64)?
there is an existing model for that, or a new training is necessary?

作者工作十分优秀，在代码复现的过程中，有几个问题请作者解答。1.hq_demo中的推理和其文件夹之外的推理使用不同的权重不同的配置文件，这是为什么？2.模糊的数据，不是imagenet,也不是人脸数据，应该如何恢复，还需要在guided_diffusion 上训练权重吗？

Thanks for this great new work.

Is there a way to adapt this to images >256 pixels size? Like using tiles or mosaic?

Hi, I try to reproduce the results, and I follow your guide, run the evaluation.sh, here are the result I got.

I have two questions:
(1) The results reported in the paper or colorization task is evaluated via Cons metrics, but the code is via PSNR. And interestingly, I found they are close on CelebA datasets.

(2) How can I reproduce the results on inpainting task? My results are much poor than yours.

The results I collected from ouput are as followings:

Thanks a lot if you could help me.

When running something like this

python main.py --ni --config imagenet_256.yml --path_y imagenet --eta 0.85 --deg "sr_averagepooling" --deg_scale 4 --sigma_y 0. -i imagenet_sr_ap_4

generated images have size 256x256

Hi @wyhuai ,

Happy new year of the rabbit.

I like your work very much. Recently, I tried it for medical image super-resolution and denoising but suffered from quality issues.

I trained a DDPM on my dataset and verified that it can generate very good images.

Then, I applied the DDNM for denoising and SR.

Original image:

128x128

Reconstructed image

The results still have large noise and obvious block effects. I have tried different sigma_y but it didn't improve the quality.

I modified the code https://github.com/wyhuai/DDNM/blob/main/hq_demo/main.py as follows

Any comments are highly appreciated. BTW, the question is not urgent. Enjoy your holidays first:)

"""
This is an DDNM implementation based on RePaint. 
This implementation support arbitrary in/out image size.
At present, we only implemented SR and noisy SR tasks.
"""

import os
import argparse
import torch as th
import torch.nn.functional as F
import time
import conf_mgt
from utils import yamlread
from guided_diffusion import dist_util
import torchvision.transforms as transforms
from PIL import Image

# Workaround
try:
    import ctypes
    libgcc_s = ctypes.CDLL('libgcc_s.so.1')
except:
    pass


from guided_diffusion.script_util import (
    NUM_CLASSES,
    model_and_diffusion_defaults,
    classifier_defaults,
    add_dict_to_argparser,
    create_model_and_diffusion,
    create_classifier,
    select_args,
    args_to_dict,
)  # noqa: E402

def read_model_and_diffusion(args, model_path):
    """Reads the latest model from the given directory."""

    model, diffusion = create_model_and_diffusion(
        **args_to_dict(args, model_and_diffusion_defaults().keys()), conf=conf,
    )
    model.load_state_dict(dist_util.load_state_dict(model_path, map_location="cuda"))

    # if args.use_fp16:
    #     model.convert_to_fp16()
    model.eval()
    return model, diffusion

# def main(conf: conf_mgt.Default_Conf, args):

# if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument('--config', type=str, required=False, default="/home/jma/Documents/I2I/DDNM/hq_demo/confs/ct256.yml")
parser.add_argument('--deg', type=str, required=False, default="sr_averagepooling")
parser.add_argument('--sigma_y', type=float, required=False, default=1.5)
parser.add_argument('-i', "--save_path", type=str, required=False, default="demo15") 

"""
SR scales should be divisible by 256, e.g., 2, 4, 8, 16 ...
"""    
parser.add_argument('--scale', type=int, required=False, default=4)


"""
resize y to the same shape with the desired result
"""
parser.add_argument('--resize_y', default=True, action='store_true')
parser.add_argument('--path_y', type=str, required=False, default="data/datasets/ct128.png")
"""
no classifier guidance
"""
parser.add_argument('--class', type=int, required=False, default=323)
args = vars(parser.parse_args())

conf_arg = conf_mgt.conf_base.Default_Conf()
conf_arg.update(yamlread(args.get('config')))

def create_argparser():
    defaults = dict(
        image_size=256,
        batch_size=1,
        num_channels=64,
        num_res_blocks=3,
        num_heads=1,
        diffusion_steps=1000,
        noise_schedule='linear',
        lr=1e-4,
        clip_denoised=False,
        num_samples=1, # 10000
        use_ddim=True,
        # timestep_respacing='ddim250',
        model_path="",
    )
    ori = model_and_diffusion_defaults()
    # defaults.update(model_and_diffusion_defaults())
    ori.update(defaults)
    parser = argparse.ArgumentParser()
    add_dict_to_argparser(parser, ori)
    return parser

model_args = create_argparser().parse_args()

conf = conf_arg

print("Start", conf['name'])
print(f"{conf=}")
print(f"{args=}")

os.chdir('/home/jma/Documents/I2I/DDNM/hq_demo')
ct_model_path = './data/pretrained/ct256.pt'
device = dist_util.dev(conf.get('device'))
model, diffusion = read_model_and_diffusion(model_args, ct_model_path)
model.to(device)
if conf.use_fp16:
    model.convert_to_fp16()
model.eval()

show_progress = conf.show_progress

cond_fn = None

def model_fn(x, t, y=None, gt=None, **kwargs):
    assert y is not None
    return model(x, t, y if conf.class_cond else None, gt=gt)

print("sampling...")
all_images = []

dset = 'eval'

eval_name = conf.get_default_eval_name()

dl = conf.get_dataloader(dset=dset, dsName=eval_name)

for batch in iter(dl):

    for k in batch.keys():
        if isinstance(batch[k], th.Tensor):
            batch[k] = batch[k].to(device)

    model_kwargs = {}

    gt = Image.open(args.get("path_y")).convert('RGB')
    data_transform=transforms.Compose([
                        transforms.ToTensor(),
                        transforms.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])
                    ])
    #print("gt.size:",gt.size)
    gt = data_transform(gt).unsqueeze(0).to("cuda")
    model_kwargs["gt"] = gt
    model_kwargs['scale'] = args.get('scale')
    model_kwargs['deg'] = args.get('deg')
    model_kwargs['resize_y'] = args.get('resize_y')
    model_kwargs['sigma_y'] = args.get('sigma_y')
    model_kwargs['save_path'] = args.get('save_path')

    gt_keep_mask = batch.get('gt_keep_mask')
    if gt_keep_mask is not None:
        model_kwargs['gt_keep_mask'] = gt_keep_mask

    batch_size = model_kwargs["gt"].shape[0]

    if conf.cond_y is not None:
        classes = th.ones(batch_size, dtype=th.long, device=device)
        model_kwargs["y"] = classes * conf.cond_y
    else:
        classes = th.randint(
            low=0, high=NUM_CLASSES, size=(batch_size,), device=device
        )
        model_kwargs["y"] = classes
        
    classes = th.ones(batch_size, dtype=th.long, device=device)
    model_kwargs["y"] = classes * args.get("class") 

    sample_fn = (
        diffusion.p_sample_loop if not conf.use_ddim else diffusion.ddim_sample_loop
    )


    result = sample_fn(
        model_fn,
        (batch_size, 3, conf.image_size, conf.image_size),
        clip_denoised=conf.clip_denoised,
        model_kwargs=model_kwargs,
        cond_fn=cond_fn,
        device=device,
        progress=show_progress,
        return_all=True,
        conf=conf
    )

print("sampling complete")

Hi， the Download link 'https://image-editing-test-12345.s3-us-west-2.amazonaws.com/checkpoints/celeba_hq.ckpt' can't work. It is in File "DDNM/guided_diffusion/diffusion.py", line 134 .
I open the URL, and it shows：

This XML file does not appear to have any style information associated with it. The document tree is shown below.

AccessDenied
Access Denied
RJY67AGA9CASHPB4
k/v9atKLGKrKfqeC7bKJKzqE9Y4Mrt1LZCqYQuNHH4OphxuSg0FLNuO/LTMs6bGmHf1aH961B7uHDKnirAmWLQ==

And When I enter the command
python main.py --ni --simplified --config celeba_hq.yml --path_y celeba_hq --eta 0.85 --deg "sr_averagepooling" --deg_scale 4.0 --sigma_y 0 -i demo
an error occurs as follows：

INFO - main.py - 2023-05-31 11:21:07,706 - Using device: cuda
1.02kB [00:00, 1.57MB/s]
ERROR - main.py - 2023-05-31 11:21:11,007 - Traceback (most recent call last):
File "/DDNM/main.py", line 166, in main
runner.sample(args.simplified)
File "/DDNM/guided_diffusion/diffusion.py", line 138, in sample
model.load_state_dict(torch.load(ckpt, map_location=self.device))
File "/anaconda3/envs/ddnm/lib/python3.9/site-packages/torch/serialization.py", line 595, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/anaconda3/envs/ddnm/lib/python3.9/site-packages/torch/serialization.py", line 764, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '<'.

Hi. Thanks for the great job.
When running the main.py, I'm getting the mentioned error.
Note : I'm using torch 1.8 with cuda 10.1
Please advise.

Hi, bro. Nice to see you. When I evaluate the metrics output image, I find that the PSNR you implement is different from the code implemented by torchmetrics.

Here is your version:

 mse = torch.mean((x[0].to(self.device) - orig) ** 2)
 psnr = 10 * torch.log10(1 / mse)

But the torchmetrics is

mse = torch.mean((x[0].to(self.device) - orig) ** 2)
psnr = 10 * torch.log10(torch.max(orig)**2 / mse)

so when I use the api implemented by torchmetrics to calculate the psnr of output image

from torchmetrics import PeakSignalNoiseRatio
psnr_calculator = PeakSignalNoiseRatio()
psnr_calculator(x[0], orig.cpu())

They will lead to different output. For example, when I run this script

python main.py --ni --simplified --config imagenet_256.yml --path_y imagenet --eta 0.85 --deg "sr_averagepooling" --deg_scale 4.0 --sigma_y 0.1 -i demo

The output by torchmetrics is 30.3245, and yours is 30.3587. Although they are close, I want to know if it's important.

First, thank you for the great project. The mathematical processing is very interesting and wonderful.

Recently, I am studying image inpainting with Stable Diffusion 2. As far as I understand, since this project uses linear algebra, I think it will work regardless of the model structure if it is a diffusion model.
I've tried to apply it myself, but the size of the model and the structure of the yaml file are so different that it is difficult to apply it.

Then I found the hq_demo folder and the code looks cleaner there. Do you recommend applying custom model over hq_demo?

I am not an English speaker, so English may be awkward. I kindly ask for your understanding.

The tool link in the 'Try your own photos' section doesn't seem to be working properly. Could you please check it? I want to upscale my own photos from LR to SR.

Hi authors, thanks for your great work.

I'm trying to apply the equations to latent diffusion models but it doesn't work (as you explained in another issue). Would you mind providing some insights on how to make it work? If this can be achieved without gradients?

Here is my test result:

Hi @wyhuai ,

Thanks for sharing the great work.

What's the difference between the following two main files? I know the 2nd one is scalable to different image sizes. Is the 2nd one always better than the 1st one in terms of performance?

https://github.com/wyhuai/DDNM/blob/main/main.py

https://github.com/wyhuai/DDNM/blob/main/hq_demo/main.py

I want to use general images for image restoration tasks, such as outdoor images（The outdoor image contains many elements, unlike the example where there is only one）. According to the test on colob, there seems to be no classification number for outdoor images in the class classification. What command should I use to restore them?