zsyoaoa / difface Goto Github PK
View Code? Open in Web Editor NEWDifFace: Blind Face Restoration with Diffused Error Contraction (TPAMI, 2024)
License: Other
DifFace: Blind Face Restoration with Diffused Error Contraction (TPAMI, 2024)
License: Other
How long it takes to train the entire network?
Number of parameters: 15.79M
=> Loaded checkpoint /home/zhuchao/code/diface/code/diface/weights/SwinIR/General_Face_ffhq512.pth
Traceback (most recent call last):
File "main_sr.py", line 32, in
trainer = Trainer(configs)
File "/home/zhuchao/code/diface/code/diface/trainer.py", line 285, in init
super().init(configs)
File "/home/zhuchao/code/diface/code/diface/trainer.py", line 67, in init
self.resume_from_ckpt()
File "/home/zhuchao/code/diface/code/diface/trainer.py", line 145, in resume_from_ckpt
util_net.reload_model(self.model, ckpt['state_dict'])
KeyError: 'state_dict'
I want to finetune swinIR on a small face datasets, but I met this error.
It seems the General_Face_ffhq512.pth not save state_dict and iters_start,I don't know how to fix it.
Hope you can help me, THANKS!
adjust_lr() missing 1 required positional argument: 'ii'adjust_lr() missing 1 required positional argument: 'ii'.
In trainer_py trainer.py", line 255, in train self.adjust_lr()self.adjust_lr()
Please correct me if I am wrong? What to put there? I am trying to train model with batch size [8, 2]
followed the example code from the huggingface model card but got the error
then tried the below with the same error
from diffusers import DiffusionPipeline
model_id = "OAOA/DifFace"
# load model and scheduler
difface = DiffusionPipeline.from_pretrained(model_id)
i'm using diffusers==0.25.0
Have you ever tried to decrease the lr(the default value is 1e-4) for the training procedure of diffusion model?
Since the loss converges very quick(less than 1W iters), but the lr keeps constant during the whole training time.
python.exe D:\develop\DifFace\inference_difface.py --in_path D:\develop\DifFace\testdata\whole_imgs --out_path D:\data --gpu_id 0
Setting random seed 20000
Loading from ./weights/diffusion/iddpm_ffhq512_ema500000.pth...
Loaded Done
C:\Users***\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\actions-runner_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\TensorShape.cpp:3191.)
return _VF.meshgrid(tensors, kwargs) # type: ignore[attr-defined]
Loading from ./weights/SwinIR/General_Face_ffhq512.pth...
Loaded Done
C:\Users*****\AppData\Local\Programs\Python\Python39\lib\site-packages\torchvision\models_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
warnings.warn(
C:\Users*\AppData\Local\Programs\Python\Python39\lib\site-packages\torchvision\models_utils.py:223: UserWarning: Arguments other than a weight enum or None
for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing weights=None
.
warnings.warn(msg)
Traceback (most recent call last):
File "D:\develop\DifFace\inference_difface.py", line 160, in
main()
File "D:\develop\DifFace\inference_difface.py", line 137, in main
image_restored, face_restored, face_cropped = sampler_dist.sample_func_bfr_unaligned(
File "D:\develop\DifFace\sampler.py", line 368, in sample_func_bfr_unaligned
restored_faces = _process_batch(self.face_helper.cropped_faces)
File "D:\develop\DifFace\sampler.py", line 332, in _process_batch
restored_faces = self.sample_func_ir_aligned(
File "D:\develop\DifFace\sampler.py", line 279, in sample_func_ir_aligned
sample = self.diffusion.p_sample_loop(
File "D:\develop\DifFace\models\gaussian_diffusion.py", line 428, in p_sample_loop
for sample in self.p_sample_loop_progressive(
File "D:\develop\DifFace\models\gaussian_diffusion.py", line 484, in p_sample_loop_progressive
out = self.p_sample(
File "D:\develop\DifFace\models\gaussian_diffusion.py", line 383, in p_sample
out = self.p_mean_variance(
File "D:\develop\DifFace\models\respace.py", line 88, in p_mean_variance
return super().p_mean_variance(self._wrap_model(model), *args, **kwargs)
File "D:\develop\DifFace\models\gaussian_diffusion.py", line 278, in p_mean_variance
min_log = _extract_into_tensor(
File "D:\develop\DifFace\models\gaussian_diffusion.py", line 105, in _extract_into_tensor
res = th.from_numpy(arr).to(device=timesteps.device)[timesteps].float()
KeyboardInterrupt
I met this problem when I tried to run the command CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --standalone --nproc_per_node=4 --nnodes=1 main_diffusion.py --gpu_id 0123 --cfg_path configs/training/diffusion_ffhq512.yaml --save_dir myfolder
. Could someone help me solve it?
By running this command python inference_difface.py --in_path ~/Images/blur/blurred_1.png --out_path ~/Images/blur/unblurred
I got error
Here the python command line stack Trace
Traceback (most recent call last):
File "inference_difface.py", line 160, in <module>
main()
File "inference_difface.py", line 137, in main
image_restored, face_restored, face_cropped = sampler_dist.sample_func_bfr_unaligned(
File "~/programmation/DifFace/sampler.py", line 368, in sample_func_bfr_unaligned
restored_faces = _process_batch(self.face_helper.cropped_faces)
File "~/programmation/DifFace/sampler.py", line 328, in _process_batch
cropped_face_t = np.stack(
File "<__array_function__ internals>", line 180, in stack
File "~/anaconda3/envs/DifFace/lib/python3.8/site-packages/numpy/core/shape_base.py", line 422, in stack
raise ValueError('need at least one array to stack')
ValueError: need at least one array to stack
I have CUDA installed, here the output of nvidia-smi
command
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.105.17 Driver Version: 525.105.17 CUDA Version: 12.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:01:00.0 Off | N/A |
| N/A 42C P8 N/A / N/A | 4MiB / 4096MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 952 G /usr/lib/xorg/Xorg 4MiB |
+-----------------------------------------------------------------------------+
Hi, you have done a nice work! I am interested in your paper. Could you please release your pretrain diffusion ckpts on FFHQ?
I retrained the Train diffused estimator (SwinIR) stage and found that the saved weights are dict_keys(['iter_start','log_step','log_step_img','state_dict']) and not SwinIR's weights, so the inference reports a load model error. Would like to ask if the code in main_sr.py about saving model weights is wrong? I'm looking forward to hearing from the author!
Thank you for your great Job!!
You updated this repo in 2023/12/12 maybe, so there is no links for SwinIR model
https://github.com/zsyOAOA/DifFace/releases/download/V1.0/General_Face_ffhq512.pth
and now, I downloaded it(below link) from your release page instead of before link
https://github.com/zsyOAOA/DifFace/releases/download/V1.0/swinir_restoration512_L1.pth
but, I cant load the model for running inference.py
I trained SwinIR and saved the ckpts .
i'm trying to inference the new model after i changed the ckpts paths in "iddpm_ffhq512_swinir.yaml"
i have issue in loading new model state_dict
" RuntimeError: Error(s) in loading state_dict for SwinIR:
Missing key(s) in state_dict: "conv_first.1.weight", "conv_first .... "
note : " inference code is worked with pretrained "model /weights/SwinIR/General_Face_ffhq512.pth" .
Hi, thanks for sharing your code! Could you please tell me what and how many GPUs you used and how long you trained for?Thanks!
I met this problem when I tried to run the command CUDA_VISIBLE_DEVICES=0 python main_sr.py --cfg_path configs/training/swinir_ffhq512.yaml --save_dir save_dir_1 Could someone help me solve it?
Thank for sharing your great work!! : )
if last_batch or self.num_gpus <= 1:
loss = self.loss_fun(hq_pred, micro_data['gt']) / hq_pred.shape[0]
else:
with self.model.no_sync():
loss = self.loss_fun(hq_pred, micro_data['gt']) / hq_pred.shape[0]
=======================
Please check these lines.
@zsyOAOA how to train FFHQ on 1024x1024 resolution. I have checked the swinIR_ffhq512 the img_size : 64 (at line8).
This file is not available for model training
Hi, I am interested in your nice work, but after reading your paper, I donot understand the way how estimator work.
Therefore, I read your code, and I suppose that the estimator(swinIR) is used to predict the x0 by low quality image y0, and add noise like diffusion forward process do?
Line 247 in 35d91a1
im_hq = self.model_ir(y0)
and
Line 259 in 35d91a1
yt = self.diffusion.q_sample(
x_start=post_fun(im_hq),
t=torch.tensor([start_timesteps,]*im_hq.shape[0], device=device),
)
finally sample by ddpm
Line 279 in 35d91a1
sample = self.diffusion.p_sample_loop(
self.model,
shape=yt.shape,
noise=yt,
start_timesteps=start_timesteps,
clip_denoised=True,
denoised_fn=None,
model_kwargs=None,
device=None,
progress=False,
)
The script I run is as followings. Did I miss any details? ^_^
python inference_difface.py --aligned --in_path testdata/cropped_faces --out_path result/testdata --gpu_id 2
Your work is awesome! I have tested with your pre-trained model on CelebaTest and got amazing visual results.
However, I noticed that your paper and the VQFR's paper both provide metrics for testing VQFR on CelebaTest, and these metrics are different.
Thus, I used your model to infer on the CelebaTest dataset provided by VQFR link here and use the calculation script provided by VQFR link here, the unexpected results were obtained.
Therefore, I am very curious about the quantitative metrics mentioned in the paper. How do you calculate the metrics? Could you release your metric calculation script please?
Could you add a device number to the command line parameters?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.