Restore face without colorizing them about gfpgan HOT 18 CLOSED

The model can be fine-tuned to restore face w/o color. You could fine-tune it with the training codes (turn off the color augmentations in the config file).
We will upload such a model later~

from gfpgan.

@tachikoma777
If the training is OK, I will release it this week;-)

from gfpgan.

@tachikoma777 @syfbme
I have updated the model without colorization. Colab demo: https://colab.research.google.com/drive/1sVsoBd9AjckIXThgtZhGrHRfFI6UUYOo

from gfpgan.

The model can be fine-tuned to restore face w/o color. You could fine-tune it with the training codes (turn off the color augmentations in the config file).
We will upload such a model later~

Thanks for your quick reply. But how to turn off the color augmentations? I only find 4 settings in file train_gfpgan_v1.yml:
color_jitter_prob: 0.3
color_jitter_shift: 20
color_jitter_pt_prob: 0.3
gray_prob: 0.01

from gfpgan.

Do you mean that we set gray_prob to 0 so that model won't colorize it?

from gfpgan.

@syfbme
You can try:

have a very slight color jitter

color_jitter_prob: 0.5
color_jitter_shift: 5  # use a slight color jiter
color_jitter_pt_prob: ~  
gray_prob: 0.01
gt_gray: True

or
totally no color jitter

color_jitter_prob: ~
color_jitter_shift: 20
color_jitter_pt_prob: ~
gray_prob: 0.01
gt_gray: True

from gfpgan.

Hi @xinntao
Thanks for your advise. But i am confused. What i want is to keep black white photo the same color without colorizing them. So i think set "gray_prob" to 0 so that model won't learn the mapping from gray to color. The settings you advised is no color jitter but the model is still able to learn the mapping from gray to color(RGB). Please correct me if i am wrong. Looking forward to your reply~

from gfpgan.

we also set gt_gray, which means we also make the targets to be gray. Such a setting is for better generalization for gray photos.

from gfpgan.

The model can be fine-tuned to restore face w/o color. You could fine-tune it with the training codes (turn off the color augmentations in the config file).
We will upload such a model later~

Hi @xinntao
When do you plan to upload the model? Looking forward to it.

from gfpgan.

The model can be fine-tuned to restore face w/o color. You could fine-tune it with the training codes (turn off the color augmentations in the config file).
We will upload such a model later~

Hi @xinntao How to set the config file when fin-tuning the pretrained model.

There are 4 pretrained models but there are much more pretrained path in yml file:

How do i set each value?

Maybe just load pretrained network g and training others from beginning?

from gfpgan.

@syfbme

modify the pretrain_network_g to the path to GFPGANv1.pth
And you can train others from scratch ~

from gfpgan.

@syfbme

modify the pretrain_network_g to the path to GFPGANv1.pth
And you can train others from scratch ~

I am already doing this. Since the training is slow, i will paste the result later

from gfpgan.

Hi @xinntao
Current result is promising. I have set parameters as you suggested.
color_jitter_prob: ~ color_jitter_shift: 20 color_jitter_pt_prob: ~ gray_prob: 0.01 gt_gray: True
After 572k training steps, the result is as below:

The color is fading. However, it still colorize the below part of face. I will set the gray_prob larger(such as 0.2) to continue fine-tuning current model. Do you have any suggestions?

from gfpgan.

If you do not want to have any color enhancement, you may set color_jitter_shift: ~

from gfpgan.

If you do not want to have any color enhancement, you may set color_jitter_shift: ~

I have already set "color_jitter_prob: ~", so the "color_jitter_shift" will never be used, right?

from gfpgan.

OK， you are right.

from gfpgan.

The model can be fine-tuned to restore face w/o color. You could fine-tune it with the training codes (turn off the color augmentations in the config file).
We will upload such a model later~

May I ask when you plan to release pretrain mode w/o color change? Looking forward to it!

from gfpgan.

If you do not want to have any color enhancement, you may set color_jitter_shift: ~

Hi @xinntao @syfbme, I have set "color_jitter_prob: ~" and retrain, but the restore results of gray images are still with colorization. Would you please give some advice.

The config of dataset is listed below:

# dataset and data loader settings
datasets:
  train:
    name: FFHQ
    type: FFHQDegradationDataset
    dataroot_gt: ../../dataset/ffhq
    io_backend:
      # type: lmdb
      type: disk

    use_hflip: true
    mean: [0.5, 0.5, 0.5]
    std: [0.5, 0.5, 0.5]
    out_size: 256

    blur_kernel_size: 41
    kernel_list: ['iso', 'aniso']
    kernel_prob: [0.5, 0.5]
    blur_sigma: [0.1, 10]
    downsample_range: [0.8, 8]
    noise_range: [0, 20]
    jpeg_range: [60, 100]

    # color jitter and gray
    #color_jitter_prob: 0.3
    #color_jitter_shift: 20
    #color_jitter_pt_prob: 0.3
    #gray_prob: 0.01

    # If you do not want colorization, please set
    color_jitter_prob: ~
    color_jitter_shift: 0
    color_jitter_pt_prob: ~
    gray_prob: 0.05
    gt_gray: True

    crop_components: true
    component_path: experiments/pretrained_models/FFHQ_eye_mouth_landmarks_256.pth
    # eye_enlarge_ratio: 1.4

    # data loader
    use_shuffle: true
    num_worker_per_gpu: 12
    batch_size_per_gpu: 12
    dataset_enlarge_ratio: 1
    prefetch_mode: ~

from gfpgan.