yan9-y / lafin Goto Github PK

thanks for sharing your code
i use google colab and when i type this line to test model

!python test.py --model 2 --checkpoints checkpoints/celeba_models

i get this result

Loading InpaintingModel generator...

start testing...

End Testing

but folder result is empty

i use this config.yml

config.zip

please help me

Traceback (most recent call last):
File "train.py", line 2, in
main(mode=1)
File "/home/wp/main_work_inpainting/Facial_inpainting/Facial_inpainting_JPU/main.py", line 56, in main
model.train()
File "/home/wp/main_work_inpainting/Facial_inpainting/Facial_inpainting_JPU/src/lafin.py", line 108, in train
dataset=self.train_dataset,
AttributeError: 'Lafin' object has no attribute 'train_dataset'

Hello, I have such a problem when training model 3, can you help me?

Hi @YaN9-Y.
Thank you for providing your codes.

I have two questions about the training of stage 1.

Firstly,
According to the TRAIN_LANDMARK_LANDMARK_FLIST in config.yml.example, the value should be set to wflw_train_landmarks, ok.

However, in this setting, because the wflw landmark has the 196 points, should we set LANDMARK_POINTS to 98?
The default value is set to 68 and you write the 68 in the paper.

Secondly,
When we want to use the default value of 68, which landmark dataset should we use?

Thanks.

hi, I find some white points on generated images. And i don't know why.

Does this happen in the pictures you generate? how to solve this problem

Hi, I am interested in the landmarkloss, and when I try to apply it, I found the torch.max(x) will return two result, one is the max value with grad and the other is the index of the max value.

For landmark loss, we need the index of the max value to calculate the L2(I_sr, I_gt). However, I found the index has no grad at all. So it will throw an error when loss.backward()

when i try to train
!python3 train.py --model 1 --checkpoints ./checkpoints/celeba
i get this error

./checkpoints/celeba/config.yml
Model configurations:

MODE: 1 # 1: train, 2: test, 3: eval
MODEL: 2 # 1: edge model, 2: inpaint model, 3: edge-inpaint model
MASK: 4 # 0: no mask(for testing landmark detector) 1: random block, 2: center mask, 3: external, 4: 50% external, 50% random block, 5: (50% no mask, 25% ramdom block, 25% external) 6: external non-random
SEED: 10 # random seed
GPU: [0] # list of gpu ids
AUGMENTATION_TRAIN: 0 # 1: use augmentation to train landmark predictor 0: not use
LANDMARK_POINTS: 68 # 68 for celeba, celeba-hq and 300w, 98 for wflw

TRAIN_INPAINT_IMAGE_FLIST: ./datasets/exampl_images.flist #training image set for inpaint model(stage 2 and 3)
VAL_INPAINT_IMAGE_FLIST: ./datasets/celeba_val_images.flist #validation image set for inpaint model(stage 2 and 3)
TEST_INPAINT_IMAGE_FLIST: ./datasets/celeba_test_images.flist #testing image set for inpaint model(stage 2 and 3)

TRAIN_INPAINT_LANDMARK_FLIST: ./datasets/celeba_train_landmarks.flist #training landmark set for inpaint model (stage 2 and 3)
VAL_INPAINT_LANDMARK_FLIST: ./datasets/celeba_val_landmarks.flist #validation landmark set for inpaint model (stage 2 and 3)
TEST_INPAINT_LANDMARK_FLIST: ./datasets/celeba_test_landmarks.flist #testing landmark set for inpaint model (stage 2)(test on stage 3 doesn't need test landmark flist)

TRAIN_MASK_FLIST: ./datasets/example_masks.flist #training mask set for all stages
VAL_MASK_FLIST: ./datasets/example_masks.flist #validation mask set for all stages
TEST_MASK_FLIST: ./datasets/example_masks.flist #testing mask set for all stages

TRAIN_LANDMARK_LANDMARK_FLIST: ./datasets/wflw_train_landmarks.flist #training landmark set for landmark model (stage 1)
TEST_LANDMARK_LANDMARK_FLIST: ./datasets/wflw_test_landmarks.flist #testing landmark set for landmark model (stage 1)
TRAIN_LANDMARK_IMAGE_FLIST: ./datasets/wflw_train_images.flist #training image set for landmark model(stage 1)
TEST_LANDMARK_IMAGE_FLIST: ./datasets/wflw_test_images.flist #testing image set for landmark model(stage 1)

LR: 0.0001 # learning rate
D2G_LR: 0.1 # discriminator/generator learning rate ratio
BETA1: 0.0 # adam optimizer beta1
BETA2: 0.9 # adam optimizer beta2
BATCH_SIZE: 4 # input batch size for training, 4 for inpaint training, 16 or more for landmark predictor, 8 or more for augmented landmark predictor
INPUT_SIZE: 256 # input image size for training.
MAX_ITERS: 1300000 # maximum number of iterations to train the model

L1_LOSS_WEIGHT: 1 # l1 loss weight
STYLE_LOSS_WEIGHT: 250 # style loss weight
CONTENT_LOSS_WEIGHT: 0.1 # perceptual loss weight
INPAINT_ADV_LOSS_WEIGHT: 0.01 # adversarial loss weight
TV_LOSS_WEIGHT: 0.1 # total variation loss weight

GAN_LOSS: lsgan # nsgan | lsgan | hinge
GAN_POOL_SIZE: 0 # fake images pool size

SAVE_INTERVAL: 1000 # how many iterations to wait before saving model (0: never)
SAMPLE_INTERVAL: 500 # how many iterations to wait before sampling (0: never)
SAMPLE_SIZE: 4 # number of images to sample
EVAL_INTERVAL: 0 # how many iterations to wait before model evaluation (0: never)
LOG_INTERVAL: 100 # how many iterations to wait before logging training status (0: never)

start training...

Traceback (most recent call last):
File "train.py", line 2, in
main(mode=1)
File "/content/drive/My Drive/Colab Notebooks/lafin-master/main.py", line 56, in main
model.train()
File "/content/drive/My Drive/Colab Notebooks/lafin-master/src/lafin.py", line 103, in train
shuffle=True
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 213, in init
sampler = RandomSampler(dataset)
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/sampler.py", line 94, in init
"value, but got num_samples={}".format(self.num_samples))
ValueError: num_samples should be a positive integer value, but got num_samples=0

Hi, thanks a lot for your work in implementing the code, but I really wonder why you use the ground truth landmarks, instead of the predicting result from your Landmark Detector?

Dear reaserchers, please also consider checking our newly introduced face inpainting method to address the symmetry problems of general inpainting methods by using swin transformer and semantic aware discriminators.
Our proposed method showed better results in terms of fid score and newly proposed metric which focus on the face symmetry compared to some of the state-of-the-art methods including lama.
Our paper is availabe at:
https://www.researchgate.net/publication/366984165_SFI-Swin_Symmetric_Face_Inpainting_with_Swin_Transformer_by_Distinctly_Learning_Face_Components_Distributions

The code also will be published in:
https://github.com/mohammadrezanaderi4/SFI-Swin

Hi lafin,
Can i use python 3.6 for this code?
Best regards,
PeterPham

when I train inpaining stage from scratch, I find the output img is alway been highlighted when using small irregular mask, however, it doesn't happen when using block or center mask and large irregular mask. I wonder if this happens because of the usage of irregular masks for training.

While I run this code in google colab:
!python3 test.py --model 3 --checkpoints "/content/lafin/checkpoints/example"

I got this error:
start testing...

THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=383 error=11 : invalid argument
Traceback (most recent call last):
File "test.py", line 2, in
main(mode=2)
File "/content/lafin/main.py", line 61, in main
model.test()
File "/content/lafin/src/lafin.py", line 367, in test
output_landmark = self.landmark_model(images * (1 - masks) + masks, masks)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/content/lafin/src/models.py", line 247, in forward
landmark_gen = self.mbnet(images_masked)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/content/lafin/src/networks.py", line 301, in forward
x = self.features(images) # C = 320
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/container.py", line 92, in forward
input = module(input)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/container.py", line 92, in forward
input = module(input)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/conv.py", line 338, in forward
self.padding, self.dilation, self.groups)
RuntimeError: cuda runtime error (11) : invalid argument at /pytorch/aten/src/THC/THCGeneral.cpp:383

Please Help!!

Colab Specification:

[name: "/device:CPU:0"
device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation: 6642222752450203202, name: "/device:GPU:0"
device_type: "GPU"
memory_limit: 14674281152
locality {
bus_id: 1
links {
}
}
incarnation: 15233849231393388862
physical_device_desc: "device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5"]