ha0tang / c2gan Goto Github PK

Hi, thanks for the great paper and work.
I am trying to reproduce the results for Market1501 dataset. I use the same hyperparameters mentioned in the paper.

• During training, after 14 epochs, the Keypoint generator stops producing any output for fake and recovered images. Have you experienced this? Any leads on how this can be fixed?
• The paper states that OpenPose is used for body keypoint output for the first few epochs. Please explain how this has been incorporated in the code. I do not see any loading of a pretrained openpose model in the training code.
• A little more detail on how the training was conducted for Market 1501 will be helpful to me (which model for netG and netD)

Thanks,
jysa01

Thank you for your excellent work,
And would it be better by add SpectralNorm in Discriminator?

I got into trouble when I reproduced your paper, can you help me?

My question is about the processing of the Radbound dataset. I downloaded your script for your Radbound dataset and found that the face images were not equal to the number of landmark pictures.

I noticed what you mentioned in your paper: 'We first remove those images in who's face is not detected correctly the public OpenFace software, leading to 5,628 training information and leading 1,407 testing image pair'. The training set and the test set add up to 7,035 images, which corresponds to the number of face images which I downloaded (there are 1005 images in each face expression folder in the data set. if all the face images are combined with lankmarks, the total number of image pairs is just 7035).

But the problem is that you filtered the undetected landmark, so there aren't enough landmarks to pair with the face images. So I'd like to ask how much data you actually used in your experiments. If 7035 images were used, as the paper says, how do you solve the problem of the missing lankmarks?

Thank you for your great repository!

I encountered error in networks RuntimeError: uniform_ expects to return a [from, to) range, but found from=1 > to=0.02

some range transformation needed to solve this issue

but I guess this could be something related to pytorch version error but same error appears on pytorch 0.4.1 :(

Hi, I met a problem during training the network.

At train phase, due to opt.display_freq=1, at each step, 8 images are generated, where real_A, real_B, input_C, input_D should correspond to the input of the network. But I observed that they did not form a pair. See the 12th epoch output.
In this picture, real_A and real_B are obviously not the same person.

I have read the code carefully, but I can’t find the reason. Do you know why this happened?

Thanks for sharing the code. The concept is good.

I want to get pose transfer of source image. For this i want to use MVC dataset (https://github.com/MVC-Datasets/MVC) contains 4 poses of images(streight view,left side view, right side view, back view). I preprocessed these images that,
1.Non-human images have to be deleted using face-detection
2.I created key-points for these images using open-pose

I want to train these images such that
image A:streight view,
image B: left view
I have a following questions which can solve my problem to train:

1. Suppose I trained for those two types images (streight, left view), to get a lefft view of test_image, how can I give its keypoints?? (I don't have left view of test_image. I want generate it)

2. How to train for high resolution images??

Thanks in Advance.
Regards,
Sandhya

The whole loss consists of:
GAN loss(both Keypoint and Image)
Cycle loss(both Keypoint and Image)
L1 loss(Only Image)

I wonder why you not add L1 loss of Keypoint. Did you experiment with that?

How to generate .pts keypoint file or extract keypoint of face from my own image.

I Can't dowload pretrain model with command bash ./scripts/download_c2gan_model.sh Radboud
Are you have link to dowload it?

Hello @Ha0Tang ,
I tried to reproduce the results of the Keypoint guided person Image generation using your C2GAN code on Market1501 data.
I downloaded the data from https://www.kaggle.com/pengcw1/market-1501/data
and organized the data into train and test set as discussed in the paper.
I use the Resnet9_block model as the Generator network and the hyperparameters are chosen from the paper for training.

During testing, I get some results which indicate that the model is learning the target pose.
I am unable to explain why or how the recovered_A image looks exactly like a copy of the real_A.
For instance, consider the below sample:

Note: The L1 loss and MSE loss are computed between the Real_A and recovered_A.

During testing,
The output Fake_B = netG(combined_realA_inputC) though not perfect, is believable
But in case of Recovered_A = netG(combined_FakeB_inputD) , it is surprising to me that the generator is able to output the texture on the shirt though only the images FakeB and inputD were provided at input.
The model has NOT been trained on this sample as both train and test sets are disjoint sets.
I spent some time analysing the code but I am unable to explain this behavior. Is this expected from the model? If yes, what was the motivation ? Kindly help me understand this behavior of the model.

Thanks and best regards,
jysa01

Hi, Hao Tang! I met with difficulties again. Could you help me to figure them out?

I tried to reproduce your experiment but got wrong result. I used the weight you provided to run the network. While the result of SSIM was only 0.82 (0.86 in your paper), PSNR was 23.53 (21.9 in your paper). I checked the generated images in the result folder while they were normal and the input images sucessfully translated into another expressions .

Here is my specific workflow:

1. Download dataset and weight
2. Pair face with landmark, and lead to a row of images [neutral_face_A, other_expression_B, other_landmark_B, neutral_landmark_A]
3. Split the whole dataset, and get 1407 images for testset. (BTW, I tried different seeds to make testset, but the value of SSIM are all around 0.82)
4. Run the test shell script with the default args you set before.
5. Calculate SSIM and PSNR of each pairs of images and take the average.

Do I need to train the network by myself to reproduce the experiment, or I just did something wrong during the test process?