penincillin / dream Goto Github PK

I have been training DREAM on different pitch angle images for a long time, and it still seems not a feasible work.
Have you ever tried on pitch angles?

Hi,
I searched the original paper of CFP, but it did not mention a way to get it. Could you share that from where did you download the data set? Thanks.

Hello, there are some problems when calling test_process_align:

1. In cfp_alignment.txt, every image has 21 pairs points, which facial landmarks points you ysed? And, how to get these points??
2. pnp.txt, hoe to get this txt file?
3. When calling ./test_process_align $pnp_file $image_prefix $alignment_file $aligned_img_file $pose_file, I have a problem like this:

Data/Images/001/frontal/01.jpg ../../data/CFP/data/50_Cent/frontal/01.jpg
OpenCV Error: Assertion failed (_src.total() > 0) in warpPerspective, file /home/ly/git/DREAM/opencv-3.4.0/modules/imgproc/src/imgwarp.cpp, line 2978
terminate called after throwing an instance of 'cv::Exception'
  what():  /home/ly/git/DREAM/opencv-3.4.0/modules/imgproc/src/imgwarp.cpp:2978: error: (-215) _src.total() > 0 in function warpPerspective

./align_cfp.sh: 行 16: 10665 已放弃               (核心已转储)./test_process_align $pnp_file $image_prefix $alignment_file $aligned_img_file $pose_file

How to solve this??
Thanks

Hi, after review your code, I find yaw information is very important. It represents how many the face has rotated.ringht？ So what is the role of yaw information play in your network, how can I get the yaw information on my own dataset? Thanks

1.you said All the face images should be aligned. Please follow the align protocol in dataset CelebA.
what is your align protocol ？
2.CFP dataset only have 30 landmark ground truth, which is half face .how can we align face by this incomplete landmark ?
thanks for you reply.

I run sh eval_cfp.sh and get different result in CFP dataset as follows:
`resnet50_naive ../../data/model/cfp_res50_naive.pth.tar
----- result for Frontal-Frontal -----
Average auc: 0.9972375510204083
Average eer: 0.018285714285714287
----- result for Frontal-Profile -----
Average auc: 0.9738016326530612
Average eer: 0.08342857142857144

resnet50_end2end ../../data/model/cfp_res50_end2end.pth.tar

----- result for Frontal-Frontal -----
Average auc: 0.9980669387755103
Average eer: 0.012857142857142859
----- result for Frontal-Profile -----
Average auc: 0.9792285714285714
Average eer: 0.06885714285714287
but i can't understand the reason of different result. I read the code and I find a flag named "end2end" and: if self.end2end:
raw_feature = self.fc1(mid_feature)#残差块
raw_feature = self.relu(raw_feature)
raw_feature = self.fc2(raw_feature)
raw_feature = self.relu(raw_feature)

        yaw = yaw.view(yaw.size(0),1)
        yaw = yaw.expand_as(raw_feature)
        
        feature = yaw * raw_feature + mid_feature

    else:
        feature = mid_feature`

Does it mean that naive model don't have DREAM block?
Can you tell me the difference of cfp_res50_naive.pth.tar model and cfp_res50_end2end.pth.tar model?Thank you very much!

Thanks for your sharing. I found that there are two yaw types "nonli" and "linear" in the ijba dataset. Please what's the meaning? Thanks very much.

Thank you very much for your paper and the source code!
I am wondering if your non-linear optimization after epnp is Levenberg-Marquardt optimization.
looking forward to your replay. thank you very much!

what is the supported version of pytorch for this project?

Hi, now I have gotten a pre-trained model on our own dataset, if i want to apply DREAM block on it and the third training strategy in your paper, then I should do as below, right ?
step 1、Generate a celebrity_name_list file and then image_list.txt
step 2、Use the executable file you provide to get the yaw information of each image.
step 3、Add DREAM block to our model, then finetune our new-model （training both DREAM block and normal CNN part).
step 4、Continue training the DREAM block separately with frontal-profile face pairs.

If I am right, how can I get the frontal-profile face pairs in step 4 ?

When run branch_test.py, it appears FileNotFoundError: [Errno 2] No such file or directory: '/home/sensetime/Downloads/ext_feat/data/frontal_feat_naive.bin'

I want to run your code on my couputer, which do not have gpu devices. I changed " checkpoint = torch.load(resume)" to "checkpoint = torch.load(resume , map_location=lambda storage, location: storage) " and missed the problem as :
resnet50_naive ../../data/model/cfp_res50_naive.pth.tar
Traceback (most recent call last):
File "eval_cfp.py", line 155, in
extract_feat(arch, model_path)
File "eval_cfp.py", line 110, in extract_feat
model.load_state_dict(checkpoint['state_dict']) #model.load_state_dict(checkpoint['state_dict'])
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 845, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for ResNet:
Missing key(s) in state_dict: "conv1.weight", "bn1.running_var", "bn1.bias", "bn1.weight", "bn1.running_mean", "layer1.0.conv1.weight", "layer1.0.bn1.running_var", "layer1.0.bn1.bias", "layer1.0.bn1.weight", "layer1.0.bn1.running_mean", "layer1.0.conv2.weight", "layer1.0.bn2.running_var", "layer1.0.bn2.bias", "layer1.0.bn2.weight", "layer1.0.bn2.running_mean", "layer1.0.conv3.weight", "layer1.0.bn3.running_var", "layer1.0.bn3.bias", "layer1.0.bn3.weight", "layer1.0.bn3.running_mean", "layer1.0.downsample.0.weight", "layer1.0.downsample.1.running_var", "layer1.0.downsample.1.bias", "layer1.0.downsample.1.weight", "layer1.0.downsample.1.running_mean", "layer1.1.conv1.weight", "layer1.1.bn1.running_var", "layer1.1.bn1.bias", "layer1.1.bn1.weight", "layer1.1.bn1.running_mean", "layer1.1.conv2.weight", "layer1.1.bn2.running_var", "layer1.1.bn2.bias", "layer1.1.bn2.weight", "layer1.1.bn2.running_mean", "layer1.1.conv3.weight", "layer1.1.bn3.running_var", "layer1.1.bn3.bias", "layer1.1.bn3.weight", "layer1.1.bn3.running_mean", "layer1.2.conv1.weight", "layer1.2.bn1.running_var", "layer1.2.bn1.bias", "layer1.2.bn1.weight", "layer1.2.bn1.running_mean", "layer1.2.conv2.weight", "layer1.2.bn2.running_var", "layer1.2.bn2.bias", "layer1.2.bn2.weight", "layer1.2.bn2.running_mean", "layer1.2.conv3.weight", "layer1.2.bn3.running_var", "layer1.2.bn3.bias", "layer1.2.bn3.weight", "layer1.2.bn3.running_mean", "layer2.0.conv1.weight", "layer2.0.bn1.running_var", "layer2.0.bn1.bias", "layer2.0.bn1.weight", "layer2.0.bn1.running_mean", "layer2.0.conv2.weight", "layer2.0.bn2.running_var", "layer2.0.bn2.bias", "layer2.0.bn2.weight", "layer2.0.bn2.running_mean", "layer2.0.conv3.weight", "layer2.0.bn3.running_var", "layer2.0.bn3.bias", "layer2.0.bn3.weight", "layer2.0.bn3.running_mean", "layer2.0.downsample.0.weight", "layer2.0.downsample.1.running_var", "layer2.0.downsample.1.bias", "layer2.0.downsample.1.weight", "layer2.0.downsample.1.running_mean", "layer2.1.conv1.weight", "layer2.1.bn1.running_var", "layer2.1.bn1.bias", "layer2.1.bn1.weight", "layer2.1.bn1.running_mean", "layer2.1.conv2.weight", "layer2.1.bn2.running_var", "layer2.1.bn2.bias", "layer2.1.bn2.weight", "layer2.1.bn2.running_mean", "layer2.1.conv3.weight", "layer2.1.bn3.running_var", "layer2.1.bn3.bias", "layer2.1.bn3.weight", "layer2.1.bn3.running_mean", "layer2.2.conv1.weight", "layer2.2.bn1.running_var", "layer2.2.bn1.bias", "layer2.2.bn1.weight", "layer2.2.bn1.running_mean", "layer2.2.conv2.weight", "layer2.2.bn2.running_var", "layer2.2.bn2.bias", "layer2.2.bn2.weight", "layer2.2.bn2.running_mean", "layer2.2.conv3.weight", "layer2.2.bn3.running_var", "layer2.2.bn3.bias", "layer2.2.bn3.weight", "layer2.2.bn3.running_mean", "layer2.3.conv1.weight", "layer2.3.bn1.running_var", "layer2.3.bn1.bias", "layer2.3.bn1.weight", "layer2.3.bn1.running_mean", "layer2.3.conv2.weight", "layer2.3.bn2.running_var", "layer2.3.bn2.bias", "layer2.3.bn2.weight", "layer2.3.bn2.running_mean", "layer2.3.conv3.weight", "layer2.3.bn3.running_var", "layer2.3.bn3.bias", "layer2.3.bn3.weight", "layer2.3.bn3.running_mean", "layer3.0.conv1.weight", "layer3.0.bn1.running_var", "layer3.0.bn1.bias", "layer3.0.bn1.weight", "layer3.0.bn1.running_mean", "layer3.0.conv2.weight", "layer3.0.bn2.running_var", "layer3.0.bn2.bias", "layer3.0.bn2.weight", "layer3.0.bn2.running_mean", "layer3.0.conv3.weight", "layer3.0.bn3.running_var", "layer3.0.bn3.bias", "layer3.0.bn3.weight", "layer3.0.bn3.running_mean", "layer3.0.downsample.0.weight", "layer3.0.downsample.1.running_var", "layer3.0.downsample.1.bias", "layer3.0.downsample.1.weight", "layer3.0.downsample.1.running_mean", "layer3.1.conv1.weight", "layer3.1.bn1.running_var", "layer3.1.bn1.bias", "layer3.1.bn1.weight", "layer3.1.bn1.running_mean", "layer3.1.conv2.weight", "layer3.1.bn2.running_var", "layer3.1.bn2.bias", "layer3.1.bn2.weight", "layer3.1.bn2.running_mean", "layer3.1.conv3.weight", "layer3.1.bn3.running_var", "layer3.1.bn3.bias", "layer3.1.bn3.weight", "layer3.1.bn3.running_mean", "layer3.2.conv1.weight", "layer3.2.bn1.running_var", "layer3.2.bn1.bias", "layer3.2.bn1.weight", "layer3.2.bn1.running_mean", "layer3.2.conv2.weight", "layer3.2.bn2.running_var", "layer3.2.bn2.bias", "layer3.2.bn2.weight", "layer3.2.bn2.running_mean", "layer3.2.conv3.weight", "layer3.2.bn3.running_var", "layer3.2.bn3.bias", "layer3.2.bn3.weight", "layer3.2.bn3.running_mean", "layer3.3.conv1.weight", "layer3.3.bn1.running_var", "layer3.3.bn1.bias", "layer3.3.bn1.weight", "layer3.3.bn1.running_mean", "layer3.3.conv2.weight", "layer3.3.bn2.running_var", "layer3.3.bn2.bias", "layer3.3.bn2.weight", "layer3.3.bn2.running_mean", "layer3.3.conv3.weight", "layer3.3.bn3.running_var", "layer3.3.bn3.bias", "layer3.3.bn3.weight", "layer3.3.bn3.running_mean", "layer3.4.conv1.weight", "layer3.4.bn1.running_var", "layer3.4.bn1.bias", "layer3.4.bn1.weight", "layer3.4.bn1.running_mean", "layer3.4.conv2.weight", "layer3.4.bn2.running_var", "layer3.4.bn2.bias", "layer3.4.bn2.weight", "layer3.4.bn2.running_mean", "layer3.4.conv3.weight", "layer3.4.bn3.running_var", "layer3.4.bn3.bias", "layer3.4.bn3.weight", "layer3.4.bn3.running_mean", "layer3.5.conv1.weight", "layer3.5.bn1.running_var", "layer3.5.bn1.bias", "layer3.5.bn1.weight", "layer3.5.bn1.running_mean", "layer3.5.conv2.weight", "layer3.5.bn2.running_var", "layer3.5.bn2.bias", "layer3.5.bn2.weight", "layer3.5.bn2.running_mean", "layer3.5.conv3.weight", "layer3.5.bn3.running_var", "layer3.5.bn3.bias", "layer3.5.bn3.weight", "layer3.5.bn3.running_mean", "layer4.0.conv1.weight", "layer4.0.bn1.running_var", "layer4.0.bn1.bias", "layer4.0.bn1.weight", "layer4.0.bn1.running_mean", "layer4.0.conv2.weight", "layer4.0.bn2.running_var", "layer4.0.bn2.bias", "layer4.0.bn2.weight", "layer4.0.bn2.running_mean", "layer4.0.conv3.weight", "layer4.0.bn3.running_var", "layer4.0.bn3.bias", "layer4.0.bn3.weight", "layer4.0.bn3.running_mean", "layer4.0.downsample.0.weight", "layer4.0.downsample.1.running_var", "layer4.0.downsample.1.bias", "layer4.0.downsample.1.weight", "layer4.0.downsample.1.running_mean", "layer4.1.conv1.weight", "layer4.1.bn1.running_var", "layer4.1.bn1.bias", "layer4.1.bn1.weight", "layer4.1.bn1.running_mean", "layer4.1.conv2.weight", "layer4.1.bn2.running_var", "layer4.1.bn2.bias", "layer4.1.bn2.weight", "layer4.1.bn2.running_mean", "layer4.1.conv3.weight", "layer4.1.bn3.running_var", "layer4.1.bn3.bias", "layer4.1.bn3.weight", "layer4.1.bn3.running_mean", "layer4.2.conv1.weight", "layer4.2.bn1.running_var", "layer4.2.bn1.bias", "layer4.2.bn1.weight", "layer4.2.bn1.running_mean", "layer4.2.conv2.weight", "layer4.2.bn2.running_var", "layer4.2.bn2.bias", "layer4.2.bn2.weight", "layer4.2.bn2.running_mean", "layer4.2.conv3.weight", "layer4.2.bn3.running_var", "layer4.2.bn3.bias", "layer4.2.bn3.weight", "layer4.2.bn3.running_mean", "feature.bias", "feature.weight", "fc.bias", "fc.weight".
Unexpected key(s) in state_dict: "module.conv1.weight", "module.bn1.weight", "module.bn1.bias", "module.bn1.running_mean", "module.bn1.running_var", "module.layer1.0.conv1.weight", "module.layer1.0.bn1.weight", "module.layer1.0.bn1.bias", "module.layer1.0.bn1.running_mean", "module.layer1.0.bn1.running_var", "module.layer1.0.conv2.weight", "module.layer1.0.bn2.weight", "module.layer1.0.bn2.bias", "module.layer1.0.bn2.running_mean", "module.layer1.0.bn2.running_var", "module.layer1.0.conv3.weight", "module.layer1.0.bn3.weight", "module.layer1.0.bn3.bias", "module.layer1.0.bn3.running_mean", "module.layer1.0.bn3.running_var", "module.layer1.0.downsample.0.weight", "module.layer1.0.downsample.1.weight", "module.layer1.0.downsample.1.bias", "module.layer1.0.downsample.1.running_mean", "module.layer1.0.downsample.1.running_var", "module.layer1.1.conv1.weight", "module.layer1.1.bn1.weight", "module.layer1.1.bn1.bias", "module.layer1.1.bn1.running_mean", "module.layer1.1.bn1.running_var", "module.layer1.1.conv2.weight", "module.layer1.1.bn2.weight", "module.layer1.1.bn2.bias", "module.layer1.1.bn2.running_mean", "module.layer1.1.bn2.running_var", "module.layer1.1.conv3.weight", "module.layer1.1.bn3.weight", "module.layer1.1.bn3.bias", "module.layer1.1.bn3.running_mean", "module.layer1.1.bn3.running_var", "module.layer1.2.conv1.weight", "module.layer1.2.bn1.weight", "module.layer1.2.bn1.bias", "module.layer1.2.bn1.running_mean", "module.layer1.2.bn1.running_var", "module.layer1.2.conv2.weight", "module.layer1.2.bn2.weight", "module.layer1.2.bn2.bias", "module.layer1.2.bn2.running_mean", "module.layer1.2.bn2.running_var", "module.layer1.2.conv3.weight", "module.layer1.2.bn3.weight", "module.layer1.2.bn3.bias", "module.layer1.2.bn3.running_mean", "module.layer1.2.bn3.running_var", "module.layer2.0.conv1.weight", "module.layer2.0.bn1.weight", "module.layer2.0.bn1.bias", "module.layer2.0.bn1.running_mean", "module.layer2.0.bn1.running_var", "module.layer2.0.conv2.weight", "module.layer2.0.bn2.weight", "module.layer2.0.bn2.bias", "module.layer2.0.bn2.running_mean", "module.layer2.0.bn2.running_var", "module.layer2.0.conv3.weight", "module.layer2.0.bn3.weight", "module.layer2.0.bn3.bias", "module.layer2.0.bn3.running_mean", "module.layer2.0.bn3.running_var", "module.layer2.0.downsample.0.weight", "module.layer2.0.downsample.1.weight", "module.layer2.0.downsample.1.bias", "module.layer2.0.downsample.1.running_mean", "module.layer2.0.downsample.1.running_var", "module.layer2.1.conv1.weight", "module.layer2.1.bn1.weight", "module.layer2.1.bn1.bias", "module.layer2.1.bn1.running_mean", "module.layer2.1.bn1.running_var", "module.layer2.1.conv2.weight", "module.layer2.1.bn2.weight", "module.layer2.1.bn2.bias", "module.layer2.1.bn2.running_mean", "module.layer2.1.bn2.running_var", "module.layer2.1.conv3.weight", "module.layer2.1.bn3.weight", "module.layer2.1.bn3.bias", "module.layer2.1.bn3.running_mean", "module.layer2.1.bn3.running_var", "module.layer2.2.conv1.weight", "module.layer2.2.bn1.weight", "module.layer2.2.bn1.bias", "module.layer2.2.bn1.running_mean", "module.layer2.2.bn1.running_var", "module.layer2.2.conv2.weight", "module.layer2.2.bn2.weight", "module.layer2.2.bn2.bias", "module.layer2.2.bn2.running_mean", "module.layer2.2.bn2.running_var", "module.layer2.2.conv3.weight", "module.layer2.2.bn3.weight", "module.layer2.2.bn3.bias", "module.layer2.2.bn3.running_mean", "module.layer2.2.bn3.running_var", "module.layer2.3.conv1.weight", "module.layer2.3.bn1.weight", "module.layer2.3.bn1.bias", "module.layer2.3.bn1.running_mean", "module.layer2.3.bn1.running_var", "module.layer2.3.conv2.weight", "module.layer2.3.bn2.weight", "module.layer2.3.bn2.bias", "module.layer2.3.bn2.running_mean", "module.layer2.3.bn2.running_var", "module.layer2.3.conv3.weight", "module.layer2.3.bn3.weight", "module.layer2.3.bn3.bias", "module.layer2.3.bn3.running_mean", "module.layer2.3.bn3.running_var", "module.layer3.0.conv1.weight", "module.layer3.0.bn1.weight", "module.layer3.0.bn1.bias", "module.layer3.0.bn1.running_mean", "module.layer3.0.bn1.running_var", "module.layer3.0.conv2.weight", "module.layer3.0.bn2.weight", "module.layer3.0.bn2.bias", "module.layer3.0.bn2.running_mean", "module.layer3.0.bn2.running_var", "module.layer3.0.conv3.weight", "module.layer3.0.bn3.weight", "module.layer3.0.bn3.bias", "module.layer3.0.bn3.running_mean", "module.layer3.0.bn3.running_var", "module.layer3.0.downsample.0.weight", "module.layer3.0.downsample.1.weight", "module.layer3.0.downsample.1.bias", "module.layer3.0.downsample.1.running_mean", "module.layer3.0.downsample.1.running_var", "module.layer3.1.conv1.weight", "module.layer3.1.bn1.weight", "module.layer3.1.bn1.bias", "module.layer3.1.bn1.running_mean", "module.layer3.1.bn1.running_var", "module.layer3.1.conv2.weight", "module.layer3.1.bn2.weight", "module.layer3.1.bn2.bias", "module.layer3.1.bn2.running_mean", "module.layer3.1.bn2.running_var", "module.layer3.1.conv3.weight", "module.layer3.1.bn3.weight", "module.layer3.1.bn3.bias", "module.layer3.1.bn3.running_mean", "module.layer3.1.bn3.running_var", "module.layer3.2.conv1.weight", "module.layer3.2.bn1.weight", "module.layer3.2.bn1.bias", "module.layer3.2.bn1.running_mean", "module.layer3.2.bn1.running_var", "module.layer3.2.conv2.weight", "module.layer3.2.bn2.weight", "module.layer3.2.bn2.bias", "module.layer3.2.bn2.running_mean", "module.layer3.2.bn2.running_var", "module.layer3.2.conv3.weight", "module.layer3.2.bn3.weight", "module.layer3.2.bn3.bias", "module.layer3.2.bn3.running_mean", "module.layer3.2.bn3.running_var", "module.layer3.3.conv1.weight", "module.layer3.3.bn1.weight", "module.layer3.3.bn1.bias", "module.layer3.3.bn1.running_mean", "module.layer3.3.bn1.running_var", "module.layer3.3.conv2.weight", "module.layer3.3.bn2.weight", "module.layer3.3.bn2.bias", "module.layer3.3.bn2.running_mean", "module.layer3.3.bn2.running_var", "module.layer3.3.conv3.weight", "module.layer3.3.bn3.weight", "module.layer3.3.bn3.bias", "module.layer3.3.bn3.running_mean", "module.layer3.3.bn3.running_var", "module.layer3.4.conv1.weight", "module.layer3.4.bn1.weight", "module.layer3.4.bn1.bias", "module.layer3.4.bn1.running_mean", "module.layer3.4.bn1.running_var", "module.layer3.4.conv2.weight", "module.layer3.4.bn2.weight", "module.layer3.4.bn2.bias", "module.layer3.4.bn2.running_mean", "module.layer3.4.bn2.running_var", "module.layer3.4.conv3.weight", "module.layer3.4.bn3.weight", "module.layer3.4.bn3.bias", "module.layer3.4.bn3.running_mean", "module.layer3.4.bn3.running_var", "module.layer3.5.conv1.weight", "module.layer3.5.bn1.weight", "module.layer3.5.bn1.bias", "module.layer3.5.bn1.running_mean", "module.layer3.5.bn1.running_var", "module.layer3.5.conv2.weight", "module.layer3.5.bn2.weight", "module.layer3.5.bn2.bias", "module.layer3.5.bn2.running_mean", "module.layer3.5.bn2.running_var", "module.layer3.5.conv3.weight", "module.layer3.5.bn3.weight", "module.layer3.5.bn3.bias", "module.layer3.5.bn3.running_mean", "module.layer3.5.bn3.running_var", "module.layer4.0.conv1.weight", "module.layer4.0.bn1.weight", "module.layer4.0.bn1.bias", "module.layer4.0.bn1.running_mean", "module.layer4.0.bn1.running_var", "module.layer4.0.conv2.weight", "module.layer4.0.bn2.weight", "module.layer4.0.bn2.bias", "module.layer4.0.bn2.running_mean", "module.layer4.0.bn2.running_var", "module.layer4.0.conv3.weight", "module.layer4.0.bn3.weight", "module.layer4.0.bn3.bias", "module.layer4.0.bn3.running_mean", "module.layer4.0.bn3.running_var", "module.layer4.0.downsample.0.weight", "module.layer4.0.downsample.1.weight", "module.layer4.0.downsample.1.bias", "module.layer4.0.downsample.1.running_mean", "module.layer4.0.downsample.1.running_var", "module.layer4.1.conv1.weight", "module.layer4.1.bn1.weight", "module.layer4.1.bn1.bias", "module.layer4.1.bn1.running_mean", "module.layer4.1.bn1.running_var", "module.layer4.1.conv2.weight", "module.layer4.1.bn2.weight", "module.layer4.1.bn2.bias", "module.layer4.1.bn2.running_mean", "module.layer4.1.bn2.running_var", "module.layer4.1.conv3.weight", "module.layer4.1.bn3.weight", "module.layer4.1.bn3.bias", "module.layer4.1.bn3.running_mean", "module.layer4.1.bn3.running_var", "module.layer4.2.conv1.weight", "module.layer4.2.bn1.weight", "module.layer4.2.bn1.bias", "module.layer4.2.bn1.running_mean", "module.layer4.2.bn1.running_var", "module.layer4.2.conv2.weight", "module.layer4.2.bn2.weight", "module.layer4.2.bn2.bias", "module.layer4.2.bn2.running_mean", "module.layer4.2.bn2.running_var", "module.layer4.2.conv3.weight", "module.layer4.2.bn3.weight", "module.layer4.2.bn3.bias", "module.layer4.2.bn3.running_mean", "module.layer4.2.bn3.running_var", "module.feature.weight", "module.feature.bias", "module.fc.weight", "module.fc.bias".

Hi,

Thanks for the great work. May I ask which method do you use to get the yaw information?

The process of calculating yaw angle of a face image in radian units as following:

Could you share your codes for these processes? Thanks.

I want to use your pretrained model to test some image, but I cannot understand the class_num = 13386 in eval_cfp.py and class_num = 87020 in eval_ijba.py. I see your define of class_num in end2end/main.py but it seems the meaning of class_num is the number of person.

But I see the number of person in CFP dataset is 500 and your class_num is 13386 ? Could you please explain what class_num mean exactly ? thx @penincillin

Thank you for your code. I have some problems when I read the code:

1. I canot find frontal_list_nonli.txt and profile_list_nonli.txt which are used in eval_cfp.py. I want to know how to generate them?
2. I can get estimate_pose.txt by running. The items in estimate_pose.txt are like follows:
   ../../data/CFP/data/50_Cent/frontal/01.jpg 23.3769 1.43566 178.145
   ../../data/CFP/data/50_Cent/frontal/02.jpg -7.93511 1.74187 -4.55453
   I want to know the means of the last three values.

I have read your paper but still have some questions:

1. inference code? there are only two evaluation codes in your project
2. the input is image data and yaw value, the output is new embedding, but how to remapping it to frontal face visually.
3. Is it possible to implement in large scale, not the interesting result in your paper? only sample dataset is given
   Thank you very much, I think the module is similar as the classical STN network, Good job~

Thanks for providing aligned IJB-A datasets. However, I have several questions about the aligned images:

1. the probe_list and gallery_list in each 1:N aligned splits didn't match the search_probe.csv and search_gallery.csv in each protocol splits.
2. The 1:1 aligned files., only part images are aligned. For example, for 1:1 split1, there should be 17997 items in protocol_11/split1/verify_metadta_1.csv , but in align_image_11/ijb_a_11_align_split1.txt, only 10055 are provided.
   Hope your positive response, thanks!

this source provides some pretrained models(4), named as cfp... ijba...,
what's meaning of this name style?
Dose that mean cfp...model can only use to evaluate the CFP face data?
Or the model is trained on the CFP?

img_dir: ../../data/msceleb/image
end2end?: True
class_num: 10
Epoch: [0][0/117] Time 2.087 (2.087) Data 0.750 (0.750) Loss 2.6016 (2.6016) Prec@1 0.000 (0.000) Prec@5 50.000 (50.000)
Epoch: [0][10/117] Time 0.036 (0.226) Data 0.000 (0.068) Loss nan (nan) Prec@1 0.000 (4.545) Prec@5 0.000 (40.909)
Epoch: [0][20/117] Time 0.038 (0.136) Data 0.000 (0.037) Loss nan (nan) Prec@1 0.000 (2.381) Prec@5 0.000 (25.000)
Epoch: [0][30/117] Time 0.039 (0.104) Data 0.000 (0.025) Loss nan (nan) Prec@1 0.000 (1.613) Prec@5 0.000 (16.935)
Epoch: [0][40/117] Time 0.037 (0.088) Data 0.000 (0.019) Loss nan (nan) Prec@1 0.000 (2.439) Prec@5 0.000 (15.854)
Epoch: [0][50/117] Time 0.039 (0.078) Data 0.000 (0.016) Loss nan (nan) Prec@1 0.000 (1.961) Prec@5 0.000 (13.235)
Epoch: [0][60/117] Time 0.038 (0.071) Data 0.000 (0.013) Loss nan (nan) Prec@1 0.000 (1.639) Prec@5 0.000 (11.066)
Epoch: [0][70/117] Time 0.040 (0.067) Data 0.000 (0.012) Loss nan (nan) Prec@1 0.000 (1.408) Prec@5 0.000 (9.507)
Epoch: [0][80/117] Time 0.045 (0.063) Data 0.000 (0.010) Loss nan (nan) Prec@1 0.000 (1.543) Prec@5 0.000 (9.877)
Epoch: [0][90/117] Time 0.035 (0.061) Data 0.000 (0.009) Loss nan (nan) Prec@1 0.000 (1.374) Prec@5 0.000 (8.791)
Epoch: [0][100/117] Time 0.034 (0.058) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (1.485) Prec@5 0.000 (10.149)
Epoch: [0][110/117] Time 0.038 (0.056) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (1.351) Prec@5 0.000 (9.234)
main.py:196: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number
losses.update(loss.data[0], input.size(0))
main.py:197: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number
top1.update(prec1[0], input.size(0))
main.py:198: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number
top5.update(prec5[0], input.size(0))
Test: [0/28] Time 2.358 (2.358) Loss nan (nan) Prec@1 25.000 (25.000) Prec@5 175.000 (175.000)
Test: [10/28] Time 0.010 (0.227) Loss nan (nan) Prec@1 0.000 (6.818) Prec@5 0.000 (38.636)
Test: [20/28] Time 0.009 (0.134) Loss nan (nan) Prec@1 0.000 (3.571) Prec@5 0.000 (20.238)

• Prec@1 2.679 Prec@5 15.179
  main.py:235: UserWarning: volatile was removed and now has no effect. Use with torch.no_grad(): instead.
  input_var = torch.autograd.Variable(input, volatile=True)
  main.py:236: UserWarning: volatile was removed and now has no effect. Use with torch.no_grad(): instead.
  target_var = torch.autograd.Variable(target, volatile=True)
  main.py:246: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number
  losses.update(loss.data[0], input.size(0))
  main.py:247: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number
  top1.update(prec1[0], input.size(0))
  main.py:248: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number
  top5.update(prec5[0], input.size(0))
  Epoch: [1][0/117] Time 0.693 (0.693) Data 0.641 (0.641) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Epoch: [1][10/117] Time 0.036 (0.100) Data 0.000 (0.059) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (4.545)
  Epoch: [1][20/117] Time 0.036 (0.070) Data 0.000 (0.031) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (2.381)
  Epoch: [1][30/117] Time 0.035 (0.059) Data 0.000 (0.022) Loss nan (nan) Prec@1 0.000 (0.806) Prec@5 0.000 (4.032)
  Epoch: [1][40/117] Time 0.038 (0.054) Data 0.000 (0.017) Loss nan (nan) Prec@1 0.000 (1.220) Prec@5 0.000 (6.098)
  Epoch: [1][50/117] Time 0.039 (0.051) Data 0.000 (0.014) Loss nan (nan) Prec@1 0.000 (1.471) Prec@5 0.000 (5.882)
  Epoch: [1][60/117] Time 0.036 (0.048) Data 0.000 (0.012) Loss nan (nan) Prec@1 0.000 (1.230) Prec@5 0.000 (4.918)
  Epoch: [1][70/117] Time 0.037 (0.047) Data 0.000 (0.010) Loss nan (nan) Prec@1 0.000 (1.056) Prec@5 0.000 (4.225)
  Epoch: [1][80/117] Time 0.037 (0.046) Data 0.000 (0.009) Loss nan (nan) Prec@1 0.000 (1.235) Prec@5 0.000 (4.630)
  Epoch: [1][90/117] Time 0.038 (0.045) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (1.374) Prec@5 0.000 (5.220)
  Epoch: [1][100/117] Time 0.040 (0.044) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (1.238) Prec@5 0.000 (4.703)
  Epoch: [1][110/117] Time 0.037 (0.043) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (1.126) Prec@5 0.000 (4.279)
  Test: [0/28] Time 0.643 (0.643) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 25.000 (25.000)
  Test: [10/28] Time 0.009 (0.070) Loss nan (nan) Prec@1 0.000 (2.273) Prec@5 0.000 (13.636)
  Test: [20/28] Time 0.009 (0.041) Loss nan (nan) Prec@1 0.000 (3.571) Prec@5 0.000 (19.048)
• Prec@1 2.679 Prec@5 14.286
  Epoch: [2][0/117] Time 0.673 (0.673) Data 0.609 (0.609) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Epoch: [2][10/117] Time 0.033 (0.096) Data 0.000 (0.055) Loss nan (nan) Prec@1 25.000 (2.273) Prec@5 75.000 (11.364)
  Epoch: [2][20/117] Time 0.038 (0.068) Data 0.000 (0.029) Loss nan (nan) Prec@1 0.000 (2.381) Prec@5 0.000 (11.905)
  Epoch: [2][30/117] Time 0.034 (0.058) Data 0.000 (0.020) Loss nan (nan) Prec@1 0.000 (2.419) Prec@5 0.000 (12.097)
  Epoch: [2][40/117] Time 0.037 (0.053) Data 0.000 (0.015) Loss nan (nan) Prec@1 0.000 (1.829) Prec@5 0.000 (9.146)
  Epoch: [2][50/117] Time 0.033 (0.050) Data 0.000 (0.013) Loss nan (nan) Prec@1 0.000 (1.471) Prec@5 0.000 (8.333)
  Epoch: [2][60/117] Time 0.036 (0.048) Data 0.000 (0.011) Loss nan (nan) Prec@1 0.000 (2.049) Prec@5 0.000 (8.607)
  Epoch: [2][70/117] Time 0.033 (0.046) Data 0.000 (0.009) Loss nan (nan) Prec@1 0.000 (2.113) Prec@5 0.000 (10.211)
  Epoch: [2][80/117] Time 0.038 (0.045) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (2.160) Prec@5 0.000 (9.568)
  Epoch: [2][90/117] Time 0.037 (0.044) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (1.923) Prec@5 0.000 (8.516)
  Epoch: [2][100/117] Time 0.032 (0.043) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (1.733) Prec@5 0.000 (8.168)
  Epoch: [2][110/117] Time 0.036 (0.043) Data 0.000 (0.006) Loss nan (nan) Prec@1 0.000 (1.802) Prec@5 0.000 (8.559)
  Test: [0/28] Time 0.690 (0.690) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Test: [10/28] Time 0.008 (0.071) Loss nan (nan) Prec@1 0.000 (2.273) Prec@5 0.000 (11.364)
  Test: [20/28] Time 0.008 (0.041) Loss nan (nan) Prec@1 0.000 (3.571) Prec@5 0.000 (17.857)
• Prec@1 2.679 Prec@5 13.393
  Epoch: [3][0/117] Time 0.659 (0.659) Data 0.599 (0.599) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Epoch: [3][10/117] Time 0.036 (0.095) Data 0.000 (0.055) Loss nan (nan) Prec@1 0.000 (2.273) Prec@5 0.000 (4.545)
  Epoch: [3][20/117] Time 0.032 (0.067) Data 0.000 (0.029) Loss nan (nan) Prec@1 0.000 (1.190) Prec@5 0.000 (2.381)
  Epoch: [3][30/117] Time 0.032 (0.057) Data 0.000 (0.020) Loss nan (nan) Prec@1 0.000 (0.806) Prec@5 0.000 (1.613)
  Epoch: [3][40/117] Time 0.043 (0.053) Data 0.000 (0.015) Loss nan (nan) Prec@1 0.000 (0.610) Prec@5 0.000 (1.220)
  Epoch: [3][50/117] Time 0.038 (0.050) Data 0.000 (0.012) Loss nan (nan) Prec@1 0.000 (0.980) Prec@5 0.000 (3.431)
  Epoch: [3][60/117] Time 0.033 (0.048) Data 0.000 (0.011) Loss nan (nan) Prec@1 0.000 (0.820) Prec@5 0.000 (2.869)
  Epoch: [3][70/117] Time 0.037 (0.046) Data 0.000 (0.009) Loss nan (nan) Prec@1 0.000 (0.704) Prec@5 0.000 (2.465)
  Epoch: [3][80/117] Time 0.039 (0.045) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (0.617) Prec@5 0.000 (3.086)
  Epoch: [3][90/117] Time 0.038 (0.045) Data 0.000 (0.008) Loss nan (nan) Prec@1 25.000 (1.099) Prec@5 125.000 (4.396)
  Epoch: [3][100/117] Time 0.037 (0.044) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (1.238) Prec@5 0.000 (5.198)
  Epoch: [3][110/117] Time 0.036 (0.043) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (1.126) Prec@5 0.000 (4.730)
  Test: [0/28] Time 0.738 (0.738) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 25.000 (25.000)
  Test: [10/28] Time 0.009 (0.076) Loss nan (nan) Prec@1 0.000 (2.273) Prec@5 0.000 (13.636)
  Test: [20/28] Time 0.009 (0.044) Loss nan (nan) Prec@1 0.000 (3.571) Prec@5 0.000 (19.048)
• Prec@1 2.679 Prec@5 14.286
  Epoch: [4][0/117] Time 0.690 (0.690) Data 0.631 (0.631) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Epoch: [4][10/117] Time 0.039 (0.097) Data 0.000 (0.057) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 50.000 (4.545)
  Epoch: [4][20/117] Time 0.038 (0.068) Data 0.000 (0.031) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (2.381)
  Epoch: [4][30/117] Time 0.037 (0.058) Data 0.000 (0.021) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (1.613)
  Epoch: [4][40/117] Time 0.036 (0.053) Data 0.000 (0.017) Loss nan (nan) Prec@1 0.000 (0.610) Prec@5 0.000 (4.268)
  Epoch: [4][50/117] Time 0.038 (0.050) Data 0.000 (0.013) Loss nan (nan) Prec@1 0.000 (0.490) Prec@5 0.000 (3.431)
  Epoch: [4][60/117] Time 0.039 (0.048) Data 0.000 (0.012) Loss nan (nan) Prec@1 0.000 (0.410) Prec@5 25.000 (3.279)
  Epoch: [4][70/117] Time 0.038 (0.046) Data 0.000 (0.010) Loss nan (nan) Prec@1 0.000 (0.704) Prec@5 0.000 (4.577)
  Epoch: [4][80/117] Time 0.038 (0.045) Data 0.000 (0.009) Loss nan (nan) Prec@1 0.000 (0.617) Prec@5 0.000 (4.012)
  Epoch: [4][90/117] Time 0.038 (0.044) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (0.824) Prec@5 0.000 (4.945)
  Epoch: [4][100/117] Time 0.038 (0.044) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (0.990) Prec@5 0.000 (4.950)
  Epoch: [4][110/117] Time 0.038 (0.043) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (1.126) Prec@5 0.000 (5.631)
  Test: [0/28] Time 0.733 (0.733) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Test: [10/28] Time 0.009 (0.075) Loss nan (nan) Prec@1 0.000 (2.273) Prec@5 0.000 (11.364)
  Test: [20/28] Time 0.009 (0.044) Loss nan (nan) Prec@1 0.000 (3.571) Prec@5 0.000 (17.857)
• Prec@1 2.679 Prec@5 13.393
  Epoch: [5][0/117] Time 0.633 (0.633) Data 0.579 (0.579) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Epoch: [5][10/117] Time 0.034 (0.091) Data 0.000 (0.053) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Epoch: [5][20/117] Time 0.038 (0.065) Data 0.000 (0.028) Loss nan (nan) Prec@1 0.000 (1.190) Prec@5 0.000 (5.952)
  Epoch: [5][30/117] Time 0.039 (0.056) Data 0.000 (0.020) Loss nan (nan) Prec@1 0.000 (0.806) Prec@5 25.000 (4.839)
  Epoch: [5][40/117] Time 0.037 (0.052) Data 0.000 (0.015) Loss nan (nan) Prec@1 0.000 (1.220) Prec@5 0.000 (9.146)
  Epoch: [5][50/117] Time 0.038 (0.049) Data 0.000 (0.012) Loss nan (nan) Prec@1 0.000 (0.980) Prec@5 0.000 (7.353)
  Epoch: [5][60/117] Time 0.035 (0.047) Data 0.000 (0.010) Loss nan (nan) Prec@1 0.000 (1.230) Prec@5 0.000 (8.197)
  Epoch: [5][70/117] Time 0.037 (0.046) Data 0.000 (0.009) Loss nan (nan) Prec@1 0.000 (1.056) Prec@5 0.000 (7.042)
  Epoch: [5][80/117] Time 0.036 (0.045) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (0.926) Prec@5 0.000 (6.481)
  Epoch: [5][90/117] Time 0.040 (0.044) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (0.824) Prec@5 0.000 (6.044)
  Epoch: [5][100/117] Time 0.034 (0.043) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (0.990) Prec@5 0.000 (6.188)
  Epoch: [5][110/117] Time 0.043 (0.043) Data 0.000 (0.006) Loss nan (nan) Prec@1 0.000 (0.901) Prec@5 0.000 (5.856)
  Test: [0/28] Time 0.708 (0.708) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 25.000 (25.000)
  Test: [10/28] Time 0.009 (0.073) Loss nan (nan) Prec@1 0.000 (2.273) Prec@5 0.000 (13.636)
  Test: [20/28] Time 0.009 (0.043) Loss nan (nan) Prec@1 0.000 (3.571) Prec@5 0.000 (19.048)
• Prec@1 2.679 Prec@5 14.286
  Epoch: [6][0/117] Time 0.623 (0.623) Data 0.566 (0.566) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Epoch: [6][10/117] Time 0.037 (0.090) Data 0.000 (0.052) Loss nan (nan) Prec@1 0.000 (2.273) Prec@5 0.000 (15.909)
  Epoch: [6][20/117] Time 0.037 (0.065) Data 0.000 (0.028) Loss nan (nan) Prec@1 0.000 (2.381) Prec@5 0.000 (14.286)
  Epoch: [6][30/117] Time 0.035 (0.056) Data 0.000 (0.019) Loss nan (nan) Prec@1 0.000 (1.613) Prec@5 0.000 (11.290)
  Epoch: [6][40/117] Time 0.036 (0.051) Data 0.000 (0.014) Loss nan (nan) Prec@1 0.000 (1.220) Prec@5 0.000 (8.537)
  Epoch: [6][50/117] Time 0.035 (0.049) Data 0.000 (0.012) Loss nan (nan) Prec@1 0.000 (0.980) Prec@5 0.000 (8.333)
  Epoch: [6][60/117] Time 0.038 (0.047) Data 0.000 (0.010) Loss nan (nan) Prec@1 0.000 (1.230) Prec@5 0.000 (9.836)
  Epoch: [6][70/117] Time 0.038 (0.045) Data 0.000 (0.009) Loss nan (nan) Prec@1 0.000 (1.408) Prec@5 0.000 (9.507)
  Epoch: [6][80/117] Time 0.037 (0.044) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (1.543) Prec@5 75.000 (9.877)
  Epoch: [6][90/117] Time 0.036 (0.044) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (1.923) Prec@5 0.000 (11.538)
  Epoch: [6][100/117] Time 0.036 (0.043) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (1.733) Prec@5 0.000 (10.396)
  Epoch: [6][110/117] Time 0.040 (0.042) Data 0.000 (0.006) Loss nan (nan) Prec@1 0.000 (1.577) Prec@5 0.000 (9.459)
  Test: [0/28] Time 0.696 (0.696) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Test: [10/28] Time 0.009 (0.072) Loss nan (nan) Prec@1 0.000 (2.273) Prec@5 0.000 (11.364)
  Test: [20/28] Time 0.009 (0.042) Loss nan (nan) Prec@1 0.000 (3.571) Prec@5 0.000 (17.857)
• Prec@1 2.679 Prec@5 13.393
  Epoch: [7][0/117] Time 0.665 (0.665) Data 0.609 (0.609) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Epoch: [7][10/117] Time 0.037 (0.095) Data 0.000 (0.055) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 25.000 (6.818)
  Epoch: [7][20/117] Time 0.039 (0.068) Data 0.000 (0.029) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (3.571)
  Epoch: [7][30/117] Time 0.033 (0.058) Data 0.000 (0.020) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (2.419)
  Epoch: [7][40/117] Time 0.037 (0.053) Data 0.000 (0.016) Loss nan (nan) Prec@1 0.000 (0.610) Prec@5 0.000 (3.659)
  Epoch: [7][50/117] Time 0.038 (0.050) Data 0.000 (0.013) Loss nan (nan) Prec@1 0.000 (0.490) Prec@5 0.000 (2.941)
  Epoch: [7][60/117] Time 0.042 (0.048) Data 0.000 (0.011) Loss nan (nan) Prec@1 0.000 (0.410) Prec@5 0.000 (3.279)
  Epoch: [7][70/117] Time 0.043 (0.046) Data 0.000 (0.010) Loss nan (nan) Prec@1 0.000 (0.352) Prec@5 0.000 (2.817)
  Epoch: [7][80/117] Time 0.035 (0.045) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (0.309) Prec@5 0.000 (2.469)
  Epoch: [7][90/117] Time 0.037 (0.044) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (0.275) Prec@5 0.000 (2.747)
  Epoch: [7][100/117] Time 0.034 (0.044) Data 0.000 (0.007) Loss nan (nan) Prec@1 25.000 (0.495) Prec@5 100.000 (3.465)
  Epoch: [7][110/117] Time 0.033 (0.043) Data 0.000 (0.006) Loss nan (nan) Prec@1 0.000 (0.450) Prec@5 0.000 (3.153)
  Test: [0/28] Time 0.727 (0.727) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 25.000 (25.000)
  Test: [10/28] Time 0.009 (0.076) Loss nan (nan) Prec@1 0.000 (2.273) Prec@5 0.000 (13.636)
  Test: [20/28] Time 0.009 (0.044) Loss nan (nan) Prec@1 0.000 (3.571) Prec@5 0.000 (19.048)
• Prec@1 2.679 Prec@5 14.286
  Epoch: [8][0/117] Time 0.689 (0.689) Data 0.637 (0.637) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Epoch: [8][10/117] Time 0.031 (0.096) Data 0.000 (0.058) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (4.545)
  Epoch: [8][20/117] Time 0.036 (0.068) Data 0.000 (0.031) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (2.381)
  Epoch: [8][30/117] Time 0.037 (0.058) Data 0.000 (0.021) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (1.613)
  Epoch: [8][40/117] Time 0.038 (0.053) Data 0.000 (0.017) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (1.220)
  Epoch: [8][50/117] Time 0.047 (0.050) Data 0.000 (0.014) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.980)
  Epoch: [8][60/117] Time 0.035 (0.048) Data 0.000 (0.011) Loss nan (nan) Prec@1 0.000 (0.820) Prec@5 0.000 (4.918)
  Epoch: [8][70/117] Time 0.037 (0.046) Data 0.000 (0.010) Loss nan (nan) Prec@1 0.000 (1.056) Prec@5 0.000 (5.986)
  Epoch: [8][80/117] Time 0.031 (0.045) Data 0.000 (0.009) Loss nan (nan) Prec@1 0.000 (0.926) Prec@5 0.000 (5.247)
  Epoch: [8][90/117] Time 0.039 (0.045) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (1.099) Prec@5 0.000 (5.769)
  Epoch: [8][100/117] Time 0.039 (0.044) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (0.990) Prec@5 0.000 (5.198)
  Epoch: [8][110/117] Time 0.038 (0.043) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (1.126) Prec@5 0.000 (5.405)
  Test: [0/28] Time 0.755 (0.755) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Test: [10/28] Time 0.008 (0.077) Loss nan (nan) Prec@1 0.000 (2.273) Prec@5 0.000 (11.364)
  Test: [20/28] Time 0.009 (0.044) Loss nan (nan) Prec@1 0.000 (3.571) Prec@5 0.000 (17.857)
• Prec@1 2.679 Prec@5 13.393
  Epoch: [9][0/117] Time 0.633 (0.633) Data 0.579 (0.579) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Epoch: [9][10/117] Time 0.038 (0.091) Data 0.000 (0.053) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Epoch: [9][20/117] Time 0.036 (0.066) Data 0.000 (0.028) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Epoch: [9][30/117] Time 0.036 (0.057) Data 0.000 (0.020) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Epoch: [9][40/117] Time 0.036 (0.052) Data 0.000 (0.015) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.610)
  Epoch: [9][50/117] Time 0.038 (0.049) Data 0.000 (0.012) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.490)
  Epoch: [9][60/117] Time 0.045 (0.047) Data 0.000 (0.010) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.410)
  Epoch: [9][70/117] Time 0.038 (0.046) Data 0.000 (0.009) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.352)
  Epoch: [9][80/117] Time 6.902 (0.130) Data 6.841 (0.093) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.617)
  Epoch: [9][90/117] Time 0.041 (0.120) Data 0.000 (0.082) Loss nan (nan) Prec@1 0.000 (0.275) Prec@5 0.000 (1.099)
  Epoch: [9][100/117] Time 0.040 (0.112) Data 0.000 (0.074) Loss nan (nan) Prec@1 0.000 (0.248) Prec@5 50.000 (1.485)
  Epoch: [9][110/117] Time 0.038 (0.105) Data 0.000 (0.068) Loss nan (nan) Prec@1 0.000 (0.450) Prec@5 0.000 (2.928)
  Test: [0/28] Time 0.681 (0.681) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 25.000 (25.000)
  Test: [10/28] Time 0.009 (0.072) Loss nan (nan) Prec@1 0.000 (2.273) Prec@5 0.000 (13.636)
  Test: [20/28] Time 0.009 (0.042) Loss nan (nan) Prec@1 0.000 (3.571) Prec@5 0.000 (19.048)
• Prec@1 2.679 Prec@5 14.286
  Epoch: [10][0/117] Time 0.653 (0.653) Data 0.597 (0.597) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Epoch: [10][10/117] Time 0.037 (0.093) Data 0.000 (0.054) Loss nan (nan) Prec@1 0.000 (2.273) Prec@5 0.000 (20.455)
  Epoch: [10][20/117] Time 0.035 (0.066) Data 0.000 (0.029) Loss nan (nan) Prec@1 0.000 (3.571) Prec@5 0.000 (22.619)
  Epoch: [10][30/117] Time 0.038 (0.057) Data 0.000 (0.020) Loss nan (nan) Prec@1 0.000 (2.419) Prec@5 0.000 (15.323)
  Epoch: [10][40/117] Time 0.038 (0.052) Data 0.000 (0.016) Loss nan (nan) Prec@1 0.000 (2.439) Prec@5 0.000 (14.024)
  Epoch: [10][50/117] Time 0.035 (0.049) Data 0.000 (0.013) Loss nan (nan) Prec@1 0.000 (1.961) Prec@5 0.000 (12.255)
  Epoch: [10][60/117] Time 0.035 (0.047) Data 0.000 (0.011) Loss nan (nan) Prec@1 0.000 (2.459) Prec@5 0.000 (13.934)
  Epoch: [10][70/117] Time 0.036 (0.046) Data 0.000 (0.010) Loss nan (nan) Prec@1 0.000 (2.113) Prec@5 0.000 (11.972)
  Epoch: [10][80/117] Time 0.043 (0.045) Data 0.000 (0.009) Loss nan (nan) Prec@1 0.000 (1.852) Prec@5 0.000 (10.494)
  Epoch: [10][90/117] Time 0.038 (0.044) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (1.923) Prec@5 0.000 (10.714)
  Epoch: [10][100/117] Time 0.038 (0.044) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (1.733) Prec@5 0.000 (9.653)
  Epoch: [10][110/117] Time 0.038 (0.043) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (1.577) Prec@5 0.000 (9.009)
  Test: [0/28] Time 0.681 (0.681) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Test: [10/28] Time 0.009 (0.071) Loss nan (nan) Prec@1 0.000 (2.273) Prec@5 0.000 (11.364)
  Test: [20/28] Time 0.008 (0.041) Loss nan (nan) Prec@1 0.000 (3.571) Prec@5 0.000 (17.857)
• Prec@1 2.679 Prec@5 13.393
  Epoch: [11][0/117] Time 0.660 (0.660) Data 0.595 (0.595) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Epoch: [11][10/117] Time 0.039 (0.096) Data 0.000 (0.054) Loss nan (nan) Prec@1 0.000 (2.273) Prec@5 0.000 (13.636)
  Epoch: [11][20/117] Time 0.038 (0.068) Data 0.000 (0.029) Loss nan (nan) Prec@1 0.000 (1.190) Prec@5 0.000 (7.143)
  Epoch: [11][30/117] Time 0.037 (0.058) Data 0.000 (0.020) Loss nan (nan) Prec@1 75.000 (3.226) Prec@5 200.000 (11.290)
  Epoch: [11][40/117] Time 0.036 (0.053) Data 0.000 (0.016) Loss nan (nan) Prec@1 0.000 (3.049) Prec@5 0.000 (11.585)
  Epoch: [11][50/117] Time 0.038 (0.050) Data 0.000 (0.013) Loss nan (nan) Prec@1 0.000 (2.941) Prec@5 0.000 (10.784)
  Epoch: [11][60/117] Time 0.049 (0.049) Data 0.000 (0.011) Loss nan (nan) Prec@1 0.000 (2.459) Prec@5 0.000 (9.016)
  Epoch: [11][70/117] Time 0.038 (0.047) Data 0.000 (0.010) Loss nan (nan) Prec@1 0.000 (2.113) Prec@5 0.000 (7.746)
  Epoch: [11][80/117] Time 0.043 (0.046) Data 0.000 (0.009) Loss nan (nan) Prec@1 0.000 (1.852) Prec@5 0.000 (7.716)
  Epoch: [11][90/117] Time 0.038 (0.045) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (1.923) Prec@5 0.000 (7.692)
  Epoch: [11][100/117] Time 0.038 (0.044) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (1.980) Prec@5 0.000 (7.426)
  Epoch: [11][110/117] Time 0.039 (0.044) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (2.027) Prec@5 0.000 (7.432)
  Test: [0/28] Time 0.634 (0.634) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 25.000 (25.000)
  Test: [10/28] Time 0.009 (0.069) Loss nan (nan) Prec@1 0.000 (2.273) Prec@5 0.000 (13.636)
  Test: [20/28] Time 0.009 (0.040) Loss nan (nan) Prec@1 0.000 (3.571) Prec@5 0.000 (19.048)
• Prec@1 2.679 Prec@5 14.286
  Epoch: [12][0/117] Time 0.675 (0.675) Data 0.612 (0.612) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Epoch: [12][10/117] Time 0.044 (0.095) Data 0.000 (0.056) Loss nan (nan) Prec@1 25.000 (4.545) Prec@5 100.000 (15.909)
  Epoch: [12][20/117] Time 0.035 (0.068) Data 0.000 (0.029) Loss nan (nan) Prec@1 0.000 (2.381) Prec@5 0.000 (8.333)
  Epoch: [12][30/117] Time 0.035 (0.058) Data 0.000 (0.020) Loss nan (nan) Prec@1 0.000 (1.613) Prec@5 0.000 (5.645)
  Epoch: [12][40/117] Time 0.036 (0.053) Data 0.000 (0.016) Loss nan (nan) Prec@1 0.000 (1.220) Prec@5 25.000 (4.878)
  Epoch: [12][50/117] Time 0.038 (0.050) Data 0.000 (0.013) Loss nan (nan) Prec@1 0.000 (0.980) Prec@5 0.000 (3.922)
  Epoch: [12][60/117] Time 0.040 (0.048) Data 0.000 (0.011) Loss nan (nan) Prec@1 0.000 (0.820) Prec@5 0.000 (3.279)
  Epoch: [12][70/117] Time 0.038 (0.047) Data 0.000 (0.009) Loss nan (nan) Prec@1 0.000 (0.704) Prec@5 0.000 (2.817)
  Epoch: [12][80/117] Time 0.034 (0.045) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (0.926) Prec@5 0.000 (3.704)
  Epoch: [12][90/117] Time 0.038 (0.045) Data 0.000 (0.008) Loss nan (nan) Prec@1 0.000 (0.824) Prec@5 0.000 (3.297)
  Epoch: [12][100/117] Time 0.038 (0.044) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (0.990) Prec@5 0.000 (3.713)
  Epoch: [12][110/117] Time 0.038 (0.043) Data 0.000 (0.007) Loss nan (nan) Prec@1 0.000 (0.901) Prec@5 0.000 (3.378)
  Test: [0/28] Time 0.648 (0.648) Loss nan (nan) Prec@1 0.000 (0.000) Prec@5 0.000 (0.000)
  Test: [10/28] Time 0.009 (0.069) Loss nan (nan) Prec@1 0.000 (2.273) Prec@5 0.000 (11.364)
  Test: [20/28] Time 0.009 (0.040) Loss nan (nan) Prec@1 0.000 (3.571) Prec@5 0.000 (17.857)
• Prec@1 2.679 Prec@5 13.393

Are the pretrained models: cfp_res50_end2end and ijba_res18_end2end- just trained end2end or is the DREAM block retrained after end2end training?

@penincillin
I noticed that in your source code, face recognition is treated as plain image classification task, without applying some latest loss function specifically designed for face recognition. For example, cosface loss or arcloss. Have you tried that?

(In latest face recognition loss function, features and weights of last fc both need to be normalized, is it because normalized feature will hurt the performance in your pose robust algorithm?)

Thank you : )

I want to test my image，But no alignment code was found。

Hi, after review your code, I find yaw information is very important. And it is one of the branch of DREAM block, ringht？ So where is the code generate the yaw information in train/val label data, how can I get the yaw information on my own dataset? Thanks

Thanks for you answers all the time. My another question is:
If we have gettd the 2D facial landmarks coordinates, how to get the 3D facial landmarks coordinate?
Could you give me some references for detials? Or some codes? Thanks again

Hi,

Thanks for this great work. I was looking at your stitching based training method and notice how you are building the inputs and targets.

If I understood correctly, you are considering images with less or equal to 20 degree yaw to be frontal images. These images or rather the mean of the feature vectors of these images become your target/label.

You consider the images with greater than 50 degree yaw to be the profile images and they become your inputs to the DREAM block along with the yaw.

Is there is any specific reason you are ignoring the images that are between 21 and 49 degrees ?. They are neither considered as inputs nor as labels/targets.

Regards & thanks
Kapil

At [https://github.com/penincillin/DREAM/blob/7bea01e5b108546a2b3a809ad17cf478607c9db5/src/stitching/branch_train.py#L52]

when i run this code, i get this error:

Traceback (most recent call last):
File "/media/lab210/Work/WH_HOME/SubjectDownload/Project/Face-reid/[ DREAM-Face ]/DREAM/src/stitching/branch_train.py", line 134, in
main()
File "/media/lab210/Work/WH_HOME/SubjectDownload/Project/Face-reid/[ DREAM-Face ]/DREAM/src/stitching/branch_train.py", line 52, in main
losses.update(loss.data[0], loss.size(0))
RuntimeError: dimension specified as 0 but tensor has no dimensions

when i replace 'loss' to 'loss.size(0)'. the code can run. is it a bug when you design your code?

hello， I'm a little confused. What is the main role of face alignment before training？

Could you share your method to demonstrate the visualization of deep features?
It's very interesting. Thanks.

Could you Share the pictures in stitching.zip?? Thanks!

Thank you for sharing code.
Q1: , how to extract feature using the released 178x218 cropped face? That is, how to transfer 178x218 to 224x224?
Thanks.

Hi Penincillin:
According to your process, the following problems occur:
(pytorch) shaojie@shaojie-System-Product-Name:/media/shaojie/Office/DREAM/src/stitching$ sh train_stitch.sh
mkdir: 无法创建目录"log": 文件已存在
THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1524586445097/work/aten/src/THC/THCTensorRandom.cu line=25 error=2 : out of memory
Traceback (most recent call last):
File "branch_train.py", line 134, in
main()
File "branch_train.py", line 37, in main
model.cuda()
File "/home/shaojie/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 249, in cuda
return self._apply(lambda t: t.cuda(device))
File "/home/shaojie/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 176, in _apply
module._apply(fn)
File "/home/shaojie/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 182, in _apply
param.data = fn(param.data)
File "/home/shaojie/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 249, in
return self._apply(lambda t: t.cuda(device))
RuntimeError: cuda runtime error (2) : out of memory at /opt/conda/conda-bld/pytorch_1524586445097/work/aten/src/THC/THCTensorRandom.cu:25

I try to change batch size=16,but it's error.can you solve this problem?

python3 branch_train.py error:RuntimeError: dimension specified as 0 but tensor has no dimensions

hello
I have download the pretrained model from baiduyun，there are four moedls ,cfp_res50_end2end.pth cfp_res50_naive.pth ijba_res18_end2end.pth ijba_res18_naive.pth,
I cant find the face recognition model you mentioned ，could you release the model ？
thank you verymuch

where is the stitching.zip ？how can i get it？

In your paper , the ResNet18 training in Naive method with MS-Celeb-1M achieve 8.40.
But when test CFP dataset with your released model, The result is 4.85.

So there is something wrong when I understand your paper?

Will you publicate your training dataset? and in this training dataset already remove the overlaps between IJB-A? Thanks

How to get the pretrained DREAM block to apply to our recognition model after training? Can we apply this after getting the embeddings from our model i.e second network which get inputs from the output of the original recognition model?

I installed python3.5 in my anaconda3 and install opencv2.4.13 in my computer successfully. But when I import cv2 in env python3.5, it returns

Traceback (most recent call last):
File "", line 1, in
ImportError: dynamic module does not define module export function (PyInit_cv2)

But I can use python2.7 import cv2.

What can I do to solve this problem? Or can I use opencv3 to run this model?

However, I didn't find this binary file in ‘’src/preprocess''.
If anyone find the binary file for calculating the yaw. Please help.

the value bigger than 0.9 in train_label.txt is not right , their images is almost front ! for example Lepa_Brena/1_0.jpg .

How to add the residuals and use yaw angle estiamator for our already trained model? In our model, like you said, it finds the near frontal images very well but for the profile images it is not good. Therefore I want to use only the pretrained yaw angle estimator and pretrained residual block? Can we get the 2 pretrained models?

Hello.
You wrote: "In this paper, we use three face datasets. We train base model and DREAM block on MS-Celeb-1M. We offer a subset of Ms-Celeb-1M with 300 celebrities, you could download from the following link Ms-Celeb-1M Subset (msceleb.zip) https://drive.google.com/file/d/1om0pbwBX4RZHVuI3QXVrBj9mLtOK2PV8/view?usp=sharing"

But I see 10 folders in msceleb.zip.