NotFoundError (see above for traceback): Key generator/decoder_5/deconv/filter not found in checkpoint.

I found that there are many difference about name scope and network between reduce_mode.py and pix2pix.py, so I am wondering how do you guys work it out...

The training went well. I am using Python 3.6 and TF 1.7.0. However when reducing the model I get the following:

2018-04-25 19:31:50.717783: W T:\src\github\tensorflow\tensorflow\core\framework\op_kernel.cc:1273] OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Not found: Key generator/decoder_1/deconv/filter not found in checkpoint
Traceback (most recent call last):
File "D:\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1327, in _do_call
return fn(*args)
File "D:\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1312, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "D:\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1420, in _call_tf_sessionrun
status, run_metadata)
File "D:\Python36\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 516, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: Key generator/decoder_1/deconv/filter not found in checkpoint
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
[[Node: save/RestoreV2/_79 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_84_save/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "reduce_model.py", line 215, in
saver.restore(sess, checkpoint)
File "D:\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1775, in restore
{self.saver_def.filename_tensor_name: save_path})
File "D:\Python36\lib\site-packages\tensorflow\python\client\session.py", line 905, in run
run_metadata_ptr)
File "D:\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1140, in _run
feed_dict_tensor, options, run_metadata)
File "D:\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1321, in _do_run
run_metadata)
File "D:\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1340, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Key generator/decoder_1/deconv/filter not found in checkpoint
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
[[Node: save/RestoreV2/_79 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_84_save/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

Caused by op 'save/RestoreV2', defined at:
File "reduce_model.py", line 213, in
saver = tf.train.Saver()
File "D:\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1311, in init
self.build()
File "D:\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1320, in build
self._build(self._filename, build_save=True, build_restore=True)
File "D:\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1357, in _build
build_save=build_save, build_restore=build_restore)
File "D:\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 809, in _build_internal
restore_sequentially, reshape)
File "D:\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 448, in _AddRestoreOps
restore_sequentially)
File "D:\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 860, in bulk_restore
return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
File "D:\Python36\lib\site-packages\tensorflow\python\ops\gen_io_ops.py", line 1541, in restore_v2
shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
File "D:\Python36\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "D:\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3290, in create_op
op_def=op_def)
File "D:\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 1654, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

NotFoundError (see above for traceback): Key generator/decoder_1/deconv/filter not found in checkpoint
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
[[Node: save/RestoreV2/_79 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_84_save/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

Well, I execute the command to Train the model on the data.

Train the model on the data

python pix2pix.py
--mode train
--output_dir face2face-model
--max_epochs 200
--input_dir photos/combined/train
--which_direction AtoB

It are running now, since 30 min ... is normal?

Thanks for your awesome work! :)
Just looking for some direction on how to scale detected face in input Webcam feed to match size of face in output video?
Based on my tests, input face ideally needs to match size of output face. This tends to generate the best results.
Also, I think a feature where frames are skipped when input face is undetected could help display better output results - Direction on how to implement this would be greatly appreciated.
Thanks in advance to those who might have time to help out! :)

Can you share the video and training images?

While trying to train i get the following error:
python face2face-demo/generate_train_data.py --file angela_merkel_speech.mp4 --num 400 --landmark-model shape_predictor_68_face_landmarks.dat

Traceback (most recent call last):
File "face2face-demo/generate_train_data.py", line 96, in
predictor = dlib.shape_predictor(args.face_landmark_shape_file)
RuntimeError: Unable to open shape_predictor_68_face_landmarks.dat

So i skipped training and used your validated model and got the following error:
python face2face-demo/reduce_model.py --model-input Desktop/combined/val --model-output Desktop/combined
2018-01-08 13:50:50.597204: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Traceback (most recent call last):
File "face2face-demo/reduce_model.py", line 215, in
saver.restore(sess, checkpoint)
File "/home/oem/.virtualenvs/facecourse-py3/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1662, in restore
raise ValueError("Can't load save_path when it is None.")
ValueError: Can't load save_path when it is None.

how can i do?

Following the "python generate_train_data.py --file angela_merkel_speech.mp4 --num 40 --landmark-model shape_predictor_68_face_landmarks.dat",
just show:

[INFO] elapsed time (total): 0.00
[INFO] approx. FPS: 0.00

The two folders are created but empty.

Someone knows why?

Thanks,

Best wishes.

Ben

When I run "generate_train_data.py", it would failed with error:

OpenCV Error: Assertion failed (ssize.area() > 0) in resize, file /Users/jenkins/miniconda/1/x64/conda-bld/conda_1486587097465/work/opencv-3.1.0/modules/imgproc/src/imgwarp.cpp, line 3229
Traceback (most recent call last):
  File "generate_train_data.py", line 98, in <module>
    main()
  File "generate_train_data.py", line 27, in main
    frame_resize = cv2.resize(frame, None, fx=1 / DOWNSAMPLE_RATIO, fy=1 / DOWNSAMPLE_RATIO)
cv2.error: /Users/jenkins/miniconda/1/x64/conda-bld/conda_1486587097465/work/opencv-3.1.0/modules/imgproc/src/imgwarp.cpp:3229: error: (-215) ssize.area() > 0 in function resize

In the environment.yml I had to change tbb version to "menpo::tbb=4.4" and on Ubuntu16.04 with miniconda the env seemed to be created. I then activated face2face-demo and tried running Generate Training Data script but it returned:

(face2face-demo) myname@mypc:~/face2face-demo$ python generate_train_data.py --file angela_merkel_speech.mp4 --num 400 --landmark-model shape_predictor_68_face_landmarks.dat
[INFO] elapsed time (total): 0.00
[INFO] approx. FPS: 0.00

The folders were created with no files?!

***Update: reformatted computer and the same issue: could it be Ubuntu requires additional libraries when building env?

Terminal Output:
m@my:~/github/face2face-demo$ conda env create -f environment.yml
Fetching package metadata .............
Solving package specifications: .
bzip2-1.0.6-3. 100% |################################| Time: 0:00:00 104.79 kB/s
jpeg-9b-0.tar. 100% |################################| Time: 0:00:01 650.52 kB/s
mkl-2017.0.3-0 100% |################################| Time: 0:00:19 6.94 MB/s
openssl-1.0.2l 100% |################################| Time: 0:00:00 7.76 MB/s
readline-6.2-2 100% |################################| Time: 0:00:00 7.55 MB/s
sqlite-3.13.0- 100% |################################| Time: 0:00:00 7.63 MB/s
tbb-4.4_201601 100% |################################| Time: 0:00:00 2.06 MB/s
tk-8.5.18-0.ta 100% |################################| Time: 0:00:00 7.47 MB/s
xz-5.2.2-1.tar 100% |################################| Time: 0:00:00 6.42 MB/s
zlib-1.2.8-3.t 100% |################################| Time: 0:00:00 9.92 MB/s
libpng-1.6.27- 100% |################################| Time: 0:00:00 8.50 MB/s
python-3.5.3-1 100% |################################| Time: 0:00:02 7.78 MB/s
boost-1.59.0-p 100% |################################| Time: 0:00:03 3.40 MB/s
numpy-1.13.0-p 100% |################################| Time: 0:00:00 7.58 MB/s
setuptools-27. 100% |################################| Time: 0:00:00 7.56 MB/s
wheel-0.29.0-p 100% |################################| Time: 0:00:00 11.11 MB/s
dlib-19.4-py35 100% |################################| Time: 0:00:00 5.08 MB/s
opencv3-3.0.0- 100% |################################| Time: 0:00:03 7.77 MB/s
pip-9.0.1-py35 100% |################################| Time: 0:00:00 7.81 MB/s
Collecting backports.weakref==1.0rc1
Downloading backports.weakref-1.0rc1-py3-none-any.whl
Collecting bleach==1.5.0
Downloading bleach-1.5.0-py2.py3-none-any.whl
Collecting html5lib==0.9999999
Downloading html5lib-0.9999999.tar.gz (889kB)
100% |████████████████████████████████| 890kB 596kB/s
Collecting imutils==0.4.3
Downloading imutils-0.4.3.tar.gz
Collecting markdown==2.6.8
Downloading Markdown-2.6.8.tar.gz (307kB)
100% |████████████████████████████████| 317kB 641kB/s
Collecting protobuf==3.3.0
Downloading protobuf-3.3.0-cp35-cp35m-manylinux1_x86_64.whl (5.7MB)
100% |████████████████████████████████| 5.7MB 242kB/s
Collecting six==1.10.0
Downloading six-1.10.0-py2.py3-none-any.whl
Collecting tensorflow==1.2.1
Downloading tensorflow-1.2.1-cp35-cp35m-manylinux1_x86_64.whl (34.5MB)
100% |████████████████████████████████| 34.5MB 48kB/s
Collecting werkzeug==0.12.2
Downloading Werkzeug-0.12.2-py2.py3-none-any.whl (312kB)
100% |████████████████████████████████| 317kB 2.5MB/s
Requirement already satisfied: setuptools in /home/m/miniconda3/envs/face2face-demo/lib/python3.5/site-packages/setuptools-27.2.0-py3.5.egg (from protobuf==3.3.0)
Requirement already satisfied: wheel>=0.26 in /home/m/miniconda3/envs/face2face-demo/lib/python3.5/site-packages (from tensorflow==1.2.1)
Requirement already satisfied: numpy>=1.11.0 in /home/m/miniconda3/envs/face2face-demo/lib/python3.5/site-packages (from tensorflow==1.2.1)
Building wheels for collected packages: html5lib, imutils, markdown
Running setup.py bdist_wheel for html5lib ... done
Stored in directory: /home/m/.cache/pip/wheels/6f/85/6c/56b8e1292c6214c4eb73b9dda50f53e8e977bf65989373c962
Running setup.py bdist_wheel for imutils ... done
Stored in directory: /home/m/.cache/pip/wheels/4b/5c/b9/5fb892e82d1f10678bd1a3f2555b6a9777e238b7a852e6df27
Running setup.py bdist_wheel for markdown ... done
Stored in directory: /home/m/.cache/pip/wheels/85/a7/08/33ee5cd488d0365d8bed79d1d4e5c28dd3fbfc7f6d0ad4bb09
Successfully built html5lib imutils markdown
Installing collected packages: backports.weakref, six, html5lib, bleach, imutils, markdown, protobuf, werkzeug, tensorflow
Successfully installed backports.weakref-1.0rc1 bleach-1.5.0 html5lib-0.9999999 imutils-0.4.3 markdown-2.6.8 protobuf-3.3.0 six-1.10.0 tensorflow-1.2.1 werkzeug-0.12.2

To activate this environment, use:

> source activate face2face-demo

To deactivate this environment, use:

> source deactivate face2face-demo

m@my:/github/face2face-demo$ ^C
m@my:/github/face2face-demo$ source activate face2face-demo

(face2face-demo) m@my:/github/face2face-demo$
(face2face-demo) m@my:/github/face2face-demo$ python generate_train_data.py --file angela_merkel_speech.mp4 --num 400 --landmark-model shape_predictor_68_face_landmarks.dat
Traceback (most recent call last):
File "generate_train_data.py", line 96, in
predictor = dlib.shape_predictor(args.face_landmark_shape_file)
RuntimeError: Unable to open shape_predictor_68_face_landmarks.dat
(face2face-demo) m@my:/github/face2face-demo$ python generate_train_data.py --file angela_merkel_speech.mp4 --num 400 --landmark-model shape_predictor_68_face_landmarks.dat
[INFO] elapsed time (total): 0.00
[INFO] approx. FPS: 0.00
(face2face-demo) m@my:/github/face2face-demo$ python generate_train_data.py --file angela_merkel_speech.mp4 --num 400 --landmark-model shape_predictor_68_face_landmarks.dat
[INFO] elapsed time (total): 0.00
[INFO] approx. FPS: 0.00
(face2face-demo) m@my:~/github/face2face-demo$

Hi. I cloned this repo, then created the conda environment from file (Mac OS) and dowloaded the dataSet (angela_merkel_speech.mp4 and shape_predictor_68_face_landmarks.dat). OBS: I placed the both files of the dataSet inside the face2face-demo folder. Finally I run the follow command inside the project folder, but something went wrong.

face2face-demo] $ python generate_train_data.py --file angela_merkel_speech.mp4 --num 400 --landmark-model shape_predictor_68_face_landmarks.dat
Traceback (most recent call last):
File "generate_train_data.py", line 2, in
import cv2
ModuleNotFoundError: No module named 'cv2'

Hey, I just finished training for ~2 days and now tried to continue with your instructions.
Since that step isn't described: I guessed I had to copy the face2face-model folder over to the root of your projects folder and then ran:
python reduce_model.py --model-input face2face-model --model-output face2face-reduced-model
Unfortunately that led to a big error message:

Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1323, in _do_call
return fn(*args)
File "/usr/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1302, in _run_fn
status, run_metadata)
File "/usr/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: Key generator/encoder_8/conv/filter not found in checkpoint
[[Node: save/RestoreV2_43 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2_43/tensor_names, save/RestoreV2_43/shape_and_slices)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "reduce_model.py", line 215, in
saver.restore(sess, checkpoint)
File "/usr/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1666, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/usr/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/usr/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1120, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run
options, run_metadata)
File "/usr/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Key generator/encoder_8/conv/filter not found in checkpoint
[[Node: save/RestoreV2_43 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2_43/tensor_names, save/RestoreV2_43/shape_and_slices)]]

Caused by op 'save/RestoreV2_43', defined at:
File "reduce_model.py", line 214, in
saver = tf.train.Saver()
File "/usr/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1218, in init
self.build()
File "/usr/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1227, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/usr/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1263, in _build
build_save=build_save, build_restore=build_restore)
File "/usr/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 751, in _build_internal
restore_sequentially, reshape)
File "/usr/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 427, in _AddRestoreOps
tensors = self.restore_op(filename_tensor, saveable, preferred_shard)
File "/usr/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 267, in restore_op
[spec.tensor.dtype])[0])
File "/usr/lib/python3.6/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1021, in restore_v2
shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
File "/usr/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/usr/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)
File "/usr/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1470, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

NotFoundError (see above for traceback): Key generator/encoder_8/conv/filter not found in checkpoint
[[Node: save/RestoreV2_43 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2_43/tensor_names, save/RestoreV2_43/shape_and_slices)]]

I have to admit that I never worked with tensorflow and used the version 1.4.1. I tried downgrading to 1.2.1 but that led to another error. Should I retrain with the older version of tensorflow or am I missing something else?

error while training pix2pix model:

2018-09-06 14:21:53.018145: E T:\src\github\tensorflow\tensorflow\stream_executor\cuda\cuda_dnn.cc:332] could not create cudnn handle: CUDNN_STATUS_ALLOC_FAILED
2018-09-06 14:21:53.023584: E T:\src\github\tensorflow\tensorflow\stream_executor\cuda\cuda_dnn.cc:332] could not create cudnn handle: CUDNN_STATUS_ALLOC_FAILED

after processing 3 to 4 epochs , "_2_batch/fifo_queue: Skipping cancelled dequeue attempt with queue not closed" error occured.
After referring stackoverflow still the batch queue error occurred.

Hi,
I've trained the model on 600 pictures and run it for 200 epochs.
But my results are far from yours.
what might be the issue?
what are the parameters you used to train\run the model?

Thanks.

Does anyone know of a way to achieve this same result with only audio as an input and having it remain in realtime?

NotFoundError (see above for traceback): Key generator/decoder_2/batchnorm/offse
t not found in checkpoint

Build fails on conda env create -f environment.yml with the error:

Solving environment: failed

ResolvePackageNotFound:

• xz==5.2.2=1
• libpng==1.6.27=0
• readline==6.2=2
• mkl==2017.0.3=0
• bzip2==1.0.6=3
• numpy==1.13.0=py35_0
• tk==8.5.18=0

python pix2pix.py
--mode train
--output_dir face2face-model
--max_epochs 200
--input_dir photos/combined/train
--which_direction AtoB

This command is executing more than 3 hours. Shall I know how much time it takes for the training to complete? Is it normal or an issue?

Hi,

I am using your nice repo for a video instead of a webcam.
As so, I don't want to sacrifice quality for speed.
Is there a way to keep the generative part of the pix to pix model without reducing it?
I see a huge difference in my validation set rather than my generation using a reduce model and I don't want that.
Is it easy to bypass this inside the code of reduce_model.py while keeping the rest of the pipeline intact?

Thanks in advance

Kindly provide the pretrained model, it is not available in the link provided

Thank you for the helpful contribution. I want to ask can this repo implement the human pose, intrinsic, and expression of face as txt file? And if it is possible, where can I find this in the repo?

Hello,

I believe the following file is missing: RuntimeError: Unable to open shape_predictor_68_face_landmarks.dat

Thanks for the repo btw, awesome work!

I'm on the first part of Step 4, when the model is supposed to be reduced. Whenever I try to reduce the model, i get the following errors:
File "reduce_model.py", line 249, in <module> saver.restore(sess, checkpoint) File "C:\Users\gladamirflint\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\training\saver.py", line 1769, in restore raise ValueError("Can't load save_path when it is None.") ValueError: Can't load save_path when it is None.

I've looked for solutions elsewhere but I don't know enough about Tensorflow to understand what the problem is exactly. In the face2face-model folder, there is a file called 'checkpoint', no file extension. There's also three files in particular, model-64000.data-00000-of-00001, model-64000.index, and model-64000.meta that are also in that folder.

Any help with this would be greatly appreciated.

The links to video and train dataset are not reachable, anyone knows why?

when I run "python run_webcam.py --source 0 --show 0 --landmark-model shape_predictor_68_face_landmarks.dat --tf-model face2face-reduced-model/frozen_model.pb"
The result showed following:
X Error: BadDrawable (invalid Pixmap or Window parameter) 9
Major opcode: 62 (X_CopyArea)
Resource id: 0x220000e
And new video window is always black.
I use the virtual machine with Ubuntu 16.04.

Does anybody encounter such a problem?

VIDEOIO ERROR:V4L:cannot open camera by index 0
Traceback (most recent call last)：
File "run_webcam.py",line 130,in
main()
File "run_webcam.py", line 57, in main
frame_resize = cv2.resize(frame, None, fx=1 / DOWNSAMPLE_RATIO, fy=1 /DOWNSAMPLE_RATIO)
cv2.error: OpenCV(3.4.2) /io/opencv/modules/imgproc/src/resize.cpp:4044: error: (-215:Assertion failed) !ssize.empty() in function ‘resize’

在笔记本上报错如上

Hi,
How can we resume the training from the last saved weights/checkpoints?

Hi Datitran,

I follow your example to write a script about 'face2face real-time face capture and reenactment of rgb videos code', but I meet the trouble that it is hard slow.

Are you interesting to help me to improve the speed issue?

Looks like the conda dep isn't there anymore for opencv3.

- jlaura::opencv3=3.0.0=py35_0

Checked jlaura's anaconda page:

Dunno if this is active, but figured I'd ask to see if you could update the deps. Really cool demos based on the GIFs though!

python reduce_model.py --model-input /home/virgil/Desktop/F2F/pix2pix-tensorflow/face2face-model --model-output face2face-reduced-model
2018-03-18 15:21:21.466217: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-03-18 15:21:21.466534: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] Found device 0 with properties:
name: GeForce GTX 970 major: 5 minor: 2 memoryClockRate(GHz): 1.253
pciBusID: 0000:01:00.0
totalMemory: 3.95GiB freeMemory: 3.70GiB
2018-03-18 15:21:21.466552: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2018-03-18 15:21:21.652455: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3436 MB memory) -> physical GPU (device: 0, name: GeForce GTX 970, pci bus id: 0000:01:00.0, compute capability: 5.2)
2018-03-18 15:21:21.761345: W tensorflow/core/framework/op_kernel.cc:1202] OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Not found: Key generator/decoder_1/deconv/filter not found in checkpoint
Traceback (most recent call last):
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1361, in _do_call
return fn(*args)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1340, in _run_fn
target_list, status, run_metadata)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 516, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: Key generator/decoder_1/deconv/filter not found in checkpoint
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
[[Node: save/RestoreV2/_7 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_12_save/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "reduce_model.py", line 215, in
saver.restore(sess, checkpoint)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1755, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 905, in run
run_metadata_ptr)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1137, in _run
feed_dict_tensor, options, run_metadata)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1355, in _do_run
options, run_metadata)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1374, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Key generator/decoder_1/deconv/filter not found in checkpoint
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
[[Node: save/RestoreV2/_7 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_12_save/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

Caused by op 'save/RestoreV2', defined at:
File "reduce_model.py", line 213, in
saver = tf.train.Saver()
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1293, in init
self.build()
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1302, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1339, in _build
build_save=build_save, build_restore=build_restore)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 796, in _build_internal
restore_sequentially, reshape)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 449, in _AddRestoreOps
restore_sequentially)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 847, in bulk_restore
return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1030, in restore_v2
shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 3271, in create_op
op_def=op_def)
File "/home/virgil/.local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1650, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

NotFoundError (see above for traceback): Key generator/decoder_1/deconv/filter not found in checkpoint
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
[[Node: save/RestoreV2/_7 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_12_save/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

As the same there are some problems when using reduce_model.py
How can I solve such problem?

Instead of running python run_webcam.py how can the model be tested on an already-made video?

Hi @datitran,
Just tried to run the generate_train_data script and I get the below error. I'm messing around with the environment.yml to see if there is some dependency in there that is not correct.

I have Anaconda3-4.2.0-MacOSX-x86_64.pkg installed from https://repo.continuum.io/archive/ (Anaconda / Python 3.5).

The error:

python generate_train_data.py --file vidfile2.mov --num 400 --landmark landmark.dat
Traceback (most recent call last):
File "generate_train_data.py", line 3, in
import dlib
ImportError: dlopen(/Applications/anaconda/envs/face2face-demo/lib/python3.5/site-packages/dlib.so, 2): Symbol not found: __ZN5boost6python7objects15function_objectERKNS1_11py_functionERKNSt3__14pairIPKNS0_6detail7keywordESA_EE
Referenced from: /Applications/anaconda/envs/face2face-demo/lib/python3.5/site-packages/dlib.so
Expected in: /Applications/anaconda/envs/face2face-demo/lib/libboost_python3.dylib
in /Applications/anaconda/envs/face2face-demo/lib/python3.5/site-packages/dlib.so

Any help would be appreciated!

-Ali

affinelayer/pix2pix-tensorflow#126

Also about your comment

Isn't use unique name for each tf.layers.batch_normalization is sufficient? (by default its name=None)
affinelayer/pix2pix-tensorflow#126 (comment)

raise ValueError("Can't load save_path when it is None.")
ValueError: Can't load save_path when it is None.

pls somebody help me out to solve this issue!!

I've made a small change but now this function doesn't work with the images i've edited. the images i put in to resize pass 2 parameters only.

have you tried training this on the 300w dataset (https://ibug.doc.ic.ac.uk/resources/300-W_IMAVIS/) instead of the merkel video ?

or the smaller 5-point training data from http://mmlab.ie.cuhk.edu.hk/archive/CNN_FacePoint.htm ? im interested in taking this to tensorflow-lite (android), so im wondering how i can modify your code to have it train on the 5-point data ?

Hi,
i'm having troubles disabling dropout for testing to get more stable results.
could you advice what is the right way to do it?

many thanks.

I've tested on the Merkel video as in the readme, first testing on a model trained to 200 epochs and also on a model trained to 1000 epochs. Both models produce similarly poor results:

My input video is a very high quality .mp4 video, though as can be seen the subject was definitely too close to the camera compared to the Merkel video. Any advice on improving results? I am wondering if there's any possibility of reproducing the extremely high quality example from the original face2face researchers (https://www.youtube.com/watch?v=ohmajJTcpNk).

download your file is the same.