dfaker / df Goto Github PK
View Code? Open in Web Editor NEWLarger resolution face masked, weirdly warped, deepfake,
License: Mozilla Public License 2.0
Larger resolution face masked, weirdly warped, deepfake,
License: Mozilla Public License 2.0
Hi, in align_images_masked.py
Where or how to install this package?
While running train.py I'm running out of GPU memory. I already tried to set the batch size down to 4 without any improvement. Can you recommend any model parameters to adapt? My GPU is a GTX 970 4GB (of which Tensorflow can only use 3.5GB).
The error is: OP_REQUIRES failed at transpose_op.cc:199 : Resource exhausted: OOM when allocating tensor with shape[4,16,16,1024] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
Allocation Stats:
Stats:
Limit: 3432906752
InUse: 3428587264
MaxInUse: 3431733248
NumAllocs: 1546
MaxAllocSize: 597213184
EDIT: I tried to set ENCODER_DIM
to 256 and I don't get Memory errors anymore, but now I'm presented with this error:
Traceback (most recent call last):
File "train.py", line 151, in <module>
figure = figure.reshape( (4,4) + figure.shape[1:] )
ValueError: cannot reshape array of size 589824 into shape (4,4,3,128,128,3)
I changed the values inside to merge_faces_larger.py
parser.add_argument( "--blurSize", type=int, default='250' )
parser.add_argument( "--erosionKernelSize", type=int, default='170' )
(Like do with the APP 1.1)
But it doesn't make any change, how i can reduce the surface size of the "fake" face i'm adding over the original face?
Tried to enable processing ok two gpus, but got an error about layers having the same name. Any easy way to get this working?
Microsoft Windows [VersiΓ³n 10.0.16299.98]
(c) 2017 Microsoft Corporation. Todos los derechos reservados.
C:\WINDOWS\system32>cd C:\df-master
C:\df-master>python train.py
running
loading data/A: 100%|βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ| 190/190 [00:00<00:00, 445.43it/s]
loading data/B: 100%|βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ| 946/946 [00:00<00:00, 18816.39it/s]
Images A (125, 256, 256, 4)
Images B (0,)
Traceback (most recent call last):
File "train.py", line 34, in
images_A[:,:,:3] += images_B[:,:,:3].mean( axis=(0,1,2) ) - images_A[:,:,:3].mean( axis=(0,1,2) )
IndexError: too many indices for array
C:\df-master>
I ask because, unlike in deepfakes/faceswap or FakeApp, my GPU seems to not have enough VRAM for the training process, spitting out the following error (uploaded to hastebin due to its sheer size): https://hastebin.com/raw/metojewuqa
It is a GTX 1060 6GB for reference, with 6GB of VRAM (as I'm sure you could tell). Is this not enough for this project? Has the model been increased to 128x128 perhaps? If so, is there a way to bump it back down to 64x64 or perhaps a middle-ground like 96x96.
Thank you.
Hi,
I'm interested in applying a swap on a particular part of a face.
e.g eyes or mouth.
what would be the best approach for that?
Thanks!
(dfaker) D:\Library\Desktop\DFaker>python train.py
running
Traceback (most recent call last):
File "train.py", line 25, in <module>
images_A,landmarks_A = load_images( images_A[:minImages] )
File "D:\Library\Desktop\DFaker\utils.py", line 14, in load_images_masked
basePath = os.path.split(image_paths[0])[0]
IndexError: list index out of range
(dfaker) D:\Library\Desktop\DFaker>
Thanks for this work, as far as I know, the original deepfakes only support 64x64 as input, which is determined by Linear layer of model ( can not change input size ). How did this repo maxium size support? 256 or ?
I'll pretend I'm swapping Trump's State of the Union vid with Nic Cage's face so we don't get confused.
I run 'run merge_faces_larger.py',
that is I type 'python merge_face_larger.py' 'path to photos ripped from State of the Union Trump video'
The terminal starts chunking through the photos, a screen pops up that shows each frame but faces are not being swapped.
When the process is done I have a file of photos called 'merged' and the photos are the exact same photos it started with. Nothing swapped.
Picture me with a Nic Cage 'A', fully trained Trump 'B' model, run on the Trump video frames and only returned Trump video frames in the 'merged' folder - just to be clear about it.
I'm trying to prepare images for train, but getting error with align_images_masked.py
python3 align_images_masked.py input_dir=/home/tuw/Desktop/df/dp2
Traceback (most recent call last):
File "align_images_masked.py", line 125, in <module>
main( parser.parse_args() )
File "align_images_masked.py", line 53, in main
assert input_dir.is_dir()
AssertionError
When i change line #112 from parser.add_argument( "input_dir" , type=str )
to
parser.add_argument( "input_dir" , type=str, nargs='?', default='/home/tuw/Desktop/df/dp2' )
It worked
How to fix it? Edit every time source code is for input dir is π©
It's happened many times now. After a few hours the preview will suddenly change from attempting to merge the face to a changing blob of solid colors. After a while the colors seem to settle on red and from then on the model becomes broken and, as far as I know, unfixable unless you have a backup. Sometimes it's the third row of each image which goes completely red and sometimes it's the second, regardless attempting the merge process while the models are on this broken red state results in blank faces in the final output.
Any particular reason replacing cnn face detection with frontal face detection?
Any obvious reason why alignment with Cuda enabled fails on larger resolution images? The model itself gives incredible results, so its so frustrating that the alignment tool doesnt work with GPU enabled.
I tried using the updated alignment tool from the deepfake/faceswap repo but I guess the .json output is different.
Maybe it could be rewritten to fit the format dfaker needs? Alignment on 56 core CPU (Dual xenon) takes forever compared to GPU
I wanted to ask if it is necessary to do the face_alignment before training? Why is it necessary? What will happen if I train the model without aligned images?
Hi @dfaker ,
Just opening this so we can discuss about adding your work as a plugin in the deepfakes/faceswap repo. I don't know if you had this in mind but I started it on my side but did not went very far (https://github.com/Clorr/df). For now I stopped, but if you are interested in helping me to make this happen, let's discuss it here ;-)
tensorflow-gpu==1.4.0 in requirements.txt
It's not compatible?
If you attempt to run train.py without having alignments.json files on both A and B folders, the training process will not be allowed to execute. I know that other implementations of this software, such as FakeApp, don't require this file for training, but I am not sure if it is necessary for this particular implementation. If it is not, it would be great if this requirement could be removed on a future version since adding more images for training would, to my knowledge, require you to run the whole extraction process again on all images.
if i want to do merging process, what i should i type in cmd?? is this the way??
python merge_faces_larger.py -i ~df/data/A/ -o ~df/output/ -m ~df/models/ --seamlessClone True --blurSize 10 --erosionKernelSize 7 .
First question, does df need python2 or python3?
And when i'm trying to run python align_images_masked.py or python3 align_images_masked.py
I'm getting
python3 align_images_masked.py
Traceback (most recent call last):
File "align_images_masked.py", line 11, in <module>
from face_alignment import FaceAlignment, LandmarksType
ImportError: No module named face_alignment
Tried to install face_alignment module, but
pip3 install face_alignment
Collecting face_alignment
Could not find a version that satisfies the requirement face_alignment (from versions: )
No matching distribution found for face_alignment
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
How i can install face_alignment module? p.s. im on Ubuntu 16.04
Save model weights
Traceback (most recent call last):
File "train.py", line 148, in
figure = figure.reshape( (4,4) + figure.shape[1:] )
ValueError: cannot reshape array of size 884736 into shape (4,4,3,128,128,3)
Hello,
I trained a model and it all looks very good from the preview window.
Now, I'm trying to create the merged faces, and when I run merge_images_larger.py on the original images folder (not the faceset), I do get a progress bar, and a "merged" folder is created. But it is empty.
I tried adding parameters etc. but to no avail.. always same thing. Any thoughts of anything I could try to troubleshoot this?
I saw that by default dfaker will max out the used images for a given data set at 2000. My A set is under 2000 while my B set is around 12000. Is this code minImages = 2000
by design to only take in 2000 at a time and it randomizes them? When the script first loads it doesn't just load the first 2000 but all of the images, only the output afterward shows that it's maxing at 2000. Does the script choose the best 2000 to use?
If it doesn't, would it be possible to set it to use every photo from each data set? So for example, using 1500 from A and 6204 from B?
When I use the commented extra like this minImages = min(len(images_A),len(images_B))*20
, it fails to run.
Hi dfaker, if you don't mind, can I ask some questions about the implementation? I'm not sure where else to ask other than here.
For the larger faces, I see that you pull out 240x240 pixels from the 256x256 cropped faces. The original code uses 160x160, like at the faceswap repo.
Is anything else in your code specific to generating larger face models? The model is more complex, but I think that most of it is related to generating a higher resolution 128x128 output, not for generating larger face models?
In other words, if I took the faceswap repo, and just change the image_augmentation to mimic yours with 240x240, would that generate large face models at 64x64 resolution? Or is there some other step that needs to be changed to make it work? Are changes in the conversion process essential to the large face model, or wouldn't just using user-defined kernel/blur etc. be sufficient?
Hi,
First of all, great job and thanks for sharing
I want to change the face area in merging process. The chin of the two faces are different height, in training preview the shape correction is right but is not carried to the conversion
Can I change any part of the code to modify the conversion area?
Thank you
I am running align_images_masked.py but when I enter "python align_images_masked.py sourceA" I get the message below:
(df) C:\df>python align_images_masked.py sourceA aligned
RuntimeError: module compiled against API version 0xc but this version of numpy is 0xa
Traceback (most recent call last):
File "align_images_masked.py", line 125, in
main( parser.parse_args() )
File "align_images_masked.py", line 68, in main
assert len( input_files ) < 0, "Can't find input files"
AssertionError: Can't find input files
sourceA is where my images are stored and there are 100 of them. I've tried converting them to png, jpg, and bmp. Every time the script says it can't find the input files, but it does create an empty aligned folder in sourceA, so I know it's partly working. I am using windows 10 x64 and anaconda. My files are setup like this:
C:/df/sourceA <------100 images in here
The script is located here: C:/df/align_images_masked.py
Images A (649, 256, 256, 4) Images B (1041, 256, 256, 4) Traceback (most recent call last): File "train.py", line 31, in images_A = images_A/255.0 MemoryError
What could be the problem here? Not enough RAM? I have 11GB free at the point of running the script.
Hello, does this currently work well on hight resolution face swap?
I don't have strong GPU, but think to try it at cloud gpu floyhub. What will be scrip to run it ?
And training stops.
Don't have this problem with original scripts.
[02:46<563:58:07, 2.03s/it]2018-02-06 19:49:27.301416: W C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\bfc_allocator.cc:273] Allocator (GPU_0_bfc) ran out of memory trying to allocate 128.00MiB. Current allocation summary follows.
2018-02-06 19:49:27.301575: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\bfc_allocator.cc:627] Bin (256): Total Chunks: 96, Chunks in use: 93. 24.0KiB allocated for chunks. 23.3KiB in use in bin. 484B client-requested in use in bin.
2018-02-06 19:49:27.301714: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\bfc_allocator.cc:627] Bin (512): Total Chunks: 11, Chunks in use: 10. 5.5KiB allocated for chunks. 5.0KiB in use in bin. 5.0KiB client-requested in use in bin.
2018-02-06 19:49:27.301983: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\bfc_allocator.cc:627] Bin (1024): Total Chunks: 27, Chunks in use: 27. 27.3KiB allocated for chunks. 27.3KiB in use in bin. 27.0KiB client-requested in use in bin.
...
ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[32,64,128,128]
[[Node: training/Adam/gradients/model_2/conv2d_16/convolution_grad/Conv2DBackpropInput = Conv2DBackpropInput[T=DT_FLOAT, _class=["loc:@model_2/conv2d_16/convolution"], data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](training/Adam/gradients/model_2/conv2d_16/convolution_grad/ShapeN, conv2d_16/kernel/read, training/Adam/gradients/model_2/conv2d_16/Sigmoid_grad/SigmoidGrad)]]
What is the default number of layer and nodes and how can I change them or other options.
It's normal or should be using the GPU?
after train the model ,i get the 'encoder.h5' 'decoder_A.h5' 'decoder_B.h5'
when i use these file to run 'merge_face_larger.py'
the input dir is data/B:
ββmerged
β β alignments.json
β β trainingimageB1.jpg
β β trainingimageB2.jpg
output dir is data/B/merged.
But the converted image is the same as the previous one.
How are others getting the face-alignment dependency to install under Windows 10? I've tried anaconda/miniconda, and the torch dependency fails during setup. I haven't been able to successfully build torch in Windows myself or get a working install independently. From looking at other people's screenshots, it looks like they are not bothering to install face-alignment?
https://www.youtube.com/watch?v=NYvxg8uDsAA
Keeping away the quality or not of the "Final Fusion", what is really concerning are the jumps that appear when MERGE is done, this is not a video render glitch, i can already see the jumps just scrolling trough the images in the MERGE folder.
So there is something that's not working like should i think.
Here you have the A folder with the base frames video, ALIGNED images and MERGED ones, so you can see that jumps start happening when do the MERGE:
https://drive.google.com/open?id=10rBAnx9KZ8wxa1PsrImDZuhfquSWgA7O
Hi,
I would like to for help with the following problem:
merge_faces_larger.py says:
2018-10-04 19:23:05.077752: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\gpu\gpu_device.cc:1195] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0, compute capability: 6.1)
Traceback (most recent call last):
File "merge_faces_larger.py", line 244, in
main( parser.parse_args() )
File "merge_faces_larger.py", line 202, in main
for image_file, face_file, mat,facepoints in tqdm( alignments[args.startframe::args.frameSkip] ):
TypeError: unhashable type: 'slice'
tqdm is 4.26.0
Could you please help me what could be the problem?
Thank you!
Br,Herold
I'm just curious why you don't utilize blur/kernel size settings during the merging process. Opencv's seamlessClone can produce decent results, but in my experience it leads to flickering faces more often than not and isn't concealing the boundary of the output face as effectively as the right blur/kernel size settings.
Consider using https://github.com/YuvalNirkin/face_segmentation library for better area coverage of the face #
When im trying to run align_images_masked.py
im getting
Traceback (most recent call last):
File "align_images_masked.py", line 2, in
import cv2
File "C:\Users\Admin\Anaconda3\lib\site-packages\cv2_init_.py", line 4, in
from .cv2 import *
ImportError: DLL load failed: %1 is not a valid Win32 application.
What the problem?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.