GitHub - iperov/DeepFaceLab: DeepFaceLab is a tool that utilizes deep learning t...

README.md

DeepFaceLab is a tool that utilizes deep learning to recognize and swap faces in pictures and videos.

Based on original FaceSwap repo. Facesets of FaceSwap or FakeApp are not compatible with this repo. You should to run extract again.

Features:

• new models

• new architecture, easy to experiment with models

• works on 2GB old cards , such as GT730. Example of fake trained on 2GB gtx850m notebook in 18 hours https://www.youtube.com/watch?v=bprVuRxBA34

• face data embedded to png files

• automatic GPU manager, chooses best gpu(s) and supports --multi-gpu (only for identical cards). Warning: dont use cards in SLI mode.

• cpu mode. 8th gen Intel core CPU able to train H64 model in 2 days.

• new preview window

• extractor in parallel

• converter in parallel

• added --debug option for all stages

• added MTCNN extractor which produce less jittered aligned face than DLIBCNN, but can produce more false faces. Comparison dlib (at left) vs mtcnn on hard case: MTCNN produces less jitter.

• added Manual extractor. You can fix missed faces manually or do full manual extract:

• standalone zero dependencies ready to work prebuilt binary for all windows versions, see below

Model types:

• H64 (2GB+) - half face with 64 resolution. It is as original FakeApp or FaceSwap, but with new TensorFlow 1.8 DSSIM Loss func and separated mask decoder + better ConverterMasked. for 2GB and 3GB VRAM model works in reduced mode.

H64 Robert Downey Jr.:

• H128 (3GB+) - as H64, but in 128 resolution. Better face details. for 3GB and 4GB VRAM model works in reduced mode.

H128 Cage:

H128 asian face on blurry target:

• DF (5GB+) - @dfaker model. As H128, but fullface model. Strongly recommended not to mix various light conditions in src faces.

• LIAEF128 (5GB+) - new model. Result of combining DF, IAE, + experiments. Model tries to morph src face to dst, while keeping facial features of src face, but less agressive morphing. Model has problems with closed eyes recognizing.

LIAEF128 Cage:

LIAEF128 Cage video:

• LIAEF128YAW (5GB+) - currently testing. Useful when your src faceset has too many side faces vs dst faceset. It feeds NN by sorted samples by yaw.

• MIAEF128 (5GB+) - as LIAEF128, but also it tries to match brightness/color features.

MIAEF128 model diagramm:

MIAEF128 Ford success case:

MIAEF128 Cage fail case:

• AVATAR (4GB+) - non GAN, 256x256 face controlling model.

Video:

Usage:

src - controllable face (Cage)

dst - controller face (your face)

converter --input-dir must contains extracted dst faces in sequence to be converted, its mean you can train on for example 1500 dst faces, but use for example 100 faces for convert.

• Video comparison of different Cage facesets. Vertical: 1 - mix of various Cage face shape and light conditions. 2,3,4 - without mix. Horizontal: 1 - DF, 2 - LIAEF128.

Conclusion: better not to mix and use only same shape faces with same light

Sort tool:

blur places most blurred faces at end of folder

hist groups images by similar content

hist-dissim places most similar to each other images to end.

hist-blur sort by blur in groups of similar content

brightness

hue

black Places images which contains black area at end of folder. Useful to get rid of src faces which cutted by screen.

Best practice for gather src faceset:

1. delete first unsorted aligned groups of images what you can to delete. Dont touch target face mixed with others.
2. black -> delete faces cutted by black area at end of folder
3. blur -> delete 30-50% at end of folder
4. hist -> delete groups of similar and leave only target face
5. hist-dissim -> leave only first 1500 faces
6. face-yaw -> just for finalize faceset

Best practice for dst faces:

1. delete first unsorted aligned groups of images what you can to delete. Dont touch target face mixed with others.
2. hist -> delete groups of similar and leave only target face

Ready to work facesets:

• Nicolas Cage 4 facesets (1 mix + 3 different)

• Steve Jobs

download from here: https://mega.nz/#F!y1ERHDaL!PPwg01PQZk0FhWLVo5_MaQ

Build info

dlib==19.10.0 from pip compiled without CUDA. Therefore you have to compile DLIB manually.

Command line example for windows: python setup.py install -G "Visual Studio 14 2015" --yes DLIB_USE_CUDA

CPU only mode

CPU mode enabled by arg --cpu-only for all stages. Follow requirements-cpu.txt to install req packages. Do not use DLIB extractor in CPU mode, its too slow. Only H64 model reasonable to train on home CPU.

Prebuilt windows app:

Windows 7,8,8.1,10 zero dependency (just install/update your GeForce Drivers) prebuilt DeepFaceLab (include GPU and CPU versions) can be downloaded from

1. torrent https://rutracker.org/forum/viewtopic.php?p=75318742 (magnet link inside).
2. https://mega.nz/#F!b9MzCK4B!zEAG9txu7uaRUjXz9PtBqg

Video tutorial: https://www.youtube.com/watch?v=K98nTNjXkq8

**Windows 10 memory problem:

Windows 10 consumes % of VRAM even if card unused for video output.

Problem of the year:

algorithm of overlaying neural face onto video face located in ConverterMasked.py. Can someone implement adaptive histogram matching to prevent glares when a dark eyes face merges onto a light eyes face ?