Facial Reconstruction using Autoencoders

Learn how to reconstruct/denoise images using autoencoders from scratch

Image by Author

Many of you may have witnessed the application of facial recognition or reconstruction in crime dramas or spy thrillers. Yes, it helps to solve a case so why not build your own. In this article, we will be focusing on how to reconstruct faces from noise containing or damaged images. This is also known as Image Denoising. Using unsupervised deep learning techniques or rather self-supervised deep learning techniques of autoencoders, this program helps to reconstruct or denoise facial images.

Image by Author

What are autoencoders ?

Autoencoders, as the name suggests, encodes itself. It takes inputs and makes them go through hidden layers that are supposed to give an output similar to the input. Hence, the whole aim is to get identical results to input fro the output. There are two components of an autoencoder i.e. encoder and decoder. The encoder has the job to compress the input data into smaller encodings. Whereas decoders learn to build output, the same as the input using the encodings. They have a wide range of applications from recommender systems to classification problems too. Here we will be using convolutional autoencoders (CAEs) which are built upon standard network components, i.e., convolutional layers and skip connections that can outperform state-of-the-art methods which employ adversarial training and sophisticated loss functions.

Data: Dataset and Pre-processing

We will be using the LFW Dataset for this project. It is a database of face photographs designed for studying the problem of unconstrained face recognition. The data set contains more than 13,000 images of faces collected from the web. Each face has been labeled with the name of the person pictured. 1680 of the people pictured have two or more distinct photos in the data set. For preprocessing we will covert raw matrices into images obtained from the dataset and change the color system into RGB. We will also be taking into consideration, various attributes related to the facial image like the shape of the nose, heavy makeup, smiling, etc.

Building the Model

Gist by Author

Here we build a custom model builder where we get to specify the image dimensions and output representation. We have used a Keras based model to do so as it is easy to apply. In the code snippet, img_shape signifies the image dimensions and code_space is the size of output representation.

The code_space needs to be appropriately chosen as smaller the number, it will compress the image that much, but fewer features will be considered and we won’t be able to get accurate results. In this code, I have taken a code_space of 10000 for demonstration purposes.

We build the encoder and decoder separately in the same function which are a single-layered neural network respectively.

Training and Testing

Gist by Author

Since the model is ready and data is processed, its time to train!

We will be training the model on regular faces that are present in the dataset with the aim that the model will reconstruct the same face. I tried and visualized the results with a different number of epochs. I finally concluded that 20 epochs were enough to get a good result with training loss at 0.0020 and testing loss at 0.0025.

Image by Author

Now that we have a good enough model, we will try to add noise to the original image and map it through our model to give a clearer one. For adding noise we will use the method of Gaussian noise and retrain are model over it.

Image by Author

After retraining the model to map noisy images to a clearer one we get the training loss as well as the testing loss of 0.0038 after 25 epochs. Visualized results are as followed:

Image by Author

More Applications of Autoencoders

Another cool application of autoencoders in the same field is of making DeepFakes . This is used in numerous ways such as face swaps where the encoder and decoder are from different models.

Another application of autoencoders is recommender systems where it can accurately rate, let’s say movies on a scale of 1–5 based on previously reviewed movies by a user.

Other applications include dimensionality reduction, image compression (as used in this case too), feature extraction, image generation, and sequence to sequence prediction.

Source Codes and Additional Resouces

Check out the entire code on my GitHub with relevant links to the data set as well.

Addition Reading links for autoencoders.