DeOldify: GAN based Image Colorization

B lack and white photography bears its emotional impact. You get rid of populist distractions or kitschy stereotypes of colors. The skies are blue, and the roses are red. That’s for sure. Obvious thing. But what is within us? The lights, the shadows, the contrasts. Rich world. That’s what the black&white photography expresses directly. Under your skin, into your heart and soul.”

This is what my Dad used to tell me. He knew it better, as he took photos his whole life. He was a master of psychological photo portrait. Yet he documented everything — the everyday life, the culture, the absurdity of the USSR, from the 1950ies till the 1990ies.

I was agreed with the emotional power of B&W. But at the same time, I wondered, how did look his world like, as he eternalized its contrasts and nuances. Luckily, Deep Learning can help.

#DeOldify

This model is created and released by Jason Antic . He is instantly working on DeOldify ( GitHub ) — with astonishing results and great mission:

The mission of this project is to colorize and restore old images and film footage. ( source )

DeOldify is using Generative Adversarial Networks with the iterative interplay between two Neural Networks Generator and Discriminator (like inArtBreeder). But differently to the last model, the images in DeOldify aren’t modified or generated in their form. Power of GAN brings colors — Generator applies colors to the recognized objects he’s trained on, and Discriminator tries to criticize the color choice. DeOldify is based on the fast.ai library — which brings more power and optimization for deep learning developers.

In this case, the skies are blue. The roses are red. (But not always. “It depends,” as Discriminator uses to claim).

Jason worked on this model for a long time — with fails and successes — a personal journey with emotional peaks. And with WOW moments.

In his interview for Hacker Noon he tells about his first impact of his model’s result, seeing the cup in the hands of Evely Nesbit in this DeOldified photo:

I do acknowledge the flaws in this image. But I knew I was on to something after seeing this.

So he kept digging, and so he could colorize old photos:

re-colorize paintings:

even bring colors to historical video footage:

An exciting feature for researches, who should keep in mind that the results don’t necessarily represent the original colors.

Bringing colors to historical images and video footage gives us an extraordinary opportunity to re-imagine the world, delivered in monochrome.

You can find the model in GitHub , and also in two notebooks: for Images ( Colab Notebook ) and Videos ( Colab Notebook ) — try them out!

Here is a short video tutorial for working with DeOldify Notebooks:

B&W or Colors?

I had my own WOW moment, colorizing my father’s black&white photography. Look at these flowers — diversity of their colors, not visible in b&w:

In the photo in the header of this article you also can see the colors on the painting — re-imagining imagination with AI. Green grass — and echoing of monastery colors in the picture the girls is drawing.

I still agree with the psychological energy of black & white portraits. But from my very personal side, I am thankful to wonderful possibility of DeOldify to bring us back in time. Rebirth of my Dad’s memories. The colors are always vivid. In our memories — and now, in immortalized documents of the past. With help of Deep Learning.

Follow Jason Antic on Twitter for #DeOldify-news and colors: https://twitter.com/citnaj/