Is PyTorch Catching TensorFlow?

Is PyTorch Catching TensorFlow?

The State of Deep Learning Frameworks in 2020

It’s been 10 months since I looked at landscape for the major deep learning frameworks. PyTorch and TensorFlow have emerged as the clear frameworks of choice. TensorFlow had a two year head start on PyTorch, but PyTorch has been closing the gap in many areas. Let’s see where things stand now! :+1:

What’s New?

TensorFlow and PyTorch look more and more like each other. TensorFlow has adopted PyTorch innovations and PyTorch has adopted TensorFlow innovations. Notably, now both languages can run in a dynamic eager execution mode or a static graph mode.

Both frameworks are open source, but PyTorch is Facebook’s baby and TensorFlow is Google’s baby.

Let’s look at a few other notable differences and updates. :+1:

PyTorch

PyTorch is on version 1.4 as of this writing. It has been adding features to increase adoption by industry. PyTorch can now be run more easily on Google Cloud’s Tensor Processing Units (TPUs) — the fastest way to train complex deep learning models.

Also, the maintainers of the Chainer framework, Preferred Networks, recently brought their team to PyTorch. Finally, if you are into C++ or Java, PyTorch has a version for you. :grinning:

TensorFlow

TensorFlow 2.0 introduced many improvements that make the API more streamlined and brain friendly. It tightly integrated Keras as it’s front-end, high-level API. Keras can still be used with some other frameworks, but it generally makes sense to use it as part of TensorFlow.

Also, if you prefer JavaScript to Python, TensorFlow has a JavaScript version for you. :grinning:

TensorFlow still has more bells and whistles for deep learning in production and on the edge than PyTorch does, but PyTorch is getting closer to feature parity.

Method

PyTorch and TensorFlow are the two games in town if you want to learn a popular deep learning framework. I’m not looking at other frameworks because no other framework has been widely adopted.

In my previous analysis, I looked at many metrics. The results showed TensorFlow and PyTorch were both growing at a decent clip. In this article, I’m going to focus on the four metrics that I think matter most: job listings, research use, online search results, and self-reported use.

Let’s jump in!:rocket:

Job Listings

I searched Indeed , Monster , SimplyHired , and LinkedIn for the keywords TensorFlow and PyTorch in the United States on January 26, 2020.

TensorFlow appeared in an average of twice as many search results on each job listing website. Here are the raw numbers:

And here are the results as percentages of the total.

In my analysis ten months ago , I noted that TensorFlow appeared in three times as many listings as PyTorch. Now TensorFlow’s advantage is down to 2x. :chart_with_downwards_trend:

Research

PyTorch is now the leader in terms of papers in top research conferences. Horace He, a former PyTorch intern, researched and discussed the topic in this article . Here’s a chart showing the growth.

Source: https://chillee.github.io/pytorch-vs-tensorflow/

Anything over 50% means more mentions for PyTorch than TensorFlow for that conference.

As a more detailed sample, here are the raw scores from the NeurlIPS conference, with PyTorch eclipsing TensorFlow.

Source: https://chillee.github.io/pytorch-vs-tensorflow/

In the recent NerulIPS conference, PyTorch was in 166 papers and TensorFlow was in 74. PyTorch went from being in fewer papers than TensorFlow in 2018 to more than doubling TensorFlow’s number in 2019.

I found it surprising that PyTorch surpassed TensorFlow so quickly. It will be interesting to see if PyTorch continues to extend its lead in this area. :chart_with_upwards_trend:

Let’s look at online searches next.

Search Popularity

I used Google Trends to find the relative number of searches for PyTorch (Software) and TensorFlow (Computer application) in the USA from January 26, 2017 to January 26, 2020.

See the results below with TensorFlow in blue and PyTorch in red:

Google search interest. TensorFlow in blue. PyTorch in red.

Apparently no one wants to search for deep learning frameworks around the winter holidays. :christmas_tree::grinning:

Here’s the same data with linear trend lines.

PyTorch has closed the gap considerably. It will be interesting to see if the trend holds.

Finally, let’s look at what folks say they are using.

Reported Use

In the Stack Overflow Developer Survey from 2019, 10.3% of respondents reported using TensorFlow while 3.3% reported using Torch/PyTorch. The subset of professional developers had similar scores (9.4% vs. 2.9%). No other deep learning frameworks were reported.

It’s worth noting that the survey was conducted in early 2019, so we’ll have to wait and see what the 2020 results show. :bar_chart:

Recap

TensorFlow is still mentioned in many more job listings that PyTorch, but the gap is closing. PyTorch has taken the lead in usage in research papers at top conferences and almost closed the gap in Google search results. TensorFlow remains three times more common in usage according to the most recent Stack Overflow Developer Survey.

Overall, TensorFlow still leads in most areas, but PyTorch is gaining. :racehorse:

What Should You Focus On?

This project was born out of my desire to spend my time using and educating with the deep learning framework that makes the most sense for people to learn. I’m still undecided as to whether the answer to the next two years is PyTorch or TensorFlow.

It appears likely that PyTorch will continue on its trajectory toward parity. However, TensorFlow is still the safe choice. It will be in demand for the foreseeable future, particularly by employers.

It’s worth noting that both frameworks continue to innovate in a spirit of cooperative competition. The coming innovations will shape their popularity and provide benefits for users. :tada:

Resources

If you are ready to learn deep learning, I suggest the following resources:

The fast.ai MOOC has great videos and a supportive community. It’s especially useful if you want to learn PyTorch.

Chollet’s Deep Learning with Python book is a great primer on deep learning. He’s the original author of Keras. It’s an especially good option if you want to learn TensorFlow.

Both TensorFlow and PyTorch have nice tutorials and guides in their docs.

Prerequisites

Before you tackle deep learning, I suggest you learn Python. If you are brand new to Python, might I suggest my Memorable Python book? :grinning:

Once you are comfortable with Python basics, I suggest you learn the NumPy and pandas Python libraries for data cleaning and analysis. McKinney’s Python for Data Analysis book covers NumPy and pandas well. I also have pandas book in the works, so subscribe to my Data Awesome mailing list to be the first to learn when it’s released.

Finally, before jumping into the deep learning branch of machine learning, I suggest you learn to use more general machine learning algorithms with the scikit-learn library. I like Müller and Guido’s Introduction to Machine Learning with Python . Also, the second edition of Aurélien Géron’s Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow covers machine learning and deep learning with TensorFlow 2.0.

Wrap

I hope you found this evaluation of the state of the most popular deep learning frameworks useful. If you did, please share it on your favorite social media so other folks can find it, too. :+1:

I write about Python , Docker , SQL , and other tech topics. If any of that’s of interest to you, follow me and read more here .

Happy Deciding! ⚖️