79

The performance of Intel vs. Anaconda vs. vanilla Python – my personal benchmark

 5 years ago
source link: https://www.tuicool.com/articles/hit/UVBry2v
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

The Python programming is our daily bread. We develop frameworks, which are afterwards deployed on the customers’ infrastructures.  And in some cases, there is an emphasis on performance, such as in the recent case with a recommender engine, which should load an individual recommendation in less than 30 ms.  And then faster calculation might be helpful, especially since the use of a specific distribution requires no changes to the underlying python code

Two weeks ago, Martin found that an  Intel distribution for Python  exists, so I decided to hav e a look. Intel claims that this distribution is faster in every way, and shares its benchmark. So apart from conducting the intel benchmark only, I decided to test the distributions using my own benchmark to determine the performance on typical cases often performed in a Data Science pipeline.

fAnuE3U.png!web

Intel Distribution for Python

The Intel distribution is built on top of Anaconda, and both distributions already come with the Intel MKL library. However, Intel also provides other libraries, such as Data Analytics Acceleration Library (DAAL) and Threading Building Blocks (TBB) among others. So how does a clean Python 3.6 with Anaconda distribution compare, and does Intel bring any other additional benefit to this setup?

Regarding methodology, I decided to create three Azure VM instances DS11v2, all with Ubuntu 18.04 installed. DS11v2 is an instance optimized for data science workloads, and has 14GB of RAM. It comes with 2 cores from Intel Xeon 2673v4.

These information might be useful for bughunting:

  • Always check, whether numpy is paired with MKL library using 
    numpy.show_config()
    • If it is not, you will see rows like 
      blas_mkl_info:
NOT AVAILABLE

lapack_mkl_info:
NOT AVAILABLE
  • If you cannot pair the numpy with MKL (and other libraries), even though you have sourced the intelpython or anaconda
    • Create virtual environment with conda 
      conda create -n venv
    • Install numpy, sklearn, scipy using conda install (not pip!)
  • To activate the Intel distribution, these are the main steps:
    • Install all Intel performance libraries
    • Source the Intel distribution using 
      source /opt/intel/intelpython3/bin/activate
    • Create the virtual environment using 
      conda create -n intelpython --override-channels --channel intel python=3.6 intelpython scipy pydaal scikit-learn numpy pandas

More about the installation can be found here .

Then I created three different setups.  First one is a clean Ubuntu with clean Python 3.6.5, second one was running the Anaconda distribution, and the third had a full Intel Python distribution installed. To perform the tests, I activated the distribution (for both Anaconda and Intel distro) and created a virtual environment, and installed all main dependencies using conda – namely sklearn, scipy, numpy and pandas. Intel distribution takes these from its own channels.

What strikes me the most?

  1. You are definitely better off with a custom distribution than with the vanilla Python. This is mainly caused by the Intel MKL library, and I recommend you to read more about it in this article.
  2. It seems that on such a small instance of VM the Intel distribution does not show its full capabilities.
  3. The Intel distribution seems to have a small benefit for numpy calculations versus the Anaconda.
  4. In some cases, e.g. on pandas.DataFrames, the Intel distribution performs worse.
  5. Other interesting thing is that the same calculations using numpy take ~400 times less time than using pandas.

See the attached graphs for detailed results, and you can find the complete results here , including testing scripts.

Overall, it seems that for our use case the Intel distribution is not worth it over the Anaconda distribution. Our cloud instances are similar to the instances used in this article, and we haven’t seen any significant benefits in the use of the Intel distribution. Apart from that, its installation process is quite tedious in contrast with Anaconda distribution, which is just an installation of a single shell script, which does everything itself. In conclusion, if you are running vanilla Python only, go and install Anaconda. If you are playful and have a high performance environment, you can have a look at the Intel Python, or you can try something else .


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK