Python 3.10 is now available–but should you switch to it immediately? And if not now, when?

The short answer is, no, you probably don’t want to switch immediately; quite possibly you can’t switch immediately. To understand why, we need to consider Python packaging, the software development process, and take a look at the history of past releases.

We can then make a guess about when Python 3.10 will actually be usable.

The problems with a new major Python release

I’m writing this article on October 5th, 2021, the day after 3.10 was released… and far too early to start using Python 3.10.

As with many open source projects, Python, Python libraries, and most of the toolchain and packages mentioned in this article are based on volunteer labor. None of this should be read as a complaint towards the people doing the maintenance, they’re doing hugely valuable work for free, and everything takes time. In addition, because upgrades involve so many different groups, coordination and releases take even more time.

With that in mind, let’s consider the problems with using 3.10 today:

1. Missing Docker images

Let’s try running Python 3.10 under Docker:

$ docker run -it python:3.10
Unable to find image 'python:3.10' locally
docker: Error response from daemon: manifest for python:3.10 not found: manifest unknown: manifest unknown.

As it turns out, there is no Python 3.10 Docker image for the final release. If you want to use Python 3.10, you can use a release candidate, e.g. python:3.10-rc.

No doubt it will be available within the next few days, maybe even today, but early on even installing a new release can be difficult. And that’s just our first problem.

2. Missing binary packages

Another problem is that not all packages support 3.10 yet. For example, psycopg2-binary provides bindings to the PostgreSQL database; it had 17 million downloads from PyPI over the last month. When I try to install it on Python 3.10 things don’t go well:

$ docker run python:3.10-rc-slim pip install psycopg2-binary
...
   Error: pg_config executable not found.
    
    pg_config is required to build psycopg2 from source.
...

What’s going on?

Typically, Python package maintainers upload compiled versions of their packages–known as “wheels”–to PyPI. When you install a package, you can just download the binary wheel and don’t need to compile it (unless you’re using Alpine Linux).

But it’s so soon after 3.10’s release, many packages don’t have wheels for Python 3.10 yet. In this case, psycopg2-binary still doesn’t have wheels for 3.10.

To be fair, things seem to be improving on this front: for the 3.9 release, NumPy and Pandas did not have wheels available this soon after the release. This time around, they do. But there are plenty of projects that still don’t; to pick another random example, matplotlib (with 27M downloads a month from PyPI) is also missing a 3.10 binary wheel.

Important: Make sure you upgrade pip, otherwise you might not get the latest binary wheels for 3.10 on Linux, and instead pip will try to build from source.

3. Incompatible packages

Sometimes it’s just a matter of recompiling the code. That means you can can compile it yourself, with enough work, and the maintainer will usually release a new wheel pretty quickly. In other cases, however, the code is incompatible and requires some work to support new Python releases.

For example, I’ve started porting the Fil memory profiler to 3.10, and the tests are failing. I expect to figure this out soon, but it still means extra delays, and many Python packages are maintained by volunteers with limited free time.

4. Lack of Conda support

If you’re using Conda, Python 3.10 is not available yet. Conda-Forge has started the work, but the main Python package still hasn’t had a 3.10 release yet, and once it does all Python packages will need to be updated and rebuilt for 3.10.

5. Bugs in Python

Python 3.9.0 was released in October 2020. 3.9.1 was released 2 months later, with a long list of bugfixes.

Of course, there’s always another bug fix release in the future, but given a 12-month window between major releases, lots of new code will get written but not extensively used. Which means a follow-up release with plenty of bug fixes. Python 3.10.1 will likely have many bug fixes as libraries and then applications start switching and encountering more issues.

6. Lack of toolchain support

New versions of Python often have new syntax, and that is the case with Python 3.10: you can use structural pattern matching. However, other tools need to support the new syntax too—autoformatters, linters, and so on. Until they do, you can’t use the new syntax.

For example, as I’m writing this the black code formatter doesn’t yet support the pattern matching syntax.

What version should you be using?

Given that it takes work to upgrade—some additional testing, some tweaks to your code—it can be tempting to put off upgrading Python versions indefinitely. Why worry about incompatibilities, new versions, and what not, when you can just stick with your current version indefinitely?

The problem is that Python isn’t supported indefinitely, nor do libraries support all Python versions indefinitely. So if you’re running on a 5-year-old version of Python, switching becomes a Big Deal—you’ll often end up dealing with more significant cross-version changes in both Python and in libraries at the same time. Which makes upgrading scary. And sooner or later you will have to upgrade.

Python 3.6, for example, is going to end security updates in December 2021. That’s just two months from the time I’m writing this article. It’s true, Linux distributions that ship it, like RHEL 8, will continue to provide security updates. But RHEL 8 also packages Python 3.8 and 3.9; you really shouldn’t be using 3.6 any more.

Instead of one massive scary upgrade event every few years, it’s much safer to have a continuous, ongoing process of smaller upgrades. Whenever a new major Python version comes out, or a new major library version, wait a bit, and then switch.

Which is to say that at the moment, you really should be using 3.9 if you can. If you’re far enough behind, do a series of upgrades: 3.7, 3.8, 3.9, and eventually, 3.10.

When should you switch to Python 3.10?

At a minimum, you will need to wait until:

1. All your libraries explicitly support the new Python release.
2. All the tools you rely on explicitly support the new Python release.

It’s probably worth waiting a couple months, until mid-December 2021, before even bothering to try your application. In addition to dependency availability and toolchain support, at that point you may also have access to the first point release, 3.10.1. If dependencies are still missing, keep trying again every month.