Effective Python Testing With Pytest

Testing your code brings a wide variety of benefits. It increases your confidence that the code behaves as you expect and ensures that changes to your code won’t cause regressions. Writing and maintaining tests is hard work, so you should leverage all the tools at your disposal to make it as painless as possible. pytest is one of the best tools you can use to boost your testing productivity.

In this tutorial, you’ll learn:

• What benefits pytest offers
• How to ensure your tests are stateless
• How to make repetitious tests more comprehensible
• How to run subsets of tests by name or custom groups
• How to create and maintain reusable testing utilities

Free Bonus: 5 Thoughts On Python Mastery , a free course for Python developers that shows you the roadmap and the mindset you'll need to take your Python skills to the next level.

How to Install pytest

To follow along with some of the examples in this tutorial, you’ll need to install pytest . As with most Python packages, you can install pytest in avirtual environment fromPyPI using pip :

$ python -m pip install pytest

The pytest command will now be available in your installation environment.

What Makes pytest So Useful?

If you’ve written unit tests for your Python code before, then you may have used Python’s built-in unittest module. unittest provides a solid base on which to build your test suite, but it has a few shortcomings.

A number of third-party testing frameworks attempt to address some of the issues with unittest , and pytest has proven to be one of the most popular . pytest is a feature-rich, plugin-based ecosystem for testing your Python code.

If you haven’t had the pleasure of using pytest yet, then you’re in for a treat! Its philosophy and features will make your testing experience more productive and enjoyable. With pytest , common tasks require less code and advanced tasks can be achieved through a variety of time-saving commands and plugins. It will even run your existing tests out of the box, including those written with unittest .

As with most frameworks, some development patterns that make sense when you first start using pytest can start causing pains as your test suite grows. This tutorial will help you understand some of the tools pytest provides to keep your testing efficient and effective even as it scales.

Less Boilerplate

Most functional tests follow the Arrange-Act-Assert model:

1. Arrange , or set up, the conditions for the test
2. Act by calling some function or method
3. Assert that some end condition is true

Testing frameworks typically hook into your test’sassertions so that they can provide information when an assertion fails. unittest , for example, provides a number of helpful assertion utilities out of the box. However, even a small set of tests requires a fair amount of boilerplate code .

Imagine you’d like to write a test suite just to make sure unittest is working properly in your project. You might want to write one test that always passes and one that always fails:

# test_with_unittest.py

from unittest import TestCase

class TryTesting(TestCase):
    def test_always_passes(self):
        self.assertTrue(True)

    def test_always_fails(self):
        self.assertTrue(False)

You can then run those tests from the command line using the discover option of unittest :

$ python -m unittest discover
F.
============================================================
FAIL: test_always_fails (test_with_unittest.TryTesting)
------------------------------------------------------------
Traceback (most recent call last):
  File "/.../test_with_unittest.py", line 9, in test_always_fails
    self.assertTrue(False)
AssertionError: False is not True

------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (failures=1)

As expected, one test passed and one failed. You’ve proven that unittest is working, but look at what you had to do:

1. Import the TestCase class from unittest
2. Create TryTesting , asubclass of TestCase
3. Write a method in TryTesting for each test
4. Use one of the self.assert* methods from unittest.TestCase to make assertions

That’s a significant amount of code to write, and because it’s the minimum you need for any test, you’d end up writing the same code over and over. pytest simplifies this workflow by allowing you to use Python’s assert keyword directly:

# test_with_pytest.py

def test_always_passes():
    assert True

def test_always_fails():
    assert False

That’s it. You don’t have to deal with any imports or classes. Because you can use the assert keyword, you don’t need to learn or remember all the different self.assert* methods in unittest , either. If you can write an expression that you expect to evaluate to True , then pytest will test it for you. You can run it using the pytest command:

$ pytest
================== test session starts =============================
platform darwin -- Python 3.7.3, pytest-5.3.0, py-1.8.0, pluggy-0.13.0
rootdir: /.../effective-python-testing-with-pytest
collected 2 items

test_with_pytest.py .F                                          [100%]

======================== FAILURES ==================================
___________________ test_always_fails ______________________________

    def test_always_fails():
>       assert False
E       assert False

test_with_pytest.py:5: AssertionError
============== 1 failed, 1 passed in 0.07s =========================

pytest presents the test results differently than unittest . The report shows:

pytest
rootdir

The output then indicates the status of each test using a syntax similar to unittest :

• A dot ( . ) means that the test passed.
• An F means that the test has failed.
• An E means that the test raised an unexpected exception.

For tests that fail, the report gives a detailed breakdown of the failure. In the example above, the test failed because assert False always fails. Finally, the report gives an overall status report of the test suite.

Here are a few more quick assertion examples:

def test_uppercase():
    assert "loud noises".upper() == "LOUD NOISES"

def test_reversed():
    assert list(reversed([1, 2, 3, 4])) == [4, 3, 2, 1]

def test_some_primes():
    assert 37 in {
        num
        for num in range(1, 50)
        if num != 1 and not any([num % div == 0 for div in range(2, num)])
    }

The learning curve for pytest is shallower than it is for unittest because you don’t need to learn new constructs for most tests. Also, the use of assert , which you may have used before in your implementation code, makes your tests more understandable.

State and Dependency Management

Your tests will often depend on pieces of data or test doubles for some of the objects in your code. In unittest , you might extract these dependencies into setUp() and tearDown() methods so each test in the class can make use of them. But in doing so, you may inadvertently make the test’s dependence on a particular piece of data or object entirely implicit .

Over time, implicit dependencies can lead to a complex tangle of code that you have to unwind to make sense of your tests. Tests should help you make your code more understandable. If the tests themselves are difficult to understand, then you may be in trouble!

pytest takes a different approach. It leads you toward explicit dependency declarations that are still reusable thanks to the availability of fixtures . pytest fixtures are functions that create data or test doubles or initialize some system state for the test suite. Any test that wants to use a fixture must explicitly accept it as an argument, so dependencies are always stated up front.

Fixtures can also make use of other fixtures, again by declaring them explicitly as dependencies. That means that, over time, your fixtures can become bulky and modular. Although the ability to insert fixtures into other fixtures provides enormous flexibility, it can also make managing dependencies more challenging as your test suite grows. Later in this tutorial, you’ll learnand try a few techniques for handling these challenges.

Test Filtering

As your test suite grows, you may find that you want to run just a few tests on a feature and save the full suite for later. pytest provides a few ways of doing this:

• Name-based filtering : You can limit pytest to running only those tests whose fully qualified names match a particular expression. You can do this with the -k parameter.
• Directory scoping : By default, pytest will run only those tests that are in or under the current directory.
• Test categorization : pytest can include or exclude tests from particular categories that you define. You can do this with the -m parameter.

Test categorization in particular is a subtly powerful tool. pytest enables you to create marks , or custom labels, for any test you like. A test may have multiple labels, and you can use them for granular control over which tests to run. Later in this tutorial, you’ll see an example of how pytest marks work and learn how to make use of them in a large test suite.

Test Parametrization

When you’re testing functions that process data or perform generic transformations, you’ll find yourself writing many similar tests. They may differ only in the input or output of the code being tested. This requires duplicating test code, and doing so can sometimes obscure the behavior you’re trying to test.

unittest offers a way of collecting several tests into one, but they don’t show up as individual tests in result reports. If one test fails and the rest pass, then the entire group will still return a single failing result. pytest offers its own solution in which each test can pass or fail independently. You’ll seehow to parametrize testswith pytest later in this tutorial.

Plugin-Based Architecture

One of the most beautiful features of pytest is its openness to customization and new features. Almost every piece of the program can be cracked open and changed. As a result, pytest users have developed a rich ecosystem of helpful plugins.

Although some pytest plugins focus on specific frameworks like Django , others are applicable to most test suites. You’ll seedetails on some specific pluginslater in this tutorial.

Fixtures: Managing State and Dependencies

pytest fixtures are a way of providing data, test doubles, or state setup to your tests. Fixtures are functions that can return a wide range of values. Each test that depends on a fixture must explicitly accept that fixture as an argument.

When to Create Fixtures

Imagine you’re writing a function, format_data_for_display() , to process the data returned by an API endpoint. The data represents a list of people, each with a given name, family name, and job title. The function should output a list of strings that include each person’s full name (their given_name followed by their family_name ), a colon, and their title . To test this, you might write the following code:

def format_data_for_display(people):
    ...  # Implement this!

def test_format_data_for_display():
    people = [
        {
            "given_name": "Alfonsa",
            "family_name": "Ruiz",
            "title": "Senior Software Engineer",
        },
        {
            "given_name": "Sayid",
            "family_name": "Khan",
            "title": "Project Manager",
        },
    ]

    assert format_data_for_display(people) == [
        "Alfonsa Ruiz: Senior Software Engineer",
        "Sayid Khan: Project Manager",
    ]

Now suppose you need to write another function to transform the data into comma-separated values for use in Excel. The test would look awfully similar:

def format_data_for_excel(people):
    ... # Implement this!

def test_format_data_for_excel():
    people = [
        {
            "given_name": "Alfonsa",
            "family_name": "Ruiz",
            "title": "Senior Software Engineer",
        },
        {
            "given_name": "Sayid",
            "family_name": "Khan",
            "title": "Project Manager",
        },
    ]

    assert format_data_for_excel(people) == """given,family,title
Alfonsa,Ruiz,Senior Software Engineer
Sayid,Khan,Project Manager
"""

If you find yourself writing several tests that all make use of the same underlying test data, then a fixture may be in your future. You can pull the repeated data into a single function decorated with @pytest.fixture to indicate that the function is a pytest fixture:

import pytest

@pytest.fixture
def example_people_data():
    return [
        {
            "given_name": "Alfonsa",
            "family_name": "Ruiz",
            "title": "Senior Software Engineer",
        },
        {
            "given_name": "Sayid",
            "family_name": "Khan",
            "title": "Project Manager",
        },
    ]

You can use the fixture by adding it as an argument to your tests. Its value will be the return value of the fixture function:

def test_format_data_for_display(example_people_data):
    assert format_data_for_display(example_people_data) == [
        "Alfonsa Ruiz: Senior Software Engineer",
        "Sayid Khan: Project Manager",
    ]

def test_format_data_for_excel(example_people_data):
    assert format_data_for_excel(example_people_data) == """given,family,title
Alfonsa,Ruiz,Senior Software Engineer
Sayid,Khan,Project Manager
"""

Each test is now notably shorter but still has a clear path back to the data it depends on. Be sure to name your fixture something specific. That way, you can quickly determine if you want to use it when writing new tests in the future!

When to Avoid Fixtures

Fixtures are great for extracting data or objects that you use across multiple tests. They aren’t always as good for tests that require slight variations in the data. Littering your test suite with fixtures is no better than littering it with plain data or objects. It might even be worse because of the added layer of indirection.

As with most abstractions, it takes some pr