Unit Testing a Flask Application

Introduction

This blog post provides an introduction to unit testing a Flask application.  I’ve been on a big unit testing kick at work recently which is spilling over into updating the unit tests for my personal projects.  Therefore, I thought it would be a good time to document the basics of unit testing a Flask application.

There seems to be a lot of opinions about unit testing amongst software developers, but I believe the following:

• 100% code coverage using unit tests does not mean your code works, but it’s a good indicator that the code has been developed well
• No unit tests mean that the code does not work and cannot be expanded upon

The second belief might be a bit extreme, but if I inherit a software module that does not have unit tests, I’m already assuming that it does not run properly and has not been maintained.  Simply put, the lack of unit tests leads to a negative impression of the software module.

I believe that you should create an environment for developing unit test that is both easy to use and fun to use. These may sound like very fluffy words, but they have meaning for me:

• create helper methods to allow quick generation of unit tests
• provide quick feedback on progress of unit testing
• create an environment that allows for tests to be added quickly
• make sure that the unit tests can be executed with a single command
• provide visual feedback of path coverage

As with so many things in software development, if it’s easy to use, it will be used. If it’s tough to use, it won’t be used as much as it should be.

Unit Test Frameworks

The two unit test frameworks for python that I’ve used are:

• unittest – built-in unit test framework that is based on the xUnit framework
• py.test – module for building unit tests

Both work really well, but I tend to prefer unittest since it’s built-in to the python library (there is a wealth of incredible modules built-in to python!).  I’ll be using unittest throughout this blog post.

If you want to see just how many options there are for tools to help with unit testing in python, check out this listing of python testing tools.  An incredible number of tools available!

Where to Store Your Unit Tests

Based on the flexibility that using unit test runners gives you, you could probably store your unit test files in any location in your Flask application. However, I find it best to store the unit tests (in this case located in the ‘tests’ directory) at the same level as the files that you are going to be testing:

In this structure, the unit tests are stored in the ‘tests’ directory and the unit tests are going to be focused on testing the functionality in the ‘recipes’ and ‘users’ modules, which are at the same level.

Creating a Basic Unit Test File

There are a lot of important principles to follow when writing unit tests, but I really believe it’s important to create an environment that makes writing unit tests easy. I’m a fan of writing unit tests, but I know a lot of software developers that just don’t like it. Therefore, I think it’s important to develop a unit testing structure that has a lot ‘helper’ functions to facilitate writing unit tests.

With that in mind, let’s create a simple unit test file that is not testing our Flask application, but is simply showing the structure of a unit test:

This unit test file creates a class, BasicTests, based on the unittest.TestCase class. Within this class, the setUp() and tearDown() methods are defined. This is really critical, as the setUp() method is called prior to each unit test executing and the tearDown() method is called after each unit test finishes executing.

Since we want to test out the functionality of the web application, this unit test uses a SQLite database for storing data instead of the typical Postgres database that we’ve been using. Why I’ve seen lots of discussions about whether or not this is a good idea, I feel like it really simplifies the development of the unit tests, so I’m in favor of it.

Running the Unit Tests

If you want to run the unit test that was just created, the easiest way is to just execute that file:

Please take note of the directory that the unit test was run from (top-level folder of the Flask application). Why can’t we just run from the directory where the unit test file is located? We would have this option available for simple unit test, but since we’ll importing the ‘app’, ‘db’, and ‘mail’ objects, we need to be at a location where those are discoverable. Therefore, running at the top-level directory of the Flask application allows the python interpreter to find these objects within the ‘project’ directory.

Taking it one step further, I’d highly recommend using Nose2 as the unit test runner. A unit test runner provides the ability to easily detect the unit tests in your project and then execute them.

The easiest way to run Nose2 is simply to call the executable from the top-level directory:

This command will find all of the unit tests (as long as the files start with test_*.py) and execute them. I’m a bit preferential to using the verbose mode:

If you just want to run a single unit test file, you can still use Nose2:

I’ve found Nose2 to be so easy to use and I highly recommend using it for running your unit tests.

Adding Helper Methods to Facilitate Developing More Unit Tests

For the unit testing in this blog post, we’re going to be focused on testing some of the user management aspects of the Flask application. As such, we’re likely going to be doing a lot of logging in, logging out, and registering for a new account. Let’s simplify the development of future unit tests by creating some helper methods to perform these steps:

The ‘register’ method registers a new user by sending a POST command to the ‘/register’ URL of the Flask application. The ‘login’ method logs a user in by sending a POST command to the ‘/login’ URL of the Flask application. The ‘logout’ method logs a user out of the application by sending a GET command to the ‘/logout’ URL of the Flask application. These will come in handy when we…

…add a test to make sure we can register a new user:

This unit test checks that we can successfully register a new user and receive a positive confirmation. We’re testing a nominal situation here, but it’s also good to check what happens when off-nominal data is provided to the application. Let’s try registering a new user where the confirmation password does not match the original password:

Try running the unit tests and you should see that all three unit tests pass:

One of the important checks that is in the application is to prevent duplicate emails being used for registration. Let’s check that this functionality works:

Running the unit tests:

Excellent!

Please take note that the order that the unit tests are listed in test_*.py does not determine the order in which the unit tests are run. This is really important to understand, as each unit test needs to be self-contained. Don’t count on unit test #1 registering a new user and unit test #2 logging that user in.

Code Coverage

In order to check the code coverage of your unit tests, I’d recommend using the Coverage module. This module works great with unit tests that have been written using the unittest model.

Start by installing the Coverage module and updating your list of modules for your project:

The order of commands to execute when using the coverage module is:

1. coverage run …
2. coverage report …

The ‘run’ command runs the unit tests and collects the data for determining the code coverage. The ‘report’ command shows a basic text output of the code coverage.

For example, I’ve written some unit tests for the application and the here is how I use the Coverage module:

If you add the -m flag, you can see which lines are not being tested:

This text-based output is nice to get a summary of the code coverage, but just seeing line numbers that are not being testing in a file is just not that beneficial. Luckily, the Coverage module is able to generate HTML to provide better insight into which lines are being tested and which are not. You can generate the HTML output by running:

Now you can go to navigate to your project’s folder, open the newly created ‘htmlcov’ directory, and open the index.html file. In your web browser, you should see a summary of the path coverage for this directory:

By clicking on views.py, you get a detailed view of the file contents including a color-coded line-by-line view of which lines are tested by the unit tests and which are not:

Conclusion

This blog post was intended to show you how to develop unit tests for a Flask application with a focus on using the right tools to make the process of writing and running unit tests as easy as possible. The recommended modules for unit testing a Flask application are:
– unittest – built-in python module for developing unit tests
– nose2 – runner for identifying and running the unit tests
– Coverage – seeing the code coverage of your unit tests

To recap, I think that the following aspects are needed to have a functioning setup for writing and running unit tests:
– create helper methods to allow quick generation of unit tests
– provide quick feedback on progress of unit testing
– create an environment that allows for tests to be added quickly
– make sure that the unit tests can be executed with a single command
– provide visual feedback of path coverage

I’ve found that unit tests are critical to testing out any software module that I write. Writing unit tests should be an efficient process that can be greatly simplified by setting up a proper infrastructure.

For the source code utilized in this blog post, please see my GitLab repository.