Software Development Checklist for Python Applications

Introduction

After resurrecting one of the first python applications that I wrote, I realized that I had learned a lot about the python language and ecosystem since my first adventure with the language. After going through the updates that I wanted to make to this application, I realized that I had created a checklist of software development concepts that would be beneficial to most python projects.

This list is inspired by the classic article by Joel Spolsky entitled “The Joel Test: 12 Steps to Better Code“. I provide some narrative around each checklist item, including links to more detailed descriptions or tutorials to help with learning more about the concept.

Checklist

1. Are you using virtual environments to manage the packages used by your application?

Virtual environments are a must for every python project. Virtual environments isolate the packages used for a specific application, so you can easily use different versions of packages for different projects.

I’ve been really happy using virtualenvwrapper. I wrote a blog post about how to use the virtualenvwrapper module.

As of python 3.3, there is now a standard library module (venv) available for creating virtual environments.

2. Are you utilizing a source control tool to manage your source code?

The de facto source control tool is git. It is widely used because it is a great tool. I highly recommend learning the more complex aspects of git to gain the benefits of this powerful tool. Git Pro is a great book for learning the details of git.

3. Are you using a centralized source repository (GitLab, GitHub, BitBucket) to enable collaboration and sharing?

I’m a big fan of GitLab as it has so many features combined (source control, bug tracking, and CI). However, all of the popular choices (GitLab, GitHub, and BitBucket) are good choices.

For an example of a GitLab project, here is the project that I’ve updated as I developed this checklist: https://gitlab.com/patkennedy79/picture_video_organizer

4. Do you have a README file to assist both active developers and first-time viewers?

The README file is often the first impression that people get of your project.

I’ve found the a good README file can benefit both the developers of a project and the people viewing the project for the first time. A good README file should appeal to both groups by having the following:

• Purpose of the project
• Screenshot or visual aid of the project
• How to run the application and run the tests for the project
• Current status of the project (test status: pass/fail, test coverage)

For a good overview of what the README should contain, I recommend the following blog post from Dan Bader: Write a Great README.

5. Are you testing (unit/integration/function) the application?

Testing a python application has become quite easy and there are lots of great options to help with quickly writing and running tests. In terms of frameworks, both unittest (part of standard library) and pytest are great choices. Additionally, I’ve used nose2 as a unit test runner to easily run my unit tests.

I wrote a blog post about how to use the unittest module for writing unit tests.

If you’re using the Flask web framework, check out my blog post about writing tests for a Flask application.

6. Are you using at least one static analysis tool to check for common errors in your application?

Linters are static analysis tools that check your source code for coding style errors, design issues, and bad design patterns. There are a lot of great options available, but I would recommend the following modules:

– flake8
– pydocstyle

The best way to run these tools is via a Continuous Integration (CI) tool…

7. Are you using Continuos Integration (CI) to automate your testing?

Setting up a Continuous Integration (CI) process just makes life easier. It’s easy to set up a CI process to run all your tests (unit/integration) and run linters. You no longer have to worry about remembering to run all your test cases and check the linter results manually. There are lots of great options for CI: TravisCI, Jenkins, GitLab, etc.

I have a CI system configured on GitLab for a command-line (CLI) application that runs the unit tests and executes two linters (flake8 and pydocstyle) whenever a new commit is made. This is all defined in a single file (.gitlab-ci.yml).

8. Are you thoroughly documenting your code, including the use of a document generation tool (Sphinx)?

The documentation of your project is important for getting other people to understand your source code, but it also really important for the developers who are trying to remember what they did in a specific module six months ago.

I recommend using Sphinx to help generate nice documentation. I wrote a blog post about how to setup Sphinx for a python project.

9. Are you using a logging tool to track the execution of your application?

Using print statements to debug an application might work for getting something to work, but I really prefer using a logging tool to track the execution of an application. The standard library module (logging) is a bit complex to setup, but it does work quite well.

Check out my blog post about how to use the logging module for more information.

10. Are you tracking bugs and enhancement ideas?

Assuming that you’re using a centralized source repository (GitLab, GitHub, BitBucket), there should be a built-in system for tracking defects and enhancements. This is pretty simple… you need to document what bugs there are with your project so that you can work down these issues; you need to document good ideas to enhance your project so that you know what improvements to make.

11. Can you easily run the application with limited introduction to the tool?

The ideal situation for an application is being able to run it with a single command. If you need to execute multiple, manual steps, you’re just asking for weird problems to arise and for people to loss interest in your project.

While there might need to be some configuration for your application, this process should be easy to follow. You should ideally be able to run your application with a single command.

Honorable Mentions (or Future Checklist Items)

1. Docker

The use of containers for running and deploying applications is a great idea, especially for web applications. There is a lot to learn when it comes to Docker, so I would recommend starting out with the book “Docker for Developers” by Chris Tankersley. This book provides a great introduction into the history of containers and clearly describes all the different aspects/components of Docker.

I’ve recently switched to use Docker for developing and deploying my Flask web applications, which I documented the why and how in separate blog posts.

2. Debugging

At some point in the development of a project, you are likely to run into a difficult problem that can’t be easily solved using print statements or logging statements. This might be the time to use a debugger to understand the details of how the program is executing. The pdb module is a standard library module for debugging python source code.

A great resource for learning about pdb is the tutorial from the Python Module of the Week site: https://pymotw.com/2/pdb/

3. Profiling

Profiling is a dynamic (ie. while your application is running) analysis that measures how long it takes all the pieces of your application to run. This analysis can be really beneficial to help identify parts of your source code that are taking up large amounts of execution time (it’s often not the parts that you suspect!).

Conclusion

As I was coming up with this list, I realized just how amazing the python ecosystem has become. The number of great tools that are available for building web applications, writing unit tests, performing static analysis, etc. is just incredible.

Having a good understanding of the python modules that are available for you can really help save you tons of time in terms of testing, documenting, and maintaining your source code.