Python's __all__: Packages, Modules, and Wildcard Imports

Importing Objects in Python

When creating a Python project or application, you’ll need a way to access code from the standard library or third-party libraries. You’ll also need to access your own code from the multiple files that may make up your project. Python’s import system is the mechanism that allows you to do this.

The import system lets you get objects in different ways. You can use:

• Explicit imports
• Wildcard imports

In the following sections, you’ll learn the basics of both strategies. You’ll learn about the different syntax that you can use in each case and the result of running an import statement.

Explicit Imports

In Python, when you need to get a specific object from a module or a particular module from a package, you can use an explicit import statement. This type of statement allows you to bring the target object to your current namespace so that you can use the object in your code.

To import a module by its name, you can use the following syntax:

import module [as name]

This statement allows you to import a module by its name. The module must be listed in Python’s import path, which is a list of locations where the path based finder searches when you run an import.

The part of the syntax that’s enclosed in square brackets is optional and allows you to create an alias of the imported name. This practice can help you avoid name collisions in your code.

As an example, say that you have the following module:

def add(a, b):
    return float(a + b)

def subtract(a, b):
    return float(a - b)

def multiply(a, b):
    return float(a * b)

def divide(a, b):
    return float(a / b)

This sample module provides functions that allow you to perform basic calculations. The containing module is called calculations.py. To import this module and use the functions in your code, go ahead and start a REPL session in the same directory where you saved the file.

Then run the following code:

>>> import calculations

>>> calculations.add(2, 4)
6.0
>>> calculations.subtract(8, 4)
4.0
>>> calculations.multiply(5, 2)
10.0
>>> calculations.divide(12, 2)
6.0

The import statement at the beginning of this code snippet brings the module name to your current namespace. To use the functions or any other object from calculations, you need to use fully qualified names with the dot notation.

import calculations as calc

import numpy as np
import pandas as pd

You can also use a similar syntax to import a Python package:

import package [as name]

In this case, Python loads the content of your package’s __init__.py file into your current namespace. If that file exports objects, then those objects will be available to you.

Finally, if you want to be more specific in what you import into your current namespace, then you can use the following syntax:

from module import name [as name]

With this import statement, you can import specific names from a given module. This approach is recommended when you only need a few names from a long module that defines many objects or when you don’t expect name collisions in your code.

To continue with the calculations module, you can import the needed function only:

>>> from calculations import add

>>> add(2, 4)
6.0

In this example, you only use the add() function. The from module import name syntax lets you import the target name explicitly. In this case, the rest of the functions and the module itself won’t be accessible in your namespace or scope.

Wildcard Imports on Modules

When you’re working with Python modules, a wildcard import is a type of import that allows you to get all the public names from a module in one go. This type of import has the following syntax:

from module import *

The name wildcard import derives from the asterisk at the end of the statement, which denotes that you want to import all the objects from module.

Go back to your terminal window and restart your REPL session. Then, run the following code:

>>> from calculations import *

>>> dir()
[
    ...
    'add',
    'divide',
    'multiply',
    'subtract'
]

In this code snippet, you first run a wildcard import. This import makes available all the names from the calculations modules and brings them to your current namespace. The built-in dir() function allows you to see what names are available in the current namespace. As you can confirm from the output, all the functions that live in calculations are now available.

When you’re completely sure that you need all the objects that a given module defines, using a wildcard import is a quick solution. In practice, this situation is rare, and you just end up cluttering your namespace with unneeded objects and names.

Using wildcard imports is explicitly discouraged in PEP 8 when they say:

Wildcard imports (from <module> import *) should be avoided, as they make it unclear which names are present in the namespace, confusing both readers and many automated tools. There is one defensible use case for a wildcard import, which is to republish an internal interface as part of a public API (for example, overwriting a pure Python implementation of an interface with the definitions from an optional accelerator module and exactly which definitions will be overwritten isn’t known in advance). (Source)

The main drawback of wildcard import is that you don’t have control over the imported objects. You can’t be specific. Therefore, you can confuse the users of your code and clutter their namespace with unnecessary objects.

Even though wildcard imports are discouraged, some libraries and tools use them. For example, if you search for applications built with Tkinter, then you’ll realize that many of the examples use the form:

from tkinter import *

This import gives you access to all the objects defined in the tkinter module, which is pretty convenient if you’re starting to learn how to use this tool.

You may find many other tools and third-party libraries that use wildcard imports for code examples in their documentation, and that’s okay. However, in real-world projects, you should avoid this type of import.

In practice, you can’t control how the users of your code will manage their imports. So, you better prepare your code for wildcard imports. You’ll learn how to do this in the upcoming sections. First, you’ll learn about using wildcard imports on packages.

Wildcard Import and Non-Public Names

Python has a well-established naming convention that allows you to tell the users of your code when a given name in a module is for internal or external use.

If an object’s name starts with a single leading underscore, then that name is considered non-public, so it’s for internal use only. In contrast, if a name starts with a lowercase or uppercase letter, then that name is public and, therefore, is part of the module’s public API.

When you have non-public names in a given module, you should know that wildcard imports won’t import those names. Say that you have the following module:

from math import pi as _pi

class Circle:
    def __init__(self, radius):
        self.radius = _validate(radius)

    def area(self):
        return _pi * self.radius**2

class Square:
    def __init__(self, side):
        self.side = _validate(side)

    def area(self):
        return self.side**2

def _validate(value):
    if not isinstance(value, int | float) or value <= 0:
        raise ValueError("positive number expected")
    return value

In this module, you have two non-public objects _pi and _validate(). You know this because they have a leading underscore in their names. If someone runs a wildcard import on this module, then the non-public names won’t be imported:

>>> from shapes import *

>>> dir()
[
    'Circle',
    'Square',
    ...
]

If you take a look at the output of dir(), then you’ll note that only the Circle and Square classes are available in your current namespace. The non-public objects, _pi and _validate(), aren’t available. So, wildcard imports won’t import non-public names.

Wildcard Import on Packages

Up to this point, you know how wildcard imports work with modules. You can also use this type of import with packages. In that case, the syntax is the same, but you need to use a package name rather than a module name:

from package import *

Now, what happens when you run this type of import? You may expect that this import causes Python to search the file system, find the modules and subpackages that are present in the package, and import them.

However, doing this file system search could take a long time. Additionally, importing modules might have unwanted side effects, because when you import a module, all the executable code in that module runs.

Because of these potential issues, Python has the __all__ special variable, which will allow you to explicitly define the list of modules that you want to expose to wildcard import in a given package. You’ll explore the details in the next section.