Using stdin, stdout, and stderr in Python

Introduction

Among the popular operating systems, they have all standardized on using standard input, standard output, and standard error with file desciptors 0, 1, and 2 respectively. This allows you to pipe the inputs and outputs to different locations. Let's look at how to utilize standard input, output, and error in Python.

To learn more about piping, redirection, stdin, stdout, and stderr in general, see my tutorial STDIN, STDOUT, STDERR, Piping, and Redirecting.

Basic usage

In Python, the sys.stdin, sys.stdout, and sys.stderr are file-like objects that can perform expected operations like read() and write(). Let's look at how to use these objects. Refer to the official sys package documentation for full information.

Standard output

Standard output

print(x) is basically a shortcut for sys.stdout.write(x + '\n')

import sys

# Standard output - sys.stdout
print(type(sys.stdout))
sys.stdout.write('Hello\n')

sys.stdout is a io.TextIOWrapper objects so you can read and write to them like a regular file. See https://docs.python.org/3/library/io.html#io.TextIOWrapper for more details about the io.TextIOWrapper class.

To pipe the output of your Python program to a file, you can do it from the shell like this:

python myapp.py > output.txt

Standard error

Standard error works just like standard output and can be used the same way. Standard error has file descriptor 2 where standard output has file descriptor 1. This is beneficial if you want to separate warning and error messages from the actual output of your application. For example, if your program outputs an XML file, you don't want error strings injected in the middle of your XML file.

# Standard error - sys.stderr
print(type(sys.stderr))
sys.stderr.write("Error messages can go here\n")

To pipe standard error from the shell to a file while leaving standard output going to the terminal:

python myapp.py 2>errors.txt

To pipe standard error in to standard output, you can do:

python myapp.py 2>&1

Standard input

Standard input defaults to your keyboard in the terminal, but you can also pipe in files or the output from a previous program to your standard input. Here is a basic example of reading one byte from standard input:

# Standard input - sys.stdin
print(type(sys.stdin))
letter = sys.stdin.read(1)  # Read 1 byte
print(letter)
# Can also do things like `sys.stdin.readlines()`

If you want interactive input from the user, it is better to use input() instead of sys.stdin.read() when asking for user input, but sys.stdin.readlines() can be useful for reading a file that was piped in from the shell like this:

# Feed `input_file.txt` to `sys.stdin` of the Python script
python my_script.py < input_file.txt

To pipe the standard output of one program to the standard input of your Python program, you can do it like this:

cat data.txt | python myapp.py

Dunder properties: sys.__stdin__, sys.__stdout__, sys.__stderr__

The dunder properties sys.__stdin__, sys.__stdout__ and sys.__stderr__ always contain references to the original streams. If you re-assign sys.stdout to point somewhere else like a StringIO object, you can always assign it back to the original value with the following.

Changing sys.stdout to a StringIO object can be useful especially when unit testing. Check out my tutorial Python Use StringIO to Capture STDOUT and STDERR.

from io import StringIO
import sys

temp_output = StringIO()

# Replace stdout with the StringIO object
sys.stdout = temp_output
# Now, if you print() or use sys.stdout.write
# it goes to the string objc
print('This is going to the StringIO obecjt.')
sys.stdout.write('This is not going to the "real" stdout, yet')

# Then we can restore original stdout
sys.stdout = sys.__stdout__
print("Contents of the StringIO object")
print("===============================")
print(temp_output.getvalue())

fileinput.input() shortcut

This function will return standard input separated by line, or if file names were provided as command-line arguments, it will provide all the lines from the files provided. It is similar to ARGF in Ruby. This gives you the option to pipe in a file from the shell or to provide a list of file paths for input.

For example, you can either pipe in files via standard input or provide a list of filenames as arguments to the application:

python my_app.py < file1.txt
python my_app.py file1.txt file2.txt file3.txt

Here is an example of it in a script:

# fileinput_example.py
import fileinput

lines_of_data = fileinput.input()
print(type(lines_of_data))  # fileinput.FileInput

# One option: Join each line together to one long string
print(''.join(lines_of_data))

# Another option: Iterate through each line
# for line in lines_of_data:
#     print(line.strip())

Here is how you can run the program to pipe in files or provide file names:

# Pipe file in via stdin
python fileinput_example.py < file1.txt

# Provide list of files as arguments
python fileinput_example.py file1.txt file2.txt file3.txt

Conclusion

After reading this guide, you should know how to access and read/write from standard input, standard output, and standard error in Python. You should also know how to use a StringIO object to capture output, and use the fileinput.input() function to get data.

References