Python subprocess
last modified May 25, 2026
In this article, we show how to use the subprocess module to run external commands and manage processes in Python.
The subprocess module enables you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. It is the recommended way to run shell commands from within Python programs.
The subprocess module is part of Python's standard library, so no additional
installation is required. It replaces older modules such as os.system
and os.spawn*.
Running a simple command
The subprocess.run function is the recommended way to execute
external commands. The following example runs the system ls command
to list files in the current directory.
import subprocess
result = subprocess.run(["ls", "-l"])
print(f"Return code: {result.returncode}")
The command and its arguments are passed as a list of strings. The
run function waits for the command to complete and returns
a CompletedProcess instance.
import subprocess
We import the subprocess module.
result = subprocess.run(["ls", "-l"])
We execute the ls -l command. The command line arguments are
specified as a list, where the first element is the program name and the
following elements are its arguments.
print(f"Return code: {result.returncode}")
We print the return code of the completed process. A zero means success.
$ python main.py total 12 -rw-rw-r-- 1 jano jano 202 May 25 12:00 main.py -rw-rw-r-- 1 jano jano 182 May 25 12:00 data.txt Return code: 0
Capturing command output
By default, subprocess.run prints the output directly to the
console. To capture the output for further processing, we set
capture_output=True and text=True.
import subprocess
result = subprocess.run(["echo", "Hello from subprocess"],
capture_output=True, text=True)
print(f"stdout: {result.stdout.strip()}")
print(f"stderr: {result.stderr}")
print(f"Return code: {result.returncode}")
The capture_output=True parameter redirects stdout and stderr
into the result.stdout and result.stderr attributes.
The text=True parameter returns the output as a string instead
of bytes.
result = subprocess.run(["echo", "Hello from subprocess"],
capture_output=True, text=True)
We run the echo command and capture its output. With
capture_output=True, the output is stored in the result object.
With text=True, the output is decoded to a string.
print(f"stdout: {result.stdout.strip()}")
print(f"stderr: {result.stderr}")
We access the standard output and standard error of the process. The
strip method removes the trailing newline.
$ python main.py stdout: Hello from subprocess stderr: Return code: 0
Checking for errors
When a command fails, it returns a non-zero exit code. The
check=True parameter makes run raise a
CalledProcessError if the command exits with a non-zero status.
import subprocess
try:
subprocess.run(["ls", "/nonexistent"], check=True, capture_output=True, text=True)
except subprocess.CalledProcessError as e:
print(f"Command failed with exit code {e.returncode}")
print(f"stderr: {e.stderr.strip()}")
The program attempts to list a non-existent directory and catches the resulting error.
try:
subprocess.run(["ls", "/nonexistent"], check=True, capture_output=True, text=True)
except subprocess.CalledProcessError as e:
We wrap the call in a try/except block. With check=True,
a non-zero exit code raises CalledProcessError.
print(f"Command failed with exit code {e.returncode}")
print(f"stderr: {e.stderr.strip()}")
The exception object provides the return code and the captured stderr output.
$ python main.py Command failed with exit code 2 stderr: ls: cannot access '/nonexistent': No such file or directory
Passing input to a process
The input parameter sends data to the process's standard input.
This is useful for feeding data to commands that read from stdin.
import subprocess
data = "apple\nbanana\ncherry\napple\ndate\n"
result = subprocess.run(["sort", "-u"],
input=data, capture_output=True, text=True)
print("Sorted unique lines:")
print(result.stdout)
We pass a multi-line string to the sort -u command, which sorts
the lines and removes duplicates.
data = "apple\nbanana\ncherry\napple\ndate\n"
We prepare a string with several fruit names, one per line. Note that
apple appears twice.
result = subprocess.run(["sort", "-u"],
input=data, capture_output=True, text=True)
The input parameter feeds the string to the process's stdin.
The -u flag tells sort to output only unique lines.
$ python main.py Sorted unique lines: apple banana cherry date
Running a command in a different directory
The cwd parameter specifies the working directory for the
executed command.
import subprocess
import os
current = os.getcwd()
print(f"Current directory: {current}")
result = subprocess.run(["pwd"], capture_output=True, text=True,
cwd="/tmp")
print(f"pwd from /tmp: {result.stdout.strip()}")
result = subprocess.run(["ls", "-d", "/etc/*.conf"], capture_output=True, text=True,
cwd="/tmp")
print("\nConfiguration files in /etc:")
print(result.stdout)
The cwd parameter changes the working directory before the
command executes. The current directory of the Python script itself remains
unchanged.
result = subprocess.run(["pwd"], capture_output=True, text=True,
cwd="/tmp")
We run the pwd command with the working directory set to
/tmp. The command prints /tmp even though the
Python script runs from a different location.
$ python main.py Current directory: /home/jano/Documents/zetcode-remote/python/subprocess pwd from /tmp: /tmp Configuration files in /etc: /etc/adduser.conf /etc/ca-certificates.conf /etc/deluser.conf /etc/host.conf ...
Setting environment variables
The env parameter sets environment variables for the child
process. When provided, it completely replaces the current environment;
to extend it, pass a modified copy of os.environ.
import subprocess
import os
custom_env = os.environ.copy()
custom_env["APP_MODE"] = "production"
custom_env["DEBUG"] = "false"
result = subprocess.run(["bash", "-c", "echo Mode: $APP_MODE; echo Debug: $DEBUG"],
capture_output=True, text=True, env=custom_env)
print(result.stdout)
We create a custom environment by copying the current one and adding our own variables. The child process then uses this modified environment.
custom_env = os.environ.copy() custom_env["APP_MODE"] = "production" custom_env["DEBUG"] = "false"
We copy the current process environment with os.environ.copy()
and add two custom variables.
result = subprocess.run(["bash", "-c", "echo Mode: $APP_MODE; echo Debug: $DEBUG"],
capture_output=True, text=True, env=custom_env)
We run a bash inline script that prints the environment variables. The
env parameter passes our custom environment to the child process.
$ python main.py Mode: production Debug: false
Setting a timeout
The timeout parameter limits the execution time of a command.
If the command does not complete within the specified number of seconds,
a TimeoutExpired exception is raised.
import subprocess
import sys
try:
# This command sleeps for 10 seconds, but we only wait 3
result = subprocess.run(
["sleep", "10"],
capture_output=True, text=True, timeout=3
)
except subprocess.TimeoutExpired as e:
print(f"Command timed out after {e.timeout} seconds")
print(f"Command: {' '.join(e.cmd)}")
The program runs sleep 10 but sets a timeout of 3 seconds.
The command is killed when the timeout expires.
result = subprocess.run(
["sleep", "10"],
capture_output=True, text=True, timeout=3
)
We run the sleep command with a 3-second timeout. Since the
command needs 10 seconds to complete, it will be terminated early.
except subprocess.TimeoutExpired as e:
print(f"Command timed out after {e.timeout} seconds")
The TimeoutExpired exception has a timeout attribute
indicating how many seconds were allowed.
$ python main.py Command timed out after 3 seconds Command: sleep 10
Using shell mode
The shell=True parameter executes the command through the system
shell. This allows shell features like wildcard expansion, piping, and
environment variable expansion.
import subprocess
# Count Python files using shell pipeline and wildcards
cmd = "ls -1 *.py 2>/dev/null | wc -l"
result = subprocess.run(cmd, shell=True, capture_output=True, text=True)
print(f"Number of Python files: {result.stdout.strip()}")
When using shell=True, the command is passed as a single string.
This mode is convenient but should be used carefully with user-supplied input
to avoid shell injection vulnerabilities.
cmd = "ls -1 *.py 2>/dev/null | wc -l"
We define a shell command that uses a wildcard (*.py) and a
pipe (|). These shell features are only available with
shell=True.
result = subprocess.run(cmd, shell=True, capture_output=True, text=True)
The shell=True parameter tells subprocess to execute the command
through /bin/sh. The command is provided as a single string.
$ python main.py Number of Python files: 2
Using Popen for more control
The Popen class provides more control over process execution.
Unlike run, Popen does not wait for the process to
complete, allowing you to interact with it while it runs.
import subprocess
proc = subprocess.Popen(["ping", "-c", "3", "localhost"],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True)
print("Process is running...")
stdout, stderr = proc.communicate()
print(f"Exit code: {proc.returncode}")
print("\nOutput:")
print(stdout)
Popen starts the process and returns immediately. The
communicate method then reads all output and waits for the
process to finish.
proc = subprocess.Popen(["ping", "-c", "3", "localhost"],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True)
We create a Popen instance to run the ping command.
stdout=PIPE and stderr=PIPE redirect the output
streams so we can capture them.
stdout, stderr = proc.communicate()
The communicate method reads all output from the process and
waits for it to finish. It returns a tuple of (stdout, stderr).
$ python main.py Process is running... Exit code: 0 Output: PING localhost (127.0.0.1) 56(84) bytes of data. 64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.042 ms 64 bytes from localhost (127.0.0.1): icmp_seq=2 ttl=64 time=0.062 ms 64 bytes from localhost (127.0.0.1): icmp_seq=3 ttl=64 time=0.065 ms --- localhost ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2035ms rtt min/avg/max/mdev = 0.042/0.056/0.065/0.010 ms
Reading output line by line
With Popen, we can read output line by line as the process runs,
which is useful for long-running commands where we want to show progress.
import subprocess
proc = subprocess.Popen(["ping", "-c", "4", "localhost"],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True)
for line in proc.stdout:
print(f">> {line}", end="")
proc.wait()
print(f"\nProcess finished with exit code: {proc.returncode}")
We iterate over proc.stdout to read lines as they become
available. The wait method ensures the process has completed
before we check the return code.
proc = subprocess.Popen(["ping", "-c", "4", "localhost"],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True)
We redirect stderr to stdout using stderr=subprocess.STDOUT,
so all output is available through a single stream. The text=True
parameter returns strings rather than bytes.
for line in proc.stdout:
print(f">> {line}", end="")
Iterating over proc.stdout yields lines as the process produces
them. Each line is prefixed with >> for demonstration.
$ python main.py >> PING localhost (127.0.0.1) 56(84) bytes of data. >> 64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.048 ms >> 64 bytes from localhost (127.0.0.1): icmp_seq=2 ttl=64 time=0.062 ms >> 64 bytes from localhost (127.0.0.1): icmp_seq=3 ttl=64 time=0.097 ms >> 64 bytes from localhost (127.0.0.1): icmp_seq=4 ttl=64 time=0.054 ms >> >> --- localhost ping statistics --- >> 4 packets transmitted, 4 received, 0% packet loss, time 3066ms >> rtt min/avg/max/mdev = 0.048/0.065/0.097/0.019 ms >> Process finished with exit code: 0
Piping between processes
We can pipe the output of one process directly into another using
Popen. This replicates shell-style piping but without
invoking the shell.
import subprocess
ps = subprocess.Popen(["ps", "aux"], stdout=subprocess.PIPE, text=True)
grep = subprocess.Popen(["grep", "python"],
stdin=ps.stdout,
stdout=subprocess.PIPE,
text=True)
ps.stdout.close()
output, _ = grep.communicate()
print("Python processes:")
print(output)
The first process lists all running processes. Its stdout is piped to the stdin of the second process, which filters for lines containing "python".
ps = subprocess.Popen(["ps", "aux"], stdout=subprocess.PIPE, text=True)
grep = subprocess.Popen(["grep", "python"],
stdin=ps.stdout,
stdout=subprocess.PIPE,
text=True)
We create two processes. The ps process's stdout is connected
to grep's stdin. This creates a direct pipe between them.
ps.stdout.close()
We close the ps.stdout reference in the parent process so that
ps receives SIGPIPE if grep exits before
ps finishes writing.
$ python main.py Python processes: jano 5432 0.5 1.2 123456 78901 ? Sl 10:00 0:01 /usr/bin/python3 /usr/bin/code-server jano 8765 0.1 0.3 54321 23456 pts/0 S+ 12:00 0:00 python main.py
Suppressing output with DEVNULL
Sometimes we want to discard a process's output entirely.
subprocess.DEVNULL redirects streams to the system's null device.
import subprocess
# Run a command and discard all output
result = subprocess.run(["ping", "-c", "2", "localhost"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL)
if result.returncode == 0:
print("Ping successful (output discarded)")
else:
print("Ping failed")
Both stdout and stderr are redirected to /dev/null, so nothing
is printed to the console. We only check the return code to determine success.
$ python main.py Ping successful (output discarded)
Running a Python script as a subprocess
The subprocess module can run other Python scripts as child processes. This is useful for parallel execution or isolating tasks.
import sys
import time
name = sys.argv[1]
seconds = int(sys.argv[2])
print(f"{name}: sleeping for {seconds} seconds")
time.sleep(seconds)
print(f"{name}: done")
sys.exit(0)
compute.py is a simple script that takes a name and a number of
seconds to sleep. It prints messages before and after sleeping.
import subprocess
# Run the same Python script with different arguments
result = subprocess.run(
["python3", "compute.py", "task1", "2"],
capture_output=True, text=True
)
print("Task 1 output:")
print(result.stdout)
result = subprocess.run(
["python3", "compute.py", "task2", "1"],
capture_output=True, text=True
)
print("Task 2 output:")
print(result.stdout)
print("All tasks completed")
The main script runs compute.py twice with different arguments.
Each invocation waits for the child process to finish before proceeding.
$ python main.py Task 1 output: task1: sleeping for 2 seconds task1: done Task 2 output: task2: sleeping for 1 seconds task2: done All tasks completed
Getting system information
The following example demonstrates a real-world use case: gathering system information by running multiple shell commands.
import subprocess
def run_cmd(cmd):
"""Run a command and return its stdout, or an error message."""
try:
result = subprocess.run(cmd, capture_output=True, text=True, check=True)
return result.stdout.strip()
except subprocess.CalledProcessError:
return "N/A"
except FileNotFoundError:
return "Command not found"
except Exception as e:
return f"Error: {e}"
info = {
"Hostname": run_cmd(["hostname"]),
"Kernel": run_cmd(["uname", "-r"]),
"Uptime": run_cmd(["uptime", "-p"]),
"Memory": run_cmd(["free", "-h"]),
}
for key, value in info.items():
print(f"{key}:")
print(f" {value}")
print()
A helper function run_cmd safely executes commands and captures
their output. The results are collected into a dictionary and printed in a
structured format.
$ python sysinfo.py
Hostname:
jano-pc
Kernel:
6.8.0-40-generic
Uptime:
up 3 hours, 26 minutes
Memory:
total used free shared buff/cache available
Mem: 15Gi 4.2Gi 3.1Gi 578Mi 8.1Gi 10Gi
Swap: 2.0Gi 256Mi 1.7Gi
Source
Python subprocess documentation
In this article, we have worked with the Python subprocess module to run external commands, manage process input and output, handle errors, set timeouts, and pipe data between processes.
Author
List all Python tutorials.