Step 12 · Ubuntu · Independent

Debugging Python with pdb

A program that goes wrong will not tell you why. A debugger lets you stop it in the middle, look at every variable, and walk forward one line at a time. Python comes with one built in.

There is nothing to install for pdb

The debugger is part of Python's standard library. It works on any machine that has Python — including a server you have just logged into over ssh, where you cannot install anything at all.

This step only makes sure python3 is present and writes the two files used below.

Run this

$ curl -fsSL https://aikaryashala.com/system_setup/scripts/install_py_debug.sh | bash

It writes report.py and stats.py into ~/python-samples, then checks the debugger works on your machine.

The two sample files

Debugging gets interesting the moment a program spans more than one file, so this uses two. Both are downloadable: report.py and stats.py.

def mean(numbers):
    """The average. Raises ZeroDivisionError on an empty list."""
    return sum(numbers) / len(numbers)


def median(numbers):
    """The middle value once the numbers are in order."""
    ordered = sorted(numbers)
    middle = len(ordered) // 2

    if len(ordered) % 2 == 1:
        return ordered[middle]
    return (ordered[middle - 1] + ordered[middle]) / 2


def spread(numbers):
    """How far apart the largest and smallest values are."""
    return max(numbers) - min(numbers)


def summarise(numbers):
    """Everything above, in one dictionary."""
    return {
        "count": len(numbers),
        "mean": mean(numbers),
        "median": median(numbers),
        "spread": spread(numbers),
    }

import stats

READINGS = [12, 7, 3, 21, 9, 15]


def show(title, numbers):
    summary = stats.summarise(numbers)
    print(f"--- {title} ---")
    for key, value in summary.items():
        print(f"{key:>7}: {value}")


def main():
    show("readings", READINGS)

    # This one is empty, and mean() divides by len(numbers).
    missing = []
    show("missing", missing)


if __name__ == "__main__":
    main()

Run it as it is. It prints one report, then crashes:

$ cd ~/python-samples
$ python3 report.py
--- readings ---
  count: 6
   mean: 11.166666666666666
 median: 10.5
 spread: 18
Traceback (most recent call last):
  ...
  File "stats.py", line 10, in mean
    return sum(numbers) / len(numbers)
ZeroDivisionError: division by zero

The crash is deliberate. Finding out why, without adding a single print, is the exercise.

Notice what is missing from that output

There is no --- missing --- heading. The program asked for the second report, but never got as far as printing its title — because show() calls stats.summarise() on its first line, and that is where it died.

That absence is your first clue, and reading output for what is not there is a real debugging skill. The traceback confirms it: the last line names stats.py, not report.py.

Start the program under the debugger

$ python3 -m pdb report.py
> /home/you/python-samples/report.py(1)<module>()
-> import stats
(Pdb)

It stops before the first line runs. From here you drive it one command at a time:

(Pdb) b report.py:20          # break where show() is called with READINGS
(Pdb) c                       # continue until it gets there
(Pdb) l                       # list the source around this line
(Pdb) s                       # step INTO show()
(Pdb) s                       # step again - now inside stats.py
(Pdb) w                       # where am I? show the call stack
(Pdb) a                       # what arguments did this function get?
(Pdb) p numbers               # print one value
(Pdb) pp summary              # pretty-print a bigger one
(Pdb) r                       # run until this function returns
(Pdb) c                       # continue to the next breakpoint or the end
(Pdb) q                       # quit

s versus n — the distinction that matters

n (next) runs the whole of a function call and stops on the following line. s (step) goes inside it. Stepping into stats.summarise() from report.py is how you cross from one file into another.

Let it crash, then look around

The fastest way to understand an exception is to catch the program at the moment it died, with every variable still in place:

$ python3 -m pdb -c continue report.py
--- readings ---
...
ZeroDivisionError: division by zero
> /home/you/python-samples/stats.py(3)mean()
-> return sum(numbers) / len(numbers)
(Pdb) p numbers
[]
(Pdb) p len(numbers)
0
(Pdb) w

-c continue tells pdb to run straight through and only take control when something goes wrong. It drops you into post-mortem mode, stopped inside mean() in stats.py with numbers still visible as []. w shows the chain of calls that led there — main → show → summarise → mean.

Stop at an exact spot in the code

Instead of remembering a line number, write breakpoint() where you want to stop and just run the file normally:

def main():
    show("readings", READINGS)

    missing = []
    breakpoint()          # execution stops here, and pdb takes over
    show("missing", missing)

$ python3 report.py

pdb commands worth memorising

Command	Short	What it does
`list`	`l`	Show the source around the current line
`longlist`	`ll`	Show the whole current function
`next`	`n`	Run the next line, stepping over calls
`step`	`s`	Run the next line, stepping into calls
`return`	`r`	Run until the current function returns
`until 30`	`unt 30`	Run until line 30 — useful for escaping a loop
`continue`	`c`	Carry on until the next breakpoint
`break report.py:20`	`b report.py:20`	Break at a line in a file
`break stats.mean`	`b stats.mean`	Break whenever a function is entered
`break stats.py:3, len(numbers) == 0`		Break only when a condition holds
`tbreak`		A breakpoint that fires once, then removes itself
`clear`	`cl`	Remove breakpoints
`print expr`	`p expr`	Evaluate and show one expression
`pp expr`		The same, pretty-printed — good for dicts and lists
`args`	`a`	The arguments this function was called with
`where`	`w`	The call stack: how did we get here?
`up` / `down`	`u` / `d`	Move to the calling frame and back
`display total`		Show this value automatically every time you stop
`interact`		Open a full Python prompt with the current variables
`quit`	`q`	Stop debugging

A variable named like a command

If you have a variable called n, c or l, typing its name runs the command instead of showing the value. Use p n — or !n, which forces pdb to treat the rest of the line as Python.

Tracing every line, without stopping

Sometimes you do not want to step — you want to see the whole path the program took. Python ships that too:

# print every line as it executes
$ python3 -m trace --trace report.py

# just count how many times each line ran
$ python3 -m trace --count -C . report.py

# show which functions called which
$ python3 -m trace --trackcalls report.py

Check it worked

$ python3 -c "import pdb; print('pdb ok')"

$ cd ~/python-samples && python3 -m pdb -c continue -c "p numbers" -c quit report.py

That last one should end by printing [] — the empty list that caused the crash, read out of a program that has already died.

The same idea in C

Step 3 installed lldb, which does the same job for C. The commands differ, but breakpoints, stepping, printing a variable and reading the call stack are the same four ideas in both.