Catch Thread Errors and Stop Threads Cleanly

You start a thread to resize a folder of photos. One file is broken, and the resize raises an exception. Here is what your program sees:

import threading


def resize_all() -> None:
    raise ValueError("bad.jpg is not an image")


thread = threading.Thread(target=resize_all)
thread.start()
thread.join()
print("all done")

Exception in thread Thread-1 (resize_all):
Traceback (most recent call last):
  ...
ValueError: bad.jpg is not an image
all done

The traceback goes to standard error, and then the program prints "all done" as if nothing went wrong. You might spot the error in a log, but your code never sees it.

A thread's exception stays in that thread

When an exception escapes the function a thread runs, the thread ends. Python passes the exception to threading.excepthook, which prints it. Nothing raises it in the thread that called start() or join(). join() only waits for the thread to finish. It does not hand back the function's return value or tell you whether the function succeeded.

A thread pool's Future.result() re-raises a worker's exception for you. With a plain thread, you do that yourself. Catch the exception inside the thread, store it, and raise it after join():

errors: list[BaseException] = []


def run_and_record() -> None:
    try:
        resize_all()
    except BaseException as exc:
        errors.append(exc)


thread = threading.Thread(target=run_and_record)
thread.start()
thread.join()
if errors:
    raise errors[0]

raise errors[0] raises the original exception object, so its traceback still shows the line in resize_all that failed. Catch BaseException, not only Exception, so that every kind of error reaches the caller.

Check your understanding A worker thread raised an exception, and thread.join() returned normally. Where did the exception go? Show answer

It went to threading.excepthook, which printed it to standard error. The thread ended, but nothing raised the exception in the thread that called join(). To handle it in the caller, catch it inside the thread, store it, and raise it after join(). A thread pool's Future.result() does this for you.

Tell a thread to stop

A thread that loops forever needs a stop signal it can check. A threading.Event is a flag that threads share. set() turns the flag on, is_set() reads it, and wait(timeout) sleeps until the flag is on or the timeout passes.

import threading
import time

stop = threading.Event()


def heartbeat() -> None:
    while not stop.is_set():
        print("still alive")
        stop.wait(1.0)
    print("cleaned up")


thread = threading.Thread(target=heartbeat)
thread.start()
time.sleep(2.5)
stop.set()
thread.join()

This prints "still alive" three times, then "cleaned up". stop.wait(1.0) pauses like time.sleep(1.0), but it returns as soon as you call stop.set(), so the thread stops without finishing its sleep.

This kind of stop is cooperative. The thread stops only when its own code checks the flag. Setting the flag cannot interrupt a network call or a long computation that is already running, so give those calls their own timeouts.

Wait with a deadline, then check

thread.join(timeout=5.0) waits at most 5 seconds. It returns None whether the thread finished or not, so check afterwards:

thread.join(timeout=5.0)
if thread.is_alive():
    raise TimeoutError("worker still running after 5 seconds")

A thread that is still alive keeps running after you give up on it. It can still write files and change shared data. Python has no safe way to kill a thread. Make the work stoppable, with a stop flag and timeouts on slow calls. If the work must be killable, run it in a separate process instead.

With several threads, give them one shared deadline. If each join got the full 5 seconds, ten stuck threads would keep you waiting for 50.

Daemon threads skip their cleanup

When your main code ends, Python waits for the other threads to finish before the program exits. One stuck thread can keep your program running forever. A daemon thread, created with daemon=True, is one that Python does not wait for. When only daemon threads are left, Python exits and stops them wherever they are. Their finally blocks don't run, so a half-written file stays half written.

Threads are non-daemon by default. Keep that for any thread that writes data or cleans up, and shut it down with a stop flag and join(). Make a thread a daemon only when its work can be lost at any moment, such as a progress display.

Decide what a failure means

A worker that catches every exception and loops again can hide a problem forever. A wrong password fails on every attempt, and the log fills with the same error. Retry only failures that can go away, such as a timeout, and only a few times with a growing pause. Then stop the worker and make the failure visible, for example by marking the service unhealthy.

Going further

You can replace threading.excepthook with your own function, for example to log each failure with the thread's name.
A thread that calls sys.exit() ends quietly, because the default hook ignores SystemExit.

See the threading documentation.