Thread Lifecycle: Daemons, Swallowed Exceptions, Graceful Shutdown

Two production incidents, one root cause.

“The background thread just stopped doing work, and nobody noticed for three weeks.” A refresher thread woke every 30 seconds, hit a transient ConnectionError once, and died. The process kept running. Health checks kept passing. The cache it maintained kept serving three-week-old data.

“We lose the last 200 log lines on every deploy.” The log shipper ran on a daemon thread. On shutdown, daemon threads are stopped abruptly — no finally, no __exit__, no flush. Everything still in the buffer went away, which is precisely the part you wanted during a bad deploy.

An uncaught exception does not stop your program

When Thread.run() raises, the exception goes to threading.excepthook, whose default prints a traceback to stderr and returns. The thread ends. The process carries on, believing the work happened.

Nothing else notices, because nothing else is looking:

• thread.join() returns normally. Join means “the thread finished”, not “the thread succeeded”.
• thread.is_alive() is False — correctly, and unhelpfully.
• There is no result object to inspect, because Thread has no concept of a result. Thread(target=..., args=...) gives you no channel back at all.

That last point is the honest argument for concurrent.futures over raw Thread: a Future holds either a value or an exception, and .result() makes you confront which. Raw Thread has neither, so you have to build the channel, and building it is the exercise in this section.

Two more things worth knowing about excepthook: it fires for Exception but also for SystemExit — which is silently ignored, by design — and the hook receives the thread object, so a service-wide threading.excepthook that logs structured events (with the thread name) is a five-line change that converts “silently stopped” into an alert.

💡A worker thread does try: work() except Exception: log.exception("worker failed") and loops. Is the thread now safe? click to reveal

Safer, and still not safe, in two ways.

First, except Exception does not catch BaseException. KeyboardInterrupt and SystemExit pass straight through and kill the loop — which is usually what you want on shutdown, but means “the loop is protected” is not true unconditionally. It also does not catch MemoryError cleanly in practice.

Second, and more common: catching-and-continuing turns a fail-stop into a fail-slow. If work() raises every iteration because a config value is wrong, you now have a thread that logs an exception 200 times a second forever and a service that looks alive. That is worse than dying, because dying is visible to the orchestrator and gets you a restart.

The shape that works is: catch, log with context, and count. After N consecutive failures, stop the loop and set a flag the health check reads. Let the supervisor restart you. “Retry forever, silently” is not resilience, it is a hidden outage.

Daemon threads are not a shutdown strategy

daemon=True means “do not hold up interpreter exit”. The interpreter does not ask a daemon thread to finish; it stops it. The thread’s finally blocks do not run, its context managers do not __exit__, its buffers are not flushed.

That is fine for a thread with no state worth preserving. It is wrong for anything that writes: a log shipper, a metrics flusher, a batch accumulator, a connection pool draining checkouts. The reason people reach for daemon=True anyway is always the same — a non-daemon thread that never exits hangs the process at shutdown — and the fix is not the daemon flag, it is giving the thread a way to be told to stop.

The shape that works everywhere:

stop = threading.Event()

def loop() -> None:
    while not stop.wait(interval):   # returns True when set, False on timeout
        do_work()

t = threading.Thread(target=loop, daemon=False)
t.start()
...
stop.set()
t.join(timeout=5.0)
if t.is_alive():
    log.error("worker did not stop within 5s")

stop.wait(interval) is both the sleep and the check — one call, no polling loop, and shutdown is immediate rather than up to interval late. And join(timeout) returns None whether or not the thread finished, so the is_alive() afterwards is the only way to find out. Every join(timeout) without a following is_alive() is a bug waiting to be written.

💡stop.set(); t.join(5.0) and the thread is still alive. What now? click to reveal

Whatever you do, you cannot kill it — Python has no safe way to stop a running thread from outside. That is not an oversight; asynchronously injecting an exception into arbitrary code would break every invariant that any lock, file handle or partially-updated data structure was relying on. (The ctypes trick that appears in blog posts sets a flag checked only between bytecodes, cannot interrupt a blocking C call, and can leave a lock held forever.)

So the only real options are to log loudly and continue shutting down, or to escalate to process exit and let the orchestrator restart you. Which is right depends on whether abandoning the thread’s work is safe.

The deeper point is that this is the strongest argument for asyncio in a service: a Task can be cancelled, cooperatively and safely, at a defined set of suspension points. Track item 9.10 is entirely about the discipline that makes that reliable — and item 9.22 makes the choice explicit.