“The service leaks” is one of the least actionable bug reports there is. It names a symptom, gives you a process, and leaves you to guess. tracemalloc converts it into a file, a line number and a byte count — and it does so with no dependency to install, no root, and no ptrace capability, which is exactly what you need when the leak only reproduces inside a hardened container you cannot attach a debugger to.
The four calls
import tracemalloc
tracemalloc.start(25) # trace, keeping 25 frames per allocation
before = tracemalloc.take_snapshot()
do_the_suspicious_thing()
after = tracemalloc.take_snapshot()
tracemalloc.stop()
for stat in after.compare_to(before, "lineno")[:10]:
print(stat)
start(nframe) — the argument is how many frames of traceback to keep per allocation. More frames means better attribution (you can see which caller of a shared helper is responsible) and more overhead, in both time and the memory tracemalloc itself uses. Start at 1 to find what allocates; raise it to 20-30 to find who asked for it.
take_snapshot() — everything currently traced, with its allocation site.
compare_to(other, key) — the diff, sorted by size_diff descending. The key is 'lineno' (per file and line), 'filename' (per module) or 'traceback' (per full stack, which requires the frames you paid for at start).
get_traced_memory() returns (current, peak), and reset_peak() (3.9) resets the peak without disturbing the current figure — which is the only way to measure the peak of one phase inside a long-running process.
The demonstration that makes generators concrete
Same computation, two spellings, traced:
total = sum([x * x for x in range(1_000_000)]) # traced: 40.4 MB
total = sum(x * x for x in range(1_000_000)) # traced: 0.0 MB
The list comprehension materialises a million integers and a million-pointer array before sum sees any of it. The generator expression materialises one value at a time. The answer is identical; the peak memory differs by 40 MB, which is the difference between fitting in a 512 MB container and not.
This is the measurement that turns “prefer generators” from style advice into a number.
💡You take a snapshot before and after one request, diff it, and the top entry is a line inside json/decoder.py. Why is that almost never the actual problem, and what would you change about how you measured?
click to reveal
Because the allocating line and the retaining line are different lines, and tracemalloc reports the former. json.loads allocates the dicts; something in your code kept them. Optimising json/decoder.py is not available to you and would not help anyway.
Two changes. First, raise nframe and diff on 'traceback' rather than 'lineno'. That turns “json allocated it” into “json allocated it, called from parse_event, called from handle_batch“ — which is a line you own and can reason about.
Second, measure over more than one request. A single request’s diff is dominated by transient allocations that will be freed shortly; what you want is the residue after many iterations, which is why the standard shape is warm up, snapshot, run N times, snapshot. Anything that scales with N is a leak. Anything that does not is just how much a request costs.
The measurement shape that actually finds leaks
- Warm up. Run the function once before the first snapshot. First-call allocations — imports, lazily-built caches, compiled regexes, a connection — are one-time costs, and if you snapshot before them they dominate the diff and hide the real signal.
- Snapshot.
- Run N times.
- Snapshot and diff.
- Repeat at 2N. Growth that scales with N is a leak. Growth that does not is a fixed cost you already paid.
Step 5 is what separates a leak from a warm-up artefact, and it costs one extra run.
And whatever you do, stop the tracer on every path. tracemalloc is global process state; leaving it running after an exception means every subsequent measurement in that process is taken under a different overhead regime, and the profiler slows down the thing it is measuring. try: ... finally: tracemalloc.stop().
The limit that sends you elsewhere
tracemalloc hooks Python’s allocators. A NumPy array’s buffer, a PyTorch tensor’s storage, a compression library’s arena, a database driver’s connection buffer — none of it appears, because none of it goes through PyMem_Malloc.
So a service whose RSS is 8 GB and whose tracemalloc total is 200 MB is not lying to you; it is telling you the leak is not in Python objects. That is the moment to reach for memray, which traces native allocations and the interpreter’s own, with full native call stacks (article 11.21).
Knowing which tool answers which question is most of the skill. tracemalloc: Python allocations, everywhere, cheaply. memray: everything, Linux and macOS only, with a real install.