Your program loads a large report, uses it, and moves on. When does Python free it? In CPython, the main Python implementation, that usually happens at once, as soon as nothing points at the report any more. This article shows what "points at" means, and the one case where Python waits.
An object lives while something points at it
This class prints a line when Python frees one of its objects. __del__ is
the method Python calls just before it frees an object. Run the examples as
a script. The interactive shell keeps extra references, such as _ for the
last result, so objects live longer there.
class Tracked:
def __init__(self, name: str) -> None:
self.name = name
def __del__(self) -> None:
print(f"freed {self.name}")
a = Tracked("report")
b = a # a second name for the same object
del a
print("deleted a")
del b
print("deleted b")
It prints:
deleted a
freed report
deleted b
del a removes the name a. It does not delete the object, because b
still points at it. CPython frees the object when del b removes the last
name.
A reference is anything that points at an object: a name, a list slot, a dictionary value or an attribute. CPython keeps a reference count for every object. That is how many references point at it. When the count reaches zero, CPython frees the object straight away.
Containers and saved errors hold references too
A list, a dictionary or a cache holds a reference to each item in it:
cache: dict[str, Tracked] = {}
def load(key: str) -> Tracked:
item = Tracked(key)
cache[key] = item
return item
load("user-1")
print("load returned")
cache.clear()
print("cache cleared")
The name item goes away when load returns, but cache still holds the
object. CPython frees it during cache.clear(), so "freed user-1" prints
between the two lines. A cache that never removes entries keeps every
object it has ever held.
A saved exception is a less obvious case. Its traceback points at the function calls it passed through, and each call points at its local variables:
def handle() -> None:
big = Tracked("big buffer")
raise ValueError("bad input")
try:
handle()
except ValueError as err:
saved = err
print("handled")
del saved
print("deleted saved")
"freed big buffer" prints only after del saved. If you keep errors in a
list for later and only need the message, keep str(err) instead.
Objects that point at each other wait for the garbage collector
Two objects can point at each other. That is a cycle, and neither reference count reaches zero, even when no name in your program can reach either object.
import gc
class Node:
def __init__(self, name: str) -> None:
self.name = name
self.other: Node | None = None
def __del__(self) -> None:
print(f"freed {self.name}")
parent = Node("parent")
child = Node("child")
parent.other = child
child.other = parent
del parent, child
print("deleted both names")
gc.collect() # prints "freed parent" and "freed child"
Nothing prints at the del. The garbage collector is the part of
CPython that finds cycles nothing else can reach, and frees them. It runs on
its own from time to time, and gc.collect() runs it now. A tree whose
children point back at their parent is a common cycle.
Clean up with with, not __del__
Don't close a file or give back a database connection in __del__.
__del__ runs when the reference count reaches zero, and
you have just seen three things that keep the count up: a second name, a
container and a saved exception. A cycle delays it until the collector runs.
Other Python implementations, such as PyPy, do not count references at all,
and free objects later.
Give resources back at a point you choose. A with block does that:
with open("report.txt", "w") as handle:
handle.write("done")
# The file is closed here, even if write raised.
The file object can live on after the block, but the file is closed. The
next exercise has you write a class that works with with.
Check your understanding
You call del rows at the end of a function, but memory stays high. What should you check?
Show answer
Hide answer
Check whether something else still points at the list. del rows removes
one name. If a cache, a global list, an attribute or a saved exception also
holds the list, its reference count stays above zero and CPython keeps it.
Remove that reference, or store something smaller in its place.
Going further
sys.getrefcount(obj)returns an object's reference count. The number can include a temporary reference from the call itself, and it differs between Python versions, so use it to explore, not in tests.- Since Python 3.12, some objects, such as
Noneand small integers, are immortal. CPython never frees them, andsys.getrefcountreports a huge number for them. - The free-threaded build counts references in a different way, so that threads do not compete for one counter. CPython frees some objects later there than in the standard build.
- The gc module documentation describes the collector and its settings.