Cache Method Results Without Keeping Objects Alive

You put @lru_cache on a method to speed it up. Later you notice that objects you have finished with are still in memory. The cache is holding them:

import gc
import weakref
from functools import lru_cache

class Digest:
    def __init__(self, salt: int) -> None:
        self.salt = salt

    @lru_cache(maxsize=128)
    def value(self, n: int) -> int:
        return (self.salt + n) ** 2

item = Digest(3)
print(item.value(1))  # 16
tracker = weakref.ref(item)
del item
gc.collect()
print(tracker() is None)  # False, the object is still alive

The snippet checks with a weak reference, a reference that lets you check whether an object still exists without keeping it alive. Calling tracker() returns the object, or None once Python has freed it. Python frees an object once nothing refers to it, and gc.collect() also frees groups of objects that refer only to each other. So if the object still exists after gc.collect(), something still refers to it.

The cache keys include self

The decorator runs once, when Python creates the class, so every instance shares one cache. Each call passes self as an argument, so self is part of every key. While an entry stays in the cache, the cache holds a reference to that instance, and Python cannot free it. Clearing the cache drops that reference:

Digest.value.cache_clear()
gc.collect()
print(tracker() is None)  # True

With maxsize=128, the cache throws out old entries once it is full, so it frees instances eventually. With @cache, or maxsize=None, it throws out nothing, and every instance that ever called the method stays alive.

Sharing one cache has two more effects. The 128 entries are for all instances together, so a busy instance pushes out the entries of the others. And cache_clear() clears the entries of every instance at once.

Common mistakes

Caching a method that reads a field you later change. The key holds self, not the field's value, so after the change you get the old result.

Caching a method on a class that defines __eq__ without __hash__. Such a class has unhashable instances, so every call raises TypeError.

Give each instance its own cache

Store the results in a dictionary on the instance. The cache then goes away with the instance:

class LocalDigest:
    def __init__(self, salt: int) -> None:
        self.salt = salt
        self.calls = 0
        self._memo: dict[int, int] = {}

    def value(self, n: int) -> int:
        if n not in self._memo:
            self.calls += 1
            self._memo[n] = (self.salt + n) ** 2
        return self._memo[n]

local = LocalDigest(-4)
print(local.value(4), local.value(4), local.calls)  # 0 0 1

Test whether n is a key, not whether the stored result is truthy, because a result can be 0, as it is here. This dictionary has no size limit, so it grows with each new n for as long as the instance lives. That is fine when each instance sees a limited set of inputs.

For a value that takes no arguments, the next article's cached_property stores the result on the instance for you.

Or cache a plain function

If the result depends only on values, not on the object, cache a module-level function that takes those values:

from functools import lru_cache

@lru_cache(maxsize=128)
def digest_value(salt: int, n: int) -> int:
    return (salt + n) ** 2

print(digest_value(3, 1))  # 16

The keys are two integers, so no instance stays alive. Objects with the same salt also share results, which the method version could not do.

Check your understanding A web service creates a new Digest for each request and calls value on it. You change @lru_cache(maxsize=128) to @cache to get more hits. What happens to memory? Show answer

It grows until the service restarts. A @cache never throws an entry out, and each entry holds its Digest, so every object from every request stays alive with all its data. Give each object its own cache, or cache a plain function of salt and n.

Going further

You can also give each instance its own LRU cache by wrapping a bound method in __init__, as in self.value = lru_cache(maxsize=32)(self._value). That creates a reference cycle. The instance holds the cache, and the cache holds the instance through the bound method. Python frees such a cycle only when its garbage collector runs. The collector runs on its own from time to time, not at the moment you drop your last reference.