Skip to content

← Stdlib Mastery step 23 of 55

Hard Primitives

lru_cache on a method: fix the unbounded instance leak

Refactor a leaking class so cached results do not outlive their instance — while staying just as cached.

Define:

class Digest:
    def __init__(self, salt: int) -> None: ...
    salt: int
    calls: int          # number of times the value was actually computed
    def value(self, n: int) -> int: ...   # returns (self.salt + n) ** 2

value(n) must compute its result at most once per (instance, n), incrementing self.calls only on a real computation.

def solve(salt: int, other_salt: int, queries: list[int]) -> dict[str, object]:

solve builds a Digest(salt), evaluates every query twice, records calls, builds a second Digest(other_salt) and evaluates the queries once on it, then takes a weakref.ref to the first instance, dels it, runs gc.collect(), and returns:

{"pass_one": [...], "pass_two": [...], "computations": int,
 "other": [...], "other_computations": int, "leaked": bool}

"leaked" is tracker() is not None. It must be False.

Why @lru_cache on the method fails this. The decorator is applied to the function, at class-creation time, so there is one cache and it lives on the class. self is an ordinary first argument, so the key is (instance, n) and the cache holds a strong reference to every instance it has ever seen. After del and gc.collect(), the weakref is still live. In a long-lived service caching per request, per session or per connection, that is an unbounded leak whose profile signature is “a growing population of objects nothing references, held by a functools internal” — people lose hours to it looking for a missing close().

It is also a correctness bug in the other direction: maxsize is global across instances, so ten live objects share one 128-entry cache and evict each other. The hit rate looks perfect in a unit test with one instance and collapses in production.

The type checker says nothing about any of it. Combined with the signature erasure @cache causes, a cached method is doubly invisible. --strict proves things about types, and “this object outlives its scope” is not a type.

Note that the fix may create a reference cycle (instance -> cache -> instance). That is fine — gc.collect() collects cycles, which is exactly why the test calls it.