Everything in contextlib has an async twin, and the twins are not
interchangeable with their synchronous counterparts in either direction.
asynccontextmanager
Same shape as @contextmanager, over an async generator:
@asynccontextmanager
async def acquire(self, name: str) -> AsyncIterator[Connection]:
conn = await self.pool.get()
try:
yield conn
finally:
await self.pool.put(conn)
Used with async with. The try/finally is as non-optional as in the
synchronous version, for the same reason.
The problem aclosing solves
async def stream_rows(pool: Pool) -> AsyncGenerator[Row, None]:
async with pool.acquire() as conn:
async for row in conn.query(...):
yield row
async for row in stream_rows(pool):
if row.id == target:
break # <-- the connection is still held
Breaking out of an async for leaves the generator suspended at the
yield, inside the async with. Nothing has run its finally. The
connection goes back to the pool only when the generator is finalised — which,
for an async generator, is not refcount-driven: it happens via the event
loop’s shutdown_asyncgens hook, at loop shutdown, or when the garbage
collector schedules the aclose() coroutine on a loop that may already be
closing.
In a request handler that breaks early, the connection is held until the process shuts down. Under load the pool exhausts and the symptom is “timeouts acquiring a connection” with no slow queries anywhere.
contextlib.aclosing makes finalisation deterministic:
async with aclosing(stream_rows(pool)) as rows:
async for row in rows:
if row.id == target:
break
# aclose() awaited here: the generator's finally has run
aclosing(agen) awaits agen.aclose() on exit — on every exit path,
including break, exception and cancellation. The synchronous closing
cannot do this, because aclose() is a coroutine.
💡Why is contextlib.closing (or a plain try/finally calling
click to reveal
gen.close()) not enough for an async generator?
Because agen.aclose() returns a coroutine that must be awaited, and
neither closing.__exit__ nor a synchronous finally can await.
Calling it without awaiting produces a coroutine object that is never run —
and Python emits “coroutine ‘aclose’ was never awaited” as a RuntimeWarning,
which in most production logging configurations nobody sees. The generator’s
finally never runs, so you get the leak and a warning you have already
filtered out.
The same trap applies one level up: ExitStack.enter_context(some_async_cm)
fails, because an async CM has no __enter__. Use AsyncExitStack and
enter_async_context. --strict catches this particular one — the async
context manager does not satisfy the AbstractContextManager protocol — which
is a rare case of the checker catching a concurrency bug.
AsyncExitStack
The async ExitStack, with enter_async_context and push_async_callback
alongside the synchronous methods, so you can mix both kinds of resource in
one stack.
One signature detail that catches people: push_async_callback takes a
coroutine function plus its arguments, not an already-awaited coroutine:
stack.push_async_callback(conn.rollback) # right
stack.push_async_callback(conn.rollback()) # wrong: already called
The second form calls rollback() immediately, producing a coroutine object
that is then treated as the callback and never awaited.
Cleanup under cancellation
When a task is cancelled, CancelledError is raised at the current await.
Your finally runs — but if it itself contains an await, that await can be
cancelled too, and the cleanup does not finish.
For cleanup that must complete, asyncio.shield the awaited part, or perform
the release synchronously where you can (returning an object to a
collections.deque-backed pool needs no await). A finally that only touches
synchronous state is immune, which is a good argument for keeping release
paths as simple as possible.
💡An async with inside an async generator, plus a consumer that
click to reveal
breaks early, plus aclosing. In what order does the cleanup run, and is it
deterministic?
Deterministic, and outside-in from the consumer’s point of view.
The break leaves the async for and the async with aclosing(...) block
exits, which awaits agen.aclose(). That throws GeneratorExit into the
generator at its suspended yield. The generator’s async with block then
exits, running the finally in acquire, which returns the connection. Only
then does aclose() return and the consumer proceeds.
So by the time the statement after the async with aclosing(...) runs, the
connection is provably back in the pool — which is exactly what makes it
testable: assert on the pool’s event log immediately after the block, with no
sleeps and no waiting for the loop to shut down.
Without aclosing, the same events happen in the same order but at an
unspecified time, which is why the leak is so hard to reproduce in a test.