with handles a fixed number of resources beautifully and a variable number
not at all. There is no syntax for “open each of these paths, and close them
all however this block ends” when the list length is a runtime value. The
hand-rolled version is a try/finally with a partially-populated list and a
loop that has to tolerate None entries, and it is wrong more often than it
is right.
contextlib.ExitStack is a stack of cleanup callbacks with a context manager
interface:
with ExitStack() as stack:
handles = [stack.enter_context(open(p)) for p in paths]
...
# every handle closed, in reverse order of acquisition
The four operations
| Method | Registers |
|---|---|
enter_context(cm) |
enters cm now, will call its __exit__ |
push(cm) |
registers an __exit__ without entering |
callback(fn, *args, **kwargs) |
calls fn(*args, **kwargs) on exit |
pop_all() |
transfers everything to a new stack, disarming this one |
Unwinding is reverse order, which is the same guarantee nested with
statements give you and matters whenever resources depend on each other — a
transaction inside a connection has to commit before the connection closes.
callback has a limitation that catches people: registered callbacks cannot
suppress exceptions, because they never see them. They are called with no
arguments regardless of how the block exited. If your cleanup needs to know
whether the block failed — commit versus rollback — it has to be a real
context manager entered with enter_context, or pushed with push.
💡ExitStack is documented as reusable but not reentrant. What
click to reveal
goes wrong if you enter the same stack twice?
Reusable means you can with stack: again after it has unwound — the stack is
empty and starts fresh. Not reentrant means you cannot nest a second
with stack: inside the first.
If you do, leaving the inner with calls every callback registered so
far, including the ones registered before the inner block began. The outer
block then continues running with all of its resources already closed, and the
failure surfaces later as a use-after-close on a handle that looks perfectly
alive in the source.
The rule that avoids it: one stack per with. If you need a nested scope,
create a nested ExitStack. They are cheap, and the nesting then means what
it looks like.
The pop_all() idiom: transactional acquisition
The documented pattern for “acquire N things, and if any acquisition fails, release the ones already acquired — but on success, hand ownership to the caller”:
def open_all(paths: list[str]) -> list[TextIO]:
with ExitStack() as stack:
handles = [stack.enter_context(open(p)) for p in paths]
stack.pop_all() # disarm: success, caller owns them now
return handles
If open raises on the fourth path, the with unwinds normally and closes
the three already open. If every path opens, pop_all() transfers the
callbacks to a throwaway stack that is never entered, so the original with
cleans up nothing and the handles survive.
This is the correct shape for a constructor that acquires several resources, and it is genuinely hard to write correctly by hand.
When the resources should not escape — the usual case — you do not need
pop_all at all: yield inside the with from a @contextmanager, and
all-or-nothing acquisition plus guaranteed release both fall out.
💡Your open_all takes an opener callable rather than calling
click to reveal
open directly. Why is that worth the extra parameter?
Because it is the difference between a function you can test and one you
cannot.
With the real open, testing “the third file fails to open and the first two
are closed” requires a filesystem, a permissions trick or a monkeypatch, and
asserting closure means reaching into the handle’s private state. With an
injected opener, the test supplies a fake that records open: and close:
events and raises on a named input, and the assertion is a list comparison.
It is also better production code. The parameter makes the dependency
explicit, lets a caller substitute a different acquisition strategy (a
connection pool, an in-memory store, a retrying opener) without touching
open_all, and documents in the signature that this function is about
sequencing acquisitions rather than about files.
The typed version is better still: declare a Protocol for what a handle must
provide, and open_all becomes reusable for any resource with that shape.