Your script merges log files, and the list of files comes from the command
line. One with statement per file works only when you know the count while
you write the code. If you open them in a loop instead and the fifth open
fails, the first four stay open.
ExitStack from contextlib solves this. It is a context manager that
holds a stack of cleanups. You add each resource as you open it. When the
with block ends, the stack closes them all, the last one first.
from contextlib import ExitStack
from io import StringIO
texts = ["alpha", "beta", "gamma"]
with ExitStack() as stack:
streams = [stack.enter_context(StringIO(text)) for text in texts]
print([stream.read() for stream in streams]) # ['alpha', 'beta', 'gamma']
print(all(stream.closed for stream in streams)) # True
StringIO is an in-memory text file, so you can run this without creating
files. With real files, write
stack.enter_context(open(path, encoding="utf-8")).
Register each resource as soon as you have it
stack.enter_context(cm) does what a with statement would do. It enters
cm and returns the value you would get from as. It also puts the exit on
the stack, to run later.
Register each resource before you open the next one. If the third open
raises, the stack closes the two files it already holds, and the error
continues to the caller. A common mistake is to open everything first and
register afterwards. Then a failure in the middle leaves the earlier files
open, because nothing has registered them yet.
An object can have a close() method without being a context manager. Wrap
it as stack.enter_context(contextlib.closing(obj)), or register the method
with stack.callback(obj.close).
The stack runs cleanups in reverse order
log: list[str] = []
with ExitStack() as stack:
stack.callback(log.append, "close first")
stack.callback(log.append, "close second")
print(log) # ['close second', 'close first']
Nested with statements use the same order. The last resource opened is
the first one closed. The order matters when one resource depends on
another. A database transaction has to finish before its connection closes.
| Method | What it does |
|---|---|
enter_context(cm) |
Enters cm now and puts its exit on the stack |
callback(fn, *args) |
Puts a plain function call on the stack |
pop_all() |
Moves every pending cleanup to a new stack, without running any |
Hand the resources to the caller with pop_all
Sometimes one function opens the resources and its caller uses them. Call
pop_all() once everything is open. It moves the cleanups to a new
ExitStack and returns it, so the with block ends without closing
anything. The caller now owns the new stack and must close it.
def open_streams(texts: list[str]) -> tuple[list[StringIO], ExitStack]:
with ExitStack() as stack:
streams = [stack.enter_context(StringIO(text)) for text in texts]
return streams, stack.pop_all()
streams, owner = open_streams(["one", "two"])
with owner:
print([stream.read() for stream in streams]) # ['one', 'two']
print(all(stream.closed for stream in streams)) # True
If an open fails halfway, the first stack still closes the earlier streams,
because pop_all() has not run yet. Keep the stack that pop_all()
returns. If you drop it, nothing ever runs its cleanups.
A context manager written with @contextmanager does not need pop_all().
Put the yield inside the with ExitStack() block, and the stack closes
everything when the caller's with block ends. The exercise works this way.
Check your understanding
Why does the exercise's open_all take an opener function instead of calling open itself?
Show answer
Hide answer
The caller decides how to open each name. A test can pass a fake opener that
fails on a chosen name and records every open and close, with no files on
disk. The exercise does exactly that. Real code can pass
lambda path: open(path, encoding="utf-8").
Going further
- If a cleanup raises, the stack still runs the remaining cleanups. The caller may then see a different exception from the one the block raised.
- If a context manager's
__enter__raises, the stack never registers its exit. That context manager must undo its own half-finished work. stack.push(exit_fn)registers a function that takes the same arguments as__exit__. Unlike acallbackfunction, it receives the exception and can hide it by returningTrue.- Use one
ExitStackperwithblock. Nesting the same stack inside itself does not give you two separate scopes.