Lazy record batches: what laziness moves

Reading a 40 GB export into a list is the most common way a batch job gets OOM-killed. The fix is syntactically trivial — a [ becomes a ( — and conceptually subtle, because laziness does not just move where memory goes. It moves when work happens and when errors are raised.

Measured with tracemalloc: a list comprehension over 1,000,000 squares traced 40.4 MB; the equivalent generator expression traced 0.0 MB. Same arithmetic, same answer, four orders of magnitude apart on peak allocation.

Build a lazy record parser, then prove it is lazy.

What to write

A record line is name:count, e.g. alpha:7.

def parse_line(line: str) -> Record

Split on the first :. Raise ValueError(f"malformed record: {line!r}") if there is no separator, if the name is empty, or if the count is not all digits. Return Record(name=..., count=...).

def parse_records(lines: Iterable[str], *, chunk: int) -> Iterator[list[Record]]

Yield lists of at most chunk parsed records, in order, including a final short batch. It must never hold more than chunk records at once, and it must do nothing at all when called.

def solve(*, lines, chunk, take=None, repeat=False) -> Report

Wrap the source in the supplied CountingSource. When repeat is true the source is itertools.cycle(lines) — infinite. When take is not None, take at most take batches. Fill in the Report:

• pulled_before_first — source.pulled captured immediately after calling parse_records, before any iteration.
• batch_sizes — the length of each batch consumed, in order.
• head — the first four records of the first batch rendered as "name=count" (fewer if the batch is shorter).
• total — the sum of count over every record consumed.
• pulled — source.pulled at the end.
• error — str(exc) if a ValueError escaped during consumption, else "". A batch already yielded before the bad line still counts.

The three properties under test

1. Calling the function does no work. pulled_before_first must be 0. A yield anywhere in the body makes the entire function a generator function: calling it builds a generator object and runs none of the body. Return a list instead and the source is drained during the call, and this number becomes len(lines).

2. Consumption pulls exactly what it needs. With an infinite cycle source, chunk=2 and take=3, the run must terminate having pulled exactly 6 lines. An eager implementation hangs forever.

3. A malformed line raises at consumption time, not at call time. This is the property that surprises people in production. The frame that “validates” the input is not the frame the ValueError surfaces in — the traceback points at whoever is driving the generator, often a logging or serialisation layer far from the parse. That is a feature you must know about, not a bug you can fix.

Types

Annotating the return as list[list[Record]] is the mistake this problem exists to prevent. It is a lie the moment the body contains yield, and mypy says so. Iterator[list[Record]] is the honest annotation — and it also tells the caller the result is single-shot.

Your submission must pass mypy --strict.