Skip to content

← Orientation and the Gate step 11 of 13

Easy Primitives

Placement 4/6: generators are consumed once

Placement diagnostic, 4 of 6. About seven minutes. If this one is opaque, T7 (Laziness, Iteration and Pipelines) is where you start.


A list can be walked a hundred times. A generator can be walked once, and the second walk is not an error — it is an empty sequence. That asymmetry is invisible in the annotation Iterable[int], which is exactly why it ships to production.

The starter is a plausible draft that is wrong:

def summarise(rows: Iterable[int]) -> dict[str, list[int]]:
    count = sum(1 for _ in rows)     # pass 1 — drains `rows`
    if count == 0:
        return {"above": [], "at_or_below": []}
    mean = sum(rows) / count         # pass 2 — sums nothing, gets 0
    ...

It works perfectly in the REPL, where you tested it with a list. It returns garbage in production, where the caller hands it a generator over a 40 GB file — silently, with no exception, because a mean of 0.0 is a perfectly valid float.

Fix summarise so it is correct for a one-shot iterator:

def summarise(rows: Iterable[int]) -> dict[str, list[int]]:

Compute the arithmetic mean of rows, then return {"above": [...], "at_or_below": [...]} — the rows strictly greater than the mean, and the rest — each in original order. Empty input returns two empty lists.

solve is given and must not change. It passes summarise a genuine generator expression, so you cannot rebuild the source:

def solve(values: list[int]) -> dict[str, list[int]]:
    return summarise(value * value for value in values)

The two honest options

You get exactly two, and choosing between them is the whole skill:

  1. Materialise oncerows = tuple(rows) at the top of the function. Costs O(n) memory, buys you unlimited passes. Correct here.
  2. Restructure to a single pass — accumulate everything you need in one walk. Costs nothing extra, but “everything you need” has to be a bounded-size accumulator, and “the items above the mean” is not, because you do not know the mean until you have seen the last item.

What you may not do is quietly assume the caller passed a list. If your function needs two passes, say so in the signature — annotate the parameter Sequence[int] and let the type checker reject the generator at the call site, rather than accepting Iterable[int] and returning wrong numbers. That single annotation choice is a contract about how many times you will walk the input.

Production consequence

Every function of the form if not items: ... ; for item in items: has this bug. So does len(list(x)) followed by for _ in x. So does logging f"processing {len(rows)} rows" before the loop. They are all silent, they all pass a unit test written with a list literal, and they all produce empty output the moment someone upstream converts a return [...] into a yield — which is a refactor that looks like a pure performance win.

Loading visualization…