We can't find the internet
Attempting to reconnect
Something went wrong!
Attempting to reconnect
← Stdlib Mastery step 15 of 55
batched: page a stream, and validate before you yield
Turn a stream of records into pages, and reject a bad page size at call time, not at first iteration.
def bulk_pages(records: Iterable[str], size: int) -> Iterator[list[str]]: ...
def solve(records: list[str], size: int) -> dict[str, object]: ...
-
bulk_pagesyieldslist[str]pages ofsizerecords, with a shorter final page when the stream does not divide evenly. -
size < 1must raiseValueErrorfrom the call itself, before anything is iterated. -
Never hold more than one page at a time. Take
Iterable[str], notlist[str].
solve reports which of the two moments raised:
{"raised_on_call": bool, "raised_on_iter": bool, "pages": list[list[str]]}
It calls bulk_pages(iter(records), size) inside one try, then list(...)
the result inside a second one.
This is the assertion. A generator function defers its entire body to
the first next(), including the validation at the top. bulk_pages(records, 0)
returns a generator object quite happily and the ValueError fires later —
often in a different function, several frames from the bad argument, with a
traceback that points at the consumer. In a pipeline assembled in one place
and consumed in another that is a genuinely hard bug to read. The fix is the
standard shape: a plain function that validates and returns an inner
generator. The return type does not change, so no caller is affected.
Why not range(0, len(records), size). That version needs len() and
slicing, so it works on a list and breaks on a generator, a DB cursor, a file
handle or a paginated API — the entire set of cases where you needed to batch
in the first place, because they are the ones too large to hold in memory.
itertools.batched (3.12) works on any iterable.
A typing note you are not being tested on but should know. batched
yields tuple[T, ...] — variadic, not fixed-length — so
for a, b in batched(xs, 2) type-checks and explodes on a short final batch.
strict=True is the runtime fix, and it is 3.13+; the checker cannot help
on either version.
Loading visualization…
Stuck?
Python reference solution
Sign in to attempt this problem and reveal the reference solution.