Eight remaining tools. For each, the one situation where it is clearly the right answer, and the trap.
cycle(iterable)
Repeats forever, buffering the input as it goes so it can repeat a one-shot iterator. Round-robin assignment across workers, alternating row colours, retry backoff schedules.
Trap: cycle over a large generator holds the whole thing. And for x in cycle(xs)
without a break or an islice never terminates — including inside a
list(), which will consume all your memory rather than hang visibly.
repeat(value, times=None)
The same object, times times or forever. Its real job is as a constant
column in a zip or a map:
map(pow, values, repeat(2)) # squares, without a lambda
Trap: it repeats the same object, not copies. repeat([], 3) gives three
references to one list.
zip_longest(*iterables, fillvalue=None)
Pads the short inputs. Genuinely correct when the inputs are conceptually ragged — merging sparse columns, aligning optional fields.
Usually you want zip(..., strict=True) instead (3.10+). Plain zip
truncates to the shortest input silently, which is a data-loss bug wearing
the clothes of a normal loop. strict=True raises ValueError when the
lengths differ, turning a silent truncation into an exception. It also keeps
the element types tight: zip_longest has to admit fillvalue into every
element type, so zip_longest(names, ages) is Iterator[tuple[str | None, int | None]]
and every use site needs narrowing. If the lengths are supposed to match, say
so and get both the check and the types.
takewhile(pred, it) / dropwhile(pred, it)
takewhile stops at the first failure; dropwhile skips until the first
failure and then yields everything. Prefix and suffix of an ordered stream:
reading a header block, skipping a preamble, cutting a sorted series at a
threshold.
Trap, and it is a real one: takewhile consumes and discards the element
that failed the predicate. If you takewhile a header and then keep reading
the same iterator, the first body line is gone. There is no “peek” that would
avoid it. If you need both halves, either materialise, or use a small
push-back wrapper.
💡takewhile(lambda line: line.strip(), f) then list(f) loses a
click to reveal
line. What are the honest fixes? Three, in ascending order of machinery.
Read the blank line back into your model. Often the discarded element is a separator you did not want anyway — a blank line between headers and body, a sentinel row. If that is genuinely true, document it and move on.
Materialise, then split. lines = list(f) and slice at the index of the
first blank. Honest, obvious, and fine unless the file is large.
Wrap the iterator with one slot of push-back. A tiny class holding an
optional pending item, whose __next__ returns the pending item if present.
Ten lines, reusable, and it makes the “peek” operation the stdlib does not
give you explicit.
What does not work is itertools.tee: the tee also sees the element consumed,
because the consumption happened in the shared source before the tee existed.
filterfalse(pred, it)
The complement of filter. Its value is that it removes a not from a lambda
and makes the pair of complementary filters read symmetrically:
valid = filter(is_valid, rows)
invalid = filterfalse(is_valid, rows)
Trap: filterfalse(None, it) keeps the falsy items — the mirror of
filter(None, it) keeping the truthy ones. Both are worth a comment at the
call site, because None as a predicate reads like a mistake.
starmap(f, iterable_of_tuples)
map unpacks each element as arguments. starmap(pow, [(2,5), (3,2)]) rather
than map(lambda t: pow(*t), ...). Exactly right when you already have a
sequence of argument tuples — a parsed CSV, the output of zip, a parameter
grid.
compress(data, selectors)
Keeps data[i] where selectors[i] is truthy. The situation that justifies
it: the mask was computed elsewhere, by something vectorised or by an
earlier pass, and you want to apply it without recomputing the predicate.
Applying a boolean mask from numpy, or filtering by a precomputed validity
vector.
Trap: it stops when either input runs out, so a short selector silently truncates the data.
💡A reviewer says "this itertools chain is unreadable, just write click to reveal
the loop”. When are they right? When the pipeline has more than about three stages, or when any stage needs a lambda longer than a few tokens.
The case for the chain is that each stage is a named, testable
transformation and the whole thing streams. The case against is that Python’s
syntax puts the stages in reverse reading order —
list(islice(filterfalse(pred, chain.from_iterable(src)), 10)) is read from
the inside out — and every reader pays that cost on every visit.
The compromise that usually wins: keep the streaming, lose the nesting. Assign
each stage to a named local, or write a generator function whose body is a
plain for loop with a yield. Both preserve constant memory, and both read
top to bottom. Reserve the dense one-liner for the two-stage case where it
genuinely is clearer.