Skip to content

← Tests That Earn Their Keep step 8 of 19

Medium Framework

Reimplementing pytest's parametrize expansion

A test that loops over ten inputs and asserts inside the loop reports one failure and hides the other nine. It stops at the first bad input, so you fix it, re-run, and discover the second — ten times, one round trip each.

@pytest.mark.parametrize turns those ten inputs into ten independent test items. All ten run, all ten report, and the argument table becomes the most readable documentation of the contract that will ever exist in the codebase.

The semantics are subtle enough that people get them wrong for years. The fastest way to stop guessing is to implement them.

What to write

def expand_parametrize(
    argnames: str, argvalues: Sequence[object]
) -> list[dict[str, object]]

def stack_parametrize(
    first: Sequence[Mapping[str, object]],
    second: Sequence[Mapping[str, object]],
) -> list[dict[str, object]]

expand_parametrize returns one dict per generated test case, mapping each argument name to its value for that case.

  • argnames is a comma-separated string; whitespace around each name is stripped, so "a, b" and "a,b" are the same thing.
  • One name: each element of argvalues is that argument’s value, whole. expand_parametrize("xs", [[1, 2], [3]]) gives two cases whose xs are the lists [1, 2] and [3] — the list is not unpacked.
  • Several names: each element must be a list or tuple of exactly matching arity, and its items line up with the names positionally.

That one/several asymmetry is the single most common parametrize mistake, and it is why a table that worked yesterday explodes the moment someone adds a second argument name.

On a bad row, raise ParametrizeError naming the offending index:

argvalues[1] has 1 values but there are 2 argnames
argvalues[1] is not a list or tuple but there are 2 argnames

Note the second message. A str is a Sequencelen("xy") == 2 — so a membership test written as isinstance(row, Sequence) happily accepts "xy" for two argnames and silently binds a="x", b="y". Check for list or tuple, not for Sequence.

stack_parametrize is what two stacked decorators produce. pytest’s docs are explicit about the order:

@pytest.mark.parametrize("x", [0, 1])   # first  — the upper decorator
@pytest.mark.parametrize("y", [2, 3])   # second — the lower decorator
def test_foo(x, y): ...

runs x=0/y=2, x=1/y=2, x=0/y=3, x=1/y=3. The lower decorator is applied first and varies slowest. Merge each pair into one dict, in that order. An empty table on either side produces an empty product.

solve is provided; it calls both and puts any ParametrizeError into the report’s error field.

Why this matters beyond pytest

pytest 9’s strict_parametrization_ids makes duplicate or ambiguous case ids an error rather than a silent rename to case0, case1. Ids are generated from the values, so knowing exactly which values become one case and which become several is what makes that setting usable instead of infuriating.

Loading visualization…