This test checks a word counter on four inputs in a loop. Save it as
test_words.py and run pytest:
def word_count(text: str) -> int:
return text.count(" ") + 1
def test_word_count() -> None:
cases = [("one", 1), ("", 0), ("two words", 2), ("two words", 2)]
for text, expected in cases:
assert word_count(text) == expected
FAILED test_words.py::test_word_count - AssertionError: assert 1 == 0
1 failed in 0.01s
The loop stops at the first failing case, the empty string. You cannot tell whether the cases after it pass. In fact the last one, with two spaces, fails too.
In Catch the Broken Versions, you checked a table of cases inside one function. pytest has its own way to do that, and it runs each case as a separate test.
Write the cases as a table
@pytest.mark.parametrize takes the parameter names and a list of rows.
pytest runs the test once per row and reports each run on its own:
import pytest
def word_count(text: str) -> int:
return text.count(" ") + 1
@pytest.mark.parametrize(
("text", "expected"),
[
("one", 1),
("", 0),
("two words", 2),
("two words", 2),
],
)
def test_word_count(text: str, expected: int) -> None:
assert word_count(text) == expected
FAILED test_words.py::test_word_count[-0] - AssertionError: assert 1 == 0
FAILED test_words.py::test_word_count[two words-2] - AssertionError: assert ...
2 failed, 2 passed in 0.01s
Now you see both bugs at once. The fix is len(text.split()), which
ignores extra spaces and returns 0 for an empty string.
Use a table when only the inputs and the expected results change. When the setup or the checks differ from case to case, write separate tests.
Give each case a readable ID
The text in square brackets is the case's ID. pytest builds it from the
row's values, so the empty string gives the hard-to-read -0. Wrap a row in
pytest.param to choose the ID yourself:
import pytest
def word_count(text: str) -> int:
return len(text.split())
@pytest.mark.parametrize(
("text", "expected"),
[
pytest.param("one", 1, id="one-word"),
pytest.param("", 0, id="empty"),
pytest.param("two words", 2, id="two-words"),
pytest.param("two words", 2, id="double-space"),
],
)
def test_word_count(text: str, expected: int) -> None:
assert word_count(text) == expected
With the old word_count, the failures would read test_word_count[empty]
and test_word_count[double-space]. pytest -k empty runs only the tests
whose names contain empty.
Match each row to the names
You can write several names as a tuple, ("text", "expected"), or as one
string with commas, "text,expected". Each row then needs one value per
name, in the same order. If a row has the wrong number of values, pytest
reports an error before it runs any test.
With a single name in a string, each row is the whole value, even when the row is itself a list:
import pytest
@pytest.mark.parametrize("values", [[1, 2], [3]])
def test_not_empty(values: list[int]) -> None:
assert len(values) > 0
Here values is [1, 2] in the first test and [3] in the second. pytest
does not unpack the inner lists.
Stack decorators to combine tables
Stack two parametrize decorators, and pytest runs every combination:
import pytest
@pytest.mark.parametrize("x", [0, 1])
@pytest.mark.parametrize("y", [2, 3])
def test_pair(x: int, y: int) -> None:
assert x < y
pytest collects four tests, with (x, y) set to (0, 2), (1, 2),
(0, 3) and (1, 3), in that order. The values of the upper decorator
change fastest. Run pytest --collect-only -q to list the tests without
running them.
The number of tests is the product of the table sizes. Three decorators with 4, 5 and 12 values make 240 tests. When only some combinations matter, list those rows in one table instead.
Check your understanding
A test has @pytest.mark.parametrize("x", [1, 2, 3]) above @pytest.mark.parametrize("y", ["a", "b"]). In what order does pytest run its cases?
Show answer
Hide answer
pytest runs six cases, one for each pair. The values of the upper
decorator, x, change fastest. So the order is (1, "a"), (2, "a"),
(3, "a"), then the same three x values with "b".
Give each row its own mutable data
pytest passes each row's values as they are, without copying them. If two rows hold the same list and a test changes it, the next test sees the change:
import pytest
shared: list[str] = []
@pytest.mark.parametrize("items", [shared, shared])
def test_append(items: list[str]) -> None:
items.append("x")
assert items == ["x"]
The second case fails, because items is already ["x"] when it starts.
Give each row its own list, or build the list inside the test.
The next exercise has you expand parametrize tables into test cases yourself.
Going further
pytest.param can also mark one row. With
marks=pytest.mark.xfail(strict=True, reason="bug 123"), pytest expects
that row to fail, and the run fails if it starts to pass. With
indirect=True, pytest hands each value to the fixture of the same name as
request.param, so the fixture can build an object from it. A fixture
declared with @pytest.fixture(params=[...]) makes every test that uses it
run once per value. pytest 9 can also reject duplicate IDs, and
Turn On pytest's Strict Checks, later
in this track, shows how.
See pytest's guide How to parametrize fixtures and test functions.