Skip to content
← Back to explainers

Python explainer

Let Hypothesis Find the Inputs You Missed

About 5 min to read. Allow extra time to work through examples.

State a rule your function must follow, let Hypothesis generate the inputs, and read the small failing case it reports.

This article uses Python. You can make it your practice mode to browse related content across the site.

You write a function that splits a restaurant bill between friends, in cents. You test it with a bill of 300 cents and three people, and the test passes:

def split_bill(total: int, people: int) -> list[int]:
    share = total // people
    return [share] * people


def test_split_bill() -> None:
    assert split_bill(300, 3) == [100, 100, 100]

Now split 100 cents three ways. You get [33, 33, 33], and one cent is gone. You picked an example that divides evenly, so the test never checked a remainder.

State a rule, and let Hypothesis pick the inputs

A property is a rule your function must follow for every valid input. Here, the shares must add up to the total. A property-based test checks a property against many generated inputs, instead of a few that you chose.

Hypothesis is the Python library that generates the inputs. Install it with pip install hypothesis, and save this as test_bill.py:

from hypothesis import given, strategies as st


def split_bill(total: int, people: int) -> list[int]:
    share = total // people
    return [share] * people


@given(
    total=st.integers(min_value=0, max_value=100_000),
    people=st.integers(min_value=1, max_value=50),
)
def test_shares_add_up(total: int, people: int) -> None:
    assert sum(split_bill(total, people)) == total

st.integers(min_value=1, max_value=50) is a strategy, a description of the values Hypothesis may generate for one argument. @given calls the test many times, 100 by default, with new values each time.

Run pytest test_bill.py, and the test fails. Hypothesis reports the input that broke it. The report looks like this:

Failing test case: test_shares_add_up(
    total=1,
    people=2,
)

The first failing input Hypothesis finds can be a messy one, such as 48,213 cents split 17 ways. Before it reports, Hypothesis shrinks the input. It tries smaller and simpler values, keeps each one that still fails, and shows you the simplest failure it reaches. One cent split two ways is easy to debug.

Make the property strong enough

Here is a fix. divmod returns the share and the cents left over, and the first extra people each pay one cent more:

def split_bill(total: int, people: int) -> list[int]:
    share, extra = divmod(total, people)
    return [share + 1] * extra + [share] * (people - extra)

The test passes now. So does this broken version, where the first person pays everything:

def split_bill(total: int, people: int) -> list[int]:
    return [total] + [0] * (people - 1)

"The shares add up" is one promise. The function also promises one share per person, and shares that differ by at most one cent. Check all three. A property is strong enough when every broken version you can think of fails it. Catch the Broken Versions asked the same of your check.

from hypothesis import example, given, strategies as st


def split_bill(total: int, people: int) -> list[int]:
    share, extra = divmod(total, people)
    return [share + 1] * extra + [share] * (people - extra)


@given(
    total=st.integers(min_value=0, max_value=100_000),
    people=st.integers(min_value=1, max_value=50),
)
@example(total=1, people=2)
def test_split_is_fair(total: int, people: int) -> None:
    shares = split_bill(total, people)
    assert sum(shares) == total
    assert len(shares) == people
    assert max(shares) - min(shares) <= 1

@example adds a case that runs every time. Hypothesis saves the failures it finds in a .hypothesis folder and tries them first on the next run. Your CI server may not keep that folder, so pin a bug you fixed with @example.

Properties that come up often

With random inputs you rarely know the exact output, but you know rules that it follows. Four kinds come up often:

  • Round trip. Decoding what you encoded gives back the original, so bytes.fromhex(data.hex()) == data.
  • Compare with a simpler version. Your fast function agrees with a slow, obvious one, such as your own sort against sorted().
  • Same result twice. Running the function on its own output changes nothing, so normalize(normalize(s)) == normalize(s).
  • Nothing lost. A sort keeps every element, so Counter(result) == Counter(items).

Test only the promises your function makes. A database migration that adds a column fails when you run it twice, so "same result twice" is the wrong property there.

Common mistakes

A round trip that hides a wrong format. Say a format requires lowercase hex. An encoder that writes uppercase, paired with a decoder that accepts both cases, still passes the round trip. Also compare the encoded text with a few known examples or with a reference encoder. The next exercise has this bug.

A strategy that cannot reach the bug. Hypothesis only tries values its strategy can produce. st.text(alphabet="abc") never produces a space or an accented letter. For a function that takes user text, start with st.text(), which covers most of Unicode.

Throwing most inputs away. assume(n > 0) skips any input where n is not positive, and Hypothesis generates another input in its place. When almost every input gets skipped, Hypothesis stops with a health-check error. Generate valid values directly with st.integers(min_value=1).

Check your understanding assert sum(split_bill(total, people)) == total passes for 100 generated inputs. What has it shown? Show answer

That the shares add up to the total for the inputs Hypothesis tried. It says nothing about fairness. A version that makes one person pay the whole bill passes the same test. Add a check for each promise the function makes, and test the checks against broken versions you can think of.

The site's grader cannot run Hypothesis. So in the next exercise the driver supplies fixed sample strings, and you write the property checks.

Going further

Call hypothesis.event("remainder") inside a test to label the cases you care about. Then run pytest --hypothesis-show-statistics to see how often each label came up. A label that rarely appears means your strategy seldom reaches that case.

Before Hypothesis 6.159, released in July 2026, the report started with Falsifying example: instead of Failing test case:. Older guides show that form.

step 10 of 19

Testing with pytest and Hypothesis

Up next Check a Run-Length Codec with Four Properties