Property-Based Testing — The Practice

The video covered the theory. This is where you run it. Everything below executes Hypothesis in your browser — edit a test, hit run, watch it search the input space and shrink whatever it breaks. Nothing to install, and nothing you do here leaves the page.

Part A — Refresher

One screen of syntax, so you can start writing tests without rewinding the video. Skip straight to Part B if it's fresh.

from hypothesis import given
from hypothesis.strategies import text

@given(text(), text())
def test_concat_length(a: str, b: str) -> None:
    assert len(a + b) == len(a) + len(b)

• @given(text(), text()) — the strategies. Each names a space of values, and Hypothesis draws two fresh strings per call.
• def test_… — the body receives concrete values. No hardcoded expected output; the assertion is the property.
• On a failure, Hypothesis shrinks: it keeps re-running with simpler inputs until it finds the smallest one that still breaks the rule — k = "" rather than the 2 KB of garbage it originally generated.

Built-in strategies (text(), integers(), lists(), floats()) cover primitives. For your own types you compose one with @composite — a function handed a draw callable, which turns any strategy into a concrete value:

from hypothesis.strategies import composite, floats

@composite
def single_ball(draw) -> Ball:
    return Ball(
        x=draw(floats(0.06, 0.94)),     # draw() → a float in range
        y=draw(floats(0.06, 0.94)),
        vx=draw(floats(-2.0, 2.0)),
        vy=draw(floats(-2.0, 2.0)),
    )

@given(single_ball())                    # now usable like any strategy
def test_energy_conserved(ball: Ball) -> None:
    ...

That's how every generator in this tutorial is built: single_ball, two_balls, whole sequences of store operations. Each runner below lists the @composite generators in its file next to the tests — click one to draw sample values and see the shape of the space Hypothesis is searching.

Part B — Warm up on a couple of small ones

You don't need a custom system to write property-based tests. The Python standard library is full of operations that have properties you can test directly. test_simple.py has five of them — two boring-and-passing, three that look just as obvious and Hypothesis will break.

# Pass: useful, dull
@given(lists(integers()))
def test_sorted_idempotent(xs):
    assert sorted(sorted(xs)) == sorted(xs)

@given(lists(integers()))
def test_sorted_preserves_length(xs):
    assert len(sorted(xs)) == len(xs)

# Look uncontroversial — Hypothesis disagrees
@given(text())
def test_upper_lower_round_trip(s):
    assert s.upper().lower() == s

@given(text())
def test_lower_upper_round_trip(s):
    assert s.lower().upper() == s

@given(text())
def test_utf8_byte_length(s):
    assert len(s.encode("utf-8")) == len(s)

Run each of them in the runner below. The first two pass against 100 inputs every time. The bottom three each fail within the first dozen generated examples — and Hypothesis shrinks the failure to a single character that demonstrates the issue.

Part C — Watch a property fail on a physics sim

Some systems make their invariants obvious because the invariants are physics. Below is a 2D billiard sim: circular balls bouncing in a unit square with perfectly elastic collisions. Energy is conserved. Momentum is conserved (within collisions). Balls stay inside the box. Same initial state + same dt → same final state. Four properties without trying.

We planted one bug in the wall-bounce code, gated behind a dial called BUG_THRESHOLD. The rule is: when a ball hits a wall slower than the threshold, the bounce eats half its energy on that axis. Fast hits stay perfectly elastic — so the sim looks fine when everything is moving quickly, and the bug only shows on slow, grazing bounces. Drag the Bug threshold slider up and watch: a red ring marks each leaky bounce, and a dashed ghost ball follows the trajectory the bounce should have produced.

v = getattr(ball, component)      # velocity into the wall
expected = -v                     # a clean elastic bounce
if BUG_THRESHOLD > 0 and abs(v) < BUG_THRESHOLD:
    setattr(ball, component, expected * 0.707)   # slow hit: leak energy
else:
    setattr(ball, component, expected)           # fast hit: perfect

Now the part that matters: an example-based test on this sim almost never spots the bug. You'd write something like "after 1 second, ball at position X" and the absolute number shifts just enough to look plausible. The property energy_before == energy_after catches it the first time Hypothesis happens to generate a slow-approach input — usually within a couple of dozen examples.

First, predict

Before you run anything: test_physics.py holds four properties, and the bug is armed. Which ones actually break? Commit to an answer for each — the useful part of this exercise is being wrong about one of them.

Now go find out. The runner below has the same Bug threshold slider as the sandbox — but this one rewrites the test file. Drag it and watch the physics.BUG_THRESHOLD = … line in the source change, then run each of the four tests and see whether your predictions hold.

Part D — Take it deeper with a key-value store

Physics is great for visceral demos; a key-value store is closer to the systems you actually build. Four operations — insert, get, delete, select(regex) — and a surprising number of subtle properties. We'll play with it, list the invariants, then test them.

Play first

You can't write properties for a system you haven't touched. The playground below has a 10-item checklist that ticks itself off as you exercise each operation — work through it and the invariants in the next section will be obvious rather than abstract.

Now list the invariants

Before code, finish these in plain English:

1. After insert(k, v), calling get(k) should return …
2. After delete(k), calling get(k) should return …
3. If I insert(k, v1) and then insert(k, v2), get(k) should return …
4. delete twice in a row is the same as delete once. True or false?

Your first property

test_warmup.py has the invariants as @given stubs. Fill in the body of test_insert_then_get — two lines — and run it. Then flip the toggle at the top of the file (store.BUGGY = True) to plant a bug. Hypothesis will find that the store silently swallows empty-string keys, and shrink the failure to k = ''.

    store.insert(k, v)
    assert store.get(k) == v

# import store
# store.BUGGY = True

Where to go from here

The repo has progressively richer tests against the same store. The runner below tabs across them — pick one and click any @given button, or click a @composite generator to draw sample values from it. (test_isolation and test_trace_stateful reference sockets/threads, so they won't actually run in-browser — their source is still worth reading, and test_trace_stateful's generators still draw.)

• test_message — every Message survives serialize → deserialize unchanged. Runs in-browser.
• test_containment — round-trip against a populated multi-user store. Runs in-browser.
• test_isolation — across many random client interactions, no client ever sees another client's keys via select. (Source only.)
• test_trace_stateful — Hypothesis generates random traces (sequences of operations) and checks the server matches a Python-dict model after each step. Model-based testing. (Source only.)