You changed how your service parses JSON, and it feels faster. Before you
say so, write down the question you want answered. For example, "How long
does json.loads take on our 1,000-value payload?" The question decides
what goes inside the timer.
import json
import statistics
import time
payload = json.dumps({"values": list(range(1_000))})
samples: list[float] = []
for _ in range(20):
start = time.perf_counter()
result = json.loads(payload)
samples.append(time.perf_counter() - start)
assert result["values"][-1] == 999
print(f"first {samples[0] * 1e6:.0f} µs, median {statistics.median(samples) * 1e6:.0f} µs")
print(f"fastest {min(samples) * 1e6:.0f} µs, slowest {max(samples) * 1e6:.0f} µs")
On a laptop with Python 3.15, it printed:
first 30 µs, median 25 µs
fastest 24 µs, slowest 30 µs
time.perf_counter() reads a clock made for timing. It returns seconds as
a float, and only the difference between two readings means anything. It
never goes backwards. Don't time code with time.time(), which follows the
system clock and jumps when the clock is adjusted.
Put only your question inside the timer
The snippet builds the payload before the loop and checks the result after the clock stops. Building the payload is not part of the question, so it stays outside. If your question were "how long does the whole import job take?", you would start the clock before the job and stop it after.
Wall-clock time and CPU time
Wall-clock time is elapsed time on a clock. perf_counter measures it,
and it includes time your code spends waiting. CPU time is time the
processor spent running your code. time.process_time() measures it for
your process:
import time
wall_start = time.perf_counter()
cpu_start = time.process_time()
time.sleep(0.5) # waiting, like a network call
total = sum(range(10_000_000)) # computing
print(f"wall-clock {time.perf_counter() - wall_start:.2f}s")
print(f"CPU {time.process_time() - cpu_start:.2f}s")
This printed wall-clock 0.58s and CPU 0.07s. Users wait for wall-clock
time, so that is the number to report. Compare the two to see what your
code does. When CPU time is much smaller, the code is mostly waiting, and a
faster algorithm will not help much.
Run it many times
One run is not a result. Your computer changes speed while it works. The processor slows down when it gets hot, other programs compete for it, and the first call pays for setup that later calls reuse. That is why the first sample above is the slowest. Time many runs and look at the spread. Run the whole script a few times too, because a fresh process can behave differently.
Decide whether the first call matters to your question. A command-line tool that runs once is all first call, so include it. A server handles thousands of requests, so the later calls are what its users see.
Common mistakes
- Timing one tiny call. On the laptop above, two clock readings
with nothing between them differ by about 40 nanoseconds. Timing a
single
abs(x)gives the same 40 nanoseconds, so it measures only the clock. Time a loop of many calls and divide by the count. The next article'stimeitmodule does this for you. - Timing two versions once each, in the same order. The second one may run on a warmer, or busier, machine. Alternate them and repeat.
Check your understanding You make one function 1.4 times faster. It took 2% of a request's time. How much faster is the request? Show answer Hide answer
About 0.57% faster, if nothing else changes. The function's share falls from 2% to 2% / 1.4, which is about 1.43%, so the request saves the difference. A part that takes 2% of the time can never save more than 2%. Measure where the time goes before you choose what to speed up.
Going further
For a benchmark you will publish or act on, try pyperf. It runs your code in several fresh processes, throws away warm-up runs, and reports a mean and a standard deviation. The time module documentation describes each clock.