Time it properly

start = time.perf_counter()
result = fn()
elapsed = time.perf_counter() - start

This measures the first run, which is the one run that is not representative of anything.

What the first run includes

• Lazy initialisation. CUDA context creation, cuDNN loading, the memory allocator’s first large request.
• Autotuning. cuDNN and cuBLAS benchmark several algorithms on the first call for a given shape and cache the winner.
• Compilation. Under torch.compile, the first call traces and compiles and can take seconds.

None of that happens again. Measuring it tells you about startup, not about the operation.

The three rules

Warm up. Run it a few times and discard those. You are measuring the steady state.

Repeat and take the median. A single timed run competes with whatever else the machine is doing. The median is robust to an outlier in a way the mean is not, which matters because interference makes runs slower and never faster, so the distribution has a long right tail.

Synchronise, on a GPU. torch.cuda.synchronize() before starting the clock and before stopping it. Without it you are timing how fast Python can queue work, which is very fast and unrelated to anything. This is the .item() lesson from the devices track, needed here on purpose rather than avoided.

The tool

torch.utils.benchmark.Timer does all three, plus it handles thread counts and reports a proper distribution. Reach for it for anything you intend to quote. Write the loop by hand once so you know what it is doing.

Your task

def measure(fn, warmup: int, runs: int) -> dict

Call fn warmup times without timing it, then runs times with timing, and return the median of the timed runs plus the total number of calls made.

The starter times one run and calls it a result.