When to Write a C or Rust Extension

Your profile says most of the time goes into one small function that runs a million times. Rewriting it in C sounds like the fix. Try the cheapest version of that idea first. The standard library's ctypes module calls C functions directly. This snippet calls labs, the C library's version of abs, a million times. It runs on Linux and macOS:

import ctypes
import ctypes.util
import time

libc = ctypes.CDLL(ctypes.util.find_library("c"))
libc.labs.argtypes = [ctypes.c_long]
libc.labs.restype = ctypes.c_long

values = list(range(-500_000, 500_000))

start = time.perf_counter()
python_total = sum(abs(v) for v in values)
print(f"Python abs:        {time.perf_counter() - start:.2f}s")

start = time.perf_counter()
c_total = sum(libc.labs(v) for v in values)
print(f"C labs via ctypes: {time.perf_counter() - start:.2f}s")

assert python_total == c_total

On a laptop with Python 3.15, it printed:

Python abs:        0.02s
C labs via ctypes: 0.13s

The C version is about six times slower. Each call converts a Python int to a C long, goes through ctypes, and turns the result back into a new Python int. That round trip costs far more than labs itself. Python's built-in abs is already written in C, and it has no round trip.

Send whole batches across

Native code wins when one call does a lot of work. For example, one call can take a whole buffer of numbers and loop over them in C. A native function called once per value pays for the round trip every time. So measure the whole operation, including converting your data on the way in and the results on the way out.

A Python list passed to compiled code is still a list of separate Python objects. Packing it into a typed buffer, such as an array.array or a NumPy array, costs a copy of its own.

Try the cheaper fixes first

Before you take on an extension, check three things. Is there a better algorithm, as with the moving average earlier in this track? Can you call the slow function less often? Does a library already do this work in compiled code, such as NumPy or the standard library? Each of these is cheaper to keep than an extension.

Pick a tool that fits your team

Rust's rules prevent many memory bugs, but unsafe blocks, C libraries and threads still need review. Passing mypy --strict doesn't mean mypyc can compile your code.

Plan the builds

You must compile each extension for every operating system, CPU and Python version your users run. The results ship as wheels, the prebuilt packages that pip installs. CPython's Stable ABI lets one wheel serve several Python versions, but not several operating systems or CPUs.

Free-threaded Python builds need their own wheels. When such a build imports an extension that has not declared free-threading support, it turns the GIL back on and prints a warning. Declaring support doesn't make the code thread-safe. Check its shared state and test it with the GIL off.

Keep the Python version

Keep the pure-Python function as the reference. Test the extension against it on the same inputs, including empty input, large numbers and errors. Then benchmark both on real data, conversions included.

Going further

Start with the Cython memoryviews guide, mypyc's introduction, the PyO3 guide, maturin, CFFI's overview, CPython's C API stability page and its free-threading extension guide.