Bridging Blocking Code: to_thread, Executors, Cross-Thread Scheduling

“Our p99 spikes to eight seconds during report generation.”

The cause is almost always one line: a synchronous database call, a requests.get, a json.dumps of a 200 MB payload, a bcrypt.hashpw — executed directly on the event loop thread. And the damage is not to that request. It is to every request, every timer, and every heartbeat, because a blocking call on the loop thread freezes the entire loop for its duration.

This is the single most common way an async rewrite ends up slower than the threaded version it replaced.

Going async → blocking

Three tools, in preference order.

`asyncio.to_thread(fn, *args, kwargs)** (3.9). The default. It runsfnon the loop's defaultThreadPoolExecutorand — importantly — **propagates the currentcontextvars.Context** into the worker thread, so your request id survives the hop. It also takes keyword arguments directly, whichrun_in_executor` does not.

loop.run_in_executor(pool, fn) when you want to control which pool. This matters more than it sounds: to_thread‘s default executor is created lazily with a max_workers you cannot set from the call site, and everything in the process shares it. A burst of report generations queues behind whatever else happens to be using it, and you have built a hidden global bottleneck. A dedicated pool for a known-slow workload keeps it out of everyone else’s way.

functools.partial when you need keyword arguments with run_in_executor, whose signature is positional-only.

💡to_thread moves work off the loop. Does that make a CPU-bound function parallel? click to reveal

Only if the function releases the GIL — which is exactly the question item 9.22 is about.

For pure-Python CPU work the answer is no. The work moves off the loop thread, so the loop keeps serving other requests — which is a real and often sufficient win — but the worker thread and the loop thread now contend for the GIL, so the loop’s own throughput drops while it runs. You have traded a hard freeze for a slowdown.

For CPU work inside a C extension that releases the GIL — NumPy kernels, hashlib, zlib, Pillow, compiled ML kernels — the answer is yes, genuinely parallel, and to_thread is the right and complete answer.

So the diagnostic before reaching for to_thread on a slow function is: is the hot frame Python or C? If it is Python and the function is genuinely expensive, you want a process pool, and loop.run_in_executor(process_pool, fn) is the spelling — with the extra constraint that arguments and results have to pickle.

Going blocking → async

Everything in asyncio is documented as not thread-safe except two functions, and debug mode will raise if you use anything else from the wrong thread.

• loop.call_soon_threadsafe(callback, *args) — schedule a plain callback on the loop, from any thread. Returns a Handle, not a result.
• asyncio.run_coroutine_threadsafe(coro, loop) — schedule a coroutine and get back a concurrent.futures.Future (not an asyncio one), so a thread can .result(timeout=...) on it and block conventionally.

Note the asymmetry: async→blocking gives you an awaitable; blocking→async gives you a concurrent.futures.Future. That is deliberate — each direction hands back the future type native to the world of the caller.

Preserving the signature: ParamSpec

The wrapper that does all of this is the most-copied decorator in async Python, and it is usually typed like this:

def asyncify(fn):                      # untyped
def asyncify(fn: Callable[..., Any]) -> Callable[..., Any]:   # worse

The second is worse than untyped because it looks deliberate. Callable[..., Any] erases the parameters and the return, so every call through the wrapper type-checks — including the ones with the arguments in the wrong order — and every result is Any, which then contaminates whatever you build from it. --strict will not say a word, because explicit Any is exactly what --strict does not catch.

The typed version:

def asyncify[**P, T](fn: Callable[P, T]) -> Callable[P, Coroutine[Any, Any, T]]:
    @functools.wraps(fn)
    async def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
        return await asyncio.to_thread(fn, *args, **kwargs)
    return wrapper

P (PEP 612) captures the whole parameter list — positional, keyword, defaults, everything — and replays it on the wrapper. T carries the return type through. The *args: P.args, **kwargs: P.kwargs pairing is required and is checked: mypy rejects one without the other.

💡Why -> Callable[P, Coroutine[Any, Any, T]] and not -> Callable[P, Awaitable[T]]? click to reveal

Because the return type is a promise about what you produce, and here you produce something narrower than an Awaitable.

Coroutine[Any, Any, T] tells the caller they have a real coroutine object: they can pass it to asyncio.run, close() it, or hand it to TaskGroup.create_task — which specifically requires a Coroutine, not an Awaitable. Annotate it Awaitable[T] and group.create_task(af(x)) becomes a type error, even though it works at runtime.

The mirror rule from item 9.7 still holds and is not in tension with this one: be wide in parameters, narrow in returns. Awaitable[T] is the right annotation for something you accept, because you only intend to await it. Coroutine[Any, Any, T] is the right annotation for something you hand back, because it tells the caller everything they actually have.