We can't find the internet
Attempting to reconnect
Something went wrong!
Attempting to reconnect
← Parallel Python with Process Pools step 18 of 24
Fix Multiprocessing Type Hints That Fail at Runtime
With spawn or forkserver, each worker process imports your module again
before it runs a task. That import runs every def in the module, annotations
included. Here is a worker function that mypy accepts:
import multiprocessing
def drain(queue: multiprocessing.Queue[int]) -> None: ...
multiprocessing.Queue is a function that makes a queue, not the queue class,
so Python raises TypeError when it evaluates multiprocessing.Queue[int].
Python 3.13 and older evaluate annotations when the def runs, so the import
fails. Python 3.14 and later wait until something reads them, such as
typing.get_type_hints or a library like pydantic. The error then appears at
that later read, which can happen inside a worker.
Submodules cause a second problem. import multiprocessing does not load
multiprocessing.managers or the other submodules in the table below. An annotation that names
one can work in the parent, where some other import loaded it, and raise
AttributeError in a fresh worker. Import each submodule you name.
Annotate with the class, create through the context
Keep creating objects through a context, as in
multiprocessing.get_context("spawn").Queue(). In annotations, name the class
from its submodule.
| What you create | Class for annotations |
|---|---|
| Lock, event, semaphore | multiprocessing.synchronize.Lock, Event, Semaphore |
| Queue, simple queue | multiprocessing.queues.Queue, SimpleQueue |
| Process pool | multiprocessing.pool.Pool |
| Manager dictionary | multiprocessing.managers.DictProxy |
| Pipe end | multiprocessing.connection.Connection |
from multiprocessing.queues import Queue
from typing import get_type_hints
def enqueue(queue: Queue[int], value: int) -> None: ...
print(get_type_hints(enqueue)["queue"]) # multiprocessing.queues.Queue[int]
Connection[bytes, bytes] raises TypeError at runtime, even though
type-checker stubs describe the class as generic. Use bare Connection.
What to write
Fix the imports and the obj annotation of the eight functions in the starter.
Keep the element types Queue[int], SimpleQueue[bytes] and
DictProxy[str, int], and use bare Connection. typing.get_type_hints must
work on every function in REGISTRY, and the file must pass mypy --strict
with no # type: ignore.
Then write solve(*, functions). For each name in functions:
- Look the function up in
REGISTRY. - Call
typing.get_type_hintson it and take the"obj"entry. - If
typing.get_originreturns a class, use that class. Otherwise use the hint itself.Queue[int]has the originQueue, and a plain class has none. - Store that class's
__module__under the requested name.
For example, solve(functions=["takes_lock"]) returns
{"takes_lock": "multiprocessing.synchronize"}.
- Read each module from the annotation. Don't hard-code the strings or create
multiprocessing objects in
solve. - For an empty list, return
{}. A repeated name gives one entry. - An unknown name raises
KeyError. - Leave the input list and
REGISTRYunchanged.
The tests ask about subsets of the registry, but all eight annotations must be right.
Your code, saved here
Sign in to access your accepted solutions and saved draft history. You can keep reading the lesson without an account.
Sign in to your accountStuck?
Python reference solution
Sign in to track your attempts toward unlocking the reference solution.