We can't find the internet
Attempting to reconnect
Something went wrong!
Attempting to reconnect
← Parallel Python with Process Pools step 8 of 24
Type-Check the Arguments You Send to a Worker
This code has a bug. resize takes a path and a width, and the call passes
only the path:
def resize(path: str, width: int) -> None: ...
worker = multiprocessing.Process(target=resize, args=("photo.jpg",))
mypy --strict accepts it, because Process takes any args at all. You
find the bug only when the worker runs and raises TypeError. A small
wrapper typed with ParamSpec moves that error to type-check time.
You met ParamSpec in the typed async wrapper exercise. [**P, R] declares
P, which stands for a function's whole parameter list, and R, its return
type. Then *args: P.args, **kwargs: P.kwargs must match that list:
def call_now[**P, R](fn: Callable[P, R], *args: P.args, **kwargs: P.kwargs) -> R:
return fn(*args, **kwargs)
call_now(resize, "photo.jpg") # error: Missing positional argument "width"
What to write
def typed_submit[**P, R](
ex: Executor, fn: Callable[P, R], /, *args: P.args, **kwargs: P.kwargs
) -> Future[R]: ...
def typed_process[**P](
fn: Callable[P, None], /, *args: P.args, **kwargs: P.kwargs
) -> multiprocessing.process.BaseProcess: ...
typed_submitcallsex.submit(fn, *args, **kwargs)once and returns that future. Don't wait for the result, and don't shut the executor down. The caller owns it.typed_processbuildsmultiprocessing.Processwithtarget=fn,args=argsandkwargs=kwargs, and returns it without starting it. The target must returnNone, because a process never sends its target's return value back.- The
/makesexandfnpositional-only, so a target can have its own parameter namedfnorex.
ex.submit is already typed this way, so typed_submit adds no new check.
It is practice on an API whose types you can compare with yours. Process
is not typed this way, so typed_process adds a check you did not have.
The driver
solve(*, items, width, prefix) labels every item on a thread pool:
- Open
ThreadPoolExecutor(max_workers=max(1, width)). A zero or negative width uses one worker. - Submit
label(prefix, item)for every item withtyped_submit. Submit them all before you read any result. - Read the futures in the order you submitted them, and return their results as a tuple. Keep duplicates.
- After the
withblock ends, build a process forannounce("never runs")withtyped_process, and don't start it. Return the results and whether that process'sexitcodeisNone.
The grader cannot pickle functions you define, so solve uses threads.
typed_submit works the same way with a ProcessPoolExecutor.
Keep label and announce as they are. label uses
f"{prefix}-{value:03d}". It pads the number with zeros to three
characters, counting a minus sign, and leaves a longer number whole:
assert solve(items=[3, 1, 3], width=2, prefix="job") == (
("job-003", "job-001", "job-003"), True
)
assert solve(items=[-2, 1000], width=0, prefix="x") == (("x--02", "x-1000"), True)
assert solve(items=[], width=-1, prefix="") == ((), True)
Your code must pass mypy --strict.
Going further
exitcode stays None until a process ends, so a running process has
None too. The concurrent.futures documentation
covers the thread, process and interpreter pools that share submit.
Your code, saved here
Sign in to access your accepted solutions and saved draft history. You can keep reading the lesson without an account.
Sign in to your accountStuck?
Python reference solution
Sign in to track your attempts toward unlocking the reference solution.