Skip to content

← True Parallelism and the Runtime step 16 of 18

Medium Framework

The Three Start Methods, and What 3.14 Changed

This is the single highest-value migration hazard in the Python 3.14 upgrade.

A service that worked for five years by forking a parent that holds a 4 GB model, a Postgres pool and a metrics thread will now do one of three things: crash with a pickling error, silently re-load the model in every worker and get OOM-killed, or hand each child a copy of a connection object that no longer maps to a live socket. And it will usually do it at the first task submission under load — not at start-up, where you would have seen it.

What actually changed

On Unix platforms other than macOS, forkserver is now the default start method for multiprocessing and ProcessPoolExecutor, replacing fork. This change does not affect Windows or macOS, where spawn remains the default. (gh-84559)

Two details people get wrong:

  • fork is not deprecated and not removed. It is demoted. You can still ask for it explicitly via get_context("fork"), and sometimes you should.
  • macOS did not change. It has defaulted to spawn since 3.8. The 3.14 note is about Linux and the other Unixes.

The prior history is worth carrying: 3.4 added spawn and forkserver; 3.8 made spawn the macOS default; 3.12 made os.fork() in a multi-threaded process emit a DeprecationWarning.

Method Child starts from Inherits parent memory Cost per process
fork a copy of the parent’s address space yes, everything 0.87 ms
forkserver a fork of a small, clean server process only what the server preloaded 1.40 ms
spawn a fresh interpreter, re-importing __main__ nothing 26.2 ms

(Measured on a 10-core arm64 machine. The forkserver-to-spawn ratio is the number that matters for a recycling pool: 19x.)

One more rule that bites: objects from one context are not interchangeable with another. A Queue made by get_context("fork") cannot be handed to a process started by a spawn context. And spawn/forkserver “generally cannot be used with frozen executables on POSIX”, because there is no interpreter to re-launch.

Your task

The right answer depends on facts about the host, so take those facts as inputs — which is also exactly how you make the real thing testable.

def solve(
    *,
    platform: str,             # sys.platform
    active_threads: int,       # threading.active_count()
    available: list[str],      # multiprocessing.get_all_start_methods()
    frozen_executable: bool,   # getattr(sys, "frozen", False)
) -> tuple[StartMethod, tuple[str, ...]]:

Return the method you chose and the ordered tuple of reason codes for the rules that fired. Apply them in exactly this order:

  1. If fork is available and active_threads > 1, remove fork and append "threads-present-fork-unsafe".
  2. If frozen_executable and the platform is not win32, remove spawn and forkserver and append "frozen-posix-excludes-spawn-forkserver".
  3. Append the platform’s preference reason and set its preference order:
    • win32 -> "windows-spawn-only", order ("spawn",)
    • darwin -> "darwin-prefers-spawn", order ("spawn", "forkserver", "fork")
    • anything else -> "posix-prefers-forkserver", order ("forkserver", "spawn", "fork")
  4. Choose the first method in the preference order that is still eligible.
  5. If nothing is eligible, append "no-safe-method-forced-fork" and return "fork" — an honest report that the configuration has no safe answer, which is more useful than a plausible-looking lie.

The typing lesson

get_context("forkserver") reveals ForkServerContext, but ctx.Queue() reveals Queue[Any]. That is the trap: --disallow-any-generics rejects a bare Queue in an annotation, and says nothing at all about an Any-parameterised type inferred from a call. Your payload type silently becomes Any and every get() after it is unchecked.

q = ctx.Queue()                                  # Queue[Any] — no error
q: multiprocessing.queues.Queue[Job] = ctx.Queue()   # checked

The same discipline applies here: declare type StartMethod = Literal["spawn", "fork", "forkserver"] and keep every list and tuple of methods annotated with it. If you type them as str, the return will not narrow and mypy will reject the function — which is the checker doing its job.

Loading visualization…