Skip to content

← Parallel Python with Process Pools step 16 of 24

Medium Framework

Choose a Start Method for Each Platform

Your team's pipeline starts a process pool on Linux servers, on developers' Macs, and as a packaged Windows app. When Python 3.14 changed the Linux default from fork to forkserver, the team noticed only when a worker lost a global it used to inherit. Now they want one function that picks the start method on purpose and records why.

The function only decides. Do not start processes or change the global start method.

What to write

solve(*, platform, active_threads, available, frozen_executable) returns (method, reasons). method is a StartMethod or None, and reasons is a tuple of strings in the order the rules added them.

  • platform is "win32", "darwin", or any other string, which stands for Linux or another POSIX system.
  • active_threads is at least 1, as threading.active_count() reports it. That count misses threads that C libraries start, so 1 does not prove that fork is safe.
  • available lists the methods the machine supports, as multiprocessing.get_all_start_methods() reports them. Ignore names you don't recognize, and repeats. Don't change the list or use its order.
  • frozen_executable is True for a packaged app.

Apply these rules in order:

  1. The candidates are the recognized names in available, which are spawn, fork and forkserver.
  2. If fork is a candidate and active_threads > 1, remove it and add "threads-present-fork-unsafe". This avoids the deadlock from the start-methods article.
  3. If frozen_executable is true and platform != "win32", remove spawn and forkserver and add "frozen-posix-excludes-spawn-forkserver". Add this reason even if neither method was a candidate.
  4. Add the platform's reason from the table. Pick the first method in its preference order that is still a candidate.
  5. If none remains, add "no-eligible-method" and return None.
Platform Preference order Reason
win32 spawn windows-spawn-only
darwin spawn, forkserver, fork darwin-prefers-spawn
Any other forkserver, spawn, fork posix-prefers-forkserver

Each order starts with that platform's default in Python 3.14. Rule 3 exists because spawn and forkserver run sys.executable, which in a packaged app is your app, not Python. The team's packager handles that only on Windows.

Add the platform reason even when no candidate remains. A packaged Linux app with four threads and all three methods available returns:

(None, (
    "threads-present-fork-unsafe",
    "frozen-posix-excludes-spawn-forkserver",
    "posix-prefers-forkserver",
    "no-eligible-method",
))

Make a missing choice hard to ignore

Keep type StartMethod = Literal["spawn", "fork", "forkserver"] from the starter. Build the candidates from a tuple typed tuple[StartMethod, ...], so the type checker knows each one is a StartMethod without a cast.

None means no method fits. Check for it before you call multiprocessing.get_context, because get_context(None) returns the default context without an error, and the policy is skipped. mypy does not stop you passing None on, so the check is yours to write.