Sizing the Pool: process_cpu_count, cgroups, Oversubscription

This is the single most common cause of “our service got slower after we containerised it”, and the fix is one environment variable.

Since 3.13, multiprocessing.Pool and ProcessPoolExecutor default their worker count from os.process_cpu_count() rather than os.cpu_count(). That is an improvement — it respects CPU affinity, so taskset finally works.

But here is the trap: process_cpu_count() reflects affinity, not cgroup quota. A pod with limits.cpu: 500m scheduled on a 64-core node has no affinity mask at all. Python sees 64, spawns 64 workers, and they share half a core. You have paid for 64 process start-ups, 64 interpreter heaps and a storm of context switches to do the work of one worker, slowly.

The truth lives in cgroup v2 at /sys/fs/cgroup/cpu.max:

50000 100000      -> 50 ms of CPU per 100 ms period = 0.5 cores
max 100000        -> unlimited

and in cgroup v1 as cpu.cfs_quota_us over cpu.cfs_period_us, where -1 means unlimited.

CPython gives you the override: PYTHON_CPU_COUNT=4 or -X cpu_count=4 pins every stdlib default at once — pools, executors, os.process_cpu_count(). One variable in your deployment manifest fixes the entire process tree, including libraries you do not control.

The second trap: nested parallelism

You size a pool to 16 processes. Each one imports NumPy. NumPy’s OpenBLAS starts one thread per core. You now have 16 x 64 = 1024 threads fighting over 64 cores, and the machine spends its time in the scheduler.

Set OMP_NUM_THREADS=1 and MKL_NUM_THREADS=1 in the pool initializer before the first BLAS call — the thread pool is created lazily on first use, so the environment variable has to be in place first — or size your pool to cores // blas_threads. Pick one; do not accidentally pick neither.

Your task

The real function reads files, so make it take a cgroup_root — here, the file contents directly — and it becomes trivially testable. This is not a concession to the exercise; it is how you would make the production version testable too.

def solve(
    *,
    cgroup: dict[str, str],   # filename -> contents
    affinity: int | None,     # len(os.sched_getaffinity(0)), or None off-POSIX
    logical: int,             # os.cpu_count()
    env: dict[str, str],
) -> tuple[float | None, int | None, int, int, Source]:

Return (quota, affinity, logical, recommended_workers, source), resolving in this order:

1. PYTHON_CPU_COUNT parsing to a positive integer -> source "env". ("0" is ignored — CPython treats it as unset.)
2. cpu.max present with a numeric first field -> quota, source "cgroup_v2". A first field of max means unlimited: fall through.
3. cpu.cfs_quota_us positive, over cpu.cfs_period_us -> quota, source "cgroup_v1". -1 means unlimited: fall through.
4. affinity known -> source "affinity".
5. otherwise -> source "logical".

When a quota exists, recommend max(1, min(floor(quota), ceiling)) where the ceiling is affinity if known, else the logical count. Half a core still needs one worker; four cores of quota on a two-core affinity mask still gets two.

The typing lesson

os.process_cpu_count() -> int | None

The Optional is not decorative — it is None when the count is undeterminable, and code that writes range(os.process_cpu_count()) fails --strict for a genuinely good reason.

And os.sched_getaffinity does not exist on Windows or macOS. typeshed guards it with sys.platform, so mypy will reject an unguarded call when checking for those platforms. The fix is if sys.platform == "linux":, not a hasattr check — mypy narrows on the former and treats the latter’s result as Any.