Size a Process Pool to a Container's CPU Limit

Your service runs in a container on a 64-core server. ProcessPoolExecutor() without max_workers starts one worker per CPU that Python sees, which is 64. But the container's limit may be half a CPU's worth of time. Then 64 workers take turns on half a CPU, and each one holds its own memory. Set max_workers from the container's limit instead.

On Linux, that limit comes from the container's cgroup, short for control group. A cgroup is the kernel feature that caps the CPU time a group of processes may use. The limit is a quota of CPU time per period, kept in files:

• cgroup v2, the current version, keeps both numbers in one file named cpu.max. 50000 100000 means 50,000 microseconds of CPU time in every 100,000, which is half a CPU. max 100000 means no limit.
• cgroup v1, the older version, keeps the quota in cpu.cfs_quota_us and the period in cpu.cfs_period_us. A quota of -1 means no limit.

What to write

Write solve, a pure function that recommends a worker count. The caller has already read the files and counted the CPUs for you.

def solve(
    *,
    cgroup: dict[str, str],
    affinity: int | None,
    logical: int,
    env: dict[str, str],
) -> tuple[float | None, int | None, int, int, Source]: ...

• cgroup maps file names to well-formed contents. Ignore other names.
• logical is the machine's CPU count, always positive.
• affinity is how many of those CPUs the process may run on. It is positive, or None if unknown.
• Return (quota, affinity, logical, workers, source), with affinity and logical unchanged. source is a label from the supplied Source type.

Apply these rules in order:

1. The override. The PYTHON_CPU_COUNT environment variable lets an operator force a count. If env["PYTHON_CPU_COUNT"] is a nonempty string of decimal digits and int() turns it into a positive number, return (None, affinity, logical, value, "env") without capping the value. Leading zeros and non-ASCII decimal digits such as "٣" count. Skip any other value, such as "0", "+2", " 2 " or a string where int() raises, and go on to rule 2.
2. cgroup v2. If cpu.max exists, use it and never read the v1 files, even when it says max. When its first field is a number, the quota is the first field divided by the second, with source "cgroup_v2".
3. cgroup v1. If cpu.max is missing and both v1 files exist, the quota is cpu.cfs_quota_us divided by cpu.cfs_period_us, with source "cgroup_v1". Use it only when cpu.cfs_quota_us is positive.
4. No limit. Return None as the quota. The worker count is affinity, with source "affinity". If affinity is None, it is logical, with source "logical".

With a quota, the worker count is max(1, min(floor(quota), cap)), where cap is affinity, or logical when affinity is None.

assert solve(cgroup={"cpu.max": "50000 100000"}, affinity=64,
             logical=64, env={}) == (0.5, 64, 64, 1, "cgroup_v2")
assert solve(cgroup={"cpu.max": "400000 100000"}, affinity=2,
             logical=64, env={}) == (4.0, 2, 64, 2, "cgroup_v2")
assert solve(cgroup={}, affinity=None, logical=8,
             env={"PYTHON_CPU_COUNT": "3"}) == (None, None, 8, 3, "env")

Your submission must pass mypy --strict.

Going further

• A container can inherit a lower limit from a parent cgroup, so production code checks the parents too. See the kernel cgroup documentation.
• Python 3.13 added os.process_cpu_count() and made it the default pool size. It counts the CPUs the process may run on and ignores the cgroup quota. Python 3.13 also reads PYTHON_CPU_COUNT itself, and refuses to start when the value is invalid, such as 0.
• NumPy and similar libraries can start their own threads in each worker. Limit them with OMP_NUM_THREADS or OPENBLAS_NUM_THREADS, set before the library is first imported.