Lock and Batch Updates to a Shared Counter

Four worker processes count page hits in one shared dictionary. Each worker runs counts["hits"] = counts["hits"] + 1 250 times. The total should be 1,000, but one run printed 360.

Why updates get lost

Processes don't share memory, so multiprocessing.Manager() starts a separate server process that holds the shared dictionary. Each worker gets a proxy, an object that forwards every operation to the server and waits for the answer. Each forwarded call is a round trip between processes.

So counts["hits"] = counts["hits"] + 1 is two round trips, a read and then a write. Two workers can both read 10 and both write 11, and one hit is lost. Hold one lock across the read and the write. Create the lock once and pass the same lock to every worker. This snippet needs Python 3.13 or later, where DictProxy accepts type arguments:

import threading
from concurrent.futures import ProcessPoolExecutor
from multiprocessing import Manager
from multiprocessing.managers import DictProxy


def add_hits(counts: DictProxy[str, int], lock: threading.Lock, n: int) -> None:
    for _ in range(n):
        with lock:
            counts["hits"] = counts["hits"] + 1


def main() -> None:
    with Manager() as manager:
        counts = manager.dict({"hits": 0})
        lock = manager.Lock()
        with ProcessPoolExecutor(max_workers=4) as pool:
            jobs = [pool.submit(add_hits, counts, lock, 250) for _ in range(4)]
            for job in jobs:
                job.result()
        print(counts["hits"])  # 1000


if __name__ == "__main__":
    main()

Round trips are slow, so make fewer of them. Batching means reading every key you need in one call and writing them all back in one call, instead of making two calls per key.

What to build

The grader can't start a manager or worker processes. So the starter gives you Proxy, an in-memory stand-in, and you use threads as the workers. Each Proxy call adds 1 to proxy.calls:

Leave Proxy as it is. Its time.sleep(0) calls make threads switch often, so a missing lock usually loses counts.

Fill in SharedCounter(proxy). It owns one threading.Lock, and every method makes its proxy calls while holding that lock. That includes the reads, so a read never sees half a batch.

• incr(key, by=1) -> int reads the key, adds by, writes the result and returns it. by can be zero or negative, and incr stores the key even when the new value is 0.
• incr_many(amounts) -> None makes one read of every key in amounts, then one write of all the new values. An empty mapping still makes both calls. Leave amounts unchanged.
• get(key) -> int makes one read. A missing key returns 0 and stays missing.
• snapshot() -> dict[str, int] makes one dump. The caller can change the returned dict without changing the counts.

Then fill in solve(*, threads, per_thread, keys, batched):

1. Create one Proxy and one SharedCounter. Run threads workers that each do per_thread rounds. An unbatched round calls incr(key) for each key. A batched round calls incr_many once, with 1 for every key.
2. Wait for every worker, and let any worker's exception propagate.
3. Read proxy.calls now, before any snapshot. This is update_calls.
4. Take a snapshot, and add a new key, the check key, to the dict it returns. Then take a second snapshot. The check passes if the check key is absent from the second snapshot and get(key) agrees with the second snapshot for every key in keys.
5. Return (second_snapshot, update_calls, check_passed).

threads and per_thread are zero or more. keys holds distinct strings and can be empty. Any string can be a key, so pick your check key by looking at the first snapshot. ThreadPoolExecutor(max_workers=0) raises ValueError, so start no pool when threads is 0.

Every key ends at threads * per_thread, or the dict is empty when no update ran. update_calls is 2 * threads * per_thread * len(keys) unbatched, and 2 * threads * per_thread batched. For example, four workers doing 100 rounds over three keys give 400 per key, with 2,400 calls unbatched or 800 batched.

Going further

• A plain list you read from a dict proxy is a local copy, so counts["k"].append(1) changes only your copy. Read the list, change it and write it back, all under the lock.
• multiprocessing.Value shares one number, and counter.value += 1 still needs counter.get_lock() held around it.
• See the proxy documentation.