Atomicity Illusions: Check-Then-Act and Read-Modify-Write

The single most common concurrency defect in production Python is not a deadlock, a race on a file, or a subtle memory-ordering bug. It is a counter.

class RateLimiter:
    def __init__(self, limit: int) -> None:
        self._n = 0
        self._limit = limit

    def allow(self) -> bool:
        if self._n < self._limit:
            self._n += 1
            return True
        return False

Reviewed, merged, deployed. It is wrong, and it is wrong in the way that costs the most: it is right almost all the time.

Two shapes, one bug

Read-modify-write. self._n += 1 is LOAD, ADD, STORE. Two threads both load 41, both add, both store 42. One increment vanished. The count is now permanently low, and nothing anywhere raised.

Check-then-act. if self._n < self._limit: is a read, and the increment that depends on it is a write, and between them the interpreter is free to run another thread. Two threads at _n == limit - 1 both see room and both take it. The count is now permanently over budget — a limiter that does not limit.

The RateLimiter above has both, stacked. Under 64 threads hammering a limit of 5,000, a run will typically grant a few dozen too many and end with a count that does not match the number granted. Two different wrong numbers from five lines of code.

💡The bug is "obvious" once pointed at. Why does it survive code review, and why does it survive tests? click to reveal

It survives review because the code reads as a single decision. The English sentence “if there is room, take it” has no seam in it, so the reader does not look for one. Seeing the seam requires thinking in bytecodes, which is not the mode anyone is in while reading business logic.

It survives tests because a unit test calls allow() from one thread, and from one thread the code is perfectly correct. Even a test that spawns threads usually spawns four, does a hundred iterations, and passes — the window is nanoseconds wide and you have to hit it. The failure probability scales with contention, so the first place it ever fails is production, at peak.

That asymmetry is the argument for locking by default rather than locking when you can prove you need it. The reasoning that concludes “this one is fine” is exactly the reasoning that is unreliable, and the test that would catch a mistake in it is exactly the test nobody writes.

The published atomic list, and why not to learn it

CPython’s FAQ lists operations that are atomic today — L.append(x), D[x] = y, L1.extend(L2), x = D.pop() — and operations that are not: i = i + 1, i += 1, L.append(L[-1]), D[x] = D[x] + 1.

Reading it once is useful, because it makes the shape of the distinction concrete. Depending on it is not, for three reasons.

It is an implementation detail of one build of one interpreter. It comes from the fact that these operations bottom out in a single bytecode dispatching to a single C call. The free-threaded build makes no such promise, and the documentation says so.

It is not stable under refactoring. D[x] = y is atomic when x and y are already-evaluated locals. D[compute(x)] = y is not. And if D is a defaultdict with a Python-level factory, or any mapping whose __setitem__ you wrote, the “atomic” operation contains a Python frame and therefore contains switch points.

Most importantly, single-operation atomicity is rarely the invariant. Even if self._n += 1 were atomic, the limiter would still be broken, because the invariant spans the read and the write. Atomicity of a step tells you nothing about atomicity of a decision.

What correct looks like

import threading
from typing import Final

class RateLimiter:
    def __init__(self, limit: int) -> None:
        self._limit: Final[int] = limit
        self._lock: Final[threading.Lock] = threading.Lock()
        self._n = 0

    def allow(self) -> bool:
        with self._lock:
            if self._n >= self._limit:
                return False
            self._n += 1
            return True

Three things to notice.

The critical section is the decision, not the mutation. The read and the write are inside the same with. Locking only around self._n += 1 would fix the lost update and leave the over-grant.

Final on the lock is load-bearing. A lock that gets rebound is not a lock. Final makes “this attribute is assigned once, in __init__“ a statically checked fact rather than a convention, and mypy will reject the reassignment that a tired refactor would otherwise introduce. The same applies to the limit: a limiter whose limit can be mutated from another thread has reintroduced the problem one level up.

Do not return your mutable state. A method that hands back the internal dict lets a caller read it — and mutate it — with no lock held. Return collections.abc.Mapping, and return an actual read-only object (types.MappingProxyType over a copy taken inside the lock), not just a dict annotated as Mapping. The annotation constrains your callers if they type-check; the proxy constrains them regardless.

💡with self._lock: versus self._lock.acquire() … self._lock.release(). Is the difference only style? click to reveal

No. with releases the lock when the block exits by any path, including an exception, a return from inside the block, and a CancelledError or KeyboardInterrupt arriving mid-section.

The manual form leaks the lock on every path you did not write a finally for. And a leaked lock is the worst failure mode in this whole area: the thread that leaked it carries on, every other thread blocks forever on the next acquire(), and the stack trace you eventually capture points at the innocent blocked threads rather than at the one that walked away holding it.

There is one legitimate reason to reach for the manual form: acquire(timeout=...) returns a bool you have to check, so you can decline to block forever. Even then, the shape is if lock.acquire(timeout=1.0): try: ... finally: lock.release() — the finally is not optional.

The rule

Shared mutable state gets a lock, and the lock covers the whole decision. An uncontended Lock acquire and release is on the order of tens of nanoseconds; it will not be why your service is slow. Reasoning about which of your operations happen to compile to one bytecode will be why your numbers are wrong.