Python explainers

What mypy --strict Actually Enables — The Exact Thirteen

"We're on strict" means thirteen different things to thirteen teams. Here is the exact bundle, flag by flag, what each one costs you, the two that this course is built on, and the two everybody wrongly believes are in there.

The Eight Things --strict Does Not Catch

Passing the gate is not the same as being well typed. Eight verified holes in mypy --strict — explicit Any, laundered Any, dead code, missing @override, cast, blanket ignores, deprecated spellings, mutable defaults — and the config that closes them.

The Ruff Ruleset — And The Rules You Should Not Gate On

A linter with a bad false-positive rate does not teach discipline, it teaches # noqa. The Bronze and Silver rule selections used in this course, what each prefix actually is, and seven checks worth displaying but never blocking a merge on.

Cyclomatic vs Cognitive Complexity

A flat twelve-arm match scores 13 on McCabe and reads fine. Four levels of nesting scores 5 and is unreadable. Two metrics that measure different things, why only one predicts review pain, and what this course gates on.

mypy, pyright, ty — An Honest Comparison

"My code type-checks" is a checker-relative claim. Where mypy and pyright structurally disagree — unannotated bodies, joins vs unions, Unknown vs Any, narrowing, plugins — three divergences verified against mypy 2.3, and why running both is a real strategy.

The Version Landscape in 2026

Which Pythons are alive, what 3.12 already gives you for free, and the three-way split that actually matters: syntax you cannot feature-detect, library features you can backport, and behaviour changes that fail silently.

PEP 696: default values for type parameters

Repo[T = str] is the difference between a generic API people use and one they route around. The ordering rules, the TypeVarTuple exception, typing.NoDefault, and what it does to --disallow-any-generics.

PEP 649/749: the end of quoted forward references

Python 3.14 made annotations lazily evaluated. Three formats — VALUE, FORWARDREF, STRING — a new annotationlib module, and one rule that catches everybody: from __future__ import annotations still wins.

Protocol or ABC: choosing the right kind of interface

Two mechanisms, two different guarantees. A decision rule you can defend in review, the case for shipping both, and the honest limits of ABCMeta.register.

Disjoint bases (PEP 800): the types that cannot exist

Why mypy calls a branch unreachable and pyright does not, why isinstance against a Protocol never narrows to Never, and how @typing.disjoint_base finally makes the rule explicit.

Closed TypedDicts and typed extra items (PEP 728)

How to say this is the complete wire format, what extra_items buys beyond closed=True, and the payoff nobody expected: TypedDicts that are finally assignable to Mapping.

TypeForm: typing the values that are themselves types

Why every validator, deserializer and DI container ends up with a def parse(t: type[T], raw: object) -> T that rejects Optional, and what TypeForm changes.

py.typed and PEP 561: how a checker finds your types

You annotated the library exhaustively, published it, and every consumer still sees Any. The marker file, the naming rules, and the resolution order that silently overrides your real types.

__slots__, part 1: what it actually saves (with numbers)

Teams add slots=True everywhere for "speed" and get almost none. Measured memory, measured attribute access, why the folk numbers are a decade out of date — and the static win that is the real reason to use it.

Choosing a record type: dataclass, NamedTuple, TypedDict, attrs, pydantic, msgspec

Two symmetric mistakes — BaseModel everywhere and dict[str, Any] everywhere — with measured construction, equality and memory numbers for each option, and a decision procedure that starts from the trust boundary.

Descriptor-typed fields: the third way to compute a field

A dataclass wires the descriptor protocol straight through a field's default value. It is documented, elegant, usually the wrong tool — and it is how you read SQLAlchemy columns, Django fields and most ORM-adjacent code.

make_dataclass and building record types at runtime

Schema-driven pipelines reach for dict[str, Any] because "we don't know the columns until runtime". make_dataclass gives you real objects instead — and confining the dynamism to one factory is what stops Any spreading through everything downstream.

Migration table: dataclass behaviour from 3.10 to 3.15

Every dataclass behaviour change across six releases, with the "works on my machine, fails in CI" cases called out — including the one where a class that runs on 3.14 raises TypeError on 3.13 with no static warning.

The old way → new way reference table

Every modernisation worth making in a Python codebase, paired with the failure mode that justifies it — and the four places where the old way is still right.

Where f-strings are the wrong answer

Four places where 'modernise all string formatting' makes the code slower, less observable, untranslatable, or injectable — and the type that turns injection into a static error.

When match is better than if/elif — and when it isn't

A reviewer's decision procedure for pattern matching, plus the performance claim that is not true and the two failure modes that got match banned at some shops.

PEP 758: except A, B: without parentheses

A small 3.14 syntax relaxation that closes a Python 2 trap for good — and the one place where the parentheses are still mandatory.

PEP 798: unpacking in comprehensions

The 3.15 replacement for chain.from_iterable and the double-for flattening comprehension — with its restrictions, its precedence trap, and why it does not help dict comprehensions.

PEP 810: explicit lazy imports

The 3.15 replacement for the function-local import hack: what lazy import binds, where it is a SyntaxError, and the five ways it can surprise you at runtime.

Counter: counting, tallying, multiset arithmetic

The single most reinvented container in the standard library, the heap hiding inside most_common(n), and why a bare Counter annotation is a type error.

defaultdict vs setdefault vs dict.get

Three ways to handle a missing key, one of which mutates on read — and why returning a defaultdict from a public function exports that behaviour to every caller.

deque: the O(1) ends, maxlen, and rotate

A BFS queue or a last-N-events buffer built on a list is quadratic — fine at 100 items, unusable at 100k. Plus the maxlen annotation that --strict makes you narrow.

ChainMap: layered configuration and scope stacks

Merging config with {**defaults, **env, **cli} copies eagerly and destroys the one question you will be asked in the incident review: which layer did this value come from?

OrderedDict in 2026: the three things dict still cannot do

Reviewers see OrderedDict and assume Python 2. Three of its behaviours genuinely survived the 3.7 ordering guarantee — knowing which lets you defend it or delete it.

itertools foundations: chain, islice, and the laziness contract

list(a) + list(b) materialises both sides to concatenate them. On a 10 GB log that is the difference between working and OOM — and the element type of chain joins rather than unions.

groupby: two silent failures, one of which passes your test

Unsorted input returns plausible partial groups; list(groupby(...)) returns the right keys with empty groups. Both look correct from a distance.

batched: chunking without the off-by-one

The hand-rolled range(0, len(xs), n) version only works on sequences — it breaks on generators, cursors and file handles, which is exactly where batching matters.

pairwise and accumulate: the two loops you stop writing

for i in range(len(xs) - 1) has an off-by-one and an empty-list crash. pairwise has neither, works on iterators, and its tuple is arity-checked — unlike batched's.

tee and product: the two itertools functions that will OOM you

Both are marketed as lazy and both hold everything. tee is the worse trap, because draining one side makes its buffer strictly larger than the list() you were avoiding.

The rest of itertools, one justification each

cycle, repeat, zip_longest, takewhile, dropwhile, filterfalse, starmap, compress — the single situation that justifies each, and why zip(strict=True) beats zip_longest.

cache and lru_cache: the decorator that deletes your signature

Adding @cache to a hot function is a routine optimisation that silently removes that function from your type checker's coverage. Verified: fib("nope"), fib(1,2,3) and fib() all pass --strict.

lru_cache on methods: the leak that profiles as 'objects that should be dead'

The cache lives on the class, and self is part of the key — so it holds a strong reference to every instance it has ever seen. Verified: after del and gc.collect(), the instance is still alive.

cached_property: the 3.12 lock removal and the __slots__ crash

Code written pre-3.12 that relied on the implicit once-only guarantee under threading is now racy. And a __slots__ class with a cached_property type-checks clean and raises at runtime.

singledispatch: open extension, and the exhaustiveness you give up

The alternative is a growing isinstance ladder that every new type must edit — a closed set masquerading as an open one. The cost is that no checker can prove your dispatch is total.

partial, partialmethod, and the 3.14 change that binds self

functools.partial became a method descriptor in 3.14, so a partial stored as a class attribute now binds self. 3.13 warned; 3.14 broke. The documented fix is staticmethod.

reduce: the narrow band where it earns its place

reduce(operator.add, list_of_lists) is quadratic — a classic performance bug wearing a functional hat. Guido removed it from builtins for a reason; here is the remaining 5%.

wraps: the eight attributes a decorator has to carry

A fifteen-minute recap of the metadata functools.wraps copies, why __wrapped__ is the one that makes introspection work, and what ParamSpec adds on top.

total_ordering: six comparisons from two, and the NotImplemented rule

People return False instead of NotImplemented, which silently breaks reflected comparison and produces an asymmetric ==. Also: mypy 2.3 understands the decorator, so that advice is stale.

contextmanager: the try/finally that is not optional

A bare yield runs cleanup on the happy path and silently not when the body raises — the exact case the context manager exists for. Plus: they are single-use, and the return type is Iterator[T].

ExitStack: N resources when N is a runtime value

Opening N files where N is not known until runtime has no clean with syntax. ExitStack gives you one, plus the pop_all().close idiom for transactional acquisition.

suppress, closing, nullcontext, chdir

nullcontext eliminates the use-it-if-given-otherwise-open-one duplication. suppress abandons the rest of the block, which is not the same as except: pass. chdir is process-global.

asynccontextmanager, aclosing, AsyncExitStack

An async generator holding a DB connection and abandoned mid-iteration will not release it promptly. The loop's finalisation hook runs later — or not before shutdown.

pathlib: path: str accepts a URL, an SQL fragment, or a user's name

The correspondence table with os.path, why p / '/absolute' is a path-injection vulnerability in one line, and the difference between resolve() and absolute().

pathlib advanced: walk, glob, and the 3.13 change nobody announced

Path.walk lands in 3.12, full_match and from_uri in 3.13, copy/move in 3.14 — and 3.13 silently changed what a trailing ** matches.

StrEnum: the migration path out of a module of string constants

Every existing comparison and JSON serialisation keeps working, and you gain iteration, membership, a namespace and a parse error. Plus a verified checker disagreement about Literal.

Flag and IntFlag: bitmasks that print, iterate and validate

The hand-rolled alternative is integer constants and flags & PERM_READ, which produces unreadable debug output, allows meaningless values, and offers zero introspection.

tomllib: config parsing is where untyped data enters your program

tomllib.load returns dict[str, Any], so teaching it without the boundary conversion produces code that passes --strict while being completely unchecked.

datetime: two silent data-corruption sources in one lesson

utcnow() returns a NAIVE datetime that represents UTC, and .timestamp() on it interprets it as local time — measured 28,800 s of skew. Arithmetic on a zoned datetime is wall-clock, not absolute.

statistics, bisect, heapq: one signature trap each

quantiles returns the n-1 cut points; insort in a loop is O(n²); and a heap of (priority, payload) tuples works perfectly until two items share a priority.

Your exception hierarchy is an API

The exceptions your library raises are a public contract you never wrote down. How to design one on purpose: a package-level base, subclassing along recoverability, structured attributes instead of encoded messages, and dual-inheriting from a builtin so existing handlers survive.

Chaining: raise ... from, __cause__, __context__, from None

Implicit context, explicit cause, and deliberate suppression — which of the three you get, which you should want, and why from None is a decision about secrets and implementation details rather than about tidiness. Plus bare raise, add_note, and typing the chain.

Bare except, blind except Exception, and where to catch

Why except: catches Ctrl-C, why PEP 760 will never save you from it, and the rule that actually matters: narrowly near the raise when you can recover, broadly only at a boundary where a failed unit of work is a complete outcome.

ExceptionGroup and except*

Concurrency changed the shape of failure: when ten tasks fail differently, except makes you lose nine. How groups model a failure as a tree, why BaseExceptionGroup is deliberately not an Exception, and why except* ValueError binds an ExceptionGroup[ValueError].

Retry, backoff, and the difference between retryable and terminal

The retry storm, drawn out mechanically, and the four properties that prevent it: a budget, exponential backoff, full jitter, and a retryable/terminal split. Plus why idempotency belongs in the classification table and why injecting sleep is what makes the schedule testable.

Deprecating public API: @warnings.deprecated

PEP 702's decorator, what it does on functions versus classes, why category=None is often the right setting — and why the runtime warning is the least important channel, because DeprecationWarning is hidden by default outside __main__.

NoReturn, Never, and functions that do not come back

A cross-reference to items 1.3 and 1.4, plus the one thing that belongs in a failure track: the -> NoReturn helper idiom, and why annotating a raising method -> None instead of -> Never quietly poisons every caller's narrowing.

Typing the error path

The exception hierarchy is part of the signature and the type system cannot express it. What to do instead: a maintained raises-contract, Result at one or two boundaries (with an honest account of why it is not a house style), __exit__ -> None, and ParamSpec so decorators do not launder your types into Any.

warnings in libraries: categories, stacklevel, and filters

Warnings are for developers about code, not for operators about events — which explains the categories, the once-per-location default, and why forgetting stacklevel=2 is the reason nobody acts on your deprecation. Plus why a library must never install a global filter, and why catch_warnings is thread-unsafe.

Reading modern Python error messages

A version tour of what 3.10 through 3.15 added to tracebacks — specialised syntax errors, PEP 657 fine-grained carets, import and keyword suggestions, colour, stdlib-shadowing detection — and the half you have to write yourself: add_note().

Generators as a memory strategy

One bracket changes 40 MB into nothing. But laziness does not only move memory — it moves when work happens, when errors surface, and whether the result can be read twice. Those three are the production consequences.

Iterator, Iterable, Generator, Sequence

Four ABCs that people pick by vibes. Each one is a promise about what your function will do to the argument — and getting it wrong produces a bug that has no exception attached to it.

Async iterators and async generators

Breaking out of an async for leaves the generator suspended, and its finally runs at some later time you do not control. Here is the protocol, the finalisation problem, and the two-line fix.

Never yield inside a TaskGroup or timeout scope

The most appealing-looking abstraction in async Python — an @asynccontextmanager wrapping a TaskGroup — delivers cancellations to unrelated tasks. PEP 789 documents it precisely and has not landed, so the mitigation is a review rule.

zip(strict=True) and the silent-truncation bug

zip() stops at the shortest argument and says nothing. That is a data-corruption bug no test with equal-length fixtures will ever find, and the fix has been one keyword argument since 3.10.

pytest Project Layout: conftest, rootdir, Import Modes

Why a second file called test_utils.py breaks your suite, what rootdir actually decides, the three import modes and their real trade-offs, and the ini settings that replace sys.path hacking.

pytest 9 Configuration and Unified Strict Mode

A typo'd marker is a silent no-op that runs the tests you meant to skip. pytest 9's native TOML config, standalone pytest.toml and single strict switch, plus the eight-line baseline every project should start from.

Fixtures: Scope, Teardown, Factories, Autouse

One state-changing action per fixture, and why the pytest docs' sentence about setup exceptions is the reason. Scopes and the caching rule, yield teardown order, factory-as-fixture, and why autouse should be rare.

Why Constructor Injection Beats mock.patch

A suite built on unspec'd patches goes green after you delete the function under test. Where patching actually binds, why a bare Mock checks nothing, when patching is still right, and what autospec buys you.

parametrize: Tables of Cases Instead of Loops of Asserts

A loop over ten inputs reports one failure and hides nine. pytest.param with ids and marks, stacked decorators and their exact order, indirect parametrisation, and what pytest 9's strict ids change.

Property-Based Testing: Naming the Thing That Is Always True

The skill is identifying the property, not generating random input. The six-property taxonomy — round-trip, idempotence, invariance, metamorphic, oracle, never-crashes — with the trap that round-trip only ever compares a system with itself.

Stateful Testing: Letting the Machine Find the Sequence

The bugs in a cache are sequences nobody imagines. RuleBasedStateMachine, the model-based approach, invariants versus postconditions, bundles, and why the minimal reproducing sequence is the whole payoff.

Typing Your Test Suite Under --strict

An untyped suite lets you rename a parameter and still see green. pytest's public types, why a yield fixture is Iterator[T], why bare CaptureFixture fails, and the callback Protocol that Callable cannot express.

Deterministic Tests for Concurrency

A concurrency test that usually passes is a coin flip with a CI bill. Controllable clocks instead of sleeps, Barrier and Event to pin an interleaving, all_tasks() for leak detection, and the two flags that turn a missing await into a failure.

What to Test and What Not To

Testing private functions locks in the implementation. Coverage as a gate manufactures assertion-free tests. Mocking what you do not own encodes a belief about someone else's API. And flakiness is a design signal, not a tooling problem.

S101 and Per-File Rule Scoping

assert is correct in tests and wrong in library code, because python -O deletes it. The worked example of scoping a lint rule to where it applies instead of switching it off — the habit that keeps a lint config trusted.

What the GIL Actually Guarantees

Thread switches happen between bytecodes — that is the entire guarantee. Everything else people believe about the GIL is a consequence of it, or is false.

Atomicity Illusions: Check-Then-Act and Read-Modify-Write

The published list of atomic operations is an implementation detail you should read once and never depend on. The invariant you need almost always spans a read and a write.

Threading Primitives Beyond Lock

Condition, Event, Semaphore, Barrier — which one a problem wants, why wait_for replaces every hand-written wait loop, and why notify() stalls a producer/consumer buffer.

Thread Lifecycle: Daemons, Swallowed Exceptions, Graceful Shutdown

An uncaught exception in a thread kills the thread and leaves the program running. join() means finished, not succeeded. And a daemon thread never runs its finally block.

concurrent.futures: ThreadPoolExecutor Done Properly

The unobserved future that swallows an exception, the two things map() does that map() does not, pool sizing in a container, and what cancel_futures actually cancels.

asyncio.run, Runner, and the Death of get_event_loop

What asyncio.run cleans up that run_until_complete does not, why Runner exists, and the 3.14 change that breaks every tutorial written between 2016 and 2020.

Typing Async Code: Coroutine vs Awaitable vs AsyncIterator

Annotate parameters as Awaitable and returns concretely — and never put async def in a Protocol unless you mean to forbid caches, batchers and test doubles forever.

Tasks, Task Lifetime, and the Fire-and-Forget GC Bug

The event loop keeps only weak references to tasks. Discard the reference and the task can vanish mid-execution — and its exception surfaces hours later at collection time.

TaskGroup Supersedes gather

gather's failure mode leaves siblings running detached. TaskGroup guarantees that nothing it started outlives the block — and gather is still right for partial results.

Cancellation: CancelledError Is a Protocol, Not an Error

cancel() throws an exception into a coroutine and lets it decide. Why it inherits BaseException, what cancelling()/uncancel() are really for, and why shield() is usually a bug.

Timeouts: asyncio.timeout Supersedes wait_for

Per-call timeouts do not compose; deadlines do. Why TimeoutError can only be caught outside the block, how nesting attributes correctly, and the retry rule everyone misses.

Async Context Managers: __aenter__ Returns Self, __aexit__ Returns bool | None

A cross-reference to 5.24 and 7.4, plus the two signatures worth getting right: Self on the way in, and why annotating __aexit__ as -> bool invites silent exception suppression.

asyncio.Queue: Backpressure, join/task_done, and Shutdown

maxsize is the most important argument you are not passing. task_done in a finally is the difference between a clean shutdown and a silent hang. And 3.13's Queue.shutdown finally removes the sentinel union.

Bridging Blocking Code: to_thread, Executors, Cross-Thread Scheduling

One blocking call on the loop thread freezes every request. The two directions across the boundary, why the default executor is a hidden global bottleneck, and why the wrapper needs ParamSpec.

Graceful Shutdown of an asyncio Service

SIGTERM then SIGKILL after 30 seconds. The signal handler can only set a flag, producers must be cancelled before consumers, and serve() must return normally rather than raising CancelledError.

Debugging Async in Production

Debug mode's slow-callback warning is the fastest way to find a blocking call. On 3.14, python -m asyncio pstree PID renders the live await graph of a running process with no code change.

contextvars Across Tasks and Threads

A task copies the context, to_thread propagates it, run_in_executor does not — and 3.14's thread_inherit_context defaults differently on free-threaded and GIL builds.

Eager Tasks and asyncio Performance

The eager task factory runs a coroutine synchronously during create_task and only schedules it if it actually blocks — 2x to 5x on the right workload, and a real ordering change on the wrong one.

Concurrency Limiters Are Not Rate Limiters

A cross-reference to 7.6, plus the distinction that gets services throttled: a semaphore holds concurrency constant, and rate is whatever latency makes it.

asyncio vs anyio and trio

trio pioneered structured concurrency and asyncio adopted the shape. What still differs: the cancel scope as a first-class value, and level-triggered versus edge-triggered cancellation.

mypy vs pyright for Async Code

Where the two checkers actually diverge on async — PEP 696 defaults, bracketless except*, Coroutine vs Awaitable in overloads — and why pinning the checker version is not optional.

Choosing a Concurrency Model: A Decision Procedure

Four situations, four answers — and the one most often got wrong in both directions: CPU-bound work inside a C library that releases the GIL, where threads already work.

Deterministic Concurrency Testing

A cross-reference to 8.10, and the reason every problem in this track asserts a log, a count or an ordering — never a duration.

Why fork Plus Threads Is Genuinely Dangerous

The failure mode is a hang, not a crash — one task in ten thousand, no traceback, no exit code. Why fork() copies locks without owners, the roster of threads you did not know you had, and why glibc hides it.

Pool vs ProcessPoolExecutor, and the chunksize Trap

Executor.map defaults chunksize to 1 while Pool.map computes it for you — a ~100x IPC difference, and the usual reason people conclude the executor is slower. Plus what each API has that the other does not.

Debugging Hangs, Zombies and Dead Workers

Multiprocessing bugs are the ones on-call cannot debug, because a traceback and a log line are exactly what is missing. A triage tree: py-spy, faulthandler in the initializer, exit codes, and four signature failures with their causes.

What Subinterpreters Cost, and What They Cannot Do

"Subinterpreters remove the GIL" will be the most-repeated wrong statement about 3.14. The measured creation cost, the memory per interpreter, the extension-compatibility wall, and the three things they are not.

The CPython object model: names, objects and why assignment never copies

Every value is a PyObject with a refcount and a type. Names bind to those objects; assignment moves a pointer, never data. Almost every aliasing bug a mid-level engineer writes is this one fact, not yet internalised.

Reference counting, and the four ways it stops being deterministic

Refcounting is why __del__ appears to work. Cycles, exception tracebacks, the REPL's underscore and any global registry each break it — and on the free-threaded build prompt destruction is not even the design goal.

Reference cycles and the generational collector

Cycles are not leaks — they are unbounded-latency destruction, which is the failure behind 'our RSS sawtooths and occasionally OOMs'. How the collector finds them, what tp_traverse actually does, and why PEP 442 made __del__ in a cycle survivable.

The incremental-GC episode: how to read release notes properly

Python 3.14.0 shipped an incremental garbage collector. Python 3.14.5 reverted it. 'We are on 3.14' is not a precise enough answer in an incident review, and a large slice of the 2025-2026 blog corpus on this topic is now simply wrong.

gc.freeze, pre-fork copy-on-write, and why gc.disable() is usually a mistake

The standard fix for 'our pre-forked workers each grow to 800 MB even though the model is loaded once before fork'. Why writing a refcount dirties a shared page, what gc.freeze() moves, and the narrow case where disabling the collector is defensible.

Weak references, and why @lru_cache on a method leaks

A registry that remembers is a registry that leaks. weakref.ref, WeakValueDictionary, WeakSet and finalize — plus the two things that surprise people: not every object supports weak references, and weakref.ref[T]() returns T | None, which --strict makes you handle.

Identity vs equality: the three guarantees, and everything else

The FAQ gives exactly three cases where identity is guaranteed. Everything else — small ints, interned strings, constant folding — is implementation detail, and x = 1000; y = 1000; x is y is True at module scope and False in a function.

Interning: a real memory optimisation, and never a correctness guarantee

sys.intern genuinely cuts memory when you parse millions of records with few distinct values. It is also documented to not immortalise strings — except on the free-threaded build, where it does, turning 'intern the parsed field names' into an unbounded leak on untrusted input.

Shallow vs deep copy, the memo dict, and copy.replace

Shallow-copying a request and mutating a nested list is cross-request data corruption. Reflexively reaching for deepcopy turns a 40 ms endpoint into a 400 ms one — measured, 1.709 ms vs 0.00051 ms on a 1,000-element nested structure, about 3,300x. Both are wrong; the fix is deciding per field.

__slots__ in 2026: measuring it properly, and why attribute access is a wash

The folklore numbers for __slots__ are wrong in both directions. It saves about 30%, not 5-10x, and it does not speed up attribute access — 3.9 ns versus 3.8 ns. Getting the right answer requires tracemalloc rather than sys.getsizeof, and knowing what the specialising interpreter already did for you.

sys.getsizeof and why it lies

getsizeof reports only what is directly attributed to an object, not what it refers to. A list of 1,000 objects reports 8,856 bytes while its contents total 16,000 more — and capacity planning done on that number under-reports by one to three orders of magnitude.

tracemalloc: turning 'the service leaks' into a file and a line number

No install, no root, no ptrace permissions — which is what matters when the leak only reproduces in a locked-down container. Snapshots, compare_to, get_traced_memory, reset_peak, and the limit that sends you to memray.

Big-O traps I: list vs deque vs set vs dict, with numbers

Measured on 3.14.6: 100,000 list.insert(0, i) is 1157 ms against deque.appendleft at 1.04 ms. 10,000 membership tests in a 10,000-element list is 243 ms against 0.14 ms for a set. Identical code, only n changed — which is why it passed staging.

Big-O traps II: the string-concatenation cliff

50,000 concatenations: s += x is 1.25 ms, ''.join is 0.57 ms — and the same s += x loop with one live alias is 131 ms. A 105x cliff from a line that looks harmless, because CPython's in-place optimisation requires the next opcode to be STORE_FAST and the local to hold the only reference.

timeit: microbenchmarking that isn't a lie

Two documented behaviours everyone should know: timeit disables the GC while timing, and the docs say the min() of the result is probably the only number you should be interested in. Plus the reason to prefer the callable form in a typed codebase — mypy sees nothing at all inside a string statement.

Why wall-clock benchmarking misleads

A 3% improvement measured once on a laptop is indistinguishable from noise. Frequency scaling, IRQ handling, NUMA locality, ASLR and hash randomisation all move your number without touching your code — pyperf's docs show a maximum 39% above the mean on an untuned system.

Deterministic profiling: tottime vs cumtime, and what cProfile is not for

Optimising the wrong function is the default outcome of optimising without a profiler; reading cumtime when you needed tottime is the default outcome of using one badly. Plus the caveat that invalidates a whole genre of blog post: the profilers are not for benchmarking, and they make C code look faster than Python.

PEP 799 and Tachyon: profiling a running production process

python -m profiling.sampling attach PID, with zero measurable overhead on the target, a gil mode that tells you which thread is serialising everything, and operational constraints you must plan for before the incident — matching minor versions, matching build, and ptrace permissions.

The profiling toolbox beyond the stdlib

The stdlib gives you function-level CPU, process-attach CPU on 3.15 only, and Python-only memory. Real incidents routinely need line-level attribution, native-allocation visibility, or an interpreter older than 3.15 — which is py-spy, memray, scalene and line_profiler.

The CPython JIT: an honest status report

It is off by default in release binaries, it does not work on the free-threaded build you also want, and on 3.14 the documented range is 10% slower to 20% faster. 3.15's tracing frontend is a genuine step change at 8-9% geomean — with a per-benchmark spread from a 15% slowdown to over 100% speedup, and a not-yet-final note attached.

When to reach for NumPy, and when it makes things slower

The decision is usually framed as speed and is really about memory layout. 1,000,000 int64: a list costs 8.0 MB for the pointer array alone plus a PyObject each; array.array is 8.18 MB total; np.arange(...).nbytes is exactly 8.0 MB with zero per-element overhead. And below a few hundred elements NumPy is slower.

When to write a C or Rust extension

The most expensive optimisation available: a build toolchain, cross-platform wheels now multiplied by free-threaded variants, and either a new class of memory bugs or a new language. The framing that predicts success before you write anything is per-call boundary cost — move the loop, not the loop body.

src layout vs flat layout: the import bug you cannot see

A test suite can be green in CI while the wheel it just shipped is broken, because the tests imported the working tree and never touched the packaged artefact. src layout is the structural fix.

How Python actually resolves an import

Every ModuleNotFoundError-but-the-file-is-right-there, every tests-import-a-different-copy, and every conftest.py with sys.path.insert traces back to six rules.

__init__.py discipline and the public API surface

An __init__.py that imports everything for convenience turns import mypkg into 400 ms and a hard dependency on every optional extra you have. Here is what belongs in it and what does not.

Namespace packages: when to omit __init__.py

Most namespace packages in the wild are accidents — somebody forgot an __init__.py and nothing broke, so nobody noticed until packaging, test collection or mypy started behaving strangely.

Circular imports: diagnosis and the four real fixes

A cycle is a design smell rendered as a crash. Teams fix it with a function-local import, which converts a compile-time architectural problem into a scattered runtime one.

Layered architecture: domain, application, adapters

The alternative is a codebase where every module can import every other, so every change has unbounded blast radius, every test needs a database, and nobody can reason locally.

Dependency injection without a framework

The difference between a class you can test in microseconds and one that needs Docker is a parameter. No container, no decorators, no registry — just the collaborator arriving from outside.

The composition root

Module-level db = connect(...) singletons and import-time side effects are why a test suite cannot run without a database. One place builds the real object graph, and it is not import time.

Enforcing structure: import-graph fitness functions

A layout without enforcement is decoration. The enforcement artefact — a contract file, a namespace assertion, an import-cost budget — is the part that survives the people who wrote it.

Typing the seam: Protocols in signatures

A pointer, not a lesson: the Protocol mechanics live in track 2. What this track adds is where the Protocol goes, and which of them belongs in a public signature.

pyproject.toml as the single source of truth

setup.py plus setup.cfg plus MANIFEST.in plus .flake8 plus mypy.ini plus pytest.ini plus tox.ini is why nobody knows where a setting lives and why CI and local disagree.

Version specifiers and the upper-bound argument

One over-tight upper bound in a widely-used library strands thousands of downstream projects with no override. An unbounded >= in an application with no lockfile means the deploy pulls a different tree every time.

Build backends and editable installs

"It works when I pip install -e . but the wheel is broken" is almost always a flat layout plus an editable install papering over a missing file.

Dependency groups vs extras vs requirements.txt

Using extras for dev dependencies leaks your CI toolchain into published metadata, so every consumer sees pip install yourlib[dev] as supported and every tool you listed becomes a compatibility constraint you owe them.

Lockfiles and reproducible installs

Without a lockfile your deploy is a fresh resolution against a mutable index — the artefact you tested is not the artefact you shipped. Without hashes, a compromised index silently changes your code.

Entry points, console_scripts and __main__.py

A CLI whose logic lives inside if __name__ == "__main__": cannot be unit-tested, reused or exposed as a console script — and returning a str from main() exits non-zero on success, which every CI system reads as failure.

uv as the project workflow

The pyenv + virtualenv + pip + pip-tools + pipx + tox stack has six failure modes and no shared lockfile. One tool with one lockfile removes most of them.

Ruff: a configuration you can defend in review

Lint config is where a team encodes its standards, and a bad one costs more than none — too permissive catches nothing, too aggressive teaches everyone to write # noqa.

Beyond --strict: optional error codes and mypy 2.0's changed defaults

A mypy major upgrade that changes inference defaults produces a wall of new errors. Knowing which four flags caused it turns a week into an afternoon.

A library that does not force a start method on its users

A library calling set_start_method at import breaks every downstream application in a way the application author cannot fix without vendoring you — the concurrency equivalent of logging.basicConfig() in a library.

The CI gate: what a real pipeline runs, in what order

A gate that runs the tools but not on the artefact you ship is theatre. Here is the order, why it is that order, and the step everyone skips.

Two scores, one gate

Trust in the gate is the scarce resource, and every false positive spends it. Keep the pass/fail set small and near-perfect, and put everything else in a score that never blocks.

Seven mechanisms for testing structure

How to turn "is this well designed?" into something a machine can answer — and, just as important, which parts of that question a machine must never be asked.

Version budgeting: what you may actually use

Getting this wrong in an application means a broken image. In a library it means breaking every downstream user on an older interpreter, with an import-time crash rather than graceful degradation.

Untrusted deserialisation and hostile input

pickle across a trust boundary is arbitrary code execution by design, not by accident. Plus the resource-exhaustion attacks that need no exploit at all, and the tarfile extraction-filter version table that decides whether your library is safe by default.

Observability of the runtime itself: an incident playbook keyed by symptom

You do not start an incident knowing which tool you need — you start with a symptom. A lookup table from what you can see to what to run, plus the operational constraints on attaching a profiler that must be settled before the incident, not during it.

Publishing an API you can evolve

"Breaking change" understood as signature changes only is wrong — an exception type, an iteration order and a default argument are all breaking, and none of them change a signature. What a public surface actually consists of, how to declare it so tooling enforces it, and the deprecation cycle that reaches users through their type checker rather than a warning nobody sees.

Packaging the artefact: the bugs that only exist in the wheel

setup.py the file is not deprecated; invoking it as a CLI is. The missing py.typed marker, the src-layout bug, and why a packaging exercise that never installs the artefact leaves both invisible — plus the seven-step acceptance check that finds them.

Security linting without crying wolf

A security linter with a bad false-positive rate teaches the team to add # noqa, which is worse than no linter. Ruff's S rules sorted into three tiers — hard failure, review prompt, and per-directory — plus the checks (constant-time comparison, secrets vs random) that no rule engine can make for you.

The trust-boundary checklist

Every injection class is the same mistake at a different boundary: data from outside is a string until you have parsed it into a type. Ten rules on one page, and the NewType tainted/trusted split that turns nine of them from things a reviewer must remember into something CI proves.