Modernisation without rationale is how a file ends up with three styles and a
reviewer who cannot say which is right. “It’s newer” is not a review comment.
“datetime.utcnow() returns a naive datetime that lies about itself, and
.timestamp() on it silently applies your local offset” is.
This is the reference table for the rest of the track. Every row pairs the migration with the failure mode it removes. Read the right-hand column first; if it does not describe a bug you could plausibly ship, the migration is cosmetic and belongs in a separate commit — or nowhere.
The table
| Old | New | Since | Why (not “it’s newer”) |
|---|---|---|---|
os.path.join/dirname/splitext, glob.glob, os.walk |
pathlib.Path, /, .stem/.suffix, .glob, Path.walk |
3.4; Path.walk 3.12; Path.copy/move 3.14 |
path: str accepts a URL, an SQL fragment, and a user’s name. path: Path does not. The type is the validation. |
'%s' % x, '{}'.format(x) |
f-string | 3.6 | Faster, and the value sits next to its label. Except in logging — see below. |
pytz.timezone(...) + .localize() + .normalize() |
zoneinfo.ZoneInfo |
3.9 |
datetime(2020, 1, 1, tzinfo=pytz.timezone('America/New_York')) silently gives an LMT offset of −07:52 — a 1883-era railway offset — and survives every UTC-based test you have. |
datetime.utcnow() |
datetime.now(UTC) |
UTC alias 3.11 |
utcnow() returns a naive datetime whose value is UTC but whose type says “local”. .timestamp() on it applies the local offset, producing a silent multi-hour skew. |
configparser / hand-rolled INI |
tomllib (read-only) |
3.11 |
Real types out of the parser instead of .getint() / .getboolean() string coercion at every read site. |
os.system, os.popen, subprocess.call/check_output |
subprocess.run([...], check=True, ...) |
3.5 |
os.system(f'convert {name} out.png') with name = "a; rm -rf ~" is remote code execution. A list argv has no shell to inject into. |
| typing.List, Dict, Optional, Union | list, dict, X | None, X | Y | 3.9 / 3.10 | Deprecated — but removal is not planned. Migrate for consistency, not for survival, and know that mypy --strict will not flag the old spelling. That is ruff’s job (UP006, UP007). |
| T = TypeVar('T') + Generic[T]; X: TypeAlias = ... | def f[T](...), class C[T], type X = ... | 3.12 | Real lexical scoping, inferred variance, and lazily-evaluated aliases that kill quoted forward references. |
| asyncio.gather(*tasks) | asyncio.TaskGroup | 3.11 | On failure gather leaves the siblings running, detached, with nobody awaiting them. A TaskGroup cancels them and raises an ExceptionGroup. |
| asyncio.wait_for(coro, t) | async with asyncio.timeout(t): | 3.11 | Composes over more than one await, and supports absolute deadlines via timeout_at. |
| asyncio.get_event_loop(), event-loop policies | asyncio.run / asyncio.Runner | 3.7 / 3.11 | get_event_loop() raises on 3.14 when there is no running loop instead of quietly creating one; policies are removed in 3.16. |
| d2 = d1.copy(); d2.update(e), {**a, **b} | a | b, a |= b | 3.9 | {**a, **b} always produces a plain dict — silently downgrading a defaultdict, Counter or OrderedDict. | preserves the left operand’s type. |
| Hand-written __init__/__eq__/__repr__ | @dataclass (+ slots, kw_only, frozen) | 3.7 / 3.10 | A hand-written __eq__ that forgets a field passes every test that checks the fields it does compare. |
| STATUS_PENDING = "pending" | enum.StrEnum | 3.11 | if status == 'pendng': is always False, and nothing complains — not the runtime, not the checker. Status.PENDNG is an AttributeError immediately. |
| @mock.patch(...) on your own code | Constructor injection + a Protocol | — | A MagicMock is Any: it accepts every call, returns a mock, and type-checks. A Protocol-satisfying fake is checked against the same port as the real implementation. |
| setup.py <command>, setup.cfg | pyproject.toml [project] + python -m build | PEP 517/518/621 | requires-python is what stops a 3.12-only file reaching a 3.11 user as an import-time SyntaxError. (The setup.py file is not deprecated; invoking it as a CLI is.) |
| from __future__ import annotations | nothing — deferred by default | 3.14 | On 3.14+ the future import prevents PEP 649 semantics in that module, so get_type_hints sees strings instead of real objects. It is now a pessimisation. |
| function-local import for startup cost | lazy import | 3.15 | Module-level, visible to linters and type checkers, unlike the hack it replaces. |
| f-string SQL / HTML / shell | parameterised queries; t-strings | 3.14 | The renderer can see which parts came from the literal and which from user data. An f-string has already destroyed that distinction. |
💡{**a, **b} and a | b produce equal dicts. Name a concrete case where swapping one for the other changes program behaviour.
click to reveal
When a is a dict subclass with behaviour.
{**a, **b} builds a brand-new plain dict by unpacking both operands, so a Counter, a defaultdict, or an OrderedDict comes out the other side as a plain dict. a | b is dict.__or__, which the subclass may override — Counter.__or__ computes a multiset union (max of counts, not “right wins”), and defaultdict.__or__ returns a defaultdict with the same default_factory.
So the swap is behaviour-preserving only when both operands are plain dicts. The bug shape is: a function returns {**defaults, **overrides}, some caller does result[missing_key], and what used to be a default-producing defaultdict is now a KeyError. Nothing in the type annotations changes — both are dict[str, int] — so the checker cannot help.
Note also that Counter | Counter is deliberately NOT “right wins”; if you actually wanted override semantics on a Counter, neither form is correct and you need an explicit update.
The four places where the old way is still right
A blanket pyupgrade --py312-plus or ruff --fix --select UP over an entire
repository will make some of these worse. Each has a real justification.
1. %-formatting in logging calls is correct and recommended.
logger.info("user %s exceeded quota %d", user_id, quota) # right
logger.info(f"user {user_id} exceeded quota {quota}") # wrong
Two reasons, not one. The formatting is deferred — if INFO is disabled,
the % substitution never runs, and for a hot debug line that is real money.
More importantly, the logging record keeps record.msg and record.args
separate, and that is what a structured-logging handler emits as fields and
what an aggregator groups on. Collapse them into one string and every one of a
million distinct messages becomes its own group. Ruff’s G004 flags the
f-string form for exactly this reason.
2. gather(return_exceptions=True) remains correct for partial-results
fan-out. TaskGroup is the right default because it enforces “all or
nothing”. But when you are polling forty shards and want thirty-eight answers
plus two errors, the TaskGroup’s cancel-the-siblings behaviour is precisely
wrong. The rule is: TaskGroup when a failure invalidates the whole operation,
gather(return_exceptions=True) when it does not.
3. os.path is still fine in a hot loop. Path allocates an object per
operation. In a walk over a million files, os.path.join on strings measurably
wins. Use Path at the API boundary — where the type is doing work — and
strings inside the loop if you have measured a problem.
4. Migrating typing.List is not urgent. It is deprecated, it has been
deprecated for years, and CPython has stated that removal is not currently
planned (gh-106745). Do it because a file with both spellings is confusing,
not because something is going to break.
💡A colleague opens a PR titled "modernise logging" that converts 400 logger.info("...%s...", x) calls to f-strings. The tests all pass. What do you say in review, and what evidence would you bring?
click to reveal
Three concrete harms, in order of how expensive they are to discover later.
Cardinality explosion in the aggregator. Handlers such as structlog, python-json-logger and the OTel bridge read record.msg and record.args separately; the message template is the group key. After the change every log line is a unique key. The dashboard that said “quota exceeded: 4,812 events in the last hour” becomes 4,812 rows of one. That is not a formatting regression, it is the loss of the aggregation the logs existed for.
Unconditional evaluation. logger.debug(f"payload={payload!r}") calls repr() on every request even when DEBUG is off. On a large object in a hot path that is measurable; on an object with an expensive or side-effecting __repr__ it is a bug.
Loss of a security property, in one specific case. An f-string built from user data can contain % and {} sequences that downstream formatters interpret. Keeping the template constant and the data in args means the template is always a literal.
Evidence to bring: the logging HOWTO’s own guidance on passing arguments separately, ruff’s G004 rule and its rationale, and a one-line benchmark of logger.debug with a disabled level, formatted vs deferred. Then propose the narrow version of the PR: convert only the + string concatenations, which have no such property, and leave the %s calls alone.
How to sequence a real modernisation
Not “run the tool on everything”. In order of value per unit of risk:
-
Correctness rows first —
utcnow,pytz,os.system,get_event_loop. These are bugs. They can be found with a grep and fixed one file at a time. -
Then the type-carrying rows —
str→Path, magic strings →StrEnum, hand-rolled classes → dataclasses. Each one converts a class of runtime error into a static one, and each is independently reviewable. -
Then the concurrency rows —
gather→TaskGroup,wait_for→timeout. Riskiest, because the semantics genuinely change; do them with the cancellation tests in the same commit. -
Cosmetic rows last, mechanically, in a commit of their own —
typing.List,.format(), and so on, with the[tool.ruff]select list as the record of what you did. Never in the same commit as a behaviour change: a reviewer cannot see one real line among two hundred rewrites.
The reviewer heuristic for the whole exercise: if you cannot name the failure mode the change prevents, it is a style preference — configure the formatter and stop arguing.