Auditing the Any surface of a module

“We’re 100% typed, --strict is clean” — and still shipping AttributeErrors. Coverage was measured as annotation presence rather than annotation information content, and those are very different numbers.

The four holes that survive a fully-strict run

1. Callable[..., Any] — the ellipsis form erases every parameter.
2. **kwargs: Any — every keyword unchecked, typos silent.
3. A cast to Any — an assertion nobody verifies.
4. Any value flowing out of a module that resolved as Any (no stubs, or ignore_missing_imports).

And one more that --strict does have an opt-in flag for but which most teams never turn on: a bare generic. def f(items: list) -> None means list[Any].

Every gap-closing flag is opt-in: --disallow-any-explicit, --disallow-any-generics, --disallow-any-unimported, --disallow-any-decorated, and the nuclear --disallow-any-expr. Pyright takes a different route — it distinguishes a declared Any from an inferred Unknown, which is why pyright strict catches leakage mypy strict does not.

Your task

Write the auditor. Given a module’s source, return the sorted names of the public top-level functions whose signature admits an unchecked value.

def solve(source: str) -> list[str]:

Use ast — an AST walk of the annotation, not a grep, because dict[str, list[Any]] must be caught and dict[str, list[int]] must not.

A function leaks if any of these holds:

1. a parameter has no annotation, or the return annotation is missing (including *args / **kwargs);
2. any annotation mentions Any — either bare or as typing.Any;
3. any annotation contains Callable[..., X], i.e. a Callable subscript whose first element is a literal ...;
4. any annotation uses a name from GENERIC_NAMES bare — appearing as an annotation without a subscript.

Skip functions whose name starts with _: they are not part of the public surface, and --disallow-any-explicit on a private helper is a different argument.

Both def and async def count.

The traps in the rules

tuple[int, ...] must not be flagged. The ... rule applies only inside a Callable subscript — everywhere else a literal ellipsis is legitimate and extremely common.

type[int] must not be flagged but a bare type must, and typing.Callable[...] (an ast.Attribute, not an ast.Name) must be recognised as the same name as bare Callable.

Why a classifier rather than “make the module pass a stricter flag”

This grader runs one mypy profile. Rather than pretend otherwise, the exercise is the thing you would actually build on a real codebase: a fitness function that fails CI when the public surface acquires a new hole. That is the durable artefact — a flag you enable once is a moment, a checked-in auditor is a ratchet.