__init__.py is the only file in your package that every single user
executes, whether they need it or not. That makes it the highest-leverage
file you own and the easiest one to ruin.
What belongs in it
A curated public surface. Nothing else.
# mypkg/__init__.py — the whole file.
from mypkg.client import WidgetClient as WidgetClient
from mypkg.errors import WidgetError as WidgetError
from mypkg.models import Widget as Widget
__all__ = ["Widget", "WidgetClient", "WidgetError"]That is a design document. It says: three names are supported; everything else is internal and may change in a patch release. A reviewer can read it in five seconds and a user can read it instead of your source.
What does not belong in it
Business logic. A function defined in __init__.py cannot be imported
without importing the whole package, cannot be moved without breaking
callers, and cannot be tested in isolation.
Side effects. Anything that reads an environment variable, opens a
connection, configures logging, registers a signal handler or starts a thread
now happens on import mypkg — including inside your users’ test suites,
their linters and their documentation builders.
Expensive imports. This is the common one and it compounds. Each
convenience re-export drags in a subpackage, which drags in pandas, which
costs 300 ms and 90 MB. A CLI that starts in 50 ms and one that starts in
600 ms are different products.
sys.path surgery. If your package has to edit the path to import
itself, the layout is wrong.
💡import mypkg takes 700 ms. How do you find out where the time goes, and what do you do about it?
click to reveal
Measure first: python -X importtime -c "import mypkg" prints a tree of
cumulative and self time per module, in microseconds. The offender is
almost always one line in __init__.py pulling in a heavy subpackage.
Two fixes, in order of preference.
Drop the re-export. If mypkg.plotting needs matplotlib, let users write
from mypkg.plotting import plot. The dependency is then paid by people who
use it and by nobody else.
If the name genuinely belongs on the top-level surface, make it lazy with a
module __getattr__ (below). The name stays importable, and the cost moves
from import time to first use.
The assertion that keeps it fixed is a test:
assert "matplotlib" not in sys.modules after import mypkg, in a fresh
subprocess. That is a fitness function — it fails the moment somebody adds
the convenient import back.
The re-export rule that trips everyone
Under mypy --strict — specifically --no-implicit-reexport — an imported
name is not part of your module’s public API unless you say so. The
supported spellings, from the typing specification:
Written in __init__.py |
Re-exported? |
|---|---|
import os |
no |
import os as os |
yes |
from .core import Thing |
no |
from .core import Thing as Thing |
yes |
listed in a literal __all__ |
yes |
The X as X form looks like a typo and is not. It is the spec’s way of
saying “this name is deliberately part of my surface”, and it is what lets a
checker tell a re-export apart from an implementation detail that happens to
be imported at the top of the file.
A literal __all__ is the clearest signal and the one to prefer, because it
is also what from mypkg import * honours and what documentation tools read.
Note the word literal: __all__ = _discover_plugins() is legal Python and
completely opaque to every static tool you own.
A caution that matters for grading your own repo: mypy --strict does
not validate the contents of __all__ — a name listed there that does
not exist is not a mypy error. pyright does check it. Do not assume the
checker is covering you here.
Lazy and deprecated re-exports: module __getattr__
PEP 562 lets a module define __getattr__, called only when normal attribute
lookup fails. Two uses justify their weight:
import importlib
from typing import Any
__all__ = ["Widget", "plot"]
def __getattr__(name: str) -> Any:
if name == "plot":
return importlib.import_module("mypkg.plotting").plot
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
mypkg.plot still works; matplotlib is imported the first time somebody
actually touches it. The same hook is the clean way to keep a removed name
alive with a DeprecationWarning for one release cycle.
The honest cost: the return type is Any, so the lazy name is invisible to
the type checker unless you also declare it under if TYPE_CHECKING:. Use it
for genuinely heavy or genuinely deprecated names, not as a general style.
💡Your __init__.py has from .core import Thing and a user writes from mypkg import Thing. It works at runtime. Why does mypy reject it, and which of the two is "right"?
click to reveal
Both are right about different things, and that is the point.
At runtime, from .core import Thing binds the name Thing in the package
namespace, and any attribute of a module is reachable. Python has no notion
of private module attributes.
mypy under --no-implicit-reexport is answering a different question: did
the author intend this name to be public? An unaliased import reads as “I
needed Thing in order to write this module”, which is an implementation
detail. If you delete the line during a refactor you have silently broken
every user who imported it.
So the checker is enforcing the promise you did not make. Make it explicitly —
from .core import Thing as Thing, or list Thing in __all__ — or leave it
private and let users import from mypkg.core. What you must not do is leave
it ambiguous, because then nobody, including you, knows whether removing it is
a breaking change.