Your package reports has a small format_total function that other teams
import. One day import reports starts to take a second. The cause is one
line in reports/__init__.py:
# reports/__init__.py
from .formatting import format_total as format_total
from .charts import plot_sales as plot_sales # charts imports a plotting library
Python runs a package's __init__.py the first time anything imports the
package, and that includes import reports.formatting. Every import in that
file runs too. So everyone who wants format_total also waits for the
plotting library to load. On a machine without that library, import reports
raises ModuleNotFoundError.
Find what makes an import slow
Run the import in a fresh process with -X importtime:
python -X importtime -c "import reports"
Python writes one line per imported module to stderr. The self column is
the time spent in that module alone, and cumulative adds everything it
imported. Both are in microseconds. Follow the largest cumulative numbers down
to the import that costs the most.
Keep the file to imports and a list of names
Put the code in other modules, and keep __init__.py to the names you want
callers to use:
# reports/__init__.py
from .errors import ReportError as ReportError
from .formatting import format_total as format_total
__all__ = ["ReportError", "format_total"]
Callers who want charts write from reports.charts import plot_sales. That
still runs reports/__init__.py first, but that file no longer imports the
plotting library, so import reports loads only what everyone needs.
Keep other work out of the file too. Importing your package might open a
connection, start a thread, configure logging or read an environment
variable. Then that work runs for every caller, whether they want it or not.
If you read an environment variable at import time, a later change to it
has no effect.
Put that work in a function that callers run, such as reports.configure().
Mark the names you export
The X as X form looks redundant. It tells type checkers that you export the
name on purpose. mypy's --strict mode turns on --no-implicit-reexport,
which treats a plain import as private to the module that does it:
Import in reports/__init__.py |
Can callers import the name under --strict? |
|---|---|
from .formatting import format_total |
Only if __all__ lists it |
from .formatting import format_total as format_total |
Yes |
import json |
Only if __all__ lists it |
import json as json |
Yes |
With the plain form, from reports import format_total still works when the
code runs. mypy fails the caller's line with
Module "reports" does not explicitly export attribute "format_total".
__all__ is the list of a module's public names. It sets what
from reports import * imports, and type checkers treat every name in it as
exported. It does not hide anything. Callers can still import any name the
module has.
Common mistakes
Listing a name in __all__ that does not exist. mypy 2.4 accepts
__all__ = ["missing"] under --strict even when nothing defines missing.
Then from reports import * raises AttributeError. Test that every listed
name exists:
import reports
def test_exported_names_exist() -> None:
for name in reports.__all__:
assert hasattr(reports, name), name
Building __all__ with code. Type checkers read a list of string
literals, and a few simple changes to it, such as __all__.append("name").
They cannot read a list that a function call returns. Write __all__ as a
literal list.
Check your understanding
from reports import format_total works at runtime, but mypy --strict fails on that line. What do you change?
Show answer
Hide answer
Mark the re-export in reports/__init__.py. Write
from .formatting import format_total as format_total, or add the name to
__all__. The import works at runtime because the name is in the module.
mypy fails because a plain import is private under --no-implicit-reexport.
Change the caller instead only if reports.formatting is the path you want
people to use.
Going further
To keep from reports import plot_sales working without loading the plotting
library up front, define a module-level __getattr__. Python calls it when
the normal lookup cannot find a name:
# reports/__init__.py
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from .charts import plot_sales as plot_sales
__all__ = ["plot_sales"]
def __getattr__(name: str) -> Any:
if name == "plot_sales":
from .charts import plot_sales
globals()[name] = plot_sales
return plot_sales
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
The first use of reports.plot_sales imports reports.charts and stores the
function in the package, so the cost moves to that first use. If the library
is missing, the error now comes at that first use too. The TYPE_CHECKING
block gives type checkers the real signature. Python 3.15 adds
lazy from .charts import plot_sales as plot_sales, which defers the import
in one line. See
customizing module attribute access
and PEP 810.