Know Which Copy of Your Package You Import

Your package reads a template, mypkg/templates/report.html. The tests pass. Someone installs the package, and reading the template fails with FileNotFoundError. Your tests and the user loaded two different copies of mypkg.

Ask Python which copy it loaded

Run this with the same Python, and from the same folder, as the command that fails:

python -c "import mypkg; print(mypkg.__file__)"

/home/you/project/mypkg/__init__.py

If the path is inside your project folder, Python loaded your working copy. If it is inside site-packages, the folder where pip puts installed packages, Python loaded the installed copy. pip makes that copy from a wheel, which is a zip archive of your package's files, ready to install. A later item in this track builds one.

One case looks like the first but is not. After pip install -e ., an editable install, the installed package is a link back to your project folder. __file__ then points into your project even though the package is installed.

How a flat layout hides a missing file

Python looks for a module in each folder on its search path, sys.path, in order. When you run python -m pytest from the project folder, that folder comes first. Here is a flat layout, with the package next to pyproject.toml:

project/
  mypkg/
    __init__.py
    templates/report.html
  tests/
  pyproject.toml

From project/, import mypkg finds project/mypkg before it reaches site-packages. The tests read the template from your folder, so they pass even when the wheel left it out.

A src layout makes you install first

Move the package one level down:

project/
  src/
    mypkg/
      __init__.py
      templates/report.html
  tests/
  pyproject.toml

Now project/ holds no mypkg folder, so running from there cannot import it by accident. You have to install the package, and the tests import the installed copy.

If you install with pip install -e . while you work, the editable install still reads your folder, so the missing template stays hidden. To test what users get, build the wheel and install it without -e in a fresh virtual environment. That is a folder with its own Python and its own site-packages. Then run a test that reads the template. The wheel item later in this track walks through it, in either layout.

Leave the current folder off the path with -P

python -P starts Python without adding the script's folder, or the current folder, to the search path. Compare the two:

python    -c "import mypkg; print(mypkg.__file__)"
python -P -c "import mypkg; print(mypkg.__file__)"

If the paths differ, the current folder was hiding the installed copy. If the second command fails with ModuleNotFoundError, the current folder was the only place Python found mypkg, so it is not installed in this environment. -P keeps PYTHONPATH and editable installs, so it cannot tell you what is in the wheel.

Check your understanding Your tests pass with a flat layout. With python -P, import mypkg fails. What does that tell you? Show answer

This environment has no installed copy of mypkg, so the tests imported it from the project folder. Install the package. To test what users get, build the wheel, install it without -e in a fresh virtual environment, and run the tests from outside the project folder.

How the search treats a folder without `init.py`

The exercise below models the search Python runs over sys.path. In each folder on the path, Python looks for mypkg:

A folder mypkg/ with an __init__.py file is a regular package. Python stops and uses it.
A file mypkg.py is a module. Python stops and uses it.
A folder mypkg/ without __init__.py is a namespace portion. Python notes it and keeps looking, because a regular package or module later on the path still wins.

If the search ends with only namespace portions, Python combines them into one namespace package. The item on packages without __init__.py covers them.

Going further

Before it searches sys.path, Python checks sys.modules, the cache of modules already imported, and any import hooks that tools install. The exercise leaves both out. See the import system reference.
python -I also ignores PYTHONPATH and your user site-packages folder. See the interpreter options.
The packaging guide has a longer comparison of the two layouts.