src layout vs flat layout: the import bug you cannot see

Here is a bug report that arrives about once per company.

CI is green. Every test passes. We cut a release. Within four minutes, production is throwing ModuleNotFoundError: No module named 'mypkg.templates'.

Nobody wrote a bad test. Nobody skipped review. The failure is structural: the tests never imported the thing that shipped. They imported the working tree, which contains a file the wheel does not.

Where the working tree gets on `sys.path`

Two rules from the interpreter, both innocuous on their own:

python -m pytest — running a module — prepends the current working directory to sys.path.
python script.py prepends the script’s directory.

Now put those next to a flat layout:

myproject/
    mypkg/
        __init__.py
        core.py
        templates/report.html      <- not in MANIFEST, not in the wheel
    tests/
    pyproject.toml

You run the tests from myproject/. sys.path[0] is myproject/. import mypkg finds myproject/mypkg/ — the working tree — before it ever looks at site-packages. Every test exercises the directory, including templates/report.html, which was never added to the distribution. The wheel is missing it. Nothing you ran could have told you.

The same mechanism produces the other two classics: tests that pass locally and fail in CI because the working copy has a stale .py the wheel dropped, and a package that “works” only because a sibling directory happened to be importable.

What src layout changes

myproject/
    src/
        mypkg/
            __init__.py
            core.py
    tests/
    pyproject.toml

sys.path[0] is still myproject/. But there is no longer an importable mypkg there — the only mypkg on the path is the installed one. Every import mypkg in your tests now goes through the artefact your build backend produced. A missing file fails the first test that needs it, on your laptop, before review.

That is the entire argument, and it is a good one. Editable installs (pip install -e ., uv sync) make it painless day to day: you get the installed-copy semantics with the working-tree source.

💡A colleague says "src layout is just a folder, it cannot possibly change behaviour". What is the one-sentence rebuttal, and what experiment settles it in ten seconds? click to reveal

The rebuttal: it changes what is reachable from sys.path[0], and sys.path[0] is the first place the interpreter looks — so it changes which copy of your package every import in your test suite resolves to.

The experiment: from the project root, run

python -c "import mypkg; print(mypkg.__file__)"

Under a flat layout it prints a path inside your working tree. Under src layout it prints a path inside site-packages. Same command, same venv, different answer — and the second one is the file your users will get.

If you want to prove the failure rather than the mechanism, delete a non-.py data file from your sdist/wheel config and run the suite both ways. The flat layout stays green.

The honest cost

src layout is not free, and the trade is worth stating plainly.

You cannot run your own code from the source tree without installing first. python -m mypkg from the project root simply fails; there is no mypkg on the path. For a one-file script or a teaching repo that is real friction, and “just install it first” is a genuine extra step for a contributor.

Tooling has to be told. Ruff needs src = ["src"] to classify your own package as first-party for import sorting. Coverage needs to know where the source lives so it can map installed paths back to your files.

Both are one-line settings. The wheel-is-broken bug is not a one-line anything, because you find it in production.

The diagnostic: `-P` and `PYTHONSAFEPATH`

Python 3.11 added an off switch. python -P (and the PYTHONSAFEPATH=1 environment variable) tells the interpreter not to prepend the script directory or the cwd at all. python -I (isolated) implies it, along with -E and -s.

This makes shadowing falsifiable rather than a matter of opinion:

python    -c "import mypkg; print(mypkg.__file__)"   # working tree?
python -P -c "import mypkg; print(mypkg.__file__)"   # installed copy only

If those two print different paths, something on your machine is being shadowed. If the second one raises ModuleNotFoundError, your package is not actually installed and every green test you have ever run against it was testing the directory.

💡You add -P to your test command and the suite collapses with dozens of import errors for your own package. Is that a bug in -P? click to reveal

No — it is the correct diagnosis, and it is the finding.

-P removed the working-tree entry. If your imports stop resolving, then every one of them was resolving through the working tree, which means your suite has never once exercised the installed distribution. The wheel might be fine; you have no evidence either way.

The fix is not to remove -P. It is to install the project into the test environment (pip install -e . or uv sync) and re-run. If the suite now passes, you have upgraded from “probably ships” to “provably imports”. If it still fails, you have just found a packaging bug that would otherwise have shipped.

What the interpreter is actually doing

The problem below makes the mechanism concrete. sys.path is scanned in order; each entry is asked, in turn, whether it can provide the module. The subtlety worth internalising is that not every hit ends the scan. A directory without __init__.py is a namespace portion: it is remembered, the scan continues, and a regular package found later still wins. Only if nothing else turns up do the portions get stitched together.

That single rule explains a surprising amount of real-world confusion, and it is why “I deleted __init__.py and imports started resolving somewhere else” is a sentence people actually say.