Build backends and editable installs

For twenty years, building a Python package meant running your build script: python setup.py sdist. The build script imported setuptools, so you had to have setuptools, so it had to already be installed, in the right version, before anybody could find out what your project needed.

PEP 518 and PEP 517 broke that circle by separating description from building:

[build-system]
requires = ["hatchling>=1.27,<2"]
build-backend = "hatchling.build"

A frontend (pip, uv, build) reads those two keys, creates an isolated environment, installs requires into it, imports the backend, and calls a small set of hooks — build_wheel, build_sdist, and a few optional ones. Your project no longer executes anything to describe itself.

Pin requires with a range. Exact (==1.27.0) means you never get bug fixes and your build breaks the day that release is yanked. Unbounded means a backend major release can break your build with no change on your side.

The backends worth knowing

hatchling — the PyPA packaging tutorial’s default. Fast, no setup.py, good defaults for src layout, a plugin system for version sourcing and file inclusion. The safe modern choice for a pure-Python project.

setuptools — ubiquitous, and the one with the most legacy surface: setup.py, setup.cfg, MANIFEST.in, and decades of accumulated behaviour. Still the right answer when you build a C extension and no specialist backend covers your case, because setup.py is imperative and can drive a compiler. Note the distinction that trips people up: setup.py the file is not deprecated; invoking it as a CLI (python setup.py bdist_wheel) is.

uv_build — uv’s own backend, and its default for uv init. The fastest of the set and deliberately minimal: it does one layout well and does not try to be extensible. If you are already using uv and your project is conventional, it removes a dependency and some milliseconds.

flit-core — very small, aimed at simple pure-Python packages, reads the version and description from your module. pdm-backend — more featureful, with dynamic versioning and file-inclusion rules.

Almost any of these is fine. The decision is much less consequential than the time people spend on it — with one exception: if you ship a compiled extension, that constraint chooses for you.

💡What does the isolated build environment actually protect you from? click to reveal

From your build depending on whatever happens to be installed in the environment you are building into.

Without isolation, pip install . builds using the setuptools, wheel and Cython versions already present. Two developers with different environments produce different wheels from the same commit, and a CI machine that recently installed something unrelated can change your artefact. Worse, the build can succeed by accidentally importing a package that is present locally and absent on the build server.

With isolation, the frontend creates a fresh environment containing exactly [build-system].requires and nothing else. The build either declares what it needs or fails — which is the whole value: an undeclared build dependency becomes an error on your machine rather than a mystery on someone else’s.

This is also why --no-build-isolation is a debugging flag and not a workflow. It is useful when you are iterating on a compiled extension and do not want to rebuild the environment each time; it is not something to put in CI.

Editable installs are not one thing

pip install -e . used to have exactly one meaning: setuptools wrote a .pth file that added your source directory to sys.path. PEP 660 standardised the hooks — build_editable and friends — but deliberately did not standardise the mechanism. A backend may implement an editable install as:

• a .pth file adding a directory to sys.path (the classic);
• an import hook — a finder installed at interpreter startup that maps your package name to your source tree;
• a static mapping of the specific modules that existed at install time.

Those behave differently in one way that will eventually bite you: whether a newly added subpackage is picked up without reinstalling.

With a path-based install, src/ is on sys.path, so a new subdirectory is found immediately. With a static mapping, the new module was not in the map and is not importable until you reinstall. Neither is wrong; they trade strictness for convenience differently. Knowing which one your backend does is the difference between “add a module, it works” and twenty minutes of confusion.

Why the wheel breaks and the editable install does not

Here is the failure the title refers to, in full.

Flat layout. pip install -e . puts the project root on sys.path. Every import in your tests resolves to the working tree, which contains mypkg/templates/report.html, a conftest.py, a scratch module, and whatever else is lying around. The suite is green.

The wheel contains only what the backend was told to include. Data files need an explicit rule; a subpackage without an __init__.py may not be discovered at all. So the wheel ships without report.html, and nothing you ran could have noticed, because nothing you ran imported the wheel.

Two habits remove the whole class of bug:

• src layout, so the working tree is not importable and the editable install is the only path to your package;
• build and smoke-test the wheel in CI — uv build, then install the .whl into a clean venv and import it, or run the suite against it. This is the step everyone skips and the one that catches missing files.

💡Your wheel is missing a .json data file. Where do you look, and why did the sdist have it? click to reveal

Look at the backend’s file-inclusion configuration — [tool.hatch.build], [tool.setuptools.package-data], or your backend’s equivalent — and at MANIFEST.in if you are on setuptools.

The sdist and the wheel are built by different rules, which is the part that surprises people. An sdist is closer to “a snapshot of the source tree” and frequently sweeps up files nobody declared, especially with setuptools plus an SCM plugin that includes everything git tracks. A wheel is an installed layout, and non-Python files get in only if a rule puts them there.

So “it works from a git checkout, it works from the sdist, it fails from the wheel” is a completely coherent story, and it points straight at the inclusion rules.

The check that would have caught it: install the built wheel into a clean environment and run the suite against that, not against the source tree. It is one CI step and it fails loudly.