Design a CI Pipeline That Tests the Release

CI is green. You publish the release, and users who install it get an error because a data file is missing. CI tested your source folder and never installed the wheel. CI, continuous integration, is the set of checks that run on every push or pull request. A good pipeline answers two questions. Does the code pass your checks? And does the package you are about to release work once someone installs it?

Run the quick checks first

A sensible order is setup, formatting, linting, type checking, tests, and then building and installing the package. Formatting and lint checks are fast, so they report simple mistakes before the slow steps start. Checks that do not depend on each other can run in parallel.

For a project with src/, tests/, a committed uv.lock and a ci dependency group, run these from the project folder. They target Python 3.12 and newer:

uv sync --locked --no-default-groups --group ci
uv run --no-sync ruff format --check --diff src tests
uv run --no-sync ruff check src tests
uv run --no-sync mypy --strict --python-version 3.12 src tests
uv run --no-sync python -m pytest -ra --cov

Pick the job's Python before the first command, and make the job stop at the first command that fails.

• --locked fails the job if uv.lock is out of date, instead of updating it in the CI checkout, as the uv article showed.
• --no-default-groups --group ci installs only the ci group, plus the project and its dependencies.
• Each uv run takes its tools from the project's environment, so you never activate it. --no-sync reuses the environment that the first command built, so do not change pyproject.toml or uv.lock between the steps.
• --cov reports coverage but sets no minimum. Add a threshold if your policy needs one.

The ci group lists the tools:

[dependency-groups]
ci = ["ruff", "mypy", "pytest", "pytest-cov"]

These entries have no versions, but uv.lock records the exact version of each one, so every run uses the same tools. Choose the uv version, the Python version and the runner image on purpose too. A lockfile cannot make two operating systems identical.

Build the release and test it

After the source checks, build into an empty folder. uv build makes an sdist first and then a wheel from that sdist. Install the new wheel into a fresh virtual environment. Then, from outside the project folder, run a quick check that the installed package works. An import alone can pass while a data file is missing, so also read a resource, run a command or call a small public function.

Start that check with python -I. It ignores PYTHONPATH and leaves the current folder off sys.path, the list of folders Python searches for imports. -P alone drops the current folder but still reads PYTHONPATH. Do not use an editable install for this check, because it points back at your source. The build backends article has a complete script that builds, installs one wheel and reads a packaged file under -I.

Publish the exact files you tested, instead of building again after the checks pass. If you ship a container image, test the image too, because a wheel check says nothing about its system packages or settings.

Test every Python you support

Run the tests on each supported Python version, in a separate environment for each. If time is short, start with the oldest and the newest. Point mypy's python_version at the oldest one, as --python-version 3.12 does above. Type-check your tests too. A typed fake can catch a method whose signature changed, and an assert_type call means something only when mypy reads the file. The next article covers how to choose the oldest version.

Run local hooks in CI too

A pre-commit hook checks your code before each commit, but anyone can skip it with git commit --no-verify. CI can run the same hooks with pre-commit run --all-files, which checks every file, not only the changed ones. Then make the CI checks required in your merge rules.