The CI gate: what a real pipeline runs, in what order

Most Python pipelines run the right tools in the wrong order against the wrong thing. Here is a gate that does not.

The order

1  uv sync --locked --group ci        # exact environment, or fail
2  ruff format --check --diff         # cheapest, most deterministic
3  ruff check                         # fast, no imports needed
4  mypy --strict src tests            # slower, needs the environment
5  pytest -ra --cov                    # slowest, most informative
6  uv build                           # produce the artefact
7  install the wheel into a clean venv, import it, smoke test
8  repeat 4-5 across the matrix in requires-python

The ordering principle is fail fast on the cheapest, most deterministic check. A formatting failure should not wait behind a four-minute test suite, and a developer who has to wait four minutes to be told about a missing blank line stops running the pipeline locally.

Step 1 is the one that decides whether any of it is honest

uv sync --locked --group ci

--locked fails if the lockfile is out of date with pyproject.toml. Plain uv sync silently re-locks, tests the fresh resolution, and throws it away — so a pull request that adds a dependency and forgets to commit the lockfile goes green, and production installs a different tree from the one that was tested.

The ci group is a [dependency-groups] entry with the tool versions pinned:

[dependency-groups]
ci = ["ruff==0.16.0", "mypy==2.3.0", "pytest==9.0.0", "pytest-cov"]

This is not paranoia. Ruff 0.16 changed its default rule set and mypy 2.0 changed four inference defaults — both of those broke unpinned pipelines on the day they released, in repositories where nothing had been committed for a week. An unpinned linter means your gate can change without a commit, and a red build that nobody caused is the fastest way to teach a team to ignore red builds.

💡Your pipeline pins tool versions. Now Dependabot opens a PR bumping ruff, and it fails with 40 new findings. Is the pin working or getting in the way? click to reveal

Working, and this is exactly the interaction it was designed to produce.

Without the pin, those 40 findings would have appeared on somebody else’s unrelated feature branch, they would have had to fix them to merge, and the cause would have been invisible in the diff. With the pin, they arrive in a pull request whose entire content is “upgrade ruff”, where they can be reviewed as a set, fixed mechanically, or deferred with a scoped ignore and a reason.

The upgrade is the same work either way. The difference is whether it is scheduled and attributable or ambient and confusing — which is the same argument as never upgrading a checker in the same PR as a feature.

Steps 6 and 7: the ones everyone skips

uv build
python -m venv /tmp/smoke
/tmp/smoke/bin/pip install dist/*.whl
/tmp/smoke/bin/python -P -c "import yourpkg; print(yourpkg.__version__)"

Steps 2 to 5 all ran against your source tree. They tell you nothing about the wheel. A missing data file, a subpackage the backend never discovered, a module excluded by a stale MANIFEST.in — none of those can fail any check that imports from the working directory.

Building the wheel and importing it in a clean environment is thirty seconds and it is the only step that tests the artefact you actually ship. Note the -P: without it, if the smoke test happens to run from the project root, you are back to importing the working tree.

Better still, run the test suite against the installed wheel rather than the source. That is what src layout makes natural.

Step 4: over src/ and tests/

mypy --strict src and mypy --strict src tests are different gates. Test code that is not type-checked is where Any accumulates, where fakes drift out of sync with the Protocols they claim to implement, and where a fixture quietly returns the wrong type for a year.

It is also where the most valuable typing assertions live — assert_type(result, Iterator[Row]) does nothing if nothing checks the test file.

Step 8: the matrix is not optional

If requires-python = ">=3.12", then 3.12, 3.13 and 3.14 are supported claims. A claim you never test is a claim you will retract in a bug report. Run at least the oldest and the newest; run the middle ones if the matrix is cheap, which with uv it is.

Note that mypy’s python_version should be set to your lowest supported version, not your development interpreter — otherwise the checker validates against syntax and stdlib signatures your oldest users do not have.

pre-commit is a convenience; CI is the contract

pre-commit hooks are genuinely useful: they catch formatting before you push and save a round trip. They are not a gate, because they can be skipped (--no-verify), they run on staged files rather than the whole tree, and their versions are configured in a different file from your CI’s.

Run the same tools in both, pinned to the same versions, and let CI be the thing that decides.

💡Your gate is green on every PR and production still breaks weekly. Which step is missing, given the list above? click to reveal

Most often step 7 — nothing in the pipeline ever exercises the artefact that is deployed. Everything ran against the source tree; the wheel, the container image, or the deployed environment is a different object that no check has ever imported.

The second most common answer is that step 1 is not --locked, so the dependency tree tested and the dependency tree deployed are resolved separately and can differ.

Both have the same shape, and it is worth naming: the gate tests something adjacent to what ships. A useful diagnostic is to ask, for each artefact that reaches production — wheel, image, lockfile — which pipeline step consumed exactly that object. Any artefact with no answer is an untested artefact, however green the build is.