Test selection: filters, #[ignore], and the libtest CLI

cargo test builds your test targets and runs each one with Rust's built-in test runner, called libtest. The runner can be told to run only some tests: name filters, --skip, --exact and the #[ignore] options. This exercise models how those options combine.

pub fn select(
    tests: Vec<(String, bool)>,
    filters: Vec<String>,
    skip: Vec<String>,
    exact: bool,
    ignored_mode: String,
) -> Vec<String>

Each test is (full_path, is_ignored). The full path is the test's name as libtest prints it, such as tests::math::adds for a function adds inside module math inside module tests. is_ignored is true when the test is marked #[ignore]. Return the names of the tests that would actually run, in input order.

The rules to implement

1. Include: if filters is empty, every test is included. Otherwise a test is included when it matches at least one filter.
2. Exclude: remove any included test that matches at least one skip entry, using the same matching rule.
3. Ignore mode: of the tests that remain,
   • "default" keeps only tests that are not ignored,
   • "ignored" keeps only ignored tests,
   • "include_ignored" keeps both.

A name matches a filter when the filter is a substring of the full path (str::contains). When exact is true it matches only when the two strings are equal. Matching is case-sensitive and compares the strings exactly as given; no trimming or normalization is done.

Contract details:

• Any ignored_mode other than "ignored" or "include_ignored" is treated as "default". (This fallback is part of the exercise, not a libtest rule; libtest takes these modes as separate flags.)
• An empty filter or skip string "" is a substring of every name. Without exact, filters: [""] includes everything and skip: [""] excludes everything. With exact, "" matches only a test whose name is empty.
• filters that match nothing produce an empty result. That is different from an empty filters list, which includes everything.
• Names may repeat. Keep each entry that passes, in its original position.
• Empty tests returns an empty list.

Worked example

Tests, in order: tests::math::adds, tests::math::adds_negatives, tests::io::reads, tests::slow::big_matrix (ignored) and bench::adds.

• Filter adds, default mode: the three names containing adds, in input order. tests::slow::big_matrix is not a match, and would be dropped by default mode anyway.
• With exact and the filter tests::math::adds: only tests::math::adds.
• No filters, skip math: tests::io::reads and bench::adds. The two math tests are skipped and the ignored test is dropped by default mode.
• Filter big_matrix, default mode: nothing. Naming an ignored test in a filter does not make it run.

The same rules on the command line

A filter is a substring, so cargo test adds runs every test whose full path contains adds, not just a test named adds. To run one test, pass its full path with --exact: cargo test -- --exact tests::math::adds.

That command has a -- in it. Cargo reads the arguments before -- itself and passes the arguments after -- to the test binary. A single test-name filter is the exception: Cargo accepts one before -- and forwards it. The runner's own options must come after --:

cargo test adds                           # one filter, forwarded by Cargo
cargo test -- adds reads                  # several filters
cargo test -- --skip bench                # exclude by substring
cargo test -- --exact tests::math::adds   # whole-name match
cargo test -- --ignored                   # only #[ignore] tests
cargo test -- --include-ignored           # ignored and not ignored
cargo test -- --list                      # print test names, run nothing
cargo test -- --nocapture                 # print test output instead of capturing it

Putting one of these runner options before --, as in cargo test --nocapture, makes Cargo report an unexpected argument, because Cargo itself has no option with that name.

Two options with similar purposes are easy to confuse. cargo test -j N (or --jobs N) limits how many build jobs Cargo runs while compiling. cargo test -- --test-threads=N limits how many tests a test binary runs at the same time. If a suite passes when tests are run one at a time but fails when they run together, --test-threads=1 is a quick way to check whether tests interfere with each other.

#[ignore] compared with #[cfg]

A test marked #[ignore] (optionally with a reason, #[ignore = "slow"]) is still compiled. In a normal run it is listed and reported as ignored instead of being executed. Because it is compiled with the rest of the test build, a change that breaks its code is a compile error you see immediately. Its runtime behavior is not checked until someone runs it with --ignored or --include-ignored.

Code under #[cfg(feature = "slow")] is different: when the feature is off, the code is removed before type checking, so it can stop compiling without anyone noticing until the feature is turned on.

Shared state between tests

By default, libtest runs the tests in one test binary on several threads of a single process. Process-wide state is shared among them: environment variables, the current working directory, the panic hook and any static values. Two tests that change the same environment variable can see each other's changes depending on timing, so the suite can pass in one run and fail in another. In edition 2024, std::env::set_var is an unsafe function partly for this reason: changing the environment while other threads read it is not safe on every platform.

The same idea makes tests of concurrent code more reliable. Check a result that does not depend on scheduling, such as a final count, a sorted merge or a set of events, rather than an elapsed time or a particular interleaving.