The argument for pathlib that survives contact with a code review is not
that it is prettier. It is that path: str is not a type — it accepts a URL,
an SQL fragment, a user’s display name, an empty string, and a path. path: Path
accepts a path. Every function that takes a str and treats it as a path is a
function whose precondition lives in its docstring.
Correspondence
os.path |
pathlib |
|---|---|
os.path.join(a, b) |
Path(a) / b |
os.path.dirname(p) |
p.parent |
os.path.basename(p) |
p.name |
os.path.splitext(p)[0] |
p.stem |
os.path.splitext(p)[1] |
p.suffix |
os.path.abspath(p) |
p.resolve() (also resolves symlinks) |
os.path.exists(p) |
p.exists() |
os.path.isdir(p) |
p.is_dir() |
os.makedirs(p, exist_ok=True) |
p.mkdir(parents=True, exist_ok=True) |
open(p).read() |
p.read_text(encoding="utf-8") |
glob.glob(pat) |
p.glob(pat) |
read_text / write_text take encoding= as a named parameter, which is the
smallest and most valuable difference in the table: it is a place you cannot
forget to specify the encoding, whereas open() silently uses the locale
default and produces a program that decodes differently on your laptop and in
the container.
PurePath (and PurePosixPath / PureWindowsPath) do pure string
manipulation with no filesystem access at all. That makes them right for
validating a path you have not yet decided to touch, for manipulating remote
or archive paths, and for tests.
The one-line path injection
Path("/srv/data") / "reports/q1.csv" # /srv/data/reports/q1.csv
Path("/srv/data") / "/etc/passwd" # /etc/passwd
When the right-hand side is absolute, / discards the left side entirely.
This mirrors os.path.join and it is a genuine vulnerability whenever the
right-hand side comes from a request parameter, an upload filename, or a
config value: one leading slash and your carefully chosen root is gone.
.. is the other half. Path("/srv/data") / "../../etc/passwd" stays inside
the string but resolves outside the root, and Path does not normalise
.. on its own — it is kept as a literal component, because resolving it
correctly requires knowing whether the preceding component is a symlink.
So every “join user input onto a root” needs an explicit containment check:
def safe_join(root: PurePosixPath, *parts: str) -> PurePosixPath:
candidate = PurePosixPath(posixpath.normpath(str(root.joinpath(*parts))))
if not candidate.is_relative_to(root):
raise ValueError(f"{candidate} escapes {root}")
return candidate
💡Why is_relative_to rather than
click to reveal
str(candidate).startswith(str(root))?
Because string prefixes do not respect component boundaries.
"/srv/data2/secrets.env".startswith("/srv/data") is True, and /srv/data2
is a completely different directory. An attacker who can create a sibling
directory whose name extends yours walks straight through that check.
is_relative_to compares the parsed parts tuples, so /srv/data2/x is not
relative to /srv/data. It also gets the trailing-slash and duplicate-slash
cases right for free.
The generalisation is worth internalising: paths are sequences of
components that happen to have a string rendering, and every check written
against the rendering rather than the structure has a counterexample. Same
reason p.suffix beats name.split(".")[-1], and p.parts beats
path.split("/").
resolve() vs absolute()
resolve() makes the path absolute, normalises .., and follows
symlinks — which means it touches the filesystem. That is what you want for
a security check (a symlink inside your root pointing at /etc is an escape
that pure string manipulation cannot see), and what you do not want when the
path may not exist yet or when you are being deliberately filesystem-free.
absolute() prepends the cwd and does not resolve .. or symlinks. It is
cheap and lexical and almost never what a security check wants.
.suffix of archive.tar.gz is .gz
Singular suffix is the last one. suffixes gives ['.tar', '.gz'], and
stem is archive.tar. Code that strips a “double extension” with stem
gets it wrong by one level, every time.
💡What should a public function that takes a path actually click to reveal
annotate?
str | os.PathLike[str] at the boundary, normalised to Path on the first
line, and Path on the way out.
def load(path: str | os.PathLike[str]) -> Config:
target = Path(path)
...
The union is what open() and every stdlib function accepts, so refusing it
makes your function less usable than the standard library for no benefit —
callers would have to wrap every literal in Path(...). os.PathLike[str]
covers Path itself and any third-party path-like object.
Returning Path rather than str is the other half. A str return forces
every caller to re-parse, and re-parsing is where the /-with-absolute-input
bug gets reintroduced. Internally, hold Path everywhere; convert to str
only at the moment you hand it to something that demands one.