Handle Missing Keys with get, setdefault and defaultdict

You are grouping orders by customer. Each customer needs a list, and the first order for a customer has to create it. Here are two ways to handle that missing key:

from collections import defaultdict

orders = [("ana", 12), ("ben", 7), ("ana", 30)]

by_customer: dict[str, list[int]] = {}
for name, total in orders:
    by_customer.setdefault(name, []).append(total)

grouped: defaultdict[str, list[int]] = defaultdict(list)
for name, total in orders:
    grouped[name].append(total)

assert by_customer == grouped == {"ana": [12, 30], "ben": [7]}

Each tool does something different when the key is missing:

Call	Key present	Key missing
`d.get(key, fallback)`	Returns the value	Returns `fallback` and stores nothing
`d.setdefault(key, fallback)`	Returns the value	Stores `fallback` and returns it
`d[key]` on a `defaultdict(factory)`	Returns the value	Calls `factory()`, then stores and returns the result

A factory is a function that takes no arguments and builds a default value. The usual ones are list, int and set. defaultdict(list) calls list() for each new key, so every key gets its own empty list.

Reading a missing key adds it

from collections import defaultdict

seen: defaultdict[str, list[int]] = defaultdict(list)
assert seen.get("alice") is None
assert "alice" not in seen

bucket = seen["alice"]
assert bucket == []
assert "alice" in seen

Only d[key] on a missing key calls the factory. The get method and the in test never do. So a check such as if seen["bob"]: adds an empty list for "bob", even though the check is false. Use in or get when you only want to look.

Check your understanding Are key not in counts and counts.get(key, 0) == 0 the same test? Show answer

No. The first is true only when the key was never stored. The second is also true when the key holds the value 0. Neither one adds the key. Pick the test that matches what you mean, either "never counted" or "counted down to zero".

setdefault builds its default every time

calls = 0


def new_list() -> list[int]:
    global calls
    calls += 1
    return []


d: dict[str, list[int]] = {"a": [1]}
d.setdefault("a", new_list()).append(2)
assert calls == 1  # new_list ran, although "a" was already there
assert d == {"a": [1, 2]}

Python works out every argument before it calls a function. In d.setdefault(key, new_list()), new_list() runs even when key is already there. The same goes for a default you pass to get. A defaultdict stores the factory itself and calls it only when a key is missing.

For a cheap default such as [], setdefault is fine. When a function adds to many groups, defaultdict(list) names the factory once. For counting, use Counter from the previous article, or defaultdict(int).

The factory must build a new value on each call. A factory that returns one shared list gives every new key that same list.

Return a plain dict

from collections import defaultdict


def group_by_key(rows: list[tuple[str, int]]) -> dict[str, list[int]]:
    grouped: defaultdict[str, list[int]] = defaultdict(list)
    for key, value in rows:
        grouped[key].append(value)
    return dict(grouped)

Inside the function, the defaultdict saves a check on every row. The caller gets a plain dict, so looking up a missing key raises KeyError instead of adding an empty group. Return the defaultdict itself only when callers are meant to keep adding to it, and say so in the docstring.

The return annotation does not make this choice for you. A defaultdict is a kind of dict, so mypy accepts return grouped under the annotation dict[str, list[int]]. Only the dict(...) call changes what callers get.

dict(grouped) copies the keys but shares the lists. That is fine here, because nothing else holds grouped once the function returns.

Check your understanding Can serializing a defaultdict add keys to it? Show answer

No. Looping over the entries, as json.dumps does, reads only the keys that are there. A key appears only when some code reads a missing key with brackets. So check the bracket lookups your code makes. Plain reads of the entries change nothing.

Type the accumulator

The annotation defaultdict[str, list[int]] tells the reader and mypy what each group holds. mypy can often work out the type from how you use the defaultdict. Write the annotation anyway when it makes the code easier to read, which for an accumulator is most of the time.