Skip to content
← All Python articles

Python explainer

Group Adjacent Items with groupby

About 3 min to read. Allow extra time to work through examples.

Group adjacent items that share a key with groupby. Sort first when you want one group per key, and use each group before you move to the next.

This article uses Python. You can make it your practice mode to browse related content across the site.

You have a list of words and want them grouped by their first letter. itertools.groupby looks like the right tool, but it groups only items that sit next to each other:

from itertools import groupby

words = ["apple", "avocado", "banana", "apricot"]
runs = [(key, list(group)) for key, group in groupby(words, key=lambda word: word[0])]
assert runs == [("a", ["apple", "avocado"]), ("b", ["banana"]), ("a", ["apricot"])]

The groupby function reads the input once and starts a new group each time the key changes. A run is a stretch of neighbouring items with the same key. Here "a" appears twice, because "apricot" starts a second run.

Runs are what you want for tasks such as collapsing repeated lines or spotting when a status changes. But if you build a dict from these runs, the second "a" run replaces the first. So decide first whether you want runs or one group per key.

Sort first for one group per key

Sorting by the grouping key puts equal keys next to each other:

from itertools import groupby


def first_letter(word: str) -> str:
    return word[0]


words = ["apple", "avocado", "banana", "apricot"]
ordered = sorted(words, key=first_letter)
groups = {key: list(group) for key, group in groupby(ordered, key=first_letter)}
assert groups == {"a": ["apple", "avocado", "apricot"], "b": ["banana"]}

Use the same key for sorting and for grouping. Pass one named function, such as first_letter, to both calls, so they always use the same key. The key must also give the same answer each time for the same item. A key based on the clock or on a random number breaks the grouping.

Sorting takes O(n log n) time and needs the whole input in memory. If the input already arrives in key order, for example from a database query with ORDER BY, skip the sort.

Use each group before you move on

Each group is a one-shot iterator that reads from the same source as groupby itself. When you move on to the next group, groupby skips past the rest of the current one, and that group is empty from then on:

from itertools import groupby

outer = groupby(["a", "a", "b"])
key, first = next(outer)
assert key == "a"
assert next(first) == "a"
next_key, second = next(outer)
assert next_key == "b"
assert list(first) == []  # the rest of the "a" group is gone
assert list(second) == ["b"]

So use each group inside the loop. If you need its items later, store them now with list(group). If you need only a summary, compute it now. A group has no len, so count while you loop over it, or turn it into a list first.

list(groupby(...)) and dict(groupby(...)) have the same problem. Both run groupby to the end and keep the group iterators, which are all empty by the time you look inside them. Keeping only the keys is fine.

Summarise a group in one pass

from itertools import groupby

rows: list[tuple[str, int]] = [("a", 2), ("b", 10), ("a", 3)]


def by_name(row: tuple[str, int]) -> str:
    return row[0]


summary: dict[str, tuple[int, int]] = {}
for name, members in groupby(sorted(rows, key=by_name), key=by_name):
    count = 0
    total = 0
    for _, value in members:
        count += 1
        total += value
    summary[name] = (count, total)
assert summary == {"a": (2, 5), "b": (1, 10)}

This loop counts and sums each group in one pass. Turning each group into a list first also works, and lets you loop over its members as often as you like. What fails is looping over the same group twice, once to count and once to sum. The second loop finds nothing.

Choose groupby or a dict

On input that is already in key order, groupby handles one group at a time, so it holds at most the group you are working on. On unsorted input, a dict of running counts and totals needs memory only for the distinct keys, and no sort at all. A defaultdict(list) keeps every row instead. Choose based on the order your input arrives in and on what you need to keep.

step 16 of 55

Everyday Standard Library Tools

Up next Count and Total Rows per Key with groupby