ChainMap: layered configuration and scope stacks

Every service resolves configuration from layers: compiled-in defaults, a config file, environment variables, command-line flags. The reflex is a merge:

config = {**defaults, **from_file, **from_env, **from_cli}

That works and throws away three things.

Provenance. config["port"] is 8080. From where? The merged dict cannot say, and “which layer set this” is the first question asked when a deployment behaves differently from staging.
Liveness. The merge is a snapshot. Mutating from_env afterwards has no effect on config, which is either what you wanted or a bug you will find at 3am.
Cost. It copies every key of every layer on every merge. Config resolution inside a request handler, over a few hundred keys, is a real allocation.

collections.ChainMap keeps the layers:

from collections import ChainMap

config = ChainMap(from_cli, from_env, from_file, defaults)

Lookups walk maps front to back and return the first hit. Nothing is copied. The layers stay addressable as config.maps[i], so provenance is a loop, and mutating a source dict is immediately visible through the chain.

Note the order: maps[0] wins. That is the reverse of the {**a, **b} idiom, where the last mapping wins. Getting this backwards produces a config that looks plausible and silently prefers your defaults over the operator’s flags.

The asymmetry that is the whole design

Reads see every layer. Writes and deletes only ever touch maps[0].

config["port"] = 9000      # writes into maps[0], nothing else
del config["port"]         # KeyError if "port" is not in maps[0],
                           # even if a lower layer has it

That is not a wart, it is the point: the first map is the mutable scratch layer and everything behind it is read-only backing. It is exactly how a scope stack behaves — assignment writes to the innermost scope, lookup falls through to the outer ones. The new_child() and .parents API make that explicit, and the docs draw the analogy to nonlocal directly.

outer = ChainMap(globals_dict)
inner = outer.new_child()      # ChainMap({}, globals_dict)
inner["x"] = 1                 # local
inner.parents                  # back to `outer`'s view

💡del config["port"] raises KeyError even though click to reveal

config["port"] returns a value. Is that a bug in ChainMap? No, and the alternative is worse. Deleting through the chain would have to mean one of two things: delete from every layer (silently destroying the defaults for the whole process), or record a tombstone in maps[0] so lookups skip the lower layers.

The first is catastrophic and irreversible. The second requires inventing a sentinel value that is distinguishable from every legitimate value a config can hold, and would make len(), iteration and in inconsistent with [].

So ChainMap refuses: writes and deletes are scoped to the layer you own. If you want “make this key resolve to nothing”, the honest expression is to set maps[0][key] to your own explicit sentinel and have the reader interpret it — a decision that belongs in your code, not in the container.

Typing

ChainMap[str, object] is generic in key and value like any mapping. config.maps is a list[MutableMapping[K, V]] — a real list you can index, slice and reorder, which is what makes provenance queries and “temporarily push a layer” both trivial.

One nuisance worth knowing: because the value type is a single V across all layers, a genuinely heterogeneous config types as ChainMap[str, object] and every read needs narrowing at the point of use. That is the correct amount of friction — it is exactly the boundary where untyped external data enters your program — but it does mean ChainMap is not a substitute for parsing the config into a typed structure once.

💡When is the eager merge actually the right call? click to reveal

When the config is resolved once at startup and then read millions of times. A ChainMap lookup walks up to len(maps) dictionaries; a merged dict hashes once. Four layers means up to four hash lookups per read on the hot path.

The pattern that gets you both: resolve with a ChainMap (so you can log provenance for every key at startup, which is worth doing), then freeze the result into a typed frozen dataclass and hand that to the rest of the program. You get the diagnostics at the boundary and the flat cost afterwards, and the dataclass gives the type checker something to work with — which ChainMap[str, object] never will.