Parse a Nested Config with Placeholders and Secrets

Your service reads its settings from a JSON file. The database settings sit in their own object. The API token is not in the file at all. The file holds ${TOKEN} instead, a placeholder that you fill in from an environment variable. In Turn Outside Data into a Typed Record, you checked a flat dictionary one field at a time. This exercise adds nesting, unknown keys, placeholders, a depth limit, and a token that stays hidden when you print the config.

raw = {
    "name": "billing", "port": 8080, "debug": False,
    "database": {"url": "postgres://${HOST}/app", "pool_size": 8},
    "features": ["retry", "retry"], "token": "${TOKEN}",
}
env = {"HOST": "db.internal", "TOKEN": "private-token"}
config = load_config(raw, env=env)
config.database.url  # "postgres://db.internal/app"
config.features      # ("retry", "retry")
config.token         # Secret(***)

What to write

Fill in this function, and add helpers if you like:

def load_config(raw: Mapping[str, object], *, env: Mapping[str, str]) -> Config:

raw is decoded JSON with string keys. env maps variable names to values. Read variables from env, never from os.environ, and change neither input. Keep the dataclasses, ConfigError, Secret, _negative_fixtures and solve as they are.

Work in three steps, in this order:

1. Check the depth of raw.
2. Fill in the placeholders in every string.
3. Check the keys and types, and build the Config.

Because step 2 comes first, you report a missing variable even when the input also has a missing or unknown key.

First, limit the depth

The root has depth 0. Each value in an object, and each item in a list, is one level deeper than its container. That includes plain values such as numbers. If any value is deeper than MAX_DEPTH (64), raise a ConfigError with path "<root>" and kind "depth".

A placeholder function that calls itself once per level crashes with RecursionError on a 10,000-level input. So reject deep input before step 2 starts. Write the depth check with an explicit stack, a list of (value, depth) pairs, instead of recursion. A string is one value, so do not walk its characters.

Next, fill in placeholders

Replace each ${NAME} in a string with env[NAME], and keep the text around it. Do this for every string at any depth, inside objects and lists, including strings under keys that step 3 will reject. Leave the keys themselves as they are.

• The name is the text between ${ and the next }.
• If the name is not in env, or a ${ has no closing }, raise a ConfigError with kind "env". Use the path of that string.
• An empty value is fine.
• Insert the value as it is. If it contains ${OTHER}, do not expand that.
• The result is always a string. With PORT="8080", "${PORT}" becomes the string "8080", which then fails the integer check in step 3.

Last, check keys and types

Every field is required. At the root, first look for a key that is not in the schema. Report the first one, in the order the keys appear, with kind "unknown". Then check the fields in the order of this table. A missing field has kind "missing", and a value of the wrong type has kind "type".

Check database the same way. Look for unknown keys first, then check url as a string and pool_size as an integer that is not a boolean. Do not convert strings to numbers, and add no other checks, such as a port range or a non-empty name.

Wrap the token in the supplied Secret. Its str and repr show ***, so the token stays out of repr(config) and out of any log line that formats the config. reveal() returns the real value.

Errors

Raise ConfigError(path, kind, detail). The path joins keys with dots and puts list indexes in brackets, as in database.pool_size or features[1]. ConfigError adds the path to its message for you.

Keep input values, env values and placeholder names out of detail. Write "environment variable missing", not the variable's name. Config files hold secrets, and error messages end up in logs.

What the driver reports

When deep is above zero, solve replaces database with an object nested deep levels down, for the depth test. On success it returns the fields, with features as a list. On a ConfigError it returns "ok": False, the error's path and kind, and path_in_message, which is true when the message contains the path. leaked is true when the probe text appears in the error message or in repr(config). Every test expects it to be false.

The supplied _negative_fixtures never runs. Its last line indexes a Config like a dictionary, and that line must stay a type error. Strict mypy fails if its # type: ignore[index] comment stops being needed, so keep the function unchanged.