Skip to content

← Modelling Data with Dataclasses step 19 of 28

Hard End-to-End

Parse Untrusted JSON into a Dataclass

A web handler receives events as JSON. json.loads gives you dictionaries and lists of any shape. One request sends "count": "3", and another leaves out kind. If the program passes that data around as a plain dictionary, a bad value fails far from where it came in.

So check the data once, where it enters, and turn it into a frozen Event. The rest of the code then only ever sees a checked record. This is parsing at the boundary. The boundary is where outside data enters your program.

solve({"id": "e1", "kind": "view", "count": 0, "ratio": 0.5, "tags": ["a", "a"]})
# {"error": "", "event": {"id": "e1", "kind": "view", "count": 0,
#                         "ratio": 0.5, "tags": ["a", "a"], "meta": None}}

solve({"id": "e1", "kind": "view", "count": True, "ratio": 0.5, "tags": []})
# {"error": "count: expected int, got bool", "event": None}

Libraries such as Pydantic can do this for you. Here you write it with the standard library, so you see every check and its order.

What to write

Keep the supplied InvalidEvent, Kind and Event definitions. Fill in these two functions:

def parse_event(raw: object) -> Event:
def solve(raw: object) -> dict[str, object]:
  • raw is a decoded JSON value. Its object keys are strings, and JSON null arrives as None.
  • Check everything first, then build the Event once.
  • For bad input, raise only InvalidEvent. Do not let a KeyError from a missing key or a ValueError from Kind(...) escape.
  • Do not change raw.

Check in this order

Stop at the first error:

  1. The root must be a Mapping.
  2. Every key must be one of id, kind, count, ratio, tags and meta. If any are not, sort the unknown keys and report the first.
  3. Check the fields in the order of this table:
Field Accepted value Stored value
id a str, which may be empty the same string
kind the string "click" or "view" the matching Kind member
count an int that is not a bool the same integer
ratio a float the same float
tags a list of strings a tuple in the same order, duplicates kept
meta missing, None, or a mapping from strings to strings None, or a copy of the mapping

The first five fields must be present. Do not convert values. "3" and True are not valid counts, and the integer 1 is not a valid ratio. bool is a subclass of int, so test for it separately. Do not add other limits, such as a positive count. Empty tags and empty metadata are valid.

Exact errors

InvalidEvent(field_name, problem) builds the message "<field_name>: <problem>". Use these arguments:

Situation field_name problem
Non-mapping root "<root>" f"expected object, got {type(raw).__name__}"
Unknown root key First sorted unknown key "unknown key"
Missing required field Field name "missing"
Wrong id or kind type Field name f"expected str, got {type(value).__name__}"
Wrong count type "count" f"expected int, got {type(value).__name__}"
Wrong ratio type "ratio" f"expected float, got {type(value).__name__}"
Unknown kind string "kind" f"unknown kind {value!r}"
Non-list tags "tags" f"expected list, got {type(value).__name__}"
First non-string tag "tags" f"expected list[str], got {type(item).__name__}"
Non-mapping, non-None meta "meta" f"expected object, got {type(value).__name__}"
Non-string metadata key or value "meta" "expected object of str -> str"

Report a missing field only after the earlier steps pass. So {"zzz": 1, "aaa": 2} reports "aaa: unknown key", and {} reports "id: missing".

What solve returns

On success, return {"error": "", "event": {...}} with all six fields as plain data. kind is the enum's string value, tags is a list, and meta is a plain dictionary or None. Include "meta": None when the input had no metadata.

On InvalidEvent, return {"error": str(exc), "event": None}.

Why the parser copies

The parser turns tags into a tuple and copies meta, so a later change to the input cannot reach the record. frozen=True does not freeze a dictionary stored in a field, and the Mapping annotation makes mypy reject writes to meta. A direct Event(...) call skips every check above, so build events from outside data through parse_event.