Walk the graph

The autograd graph is not an abstraction you have to take on faith. It is a linked structure hanging off every non-leaf tensor, and you can read it.

x = torch.ones(3, requires_grad=True)
y = (x * 2 + 1).sum()

x.is_leaf        # True
x.grad_fn        # None
y.grad_fn        # <SumBackward0>
y.grad_fn.next_functions
# ((<AddBackward0>, 0),)

Each node’s next_functions names the nodes whose outputs it consumed, so following them walks back to the leaves. AccumulateGrad is the node that writes into a leaf’s .grad, which is where every path ends.

Leaf, and why it matters

A leaf is a tensor autograd did not create: your parameters and your inputs. Exactly the tensors with requires_grad=True and no grad_fn.

Only leaves get .grad populated. Ask for the gradient of an intermediate and you get None plus a warning, because autograd frees intermediate gradients as it goes. retain_grad() asks it not to, for one tensor, which is the standard way to debug where a gradient goes to zero.

Reading it is how you debug

When a parameter is not learning, the question is where the path from the loss to it breaks. Walking the graph tells you: an unexpectedly shallow graph means something detached, and a missing AccumulateGrad for a parameter means it is not in the graph at all, which is usually a .detach() or a no_grad in the wrong place, or a parameter rebuilt each forward pass rather than registered.

Your task

def graph_depth(y: torch.Tensor) -> int

Return the number of distinct nodes in the graph reachable from y, including y.grad_fn itself and the AccumulateGrad leaves. Return 0 if y has no grad_fn.