Backward twice

loss = (x * 3).sum()
loss.backward()
loss.backward()

RuntimeError: Trying to backward through the graph a second time,
but the saved intermediate results have already been freed.

Why it frees

Backward releases each node’s saved tensors as it passes through, because those tensors are usually the largest thing a training step is holding and holding them until the next forward would roughly double peak memory.

Freeing by default is the right call: the overwhelmingly common case is one backward per forward.

When you really do need two

loss.backward(retain_graph=True)
other_loss.backward()

Multiple losses over a shared graph, a GAN step that differentiates one computation twice, or a second-order method. retain_graph=True on every pass but the last keeps the saved tensors alive, and the last one frees them.

When you think you need it and do not

This error frequently means something else:

• You meant to run a new forward. The graph belongs to a specific forward pass. Two training steps should each build their own; if the second is reusing the first’s, a tensor is being carried across the loop boundary.
• You accumulated a loss without detaching. total_loss += loss keeps the whole graph alive for every step so far. total_loss += loss.detach() for logging, and call backward per step.

Reaching for retain_graph=True to silence it converts a memory leak into a slower memory leak. Check which of the three you have first.

Second derivatives are a different flag

g = torch.autograd.grad(loss, x, create_graph=True)[0]
g.sum().backward()          # differentiates the gradient

create_graph=True makes the backward pass itself differentiable, and implies retain_graph=True. That is what you want for a Hessian-vector product or MAML; retain_graph alone will not give it to you.

Your task

def two_losses(x: torch.Tensor) -> torch.Tensor

With y = x ** 2 computed once, call backward for the loss y.sum() and then for the loss (y * 3).sum(), and return the accumulated gradient.