Autograd Mechanics

PyTorch

Inspect automatic differentiation graphs and gradients. Practise leaf tensors, gradient accumulation, no_grad and detach, and understand what each changes.

8 exercises

Explore freely. Sign in to keep your progress.

Start track

Your learning sequence

1 Not solved yet. Evaluate without recording gradient history Practice · Medium
2 Not solved yet. Clear gradients between independent backward passes Practice · Medium
3 Not solved yet. Backward from a vector Practice · Medium
4 Not solved yet. Your own backward pass Practice · Hard
5 Not solved yet. Backward twice Practice · Medium
6 Not solved yet. The value backward needed Practice · Medium
7 Not solved yet. Count the backward graph’s nodes Practice · Medium
8 Not solved yet. Cached features and inference tensors Practice · Medium

Check yourself

4 questions · not saved while you're signed out

Use these questions to check your understanding and compare explanations. Your score does not affect track completion.

0 / 4 correct

Why does the second backward call fail for this loss, and how can you retain what it needs for another pass?

Choose one answer to reveal the explanations. Answers are kept only while this page is open.

import torch

x = torch.ones(3, requires_grad=True)
y = (x * 2).sum()
y.backward()
try:
    y.backward()
except RuntimeError as error:
    print(str(error).split(".")[0])

Question 1 of 4