Skip to content
← All tracks

Memory, Copies and In-Place

PyTorch

Clone, detach, `copy_`, `out=`, preallocation. When an in-place op is the right answer, when it is a correctness bug, and how to tell which situation you are in from the code alone.

0 / 6 solved
Continue →
  1. 1. Not solved yet. Clone, detach, or both
  2. 2. Not solved yet. Write into the buffer you were given
  3. 3. Not solved yet. Fill the caller's buffer
  4. 4. Not solved yet. The history that rewrites itself
  5. 5. Not solved yet. Updating a parameter by hand
  6. 6. Not solved yet. Gradient accumulation, and the buffer that never moved

Check yourself

4 questions · one attempt each

These do not count toward finishing the track. They are here to catch the things that are easy to read past.

0 / 4

How do clone() and detach() differ on a tensor that requires grad? 

z = torch.ones(2, requires_grad=True)
z.detach()
z.clone()
Question 1 of 4