Skip to content

← Memory, Copies and In-Place step 4 of 6

Medium Primitives

Save independent tensor snapshots

Jump to the task

Saving a tensor in a list does not necessarily preserve its current values. detach() stops gradient tracking, but the returned tensor still shares storage with the original.

import torch

stats = torch.tensor([0.0, 10.0])
history = []
for _ in range(3):
    stats.add_(1.0)
    history.append(stats.detach())
print([entry.tolist() for entry in history])
# [[3.0, 13.0], [3.0, 13.0], [3.0, 13.0]]

Each entry refers to the same underlying data. Later updates to stats therefore change the values seen through earlier entries too.

Keeping a snapshot

Copy the data when recording it so each entry keeps its own values:

stats = torch.tensor([0.0, 10.0])
history = []
for _ in range(3):
    stats.add_(1.0)
    history.append(stats.detach().clone())
print([entry.tolist() for entry in history])
# [[1.0, 11.0], [2.0, 12.0], [3.0, 13.0]]

Each clone allocates storage. Keeping the full history uses memory proportional to the number of entries and the size of each tensor. If you only need a running average or the latest value, you may not need to keep every snapshot.

For a one-element tensor, .item() returns a Python number instead of a tensor. That number is independent of later tensor mutations, but extracting it from a CUDA tensor requires the value to be available on the CPU and can wait for pending GPU work. It is not an unconditional performance shortcut. This exercise asks for tensors, so use tensor snapshots.

Your task

def record(stats: torch.Tensor, steps: int) -> list

stats is a floating-point tensor that does not require gradients. Run steps iterations. Each time, add 1 to every element of stats in place, then append an independent, detached snapshot to the result list. Return that list. For zero steps, return an empty list.

The caller's stats should contain the final updated values. Earlier snapshots must keep their values after later updates.