The history that rewrites itself

A training loop keeping a history of some running statistic:

history = []
for step in range(steps):
    update(stats)                 # writes into `stats` in place
    history.append(stats.detach())

At the end, every entry in history is identical, and equal to the final value.

Why

detach returns a tensor sharing the same storage. Every appended entry is a different Python object pointing at the same buffer, and update has been writing to that buffer the whole time. The list is a list of aliases.

Nothing raised. The values are all plausible. You notice when the loss curve is flat at exactly the final loss.

The fix, and its cost

history.append(stats.detach().clone())

Now each entry has its own buffer. That is a real allocation per step, which is correct: you asked to keep the history, and keeping it costs memory.

If the statistic is one number, stats.item() is cheaper still and gives a Python float that cannot alias anything. Inside a hot loop .item() synchronises, which the performance track covers; for a per-epoch metric it is fine and is what most code should do.

The general shape

Any time you store a tensor for later, ask whether anything will write to its buffer before you read it again. Replay buffers, EMA shadows, logged activations, cached masks. If the answer is yes or unknown, clone.

Your task

def record(stats: torch.Tensor, steps: int) -> list

Run steps iterations. On each one, add 1 to every element of stats in place, then record the current value. Return the list of recorded tensors, which must show the progression rather than steps copies of the final value.