How many copies did that cost

Every conversion is a full pass over the data and a new allocation. PyTorch dispatches one aten._to_copy per conversion, and they are countable, which means “how many copies did this cost” has an exact answer rather than an opinion.

x.to(torch.float64)      # 1 copy
x.to("cpu")              # 0 copies, if it is already there

The second is the important one: to is a no-op when nothing needs changing, which is the dtype problem’s lesson applying to devices too.

The shape of the mistake

Converting inside a loop, or converting per operand:

total = 0
for chunk in chunks:
    total = total + chunk.to(torch.float64)    # one copy per chunk

versus converting once at the boundary:

total = torch.zeros((), dtype=torch.float64)
for chunk in chunks:
    total = total + chunk                       # promoted, not copied

On a device transfer this is the difference between one trip across the bus and one per iteration, and a transfer is orders of magnitude slower than the arithmetic on either side of it.

The rule

Convert at the boundary. Move and cast data once, as it enters the part of the code that wants it in that form, and let everything inside assume it. Sprinkling .to(...) defensively at each use is how you end up paying for it repeatedly.

Where the conversion genuinely is per-item, as in a data loader, do it in the worker rather than in the training loop.

Your task

def summarise(chunks: list) -> torch.Tensor

Sum a list of float32 tensors, accumulating in float64, using at most one conversion in total.

The starter converts every chunk.