Convert dtype without unnecessary copies

A dtype conversion does not always copy a tensor. With the default copy=False, to() returns the original tensor when the requested dtype and device already match.

import torch

x = torch.ones(4, dtype=torch.float32)
same = x.to(torch.float32)
converted = x.to(torch.float64)
print(same is x)       # True
print(converted is x)  # False

The first call avoids allocating or copying tensor data. It still has the small overhead of a method call and checking the request. The second call allocates storage for the converted values.

Asking for a copy

copied = x.to(torch.float32, copy=True)
print(copied is x)  # False

copy=True requests a copy even when the dtype already matches. If you only want an independent copy with the same dtype and device, x.clone() expresses that directly.

When conversion has a cost

Changing a dense tensor's dtype reads its values and writes the converted values into new storage. Float64 uses eight bytes per element, compared with four for float32, so converting from float32 to float64 doubles the output's element storage. Both tensors can remain alive at once.

Repeated conversions inside a loop can add unnecessary allocation and work. Choose them based on the operations that need a particular dtype. Mixed precision does not mean casting only once: PyTorch autocast selects dtypes for eligible operations, and some operations need higher precision.

A device change can also require new storage and a data transfer. Returning the same tensor requires the requested device to match as well as the dtype. This exercise changes only the dtype and leaves the device alone.

def widen(x: torch.Tensor, dtype_name: str) -> torch.Tensor