We can't find the internet
Attempting to reconnect
Something went wrong!
Attempting to reconnect
← Dtypes and Numerics step 6 of 6
The cast that does nothing
x = torch.ones(4, dtype=torch.float32)
x.to(torch.float32) # returns x itself. Same buffer.
x.to(torch.float64) # a new buffer, twice the size.
to is a no-op when nothing needs changing. It checks the dtype and the
device, and if both already match it hands back the same tensor rather than
copying.
That makes defensive casting cheap:
def f(x):
x = x.to(torch.float32) # free if it already was
...
which is why library code is full of it and why you should not feel bad about writing it.
When you actually want the copy
x.to(torch.float32, copy=True)
Forces a new buffer even when the dtype matches. Use it when the point is to
stop sharing, not to change the type. It is clone wearing different
clothes, and clone says so more clearly.
The cost when it is not a no-op
A dtype change reads every element and writes every element, so it is a full pass over the data and a new allocation of the target size. Going up in width costs memory as well: float32 to float64 doubles it.
This is why mixed-precision training casts once at the boundary rather
than per operation, and why a stray .float() inside a hot loop on a
float16 model undoes the entire point of the model being float16.
The device is the same story, more expensive
x.to("cuda") # no-op if already there, a transfer otherwise
Same rule, and the copy crosses a bus. The devices track has more to say about that.
Your task
def widen(x: torch.Tensor, dtype_name: str) -> torch.Tensor
Return x as the named dtype. If x already has that dtype, return x
itself rather than a copy, so the caller pays nothing.
Stuck?
PyTorch reference solution
Sign in to attempt this problem and reveal the reference solution.