Labels to one-hot

A one-hot encoding gives each label its own row, with one at the label's column and zero everywhere else. For labels [2, 0, 1] and four classes:

[[0, 0, 1, 0],
 [1, 0, 0, 0],
 [0, 1, 0, 0]]

Create a zero matrix, then scatter a one into each selected column:

import torch

labels = torch.tensor([2, 0, 1], dtype=torch.long)
out = torch.zeros(3, 4)
out.scatter_(1, labels.unsqueeze(1), 1.0)
print(out.tolist())
# [[0.0, 0.0, 1.0, 0.0], [1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0]]

Read the index shape

The destination has shape (rows, classes). Scattering along dimension 1 selects a column while retaining the row position. labels.unsqueeze(1) has shape (rows, 1), so it specifies one destination per row. The scalar source 1.0 supplies the value for every selected destination.

Repeated labels do not collide here: two rows with label 1 write to out[0, 1] and out[1, 1], which are different elements. Collision behavior depends on the full destination coordinates, not merely repeated numbers in the index tensor. Ordinary scatter_ is sufficient for this construction.

In application code

torch.nn.functional.one_hot(labels, num_classes=n_classes) provides a direct encoding operation and returns an int64 tensor. Convert its dtype if you need floating-point indicators. This exercise uses scatter to make the relationship between row positions, column indices and written values explicit. More elaborate targets, such as multiple labels per row, require you to decide how repeated destinations should behave.

Select values using coordinates

A slice selects a regularly spaced region of an array. A gather selects entries using supplied indices. Check whether those indices select whole rows or columns, or a different entry within each row.

For [[10, 11], [20, 21]], choosing column 1 from the first row and column 0 from the second gives [11, 20]. Choosing columns [1, 0] from every row instead gives [[11, 10], [21, 20]]. These operations use the same index values but have different meanings and output shapes.

Scatter writes values to selected destinations

Scatter takes values and destination indices. The operation may replace existing values or combine them using a reduction such as addition.

Starting with [0, 0, 0], scatter-add of values [2, 3] at indices [1, 1] gives [0, 5, 0]. Both values contribute to the repeated destination. Repeated assignment is not equivalent to this sum; follow the operation's rule for repeated indices.

Selecting with a mask differs from replacing with a mask

For values [10, 20, 30] and mask [True, False, True], selecting masked entries gives [10, 30]. Replacing masked entries with zero gives [0, 20, 0]. Selection can change the shape; replacement preserves it. A selection whose size depends on runtime values needs particular care in compiled code with fixed output shapes.

Check whether an update changes the original array

JAX indexed updates return a new array value. In PyTorch, basic slicing can return a view that shares storage, while advanced indexing returns a copy. Modifying a selected copy does not modify the source. Direct indexed assignment is a separate operation, so distinguish changing a selection from assigning into the original.