A stride of zero

You have a row and you want it repeated down a matrix.

v = torch.arange(4).reshape(1, 4)

v.expand(3, 4)      # (3, 4), stride (0, 1), same buffer
v.repeat(3, 1)      # (3, 4), stride (4, 1), new buffer

Both give the same numbers. One of them allocated nothing.

Stride zero

Moving one step along an axis whose stride is 0 moves nowhere in memory. So all three rows of the expanded tensor read the same four floats. The repetition exists in the stride, not in the buffer.

This is the mechanism underneath all of broadcasting. When PyTorch broadcasts a (1, 4) against a (3, 4), it is expanding the first to stride (0, 1) and then doing an ordinary elementwise op. Broadcasting is not a special case in the kernel; it is a view plus a normal operation.

The one restriction

An expanded tensor aliases itself, so writing to it is refused:

RuntimeError: unsupported operation: more than one element of the
written-to tensor refers to a single memory location.

That is a good error. The alternative would be three writes racing to one address. If you need to write, you needed repeat, and repeat is the correct answer to that question rather than a worse version of expand.

Which to use

Reading, feeding into an elementwise op, adding a bias across a batch: expand, or just let broadcasting do it implicitly. Building something you will mutate, or handing to a kernel that requires contiguous input: repeat.

The common mistake is repeat where expand would do, usually written by someone who wanted to “make the shapes line up” and reached for the one that sounded more definite. On a batch of activations that is a full copy per step.

Your task

def broadcast_row(v: torch.Tensor, rows: int) -> torch.Tensor

v is (1, cols). Return a (rows, cols) tensor whose every row is v, without copying the buffer.