Skip to content

← Shapes and Strides step 4 of 9

Easy Primitives

The batch of one

squeeze() with no argument removes every axis of length 1. That sounds convenient and is the source of one of the most annoying bugs in PyTorch, because which axes have length 1 depends on your data.

The setup

A model emits (batch, 1, features) and you want (batch, features). The obvious line works all the way through development:

out.squeeze()          # (32, 1, 5) -> (32, 5)     correct

Then the last batch of the epoch has one item left over, or somebody runs a single example through, and:

out.squeeze()          # (1, 1, 5)  -> (5,)        the batch axis is gone

Two axes were length 1, so two axes were removed. Everything downstream that indexed [batch_index] is now indexing features, and the error it eventually raises will be somewhere else entirely.

The fix

Name the axis:

out.squeeze(dim=1)     # (32, 1, 5) -> (32, 5)
                       # (1, 1, 5)  -> (1, 5)      batch axis survives

squeeze(dim=...) removes that axis if it has length 1 and otherwise does nothing at all. It never removes an axis you did not name.

The general habit

This is one instance of a rule worth adopting now: prefer the form that names what it acts on. squeeze(dim=1) over squeeze(), sum(dim=0) over sum(), dim=-1 over a positive index when you care about the last axis rather than the third one. The unnamed forms are shorter and behave differently depending on data you cannot see from the line.

Your task

def drop_channel(x: torch.Tensor) -> torch.Tensor

x has shape (batch, 1, features). Return it as (batch, features), for any batch size including 1.

The starter passes every case where the batch is larger than one.