Skip to content
← All tracks

Normalization

PyTorch

BatchNorm, LayerNorm, GroupNorm, RMSNorm, dropout — when to use which, and how each shifts gradients.

0 / 6 solved
Continue →
  1. 1. Not solved yet. Implement Batch Normalization
  2. 2. Not solved yet. Implement Layer Normalization
  3. 3. Not solved yet. Implement Group Norm
  4. 4. Not solved yet. RMSNorm
  5. 5. Not solved yet. Implement Dropout
  6. 6. Not solved yet. Feature Normalization Pipeline

Check yourself

4 questions · one attempt each

These do not count toward finishing the track. They are here to catch the things that are easy to read past.

0 / 4

Over which axis does each normalise? 

nn.LayerNorm(5)(t)     # t is (4, 5)
nn.BatchNorm1d(5)(t)
Question 1 of 4