medium primitives

Implement Batch Normalization

Implement Batch Normalization for a 2D input (batch of feature vectors).

$$\hat{x}_i = \frac{x_i - \mu_B}{\sqrt{\sigma_B^2 + \epsilon}}$$ $$y_i = \gamma \hat{x}_i + \beta$$

where $\mu_B$ and $\sigma_B^2$ are the mean and variance computed over the batch dimension.

Input:

  • x: tensor of shape (N, D) (N samples, D features)
  • gamma: scale parameter of shape (D,)
  • beta: shift parameter of shape (D,)
  • eps: small constant for numerical stability (default 1e-5)

Output: Normalized tensor of shape (N, D)

Hints

normalization training neural-network
Detecting runtime...