We can't find the internet
Attempting to reconnect
Something went wrong!
Attempting to reconnect
medium
primitives
Implement Batch Normalization
Implement Batch Normalization for a 2D input (batch of feature vectors).
$$\hat{x}_i = \frac{x_i - \mu_B}{\sqrt{\sigma_B^2 + \epsilon}}$$ $$y_i = \gamma \hat{x}_i + \beta$$
where $\mu_B$ and $\sigma_B^2$ are the mean and variance computed over the batch dimension.
Input:
-
x: tensor of shape(N, D)(N samples, D features) -
gamma: scale parameter of shape(D,) -
beta: shift parameter of shape(D,) -
eps: small constant for numerical stability (default 1e-5)
Output: Normalized tensor of shape (N, D)
Hints
normalization
training
neural-network
Sign in to attempt this problem and view the solution.