Medium End-to-End

Autoencoder

An autoencoder maps an input to a latent representation, then maps that representation back to the input's shape. Its reconstruction loss measures how closely the output matches the original input. The input itself is the target, so no separate label array is needed.

Here, evaluate one encoder layer and one decoder layer using supplied parameters. Do not train the model or update its weights.

Your task

Implement autoencoder(x, W_e, b_e, W_d, b_d) for floating-point inputs:

x has shape (N, D): N examples, each with D features.
W_e has shape (D, H) and b_e has shape (H,).
W_d has shape (H, D) and b_d has shape (D,).

All dimensions are positive. H is the latent width; it need not be smaller than D. Compute:

latent = ReLU(x @ W_e + b_e)
reconstruction = latent @ W_d + b_d
loss = mean((x - reconstruction)**2)

ReLU replaces negative encoder outputs with zero. The decoder is linear: apply no ReLU, sigmoid or other activation to its output. Reconstructions may contain negative values. Use W_d as supplied; it is not necessarily the transpose of W_e.

Return a dictionary with these keys:

"latent": a tensor of shape (N, H).
"reconstruction": a tensor of shape (N, D).
"loss": a Python float, averaged over all N * D squared errors.

Preserve all inputs. Do not sum per-example errors without also dividing by D, and do not take a square root.

Example

For x = [[1, 2]], W_e = [[1], [1]] and b_e = [0], the latent representation is [[3]]. With W_d = [[1, 1]] and b_d = [0, 0], the reconstruction is [[3, 3]]. The squared errors are [4, 1], giving a loss of 2.5.

A smaller latent width provides a bottleneck, but it does not by itself guarantee useful features or accurate reconstruction. Those depend on the parameters and how the model is trained.

Understand the ideas behind this exercise. The Problem tab contains the exact implementation requirements.

Encode an input, then reconstruct it

An autoencoder has two parts. The encoder maps an input x to a representation z = f(x). The decoder maps that representation back to a reconstruction x_hat = g(z). Training minimizes a reconstruction loss, often squared error.

A smaller hidden representation, or latent space, forces the model to express the input using fewer coordinates. That alone does not guarantee useful features or prevent memorization. The model's architecture and training objective matter.

Without biases or nonlinear activations, a linear encoder and decoder combine into a linear map whose rank is limited by the latent width. Biases add an affine offset; nonlinear activations allow more general reconstructions. Follow the exercise's architecture rather than assuming every autoencoder uses the same activation.

For denoising, use noisy input and a clean target

A denoising autoencoder receives a corrupted input and tries to reconstruct the clean version. Training it to reproduce the noisy input would be a different task. For target [1, 0] and reconstruction [0.8, 0.2], squared errors are [0.04, 0.04], so their mean is 0.04.

Check whether the loss sums or averages over features and examples. This changes the gradient scale. In manual backpropagation, compute encoder gradients using the decoder weights from the same forward pass, before updating either set of weights.

Low reconstruction loss shows that the model fits these targets. Evaluate on held-out examples to check whether it also handles unfamiliar inputs or noise.

Detecting runtime…

Autoencoder

Your task

Example

Encode an input, then reconstruct it

For denoising, use noisy input and a clean target

Your code, saved here

Stuck?

PyTorch reference solution