Skip to content

← Generative Models step 2 of 11

Hard End-to-End

Simple GAN Generator

Compute a two-layer generator's output and a generator loss from supplied discriminator scores. A GAN is a generative adversarial network: its generator learns to produce samples that a discriminator judges as real.

This exercise evaluates a forward pass and a loss value. It does not run a discriminator, update weights or train a GAN.

Your task

Implement gan_generator(z, W1, b1, W2, b2, disc_score) for floating-point inputs with matching dtype and device:

  • z has shape (batch, noise_dim), with batch > 0. The noise is supplied; do not draw new random values.
  • W1 has shape (noise_dim, hidden_dim) and b1 has shape (hidden_dim,).
  • W2 has shape (hidden_dim, output_dim) and b2 has shape (output_dim,).
  • disc_score has shape (batch, 1) and contains discriminator logits for the generated samples, not probabilities. Higher logits favor the real class.

Compute the generator using row-vector matrix products:

h = ReLU(z @ W1 + b1)
fake = tanh(h @ W2 + b2)

Biases broadcast across the batch. Apply ReLU only after the first affine layer and tanh after the second. The final values lie in [-1, 1] at floating-point precision; negative values are valid outputs.

Return a dictionary with "fake", a tensor of shape (batch, output_dim) preserving z's dtype and device, and "loss", a Python float. Leave all inputs unchanged.

Non-saturating generator loss

For each supplied discriminator logit s, the loss is -log(sigmoid(s)). Average across the batch, rather than summing. Compute it stably as:

per_sample_loss = max(-s, 0) + log1p(exp(-abs(s)))
loss = mean(per_sample_loss)

This equals softplus(-s). It avoids taking the log of an underflowed sigmoid. Do not add epsilon to the sigmoid probability: that changes the loss and artificially caps the penalty for a very negative score.

At s=0, the discriminator assigns probability 0.5 to real, so the loss is approximately 0.693147. A large positive score gives a loss near zero; a large negative score gives a large positive loss.

The supplied disc_score is already computed. Return the loss from those scores; do not infer scores from fake. In a training system the discriminator would be evaluated on fake with a differentiable connection to the generator, but that computation is outside this exercise.