Medium Primitives

Implement Binary Cross-Entropy Loss

Binary cross-entropy measures how much probability a model assigns to the correct binary label. Here y_pred is the probability of label 1.

For label 1, the loss is -log(p). For label 0, it is -log(1 - p). Use the natural logarithm. A confident correct prediction has a small loss; a confident wrong prediction has a large loss.

For labels [1, 0] and probabilities [0.8, 0.25], the probabilities assigned to the correct labels are [0.8, 0.75]. The mean loss is (-log(0.8) - log(0.75)) / 2, approximately 0.255413.

Your task

Implement bce_loss(y_true, y_pred) for two nonempty 1D floating-point tensors of the same shape. Labels are 0 or 1. Predictions are probabilities in [0, 1], not logits (raw model scores).

First clamp each prediction to [1e-7, 1 - 1e-7]. Then return the scalar mean of -(y * log(p) + (1 - y) * log(1 - p)) using the clamped values. Clamp the probabilities; do not add epsilon to every logarithm argument.

Clamping makes the loss finite even for an input probability of 0 or 1. It also changes the loss near those endpoints. This exercise specifies its own probability-clamping rule, so do not substitute a library BCE loss with a different boundary rule. Build the result from basic tensor operations.

The plot starts with label 1 and shows the loss as its predicted probability changes. It uses the same probability-clamping rule as the exercise.

Loading visualization…

Understand the ideas behind this exercise. The Problem tab contains the exact implementation requirements.

Predict the probability of one of two outcomes

A linear binary classifier first computes a score, called a logit:

$$z=w^Tx+b.$$

The weights w combine the input features x. The bias b shifts the score. The sigmoid function converts it to the probability of class 1:

$$p=\frac{1}{1+e^{-z}}.$$

A logit of zero gives probability 0.5. Positive logits give probabilities above 0.5; negative logits give probabilities below it. Predicting class 1 when p >= 0.5 is equivalent to checking z >= 0. Without a bias, the boundary w^T x = 0 passes through the origin.

Train with probabilities, not thresholded predictions

For the true label y, which is either zero or one, binary cross-entropy is

$$\ell=-y\log p-(1-y)\log(1-p).$$

If y = 1, only -log(p) remains. If y = 0, only -log(1-p) remains. In both cases, assigning a higher probability to the true class reduces the loss.

A numerically stable form works directly with the logit:

$$\ell=\max(z,0)-yz+\log(1+e^{-|z|}).$$

This avoids taking logs of probabilities that have rounded to zero or one. The derivative with respect to z is p - y. For mean loss, average this error to get the bias gradient, and average its product with each feature to get that feature's weight gradient.

At zero weights and bias, all probabilities are 0.5 and mean loss is log(2). Lower training loss means the model assigns better probabilities to the training labels. It does not guarantee better results on new data, and a linear boundary cannot fit every pattern.

Apply the decision threshold when making predictions. Replacing probabilities with hard class labels before computing the training loss discards information about how confident the model is.

Detecting runtime…

Implement Binary Cross-Entropy Loss

Your task

Predict the probability of one of two outcomes

Train with probabilities, not thresholded predictions

Your code, saved here

Stuck?

PyTorch reference solution