Easy End-to-End

Binary Classifier

Evaluate a binary classifier using supplied weights and bias. Its linear score, called a logit, becomes a probability for class 1 after applying sigmoid. This exercise computes predictions and a loss; it does not train or update the parameters.

Your task

Implement binary_classifier(x, W, b, y):

x has shape (N, features), with N >= 1.
W has shape (features, 1) and b has shape (1,).
y has shape (N, 1) and contains labels 0 or 1.

Compute z = x @ W + b, then p = sigmoid(z). Return a dictionary with "prediction" holding p of shape (N, 1) and "loss" holding a Python float. Keep probabilities as probabilities; do not threshold them into class labels. Preserve the inputs.

Loss convention

Use this epsilon-adjusted binary cross-entropy formula, with natural logs:

eps = 1e-7
loss = -mean(y * log(p + eps) + (1 - y) * log(1 - p + eps))

Average over the N examples. Add epsilon inside both logarithms; do not clamp the probabilities or add epsilon to the sigmoid. This convention keeps the logarithms finite when a probability rounds to zero or one. It slightly changes ordinary BCE: a confidently wrong prediction has a loss capped near -log(eps), and a perfectly confident correct prediction can have a tiny negative loss. Do not clip that loss to zero.

Example

For x = [[0], [1]], W = [[2]], b = [0] and y = [[1], [0]], the logits are [[0], [2]] and predictions are approximately [[0.5], [0.880797]]. The second prediction is confidently wrong. The mean adjusted loss is approximately 1.410037.

Keep y as a column. A label vector of shape (N,) would broadcast against p to form an unwanted (N, N) matrix.

Understand the ideas behind this exercise. The Problem tab contains the exact implementation requirements.

Predict the probability of one of two outcomes

A linear binary classifier first computes a score, called a logit:

$$z=w^Tx+b.$$

The weights w combine the input features x. The bias b shifts the score. The sigmoid function converts it to the probability of class 1:

$$p=\frac{1}{1+e^{-z}}.$$

A logit of zero gives probability 0.5. Positive logits give probabilities above 0.5; negative logits give probabilities below it. Predicting class 1 when p >= 0.5 is equivalent to checking z >= 0. Without a bias, the boundary w^T x = 0 passes through the origin.

Train with probabilities, not thresholded predictions

For the true label y, which is either zero or one, binary cross-entropy is

$$\ell=-y\log p-(1-y)\log(1-p).$$

If y = 1, only -log(p) remains. If y = 0, only -log(1-p) remains. In both cases, assigning a higher probability to the true class reduces the loss.

A numerically stable form works directly with the logit:

$$\ell=\max(z,0)-yz+\log(1+e^{-|z|}).$$

This avoids taking logs of probabilities that have rounded to zero or one. The derivative with respect to z is p - y. For mean loss, average this error to get the bias gradient, and average its product with each feature to get that feature's weight gradient.

At zero weights and bias, all probabilities are 0.5 and mean loss is log(2). Lower training loss means the model assigns better probabilities to the training labels. It does not guarantee better results on new data, and a linear boundary cannot fit every pattern.

Apply the decision threshold when making predictions. Replacing probabilities with hard class labels before computing the training loss discards information about how confident the model is.

Detecting runtime…

Binary Classifier

Your task

Loss convention

Example

Predict the probability of one of two outcomes

Train with probabilities, not thresholded predictions

Your code, saved here

Stuck?

PyTorch reference solution