Easy End-to-End

Linear Regression

A one-feature linear model predicts a target from an input using a weight and a bias. This exercise evaluates supplied parameters; it does not fit weights or run an optimizer.

Your task

Implement linear_regression(x, w, b, y) for floating-point arrays:

x and y have shape (N, 1), with N positive.
w has shape (1, 1) and contains one weight.
b has shape (1,) and contains one bias.

Compute predictions and mean squared error:

prediction = x @ w + b
loss = mean((y - prediction)**2)

Return a dictionary with exactly two keys:

"prediction": an array of shape (N, 1).
"loss": the scalar mean squared error.

Keep the column dimension in prediction and leave inputs unchanged. The bias broadcasts across all N rows. Square each error before averaging; squaring the average error would let opposite-sign errors cancel.

Example

With x = [[1], [2]], w = [[2]] and b = [1], predictions are [[3], [5]]. If targets are [[4], [3]], the errors are 1 and -2, so the loss is (1**2 + (-2)**2) / 2 = 2.5.

This is a mean over examples, with no square root, regularization or extra factor of one-half. Perfect predictions give zero loss.

Understand the ideas behind this exercise. The Problem tab contains the exact implementation requirements.

Predict a numerical value

Regression predicts a number, such as a price or a temperature. A linear model combines the input features using weights and a bias:

$$\hat y=Xw+b.$$

Mean squared error (MSE) averages the squared differences between predictions and targets. For N examples with one target each,

$$L=\frac{1}{N}\sum_i(\hat y_i-y_i)^2.$$

Squaring makes larger errors count more. An error of 4 contributes 16 to the sum, while an error of 2 contributes 4.

The gradient with respect to each prediction is 2 * (prediction - target) / N. For several outputs per example, check whether the loss averages over examples only or over all output elements. The denominator, and therefore the gradient, will differ.

More features can represent curves

Polynomial regression uses features such as [1, x, x²]. The predicted curve can be nonlinear in x while remaining linear in its learned coefficients. Large powers can create very different feature scales, making the fitting problem more sensitive to numerical error and learning-rate choices.

Regularization discourages large weights

Adding the penalty lambda * sum(w²) adds 2 * lambda * w to the weight gradient. Some definitions include a factor of one half, so follow the stated loss rather than assuming a convention.

This penalty is used in ridge regression. It can make fitting more stable when features are strongly correlated, but feature scaling still matters: rescaling a feature changes the coefficient needed to represent the same prediction.

Why squared error favors the mean

For targets [1, 3], predicting 2 for both gives squared errors [1, 1] and MSE 1. Predicting 1 for both gives errors [0, 4] and MSE 2. The mean minimizes squared error for a constant prediction. More generally, when the relevant expectations exist, the best prediction under expected squared loss is the target's conditional mean given the input.

Large outliers can strongly affect this mean and the fitted model. Evaluate on held-out data to check whether a low training loss carries over to new examples.

Detecting runtime…