We can't find the internet
Attempting to reconnect
Something went wrong!
Attempting to reconnect
← CNNs from Scratch step 9 of 15
Simple CNN
Implement the forward pass of a small convolutional neural network (CNN) for one one-dimensional signal. It has one filter and returns one number: filter the signal, apply ReLU, keep the largest activation, then apply a scalar weight and bias.
The parameters are supplied. Do not train the network or update them.
Inputs and output
- x has shape (L,) and conv_w has shape (K,), with 1 <= K <= L. They are floating-point tensors with the same dtype and device.
- conv_b, fc_w and fc_b are Python floats.
- Return the prediction as a Python float, leaving the inputs unchanged.
Layer sequence
- Convolution: slide conv_w over x with stride 1 and no padding. For i from 0 through L - K, compute conv[i] = sum(x[i + j] * conv_w[j] for j in range(K)) + conv_b. This gives L - K + 1 values. Use the kernel in its given order, without reversing it.
- ReLU: replace each negative convolution value with zero.
- Global max pooling: take the maximum over all remaining positions. The result is one scalar, not a shorter sequence of local maxima.
- Output layer: return pooled * fc_w + fc_b. Apply no activation after this step, so the prediction can be negative and is not a probability.
For x=[2, -1, 3], conv_w=[1, 2] and conv_b=1, the convolution gives [1, 6]. ReLU keeps both values and global pooling returns 6. With fc_w=0.5 and fc_b=2, the prediction is 5.0.
If every convolution value is nonpositive, pooling returns zero and the prediction equals fc_b. If K=L, there is only one convolution position, but all four steps still apply.
Understand the ideas behind this exercise. The Problem tab contains the exact implementation requirements.
Apply the same filter at many locations
A convolutional layer slides a learned kernel over its input. At each location, it multiplies the covered values by the kernel weights and sums them. Deep-learning libraries usually implement cross-correlation under the name convolution: the kernel is not reversed.
For input [1, 2, 3] and kernel [1, -1], with no padding and stride one,
the outputs are 1*1 + 2*(-1) = -1 and 2*1 + 3*(-1) = -1.
Work out the output shape
For one spatial axis, the output length is
$$\left\lfloor\frac{L+2P-D(K-1)-1}{S}\right\rfloor+1.$$
L is the input length, K the kernel size, P the padding on each end,
S the stride between windows, and D the spacing between kernel elements
(dilation). For a 2D input, apply the formula separately to height and width.
Check the channel layout too: an ordinary convolution sums over both spatial
positions in the window and input channels.
A depthwise convolution filters each input channel separately. A following pointwise convolution, using a 1-by-1 spatial kernel, mixes channels. This uses a different set of parameters from a full spatial convolution; the two constructions are not generally interchangeable.
Pool nearby features
Max pooling keeps the largest value in each window. Average pooling takes the window's mean. During backpropagation, max pooling sends the gradient to the selected position, while average pooling distributes it across the values that contributed to the mean. Overlapping windows can contribute more than once to the same input position, so add those gradients.
Follow the exercise's rules for tied maxima and whether padding counts in an average. For a small CNN, also save the forward-pass values needed for activation and pooling derivatives. Add each shared kernel's gradient contributions across all locations, then update weights after computing all gradients.
Your code, saved here
Sign in to access your accepted solutions and saved draft history. You can keep reading the lesson without an account.
Sign in to your accountStuck?
PyTorch reference solution
Sign in to track your attempts toward unlocking the reference solution.