We can't find the internet
Attempting to reconnect
Something went wrong!
Attempting to reconnect
← CNNs from Scratch step 2 of 15
Implement Conv2d
A 2D cross-correlation slides a kernel across an image or feature map. At each location, it multiplies matching values and sums the products. For several input channels, sum their contributions too. Each output channel has its own set of kernel weights.
For a single input channel [[1, 2, 3], [4, 5, 6]] and kernel
[[1, 0], [0, -1]], with stride 1 and no padding, there are two windows.
The first gives 1*1 + 2*0 + 4*0 + 5*(-1) = -4, and the second also
gives -4. The output has one row: [[-4, -4]].
The kernel stays in its given order. Despite the common name convolution, this exercise computes cross-correlation, with no spatial kernel flip.
Your task
Implement conv2d(x, w, stride=1, padding=0):
xhas shape(N, C_in, H, W): examples, input channels, height, width.whas shape(C_out, C_in, kH, kW). Each output channel uses all input channels. The input and weights have matching floating-point dtypes and devices.strideis a positive integer used for both spatial axes.paddingis a nonnegative integer. Add that many zeros to each of the four spatial edges of every input channel. Do not pad batch or channel axes.
All dimensions are positive, and the padded input is at least as large
as the kernel in each spatial dimension. The output shape is
(N, C_out, H_out, W_out), where
H_out = (H + 2 * padding - kH) // stride + 1
W_out = (W + 2 * padding - kW) // stride + 1
On the padded input, output position (i, j) uses the patch starting at
(i * stride, j * stride). For each output channel, multiply that patch
by its weights and sum over all input channels and kernel positions.
There is no bias, dilation, or channel grouping. Trailing partial windows
are omitted. Rectangular images and kernels are valid.
For example, a 1-by-1 kernel with input-channel weights [2, -1] applied
to channel values [3, 5] returns 2*3 - 5 = 1 at that position.
This is one output value, not separate values for each input channel.
The diagram illustrates window movement along one spatial axis. The same indexing rule applies independently to height and width here. Keep dilation at 1; the diagram does not show channel summation.
Loading visualization…
Understand the ideas behind this exercise. The Problem tab contains the exact implementation requirements.
Apply the same filter at many locations
A convolutional layer slides a learned kernel over its input. At each location, it multiplies the covered values by the kernel weights and sums them. Deep-learning libraries usually implement cross-correlation under the name convolution: the kernel is not reversed.
For input [1, 2, 3] and kernel [1, -1], with no padding and stride one,
the outputs are 1*1 + 2*(-1) = -1 and 2*1 + 3*(-1) = -1.
Work out the output shape
For one spatial axis, the output length is
$$\left\lfloor\frac{L+2P-D(K-1)-1}{S}\right\rfloor+1.$$
L is the input length, K the kernel size, P the padding on each end,
S the stride between windows, and D the spacing between kernel elements
(dilation). For a 2D input, apply the formula separately to height and width.
Check the channel layout too: an ordinary convolution sums over both spatial
positions in the window and input channels.
A depthwise convolution filters each input channel separately. A following pointwise convolution, using a 1-by-1 spatial kernel, mixes channels. This uses a different set of parameters from a full spatial convolution; the two constructions are not generally interchangeable.
Pool nearby features
Max pooling keeps the largest value in each window. Average pooling takes the window's mean. During backpropagation, max pooling sends the gradient to the selected position, while average pooling distributes it across the values that contributed to the mean. Overlapping windows can contribute more than once to the same input position, so add those gradients.
Follow the exercise's rules for tied maxima and whether padding counts in an average. For a small CNN, also save the forward-pass values needed for activation and pooling derivatives. Add each shared kernel's gradient contributions across all locations, then update weights after computing all gradients.
Your code, saved here
Sign in to access your accepted solutions and saved draft history. You can keep reading the lesson without an account.
Sign in to your accountStuck?
PyTorch reference solution
Sign in to track your attempts toward unlocking the reference solution.