We can't find the internet
Attempting to reconnect
Something went wrong!
Attempting to reconnect
← CNNs from Scratch step 6 of 15
Transposed Conv2d
A transposed convolution turns each input value into a weighted output patch. Patches can overlap, so their contributions must be added.
Implement transposed_conv2d(x, w, stride=1). This exercise uses no bias, padding or output padding, dilation 1, and a single group.
Inputs and output
- x has shape (N, C_in, H, W): batch, input channels, height and width.
- w has shape (C_in, C_out, kH, kW). The input-channel axis comes first.
- stride is a positive integer used along both spatial axes.
- All dimensions are positive. x and w share a floating-point dtype and device.
Return shape (N, C_out, H_out, W_out), preserving the input dtype and device, where:
H_out = (H - 1) * stride + kH
W_out = (W - 1) * stride + kW
Place and add the patches
Start with a zero output. For each example n, input channel ci and input position (i, j), multiply x[n, ci, i, j] by every output-channel kernel w[ci, co]. Place each resulting patch with its top-left corner at (i * stride, j * stride):
out[n, co, i * stride + a, j * stride + b] += x[n, ci, i, j] * w[ci, co, a, b]
Apply this for every output channel co and kernel position (a, b). Use w in its given spatial order. Add contributions from all input channels and positions; do not overwrite overlaps or mix batch examples.
For a single input channel containing one row [1, 2], a kernel with one row [1, 3], and stride 1, the patches are [1, 3] and [2, 6], offset by one position. Their sum is [1, 5, 6]. At stride 3, the result is [1, 3, 0, 2, 6]: positions reached by no patch stay zero.
Transposed convolution is often used to enlarge feature maps. It does not always enlarge them: a 1-by-1 kernel with stride 1 keeps the spatial shape. It applies the transpose of a convolution's linear map, not its inverse, so it does not generally recover the original input. You are implementing the forward operation here, not computing gradients.
Leave x and w unchanged.
Understand the ideas behind this exercise. The Problem tab contains the exact implementation requirements.
Apply the transpose of a convolution's linear map
With fixed weights and no bias, a convolution can be written as y = Ax.
A transposed convolution applies A^T instead. This is the operation used to
send an output gradient back to the input of that convolution. It generally
does not undo the convolution or recover the original image.
Ordinary convolution combines a neighborhood into each output value. Its transpose sends each input value through the kernel into an output neighborhood. Where contributions overlap, they are added.
For example, a stride-one kernel [1, 2] applied in this transposed direction
to [3, 4] gives [3, 10, 8]: the first value contributes [3, 6] and the
second contributes [4, 8] one position later. The middle entry is 6 + 4.
Replacing rather than adding overlapping contributions would lose information.
Match the intended output shape
For input length L, stride S, padding P, dilation D, kernel size K,
and output padding O, one output dimension is
$$(L-1)S-2P+D(K-1)+O+1.$$
Output padding resolves shape ambiguity caused by the forward convolution's stride. It does not mean appending learned values or reflected pixels.
To check a matching convolution and its transpose, use
dot(Ax, y) = dot(x, A^T y), allowing for floating-point rounding. Check channel
order and kernel orientation as well as the output size. Input and output
channel roles swap, and the library's stored weight layout may differ from the
matrix notation above.
Your code, saved here
Sign in to access your accepted solutions and saved draft history. You can keep reading the lesson without an account.
Sign in to your accountStuck?
JAX reference solution
Sign in to track your attempts toward unlocking the reference solution.