Transposed convolution

The learnable upsampling layer: each input pixel stamps a scaled copy of the kernel into the output at stride s. It is the transpose (adjoint) of a strided convolution, not its inverse, and when s does not divide k the uneven overlaps cause checkerboard artifacts.

12 min read1048 wordscore0 viewsupdated 2026-10-11

Definition

Transposed convolution (also fractionally-strided convolution, often mis-called deconvolution) is the linear map whose matrix is the transpose of a strided convolution's matrix. If a stride-ss convolution maps a large input zz to a small output CzCz, the transposed convolution with the same kernel maps a small input xx to a large output C⊤xC^{\top}x. Mechanically, every input value x[i]x[i] multiplies the whole kernel and is added into the output starting at position s is\,i.

Encoder-decoder networks need a layer that goes the other way: from a coarse feature map back to a fine one. Fixed interpolation (nearest, bilinear) can do this but cannot learn. A transposed convolution is the learnable version, and it also appears automatically in every CNN, because it is exactly what backpropagation computes for the gradient of a strided convolution with respect to its input.

The name "deconvolution" comes from Zeiler, Krishnan, Taylor and Fergus's Deconvolutional Networks (2010) and from Zeiler and Fergus's 2014 "deconvnet" visualisations, and the term stuck. It is a misnomer: deconvolution in signal processing means inverting a convolution, while C⊤C^{\top} is only the adjoint and does not undo CC. Long, Shelhamer and Darrell's FCN (CVPR 2015) called it "backwards strided convolution" and used it, initialised to bilinear interpolation, to upsample segmentation scores; Dumoulin and Visin's A guide to convolution arithmetic for deep learning (2016) fixed the modern name and the shape rules. Radford, Metz and Chintala's DCGAN (ICLR 2016) built its generator entirely from them, and U-Net (Ronneberger, Fischer and Brox, 2015) uses 2×22\times2 "up-convolutions" in its decoder.

Formula and output size

y[o]=∑i∑t=0k−11[s i+t−p=o]  w[t] x[i],nout=(n−1) s−2p+k+output_paddingy[o]=\sum_{i}\sum_{t=0}^{k-1}\mathbb 1\big[s\,i+t-p=o\big]\;w[t]\,x[i],\qquad n_{\text{out}}=(n-1)\,s-2p+k+\text{output\_padding}
ss
stride: how far apart consecutive input pixels stamp their kernel copies in the output (the upsampling factor)
pp
padding: here it crops pp pixels from each side of the full output (the opposite of padding in a convolution)
kk
kernel size; each input pixel touches kk outputs per axis
output_padding\text{output\_padding}
extra rows/columns added on one side only, 0≤0\le value <s<s, to choose among the input sizes that a stride-ss conv maps to the same nn

For 2-D the same holds per axis. Common configurations: k=4, s=2, p=1k=4,\ s=2,\ p=1 gives exactly 2n2n (4→84\to8; the DCGAN layer); k=2, s=2, p=0k=2,\ s=2,\ p=0 also gives 2n2n with no overlap (U-Net). The 3×33\times3, stride-2, padding-1 convolution maps both 1313 and 1414 to 77, so its transpose is ambiguous: (7−1)⋅2−2+3=13(7-1)\cdot2-2+3=13, and output_padding=1 selects 1414.

Three equivalent views

The same operation, three ways

  1. 1Scatter. For each input pixel, add x[i]⋅wx[i]\cdot w into the output at offset s is\,i; overlapping stamps are summed; then crop pp from each side.
  2. 2Insert zeros, then convolve. Put s−1s-1 zeros between input pixels (length (n−1)s+1(n-1)s+1), pad each side with k−1−pk-1-p zeros (plus output_padding on one side), and run an ordinary stride-1 convolution with the flipped kernel. The length works out to (n−1)s−2p+k+op(n-1)s-2p+k+\text{op}. This is why it is called fractionally strided: it behaves like a convolution with stride 1/s1/s.
  3. 3Matrix transpose. Write the stride-ss conv as a sparse matrix CC (one row per output, the kernel shifted by ss per row). The transposed conv multiplies by C⊤C^{\top}. For any zz and xx, ⟨Cz, x⟩=⟨z, C⊤x⟩\langle Cz,\,x\rangle=\langle z,\,C^{\top}x\rangle.

The third view gives the gradients for free. If y=C⊤xy=C^{\top}x and g=∂L/∂yg=\partial L/\partial y, then ∂L/∂x=C g\partial L/\partial x=C\,g: the backward pass of a transposed convolution is an ordinary strided convolution with the same kernel, and vice versa. The weight gradient is ∂L/∂w[t]=∑ix[i] g[s i+t−p]\partial L/\partial w[t]=\sum_i x[i]\,g[s\,i+t-p].

y=C⊤x⟹∂L∂x=C ∂L∂y,∂L∂w[t]=∑ix[i]  ∂L∂y[s i+t−p]y=C^{\top}x\quad\Longrightarrow\quad \frac{\partial L}{\partial x}=C\,\frac{\partial L}{\partial y},\qquad \frac{\partial L}{\partial w[t]}=\sum_i x[i]\;\frac{\partial L}{\partial y[s\,i+t-p]}
CC
the matrix of the stride-ss convolution with kernel ww, shape n×noutn\times n_{\text{out}}
C⊤C^{\top}
its transpose: the transposed convolution, shape nout×nn_{\text{out}}\times n

Worked example in 1-D

Let x=(1,2,3)x=(1,2,3), w=(1,2,1)w=(1,2,1), s=2s=2, p=0p=0. The output length is (3−1)⋅2+3=7(3-1)\cdot2+3=7. Input 0 stamps (1,2,1)(1,2,1) at positions 0–2, input 1 stamps (2,4,2)(2,4,2) at 2–4, input 2 stamps (3,6,3)(3,6,3) at 4–6:

output position0123456
x[0]·w121
x[1]·w242
x[2]·w363
y (sum)1234563
stamps overlapping1121211
Scatter-add of three kernel stamps (verified in Node against the figure's library)

Positions 2 and 4 receive two stamps, the others one: with k=3k=3 and s=2s=2 the overlap alternates 1,2,1,21,2,1,2. With p=1p=1 the output is cropped to (2,3,4,5,6)(2,3,4,5,6), and with p=1p=1, output_padding=1 it is (2,3,4,5,6,3)(2,3,4,5,6,3). The zero-insertion view gives the same numbers: (0,0,1,0,2,0,3,0,0)(0,0,1,0,2,0,3,0,0) correlated with the (here symmetric) flipped kernel.

Adjoint check and gradients, upstream gradient g = (1, 0, 2, 1, 0, 1, 3)

  1. 1The stride-2 conv of gg with ww reads windows g[0..2], g[2..4], g[4..6]g[0..2],\,g[2..4],\,g[4..6]: Cg=(3,4,5)Cg=(3,4,5).
  2. 2Adjoint identity: ⟨Cg,x⟩=3+8+15=26\langle Cg,x\rangle=3+8+15=26 and ⟨g,C⊤x⟩=1+0+6+4+0+6+9=26\langle g,C^{\top}x\rangle=1+0+6+4+0+6+9=26.
  3. 3So for L=⟨g,y⟩L=\langle g,y\rangle the input gradient is ∂L/∂x=Cg=(3,4,5)\partial L/\partial x=Cg=(3,4,5): a strided convolution.
  4. 4Weight gradient: ∂L/∂w[t]=∑ix[i] g[2i+t]\partial L/\partial w[t]=\sum_i x[i]\,g[2i+t] gives (1+4+0,  0+2+3,  2+0+9)=(5,5,11)(1+4+0,\;0+2+3,\;2+0+9)=(5,5,11).

Checkerboard artifacts

Feed an all-ones input through an all-ones kernel and every output pixel counts how many stamps land on it. Away from the border these counts take the values ⌊k/s⌋\lfloor k/s\rfloor and ⌈k/s⌉\lceil k/s\rceil per axis, so they are uniform only if ss divides kk. In 2-D the count map is the outer product of the 1-D counts: for k=3, s=2k=3,\ s=2 the interior alternates 1,2,41,2,4, a checkerboard with a 4:1 ratio. Odena, Dumoulin and Olah ("Deconvolution and Checkerboard Artifacts", Distill 2016) showed that GAN and super-resolution outputs inherit exactly this pattern, and that stacking layers compounds it at several scales.

ConvTranspose2d(k=3, s=2, p=0, output_padding=0) · all-ones input and kernelinput 4×4output = overlap count per pixel112121211112121211224242422112121211224242422112121211224242422112121211112121211input (1,1)stamps 9 outputsclick any celloutput9 × 9(n−1)s − 2p + k + op6 − 0 + 3 + 0interior counts1 … 4max / min4.00k mod s1 → checkerboardzero-insert + pad7 + 2·2 = 11then 3×3 conv, s=1→ 9□ selected cell and its links · ┄ interior (no border effects) · darker = more overlap
fig.A 4×4 all-ones input through ConvTranspose2d with an all-ones kernel: each output cell shows how many kernel stamps overlap there. The default k = 3, s = 2, p = 0 gives a 9×9 output, (4−1)·2 − 0 + 3 = 9, and the dashed interior alternates between 1, 2 and 4 overlaps (max/min 4.00): the checkerboard. Click an input cell to see the 3×3 block of outputs it stamps, or an output cell to see which inputs it sums. Set k = 4 (k mod s = 0) and the interior becomes a uniform 4; the k4 s2 p1 preset gives an exact 8×8 doubling. Switch to resize-conv (nearest-neighbour ×2, then a 3×3 conv) and every interior output reads exactly 9 taps.

Choosing kk divisible by ss (k=4, s=2k=4,\ s=2; k=2, s=2k=2,\ s=2) evens out the overlap, but the network can still learn uneven kernels that bring the artifacts back. Odena et al. recommend resize-convolution instead: upsample with nearest-neighbour or bilinear interpolation, then apply an ordinary stride-1 convolution. Every output then reads the same number of taps by construction. A third option is sub-pixel convolution (pixel shuffle, Shi et al. 2016): a stride-1 conv produces s2s^2 times the channels and nn.PixelShuffle rearranges them into an s×s\times larger map; it is equivalent to a transposed convolution with a kernel of size s⋅k′s\cdot k', and benefits from an initialisation that makes the s2s^2 sub-kernels identical (ICNR, Aitken et al. 2017).

python
1import torch, torch.nn as nn, torch.nn.functional as F
2
3# Note the weight layout: (in_channels, out_channels // groups, kH, kW), reversed vs Conv2d
4up = nn.ConvTranspose2d(512, 256, kernel_size=4, stride=2, padding=1)
5print(up.weight.shape) # torch.Size([512, 256, 4, 4])
6print(up(torch.randn(1, 512, 8, 8)).shape) # torch.Size([1, 256, 16, 16])
7print(sum(p.numel() for p in up.parameters())) # 2097408
8
9# output_padding picks between the sizes a stride-2 conv would collapse
10t = nn.ConvTranspose2d(8, 8, 3, stride=2, padding=1)
11print(t(torch.randn(1, 8, 7, 7)).shape) # 13 x 13
12print(t(torch.randn(1, 8, 7, 7), output_size=(14, 14)).shape) # 14 x 14 (sets output_padding=1)
13
14# 1-D worked example
15x = torch.tensor([[[1., 2, 3]]])
16w = torch.tensor([[[1., 2, 1]]])
17print(F.conv_transpose1d(x, w, stride=2).flatten().tolist()) # [1, 2, 3, 4, 5, 6, 3]
18
19# Adjoint: <conv(g), x> == <g, conv_transpose(x)>
20g = torch.tensor([[[1., 0, 2, 1, 0, 1, 3]]])
21print((F.conv1d(g, w, stride=2) * x).sum(), (g * F.conv_transpose1d(x, w, stride=2)).sum()) # 26, 26
22
23# Checkerboard-free alternative (Odena et al., 2016): resize, then convolve
24resize_conv = nn.Sequential(nn.Upsample(scale_factor=2, mode="nearest"),
25 nn.Conv2d(512, 256, kernel_size=3, padding=1))
26print(resize_conv(torch.randn(1, 512, 8, 8)).shape) # torch.Size([1, 256, 16, 16])
27
28# DCGAN-style generator (PyTorch-tutorial widths): 1x1 -> 4 -> 8 -> 16 -> 32 -> 64
29G = nn.Sequential(
30 nn.ConvTranspose2d(100, 512, 4, 1, 0), nn.BatchNorm2d(512), nn.ReLU(True),
31 nn.ConvTranspose2d(512, 256, 4, 2, 1), nn.BatchNorm2d(256), nn.ReLU(True),
32 nn.ConvTranspose2d(256, 128, 4, 2, 1), nn.BatchNorm2d(128), nn.ReLU(True),
33 nn.ConvTranspose2d(128, 64, 4, 2, 1), nn.BatchNorm2d(64), nn.ReLU(True),
34 nn.ConvTranspose2d(64, 3, 4, 2, 1), nn.Tanh())
35print(G(torch.randn(2, 100, 1, 1)).shape) # torch.Size([2, 3, 64, 64])
nn.ConvTranspose2d: shapes, the worked example, the adjoint, and a resize-conv alternative

Where it's used

Segmentation decoders

FCN upsamples coarse class scores with transposed convolutions initialised to bilinear interpolation (for ×2\times2: k=4k=4, weights 0.25,0.75,0.75,0.250.25,0.75,0.75,0.25 per axis). U-Net's decoder uses 2×22\times2, stride-2 up-convolutions that halve the channels before concatenating the skip connection.

Generators

DCGAN's generator is a stack of 4×44\times4, stride-2 transposed convolutions with Batch normalization and ReLU; many later GANs and VAE decoders kept the pattern or switched to resize-convolution to avoid checkerboards.

Backprop of every strided conv

Autograd computes the input gradient of a strided Conv2d with a transposed convolution, so checkerboard patterns also appear in gradient-based visualisations and adversarial perturbations of strided networks.

Use when

  • +Learnable upsampling with the same parameter count as a conv
  • +Exactly the adjoint of a strided conv: clean shape and gradient algebra
  • +One fused op, fast on GPUs; k = 4, s = 2, p = 1 is a reliable exact ×2

Avoid when

  • −Checkerboard artifacts when s does not divide k, and sometimes even when it does
  • −Confusing size rules: padding crops, output_padding disambiguates
  • −Not an inverse, despite the name 'deconvolution'
  • −Resize-conv or pixel shuffle are often preferred in modern generators and super-resolution

Check yourself

A ConvTranspose2d has kernel 4, stride 2, padding 1. What output size does it produce from a 16 × 16 input?

Check yourself

Which kernel/stride pair gives non-uniform overlap (a checkerboard) for an all-ones input?

Remember

  • ✓A transposed conv scatters x[i]⋅wx[i]\cdot w into the output at stride ss: nout=(n−1)s−2p+k+output_paddingn_{\text{out}}=(n-1)s-2p+k+\text{output\_padding}.
  • ✓It is the matrix transpose of a strided conv; its backward pass is that strided conv. It is not an inverse.
  • ✓Equivalent to inserting s−1s-1 zeros between inputs, padding by k−1−pk-1-p, and convolving with the flipped kernel.
  • ✓Overlap counts are uniform only when ss divides kk; otherwise you get checkerboard artifacts (Odena et al., 2016).
  • ✓Alternatives: resize-convolution (upsample, then conv) and pixel shuffle. PyTorch: nn.ConvTranspose2d, weight shape (Cin,Cout,k,k)(C_{in},C_{out},k,k).
← back to Deep Learning roadmap