Deep convolutional GAN (DCGAN)

Radford, Metz and Chintala (2016) made convolutional GANs train reliably: strided and transposed convolutions instead of pooling, batch norm except at G's output and D's input, ReLU and a tanh output in G, LeakyReLU(0.2) in D, and Adam with lr 2e-4 and β₁ = 0.5.

13 min read983 wordscore0 viewsupdated 2026-10-12

Definition

Deep convolutional GAN (DCGAN) DCGAN is the convolutional architecture and training recipe from Radford, Metz and Chintala, "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks" (ICLR 2016). The generator turns a noise vector into an image through a stack of fractionally-strided (transposed) convolutions. Each one doubles the resolution and halves the channel count. The discriminator mirrors it with strided convolutions. Both use batch normalization and avoid pooling and hidden fully connected layers. The loss is the standard GAN objective of Generative adversarial network (GAN).

The original GANs of 2014 were multilayer perceptrons. Attempts to scale them up with the CNNs used for classification mostly failed: training diverged, or the generator produced noise. LAPGAN (Denton et al., 2015) worked around this by generating a Laplacian pyramid one scale at a time. Alec Radford, Luke Metz and Soumith Chintala instead searched for a single architecture that trained stably across datasets and at 64×64 resolution. Their paper is mostly an empirical recipe. It also showed that the trained networks learn useful representations. The discriminator's features are good for classification, and the generator's latent space supports smooth interpolation and vector arithmetic.

The architecture guidelines

Section 3 of the paper, verbatim in substance

  1. 1Replace pooling with strided convolutions in DD and fractionally-strided convolutions in GG, so each network learns its own down- and up-sampling (the all-convolutional net of Springenberg et al., 2015).
  2. 2Use batch normalization in both networks, except the generator's output layer and the discriminator's input layer. Applying it to every layer caused sample oscillation and instability.
  3. 3Remove fully connected hidden layers. GG's first layer is a matrix multiplication of zz whose result is reshaped into a 4×44\times4 feature map. DD's last convolution is flattened into a single sigmoid output.
  4. 4Use ReLU in every generator layer except the output, which uses tanh. Images are scaled to [−1,1][-1,1] to match.
  5. 5Use LeakyReLU with slope 0.2 in every discriminator layer.

A transposed convolution is the gradient of a convolution with respect to its input. Each input pixel stamps a scaled copy of the kernel into the output, the copies step ss pixels apart, and overlapping copies add up. With stride ss, padding pp and output padding opo_p, the two layer types change size as follows:

HoutconvT=(Hin−1) s−2p+k+opHoutconv=⌊Hin+2p−ks⌋+1H_{\text{out}}^{\text{convT}}=(H_{\text{in}}-1)\,s-2p+k+o_p\qquad\qquad H_{\text{out}}^{\text{conv}}=\Big\lfloor\frac{H_{\text{in}}+2p-k}{s}\Big\rfloor+1
k=4, s=2, p=1k=4,\ s=2,\ p=1
the standard DCGAN resizing layer: (H−1)⋅2−2+4=2H(H-1)\cdot2-2+4=2H for the transposed conv, ⌊(H+2−4)/2⌋+1=H/2\lfloor(H+2-4)/2\rfloor+1=H/2 for the conv
k=4, s=1, p=0k=4,\ s=1,\ p=0
G's first layer (1×1→4×41\times1\to4\times4, the "project and reshape" of zz) and D's last (4×4→1×14\times4\to1\times1)
opo_p
extra rows on one side of a transposed conv's output, to pick among the input sizes that a strided conv would map to the same output

Training details from the paper

Optimiser
Adam, learning rate 2⋅10−42\cdot10^{-4} (the suggested 10−310^{-3} was too high), β1=0.5\beta_1=0.5 (0.9 caused oscillation and instability)
Batches and init
mini-batch SGD with batch size 128; all weights from N(0, 0.022)\mathcal N(0,\,0.02^2); no pre-processing beyond scaling to [−1,1][-1,1]
Noise
z∈R100z\in\mathbb R^{100} from a uniform distribution
Paper's generator
z→4×4×1024→8×8×512→16×16×256→32×32×128→64×64×3z\to4\times4\times1024\to8\times8\times512\to16\times16\times256\to32\times32\times128\to64\times64\times3
Datasets
LSUN bedrooms (about 3M images), ImageNet-1k (32×32 crops), and a scraped set of 3M face images of 10K people
Features for classification
DD trained on ImageNet-1k, all layers max-pooled to 4×44\times4 and fed to an L2-SVM: 82.8% on CIFAR-10. On SVHN with 1000 labels: 22.48% test error
generator: z ∈ R¹⁰⁰ → 4 fractionally-strided convs → 3×64×64 image · ngf = ndf = 64layerin → outk s p opBNactivationparamsMACsproject z100×1×1 → 512×4×44 1 0 0yesReLU820,2240.82Mup 1512×4×4 → 256×8×84 2 1 0yesReLU2,097,66433.55Mup 2256×8×8 → 128×16×164 2 1 0yesReLU524,54433.55Mup 3128×16×16 → 64×32×324 2 1 0yesReLU131,20033.55Mto RGB64×32×32 → 3×64×644 2 1 0notanh3,0723.15Mtotal G: 3,576,704 params, 104.63M MACs per image · D at the same k: 2,765,568 params100×1²512×4²256×8²128×16²64×32²3×64²up 2: kernel taps landing on each output pixel (8×8 crop)4444444444444444444444444444444444444444444444444444444444444444k = 4, s = 2: k divisible by s → every pixel gets 4uniform overlap: no built-in checkerboardH_out = (H_in − 1)·s − 2p + k + op = (8 − 1)·2 − 2 + 4 + 0 = 16for k = 4: padding p = 1, output padding op = 0 keeps exact ×2 sizesG: BN + ReLU everywhere except tanh output (no BN)no pooling, no hidden fully connected layersAdam lr 2·10⁻⁴, β₁ = 0.5, batch 128, init N(0, 0.02)
network
layer

The PyTorch reference DCGAN uses k = 4, stride 2, padding 1 for every resizing layer. The first G layer and the last D layer are 4×4, stride 1, no padding.

fig.The 64×64 DCGAN of the PyTorch reference implementation (ngf = ndf = 64, nz = 100, so 512 channels at 4×4 instead of the paper's 1024), layer by layer. Counts include BatchNorm's γ and β; the convolutions have no bias. With k = 4 the generator has 3,576,704 parameters and needs 104.63M multiply-accumulates per image. Each of the three middle upsampling layers costs 33.55M, and the first, 'up 1', holds 2,097,664 of the parameters. The discriminator has 2,765,568 parameters and 103.82M MACs. The grid shows how many kernel taps land on each output pixel of the selected transposed conv. With k = 4 and s = 2, every interior pixel receives exactly 4 (2 per axis), so the layer has no built-in pattern. Set k = 3 (p = 1, output padding 1) and the counts alternate 1, 2, 4. That is the checkerboard of Odena et al. (2016): even with constant weights and a constant input, the output already has a period-2 pattern with max/min = 4. k = 5 gives 4, 6, 9 (ratio 2.25). k = 6 is uniform again at 9. For the discriminator, the grid counts how many outputs read each input pixel, which is the same uneven pattern in the backward pass. k = 3 shrinks G to 2,371,136 parameters; k = 5 grows it to 5,126,720.

Worked example: sizes, parameters and overlap

layerin → outk, s, pweightsBN γ, βparamsMACs
project z100×1×1 → 512×4×44, 1, 0819,2001,024820,2240.82M
up 1512×4×4 → 256×8×84, 2, 12,097,1525122,097,66433.55M
up 2256×8×8 → 128×16×164, 2, 1524,288256524,54433.55M
up 3128×16×16 → 64×32×324, 2, 1131,072128131,20033.55M
to RGB + tanh64×32×32 → 3×64×644, 2, 13,072none3,0723.15M
Generator of the figure at k = 4 (counts from the figure's code, totals checked with PyTorch)

Reading the table

  1. 1A transposed conv has cin cout k2c_{\text{in}}\,c_{\text{out}}\,k^2 weights: up 1 has 512⋅256⋅16=2,097,152512\cdot256\cdot16=2{,}097{,}152. Every input pixel multiplies the whole kernel, so it costs cincoutk2Hin2c_{\text{in}}c_{\text{out}}k^2H_{\text{in}}^2 MACs: 2,097,152⋅42=33.552{,}097{,}152\cdot4^2=33.55M. Halving channels while doubling resolution keeps the middle layers at the same cost.
  2. 2Total: 3,576,7043{,}576{,}704 parameters, which is what sum(p.numel() for p in G.parameters()) prints for the PyTorch model below. More than half sit in up 1, where the channels are widest.
  3. 3Overlap in 1-D. Upsampling a length-4 input of ones with a kernel of ones at k=3k=3, s=2s=2, p=1p=1, op=1o_p=1 gives [1, 2, 1, 2, 1, 2, 1, 1]. With k=4k=4, p=1p=1 it gives [1, 2, 2, 2, 2, 2, 2, 1]: uniform away from the border.
  4. 4In 2-D the counts multiply, so the k=3k=3 interior repeats the block 1224\begin{smallmatrix}1&2\\2&4\end{smallmatrix}. The k=4k=4 interior is all 4. The network can learn weights that cancel the k=3k=3 pattern, but it must spend capacity doing so, and in practice the pattern often survives as checkerboard artefacts.
  5. 5The backward pass of a strided conv with k=3k=3, s=2s=2 has the same [1, 2, 1, 2, …] pattern in its input gradient. So an uneven discriminator also pushes an uneven gradient into GG.

What the latent space learned

The paper's most cited figures probe GG's input space. Walking along a line between two zz gives smooth changes: a TV slowly turns into a window, and a room without a window gains a large one. Sudden jumps would suggest memorised images, and the walks showed none. Vector arithmetic on single zz was unstable, but averaging the zz of three samples per concept worked. zˉsmiling woman−zˉneutral woman+zˉneutral man\bar z_{\text{smiling woman}}-\bar z_{\text{neutral woman}}+\bar z_{\text{neutral man}} produced smiling men. "Man with glasses" minus "man without glasses" plus "woman without glasses" produced women with glasses. The authors also used guided backpropagation to show that DD's filters respond to beds and windows. When they removed the generator's "window" feature maps, GG drew other objects where the windows had been.

znew=13∑i=13zi A−13∑i=13zi B+13∑i=13zi C,xnew=G(znew)z_{\text{new}}=\tfrac13\sum_{i=1}^3 z_i^{\,A}-\tfrac13\sum_{i=1}^3 z_i^{\,B}+\tfrac13\sum_{i=1}^3 z_i^{\,C},\qquad x_{\text{new}}=G(z_{\text{new}})
ziAz_i^{A}
latent vectors of three generated samples that show concept A (e.g. smiling woman), chosen by eye
13∑\tfrac13\sum
averaging the three made the arithmetic stable; single vectors often failed
xnewx_{\text{new}}
the paper also added small uniform noise to znewz_{\text{new}} to show that the result is a region of latent space, not one point
python
1import torch, torch.nn as nn
2
3nz, ngf, ndf, nc = 100, 64, 64, 3
4
5class Generator(nn.Module):
6 def __init__(self):
7 super().__init__()
8 def up(i, o): # x2 upsampling: k=4, s=2, p=1
9 return [nn.ConvTranspose2d(i, o, 4, 2, 1, bias=False), nn.BatchNorm2d(o), nn.ReLU(True)]
10 self.net = nn.Sequential(
11 nn.ConvTranspose2d(nz, ngf * 8, 4, 1, 0, bias=False), # (100,1,1) -> (512,4,4): "project and reshape"
12 nn.BatchNorm2d(ngf * 8), nn.ReLU(True),
13 *up(ngf * 8, ngf * 4), # -> (256,8,8)
14 *up(ngf * 4, ngf * 2), # -> (128,16,16)
15 *up(ngf * 2, ngf), # -> (64,32,32)
16 nn.ConvTranspose2d(ngf, nc, 4, 2, 1, bias=False), # -> (3,64,64), no BN on the output
17 nn.Tanh(), # images scaled to [-1, 1]
18 )
19 def forward(self, z):
20 return self.net(z)
21
22class Discriminator(nn.Module):
23 def __init__(self):
24 super().__init__()
25 def down(i, o, bn=True): # x1/2 downsampling, no pooling
26 return [nn.Conv2d(i, o, 4, 2, 1, bias=False), *([nn.BatchNorm2d(o)] if bn else []), nn.LeakyReLU(0.2, True)]
27 self.net = nn.Sequential(
28 *down(nc, ndf, bn=False), # (3,64,64) -> (64,32,32), no BN on the input layer
29 *down(ndf, ndf * 2), # -> (128,16,16)
30 *down(ndf * 2, ndf * 4), # -> (256,8,8)
31 *down(ndf * 4, ndf * 8), # -> (512,4,4)
32 nn.Conv2d(ndf * 8, 1, 4, 1, 0, bias=False), # -> (1,1,1): one logit
33 )
34 def forward(self, x):
35 return self.net(x).view(-1)
36
37def init_weights(m): # DCGAN: N(0, 0.02); BN gamma ~ N(1, 0.02), beta = 0
38 if isinstance(m, (nn.Conv2d, nn.ConvTranspose2d)):
39 nn.init.normal_(m.weight, 0.0, 0.02)
40 elif isinstance(m, nn.BatchNorm2d):
41 nn.init.normal_(m.weight, 1.0, 0.02); nn.init.zeros_(m.bias)
42
43G, D = Generator(), Discriminator()
44G.apply(init_weights); D.apply(init_weights)
45opt_g = torch.optim.Adam(G.parameters(), lr=2e-4, betas=(0.5, 0.999))
46opt_d = torch.optim.Adam(D.parameters(), lr=2e-4, betas=(0.5, 0.999))
47bce = nn.BCEWithLogitsLoss()
48
49z = torch.randn(128, nz, 1, 1)
50x_fake = G(z)
51print(x_fake.shape, D(x_fake).shape) # torch.Size([128, 3, 64, 64]) torch.Size([128])
52print(sum(p.numel() for p in G.parameters()), sum(p.numel() for p in D.parameters())) # 3576704 2765568
53
54def train_step(x_real): # x_real (B,3,64,64) in [-1, 1]
55 B = x_real.size(0)
56 fake = G(torch.randn(B, nz, 1, 1))
57 loss_d = bce(D(x_real), torch.ones(B)) + bce(D(fake.detach()), torch.zeros(B))
58 opt_d.zero_grad(); loss_d.backward(); opt_d.step()
59 loss_g = bce(D(fake), torch.ones(B)) # non-saturating generator loss
60 opt_g.zero_grad(); loss_g.backward(); opt_g.step()
61 return loss_d.item(), loss_g.item()
62
63print(len(train_step(torch.rand(128, 3, 64, 64) * 2 - 1))) # 2
64
65# latent arithmetic: average 3 z's per concept, then combine
66z_smile_w, z_neutral_w, z_neutral_m = (torch.randn(3, nz, 1, 1).mean(0, keepdim=True) for _ in range(3))
67with torch.no_grad():
68 G.eval()
69 print(G(z_smile_w - z_neutral_w + z_neutral_m).shape) # torch.Size([1, 3, 64, 64])
DCGAN generator and discriminator for 64×64 RGB images (PyTorch reference widths), with a training step

Use when

  • +Simple, symmetric recipe that trained stably across bedrooms, faces and ImageNet at 64×64
  • +Learned up- and down-sampling; no pooling or hidden dense layers
  • +Meaningful latent space: smooth walks and vector arithmetic on averaged codes
  • +DD's features transfer: 82.8% on CIFAR-10 with a linear SVM

Avoid when

  • −Still mode-collapses and still uses the JS-type GAN loss; long runs can oscillate
  • −Batch norm couples samples in a batch and the network to the batch size
  • −Transposed convs with kk not divisible by ss create checkerboards
  • −Does not scale well beyond 64×64 without progressive growing, better losses or normalisation

Check yourself

Which layers in DCGAN do NOT use batch normalization?

Check yourself

A transposed conv with stride 2 and kernel size 3 upsamples 8×8 to 16×16. Why does its output tend to show a checkerboard?

Remember

  • ✓DCGAN rules: strided convs in DD, transposed convs in GG, no pooling and no hidden dense layers; BN everywhere except GG's output and DD's input.
  • ✓Activations and optimiser: ReLU + tanh output in GG, LeakyReLU(0.2) in DD; Adam with lr 2⋅10−42\cdot10^{-4}, β1=0.5\beta_1=0.5, batch 128, init N(0,0.022)\mathcal N(0,0.02^2).
  • ✓Resizing layer k=4k=4, s=2s=2, p=1p=1 doubles or halves resolution exactly; transposed conv output (H−1)s−2p+k+op(H-1)s-2p+k+o_p.
  • ✓kk not divisible by ss gives uneven kernel overlap: checkerboard artefacts.
  • ✓Latent space is smooth; averaged-z vector arithmetic (smiling woman − neutral woman + neutral man) works; D's features transfer to classification.
← back to Deep Learning roadmap