VGGNet

Simonyan & Zisserman (2014) built 11–19-layer nets from nothing but 3×3 convs and 2×2 pools. Three stacked 3×3 layers see 7×7 with 45% fewer weights than one 7×7 and three ReLUs. VGG-16: 138M params (74% in fc6), 15.5 GFLOPs.

12 min read1018 wordscore0 viewsupdated 2026-10-11

Definition

VGGNet is the family of networks from Simonyan and Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition" (arXiv 2014, ICLR 2015), by Oxford's Visual Geometry Group. Every convolution is 3×33\times3 with stride 1 and padding 1, so it keeps the spatial size. Five 2×22\times2 stride-2 max-pools halve it, the width doubles after each pool from 64 to 512, and three dense layers (4096, 4096, 1000) finish the network. Configurations A to E differ only in depth, from 11 to 19 weight layers. D and E are the famous VGG-16 and VGG-19.

By 2014 the question after AlexNet and ZFNet was what actually matters in a CNN. The VGG paper isolated one variable, depth. It fixed every other design choice to the simplest option (smallest useful kernel, stride 1, same padding, one pooling type) and added layers. The result was second place in the ILSVRC-2014 classification task, with 7.3% top-5 test error from an ensemble of 7 nets, behind GoogLeNet (Inception). It was first in localisation, with 25.3% error. After the competition a 2-net ensemble of D and E reached 6.8%, and the best single net 7.0%.

VGG's real legacy is its uniformity. A VGG network is a list of numbers, like [64, 64, M, 128, 128, M, ...], and that made it the default backbone for detection, segmentation and style transfer for years. Its pretrained features still define the Perceptual loss used in super-resolution and image synthesis.

The key idea: stacks of 3×3 instead of big kernels

Two stacked 3×33\times3 convolutions (stride 1, no pooling between) have a 5×55\times5 receptive field, and three have 7×77\times7. In general nn layers see (2n+1)×(2n+1)(2n+1)\times(2n+1). With CC channels in and out, the weights compare as:

n⋅32C2⏟n stacked 3×3vs(2n+1)2C2⏟one (2n+1)×(2n+1)n=3:  27C2 vs 49C2\underbrace{n\cdot 3^2C^2}_{n\text{ stacked }3\times3}\quad\text{vs}\quad\underbrace{(2n+1)^2C^2}_{\text{one }(2n+1)\times(2n+1)}\qquad n=3:\;27C^2\text{ vs }49C^2
CC
channels in and out of each layer (biases ignored)
nn
number of stacked 3×33\times3 layers; receptive field 2n+12n+1
49/2749/27
the single 7×77\times7 needs 81% more weights, and with stride 1 also 81% more multiply-adds

The stack also has nn ReLUs instead of one, so the function it computes is more non-linear and, in the paper's words, more discriminative. The paper reads the trade as a regulariser: a 7×77\times7 filter is forced to factor through 3×33\times3 pieces with non-linearities in between. The stack also weights its receptive field differently. The centre pixel reaches the output through 7⋅7=497\cdot7=49 paths and a corner through one, so the effective receptive field is concentrated in the middle.

The paper tested this directly. Network B, with pairs of 3×33\times3 layers, beat a version in which each pair was replaced by a single 5×55\times5 layer by 7% top-1 error on a centre crop. Configuration C adds 1×11\times1 convolutions to B: extra non-linearity at the same receptive field. C beat B, but D, which uses 3×33\times3 in those places, beat C. Depth helps, and so does spatial context.

VGG config D · 16 weight layers · click a layer138,357,544 params · 15.47 GMACs224²in1.1224²641.2224²64M112²642.1112²1282.2112²128M56²1283.156²2563.256²2563.356²256M28²2564.128²5124.228²5124.328²512M14²5125.114²5125.214²5125.314²512M7²512f614096f714096f811000layerconv 3×3, 64 maps, s1 p1in → out3×224×224 → 64×224×224params1,792 (0.0%)MACs86.70M (0.6%)recept. field3×3 px, stride 1conv1_1, 3×3 stride 1 pad 1: size keptpaper Table 2: 138M paramstorchvision vgg16: 138,357,54415.47 GFLOPS · top-1 71.59% · top-5 90.38%fc6 alone: 102.76M = 74.3% of paramsconvs: 10.6% of params, 99.2% of MACs3 stacked 3×3: paths per input pixelRF 7×7 · centre 49 paths, corner 1same 7×7 receptive field, 512 → 512 channels3× 3×3 · 27C² = 7,077,888 weights · 3 ReLUs1× 7×7 · 49C² = 12,845,056 weights · 1 ReLUone big kernel needs 81% more weights and MACsstack: 55.1% of the weights, +2 non-linearitiesparamsMACs■ conv ■ LRN ■ max-pool ■ dense · labels: block.layer, M = pool, f6–f8 = fc
config
comparechannels C
fig.VGG configurations A–E layer by layer (labels are block.layer, M = max-pool). Click a block for its shape, parameters, multiply-adds and receptive field. VGG-16 (D) has 138,357,544 parameters and 15.47 G multiply-adds, exactly torchvision's vgg16 numbers. fc6 alone holds 102.76M of them (74.3%), while the convs hold 10.6% of the parameters but do 99.2% of the multiply-adds. Below: n stacked 3×3 layers against one (2n+1)×(2n+1) kernel. At n = 3 and C = 512 the stack has 27C² = 7,077,888 weights and 3 ReLUs; the 7×7 has 49C² = 12,845,056 and one ReLU, 81% more. The grid shades each input pixel by how many paths connect it to the output unit: 49 at the centre, 1 at the corners. Switch to the single kernel to see its flat footprint.

The configurations

configweight layersconvs per blockparams (lib)paper / torchvisionG MACstop-5 err.
A111-1-2-2-2132,863,336133M / vgg117.6110.4%
A-LRN11A + LRN after conv1132,863,336133M / —7.6110.5%
B132-2-2-2-2133,047,848133M / vgg1311.319.9%
C16B + one 1×1 in blocks 3–5133,638,952134M / —11.779.4%
D (VGG-16)162-2-3-3-3138,357,544138M / vgg1615.478.8%
E (VGG-19)192-2-4-4-4143,667,240144M / vgg1919.639.0%
Paper Table 1–2 and torchvision, with parameters and multiply-adds recomputed by the figure's lib (224×224 input). Top-5 error: paper Table 3, single-scale test, training scale S = 256

Three things stand out. First, LRN (A-LRN) did not help, so B–E drop it. Second, adding 8 conv layers from A to E costs only 10.8M parameters, because the dense head (123.6123.6M in every configuration) dominates. Compute, by contrast, rises from 7.6 to 19.6 G multiply-adds. Third, depth saturated: with single-scale training E was no better than D. Training with scale jittering, where each image is rescaled so its short side SS is random in [256,512][256,512] before cropping, brought D to 8.1% and E to 8.0% top-5.

Worked example: VGG-16's shapes, parameters and compute

  1. 1conv1_1: 3→643\to64 channels, 3⋅9⋅64+64=1,7923\cdot9\cdot64+64=1{,}792 parameters at 224×224224\times224, so 2242⋅27⋅64=86.7224^2\cdot27\cdot64=86.7M multiply-adds. conv1_2 (64→6464\to64): 36,92836{,}928 parameters but 2242⋅576⋅64=1.85224^2\cdot576\cdot64=1.85G multiply-adds, 12% of the whole network.
  2. 2Each pool halves the side and the next block doubles the width: 224→112→56→28→14→7224\to112\to56\to28\to14\to7 with 64, 128, 256, 512, 512 channels. A 3×33\times3 layer costs H2⋅9CinCoutH^2\cdot9C_{in}C_{out}, so halving HH and doubling CC leaves it unchanged: conv1_2, conv2_2, conv3_2, conv3_3, conv4_2 and conv4_3 each cost exactly 1,849,688,0641{,}849{,}688{,}064 multiply-adds.
  3. 3After pool5: 512⋅7⋅7=25,088512\cdot7\cdot7=25{,}088 values. fc6 has 25,088⋅4096+4096=102,764,54425{,}088\cdot4096+4096=102{,}764{,}544 parameters, more than all other layers together. fc7 adds 16.816.8M and fc8 4.14.1M.
  4. 4Receptive fields grow by 2×2\times the current stride per 3×33\times3 layer: 3,53, 5 in block 1, then 10,1410, 14 in block 2, and so on up to 196196 for conv5_3 and 212212 after pool5.
  5. 5Totals: 14,714,68814{,}714{,}688 conv parameters plus 123,642,856123{,}642{,}856 dense gives 138,357,544138{,}357{,}544. Multiply-adds: 15.4715.47G, of which 15.3515.35G are in the convs.

At a glance

Paper
Simonyan & Zisserman, arXiv 1409.1556 (2014), ICLR 2015
Building block
3×33\times3 conv, stride 1, pad 1, ReLU; 2×22\times2 max-pool, stride 2
VGG-16
13 conv + 3 dense; 138,357,544 params; 15.47 G multiply-adds at 224×224
Training
SGD, batch 256, momentum 0.9, weight decay 5⋅10−45\cdot10^{-4}, dropout 0.5 in fc6/fc7; lr 10−210^{-2} divided by 10 three times; 74 epochs; 2–3 weeks per net on 4 Titan Black GPUs
Initialisation
A was trained from random init; deeper nets started from A's first four conv layers and three dense layers. The authors later noted Glorot init works without this
ILSVRC-2014
2nd in classification (7.3% top-5, 7 nets), 1st in localisation; 6.8% after the deadline with 2 nets
torchvision
vgg16: 71.592% top-1, 90.382% top-5; vgg19: 72.376% / 90.876%; vgg16_bn: 73.36% top-1

At test time the paper converts the dense layers into convolutions: fc6 becomes a 7×77\times7 conv and fc7 and fc8 become 1×11\times1 convs. The network can then run on a whole, uncropped image of any size and produce a map of class scores, which is averaged spatially. This dense evaluation, the same idea as LeNet-5's C5, avoids recomputing overlapping crops.

python
1import torch, torch.nn as nn
2from torchvision.models import vgg16, VGG16_Weights
3
4CFG = {
5 "A": [64, "M", 128, "M", 256, 256, "M", 512, 512, "M", 512, 512, "M"],
6 "D": [64, 64, "M", 128, 128, "M", 256, 256, 256, "M", 512, 512, 512, "M", 512, 512, 512, "M"],
7 "E": [64, 64, "M", 128, 128, "M", 256, 256, 256, 256, "M", 512, 512, 512, 512, "M", 512, 512, 512, 512, "M"],
8}
9
10def make_vgg(cfg, num_classes=1000):
11 layers, c = [], 3
12 for v in cfg:
13 if v == "M":
14 layers.append(nn.MaxPool2d(2, 2))
15 else:
16 layers += [nn.Conv2d(c, v, 3, padding=1), nn.ReLU(inplace=True)]
17 c = v
18 return nn.Sequential(
19 *layers, nn.Flatten(),
20 nn.Linear(512 * 7 * 7, 4096), nn.ReLU(inplace=True), nn.Dropout(0.5),
21 nn.Linear(4096, 4096), nn.ReLU(inplace=True), nn.Dropout(0.5),
22 nn.Linear(4096, num_classes),
23 )
24
25count = lambda m: sum(p.numel() for p in m.parameters())
26print(count(make_vgg(CFG["D"]))) # 138357544, same as torchvision's vgg16
27
28# three 3x3 layers vs one 7x7, C = 512
29C = 512
30stack = nn.Sequential(*[m for _ in range(3) for m in (nn.Conv2d(C, C, 3, padding=1, bias=False), nn.ReLU())])
31big = nn.Conv2d(C, C, 7, padding=3, bias=False)
32print(count(stack), count(big)) # 7077888 12845056
33
34# frozen VGG-16 features for a perceptual loss (see "Perceptual loss")
35feats = vgg16(weights=VGG16_Weights.IMAGENET1K_V1).features[:23].eval().requires_grad_(False)
36TAPS = {3: "relu1_2", 8: "relu2_2", 15: "relu3_3", 22: "relu4_3"}
37
38def vgg_taps(x): # x: ImageNet-normalised images, N x 3 x H x W
39 out = {}
40 for i, layer in enumerate(feats):
41 x = layer(x)
42 if i in TAPS:
43 out[TAPS[i]] = x
44 return out
Building any VGG configuration from its list, the 3×3-stack arithmetic, and VGG features for a perceptual loss

Use when

  • +Extremely simple and uniform: a whole network is one list of widths
  • +Stacked 3×3 convs give large receptive fields with fewer weights and more non-linearity
  • +Strong, transferable features; still the standard backbone for perceptual and style losses
  • +Fully convolutional evaluation runs on images of any size

Avoid when

  • −138–144M parameters, 74% of them in fc6: heavy to store and to fine-tune
  • −15.5–19.6 G multiply-adds per 224×224 image, much more than GoogLeNet or ResNet-50 for lower accuracy
  • −Large activation memory in the high-resolution early layers
  • −No normalisation or shortcuts: depth beyond ~19 layers does not train well

Check yourself

Three stacked 3×3 conv layers (C channels in and out, no pooling between) are compared with one 7×7 layer. Which statement is correct?

Check yourself

Roughly what fraction of VGG-16's 138M parameters sit in the first fully connected layer (fc6)?

Remember

  • ✓Only 3×33\times3 convs (stride 1, pad 1) and 2×22\times2 max-pools; widths 64→128→256→512→512; dense head 4096-4096-1000.
  • ✓nn stacked 3×33\times3 layers see (2n+1)2(2n+1)^2 pixels with 9nC29nC^2 weights instead of (2n+1)2C2(2n+1)^2C^2, and with nn ReLUs instead of one.
  • ✓Configurations A–E: 11–19 weight layers, 133–144M parameters; D = VGG-16 (138.36M, 15.47 G multiply-adds), E = VGG-19.
  • ✓fc6 holds 74% of VGG-16's parameters; the convs do 99% of the compute. Halving the side while doubling the width keeps each layer's cost constant.
  • ✓ILSVRC-2014: 2nd in classification (7.3% top-5), 1st in localisation; scale jittering and dense evaluation were part of the recipe.
  • ✓Its frozen features are the standard backbone for perceptual and style losses.
← back to Deep Learning roadmap