# VGGNet

> Simonyan & Zisserman (2014) built 11–19-layer nets from nothing but 3×3 convs and 2×2 pools. Three stacked 3×3 layers see 7×7 with 45% fewer weights than one 7×7 and three ReLUs. VGG-16: 138M params (74% in fc6), 15.5 GFLOPs.

Source: https://www.mlfoundry.in/learn/vggnet  
Section: Deep Learning › Convolutional Neural Networks  
Level: core · 12 min read · updated 2026-10-11

**VGGNet:** is the family of networks from Simonyan and Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition" (arXiv 2014, ICLR 2015), by Oxford's Visual Geometry Group. Every convolution is $3\times3$ with stride 1 and padding 1, so it keeps the spatial size. Five $2\times2$ stride-2 max-pools halve it, the width doubles after each pool from 64 to 512, and three dense layers (4096, 4096, 1000) finish the network. Configurations **A** to **E** differ only in depth, from 11 to 19 weight layers. **D** and **E** are the famous **VGG-16** and **VGG-19**.

By 2014 the question after **AlexNet** and **ZFNet** was what actually matters in a CNN. The VGG paper isolated one variable, **depth**. It fixed every other design choice to the simplest option (smallest useful kernel, stride 1, same padding, one pooling type) and added layers. The result was second place in the ILSVRC-2014 classification task, with 7.3% top-5 test error from an ensemble of 7 nets, behind **GoogLeNet (Inception)**. It was first in localisation, with 25.3% error. After the competition a 2-net ensemble of D and E reached 6.8%, and the best single net 7.0%.

VGG's real legacy is its uniformity. A VGG network is a list of numbers, like `[64, 64, M, 128, 128, M, ...]`, and that made it the default backbone for detection, segmentation and style transfer for years. Its pretrained features still define the **Perceptual loss** used in super-resolution and image synthesis.

## The key idea: stacks of 3×3 instead of big kernels

Two stacked $3\times3$ convolutions (stride 1, no pooling between) have a $5\times5$ receptive field, and three have $7\times7$. In general $n$ layers see $(2n+1)\times(2n+1)$. With $C$ channels in and out, the weights compare as:

$$
\underbrace{n\cdot 3^2C^2}_{n\text{ stacked }3\times3}\quad\text{vs}\quad\underbrace{(2n+1)^2C^2}_{\text{one }(2n+1)\times(2n+1)}\qquad n=3:\;27C^2\text{ vs }49C^2
$$

- $C$: channels in and out of each layer (biases ignored)
- $n$: number of stacked $3\times3$ layers; receptive field $2n+1$
- $49/27$: the single $7\times7$ needs 81% more weights, and with stride 1 also 81% more multiply-adds

The stack also has $n$ ReLUs instead of one, so the function it computes is more non-linear and, in the paper's words, more discriminative. The paper reads the trade as a regulariser: a $7\times7$ filter is forced to factor through $3\times3$ pieces with non-linearities in between. The stack also weights its receptive field differently. The centre pixel reaches the output through $7\cdot7=49$ paths and a corner through one, so the effective receptive field is concentrated in the middle.

The paper tested this directly. Network B, with pairs of $3\times3$ layers, beat a version in which each pair was replaced by a single $5\times5$ layer by 7% top-1 error on a centre crop. Configuration C adds $1\times1$ convolutions to B: extra non-linearity at the same receptive field. C beat B, but D, which uses $3\times3$ in those places, beat C. Depth helps, and so does spatial context.

> Figure: VGG configurations A–E layer by layer (labels are block.layer, M = max-pool). Click a block for its shape, parameters, multiply-adds and receptive field. VGG-16 (D) has 138,357,544 parameters and 15.47 G multiply-adds, exactly torchvision's vgg16 numbers. fc6 alone holds 102.76M of them (74.3%), while the convs hold 10.6% of the parameters but do 99.2% of the multiply-adds. Below: n stacked 3×3 layers against one (2n+1)×(2n+1) kernel. At n = 3 and C = 512 the stack has 27C² = 7,077,888 weights and 3 ReLUs; the 7×7 has 49C² = 12,845,056 and one ReLU, 81% more. The grid shades each input pixel by how many paths connect it to the output unit: 49 at the centre, 1 at the corners. Switch to the single kernel to see its flat footprint.

## The configurations

Paper Table 1–2 and torchvision, with parameters and multiply-adds recomputed by the figure's lib (224×224 input). Top-5 error: paper Table 3, single-scale test, training scale S = 256

| config | weight layers | convs per block | params (lib) | paper / torchvision | G MACs | top-5 err. |
| --- | --- | --- | --- | --- | --- | --- |
| A | 11 | 1-1-2-2-2 | 132,863,336 | 133M / vgg11 | 7.61 | 10.4% |
| A-LRN | 11 | A + LRN after conv1 | 132,863,336 | 133M / — | 7.61 | 10.5% |
| B | 13 | 2-2-2-2-2 | 133,047,848 | 133M / vgg13 | 11.31 | 9.9% |
| C | 16 | B + one 1×1 in blocks 3–5 | 133,638,952 | 134M / — | 11.77 | 9.4% |
| D (VGG-16) | 16 | 2-2-3-3-3 | 138,357,544 | 138M / vgg16 | 15.47 | 8.8% |
| E (VGG-19) | 19 | 2-2-4-4-4 | 143,667,240 | 144M / vgg19 | 19.63 | 9.0% |

Three things stand out. First, LRN (A-LRN) did not help, so B–E drop it. Second, adding 8 conv layers from A to E costs only 10.8M parameters, because the dense head ($123.6$M in every configuration) dominates. Compute, by contrast, rises from 7.6 to 19.6 G multiply-adds. Third, depth saturated: with single-scale training E was no better than D. Training with **scale jittering**, where each image is rescaled so its short side $S$ is random in $[256,512]$ before cropping, brought D to 8.1% and E to 8.0% top-5.

**Worked example: VGG-16's shapes, parameters and compute**

1. conv1_1: $3\to64$ channels, $3\cdot9\cdot64+64=1{,}792$ parameters at $224\times224$, so $224^2\cdot27\cdot64=86.7$M multiply-adds. conv1_2 ($64\to64$): $36{,}928$ parameters but $224^2\cdot576\cdot64=1.85$G multiply-adds, 12% of the whole network.
2. Each pool halves the side and the next block doubles the width: $224\to112\to56\to28\to14\to7$ with 64, 128, 256, 512, 512 channels. A $3\times3$ layer costs $H^2\cdot9C_{in}C_{out}$, so halving $H$ and doubling $C$ leaves it unchanged: conv1_2, conv2_2, conv3_2, conv3_3, conv4_2 and conv4_3 each cost exactly $1{,}849{,}688{,}064$ multiply-adds.
3. After pool5: $512\cdot7\cdot7=25{,}088$ values. fc6 has $25{,}088\cdot4096+4096=102{,}764{,}544$ parameters, more than all other layers together. fc7 adds $16.8$M and fc8 $4.1$M.
4. Receptive fields grow by $2\times$ the current stride per $3\times3$ layer: $3, 5$ in block 1, then $10, 14$ in block 2, and so on up to $196$ for conv5_3 and $212$ after pool5.
5. Totals: $14{,}714{,}688$ conv parameters plus $123{,}642{,}856$ dense gives $138{,}357{,}544$. Multiply-adds: $15.47$G, of which $15.35$G are in the convs.

**At a glance**

- **Paper:** Simonyan & Zisserman, arXiv 1409.1556 (2014), ICLR 2015
- **Building block:** $3\times3$ conv, stride 1, pad 1, ReLU; $2\times2$ max-pool, stride 2
- **VGG-16:** 13 conv + 3 dense; 138,357,544 params; 15.47 G multiply-adds at 224×224
- **Training:** SGD, batch 256, momentum 0.9, weight decay $5\cdot10^{-4}$, dropout 0.5 in fc6/fc7; lr $10^{-2}$ divided by 10 three times; 74 epochs; 2–3 weeks per net on 4 Titan Black GPUs
- **Initialisation:** A was trained from random init; deeper nets started from A's first four conv layers and three dense layers. The authors later noted Glorot init works without this
- **ILSVRC-2014:** 2nd in classification (7.3% top-5, 7 nets), 1st in localisation; 6.8% after the deadline with 2 nets
- **torchvision:** `vgg16`: 71.592% top-1, 90.382% top-5; `vgg19`: 72.376% / 90.876%; `vgg16_bn`: 73.36% top-1

At test time the paper converts the dense layers into convolutions: fc6 becomes a $7\times7$ conv and fc7 and fc8 become $1\times1$ convs. The network can then run on a whole, uncropped image of any size and produce a map of class scores, which is averaged spatially. This **dense evaluation**, the same idea as **LeNet-5**'s C5, avoids recomputing overlapping crops.

```python
import torch, torch.nn as nn
from torchvision.models import vgg16, VGG16_Weights

CFG = {
    "A": [64, "M", 128, "M", 256, 256, "M", 512, 512, "M", 512, 512, "M"],
    "D": [64, 64, "M", 128, 128, "M", 256, 256, 256, "M", 512, 512, 512, "M", 512, 512, 512, "M"],
    "E": [64, 64, "M", 128, 128, "M", 256, 256, 256, 256, "M", 512, 512, 512, 512, "M", 512, 512, 512, 512, "M"],
}

def make_vgg(cfg, num_classes=1000):
    layers, c = [], 3
    for v in cfg:
        if v == "M":
            layers.append(nn.MaxPool2d(2, 2))
        else:
            layers += [nn.Conv2d(c, v, 3, padding=1), nn.ReLU(inplace=True)]
            c = v
    return nn.Sequential(
        *layers, nn.Flatten(),
        nn.Linear(512 * 7 * 7, 4096), nn.ReLU(inplace=True), nn.Dropout(0.5),
        nn.Linear(4096, 4096), nn.ReLU(inplace=True), nn.Dropout(0.5),
        nn.Linear(4096, num_classes),
    )

count = lambda m: sum(p.numel() for p in m.parameters())
print(count(make_vgg(CFG["D"])))       # 138357544, same as torchvision's vgg16

# three 3x3 layers vs one 7x7, C = 512
C = 512
stack = nn.Sequential(*[m for _ in range(3) for m in (nn.Conv2d(C, C, 3, padding=1, bias=False), nn.ReLU())])
big = nn.Conv2d(C, C, 7, padding=3, bias=False)
print(count(stack), count(big))        # 7077888 12845056

# frozen VGG-16 features for a perceptual loss (see "Perceptual loss")
feats = vgg16(weights=VGG16_Weights.IMAGENET1K_V1).features[:23].eval().requires_grad_(False)
TAPS = {3: "relu1_2", 8: "relu2_2", 15: "relu3_3", 22: "relu4_3"}

def vgg_taps(x):                       # x: ImageNet-normalised images, N x 3 x H x W
    out = {}
    for i, layer in enumerate(feats):
        x = layer(x)
        if i in TAPS:
            out[TAPS[i]] = x
    return out
```

Building any VGG configuration from its list, the 3×3-stack arithmetic, and VGG features for a perceptual loss

> **Small parameters, big activations:** VGG-16's 138M parameters (553 MB in fp32) are mostly in fc6. Its training memory, though, is dominated by activations in the first two blocks: conv1_1 and conv1_2 each output $64\times224\times224=3.2$M values per image. Summed over all layers, one forward pass stores about 15M activations per image (≈ 60 MB in fp32, before the backward pass's extra buffers). Large batches of VGG at full resolution run out of GPU memory long before parameter storage matters.

> **Why VGG features make good perceptual losses:** VGG's plain stack of $3\times3$ convs with no shortcuts builds features whose layers map cleanly onto scales, from edges at relu1_2 to object parts at relu4_3. That is why Gatys et al.'s style transfer and Johnson et al.'s **Perceptual loss** compare images in VGG-16 or VGG-19 feature space. Backbones with batch norm or residual paths have been tried as replacements, but VGG remains the usual choice.

> **What came next:** Training 16–19 plain layers already needed a careful start (initialising from net A). Going much deeper failed outright: plain networks got **worse** training error with more layers. **Batch normalization** (2015) and the shortcuts of **Residual neural network (ResNet)** (2015) fixed that, and ResNet-152 is about 8 times deeper than VGG-19 yet needs fewer multiply-adds (11.3G vs 19.6G). The $3\times3$-only rule survived in almost every later CNN.

**Strengths**

- Extremely simple and uniform: a whole network is one list of widths
- Stacked 3×3 convs give large receptive fields with fewer weights and more non-linearity
- Strong, transferable features; still the standard backbone for perceptual and style losses
- Fully convolutional evaluation runs on images of any size

**Limitations**

- 138–144M parameters, 74% of them in fc6: heavy to store and to fine-tune
- 15.5–19.6 G multiply-adds per 224×224 image, much more than GoogLeNet or ResNet-50 for lower accuracy
- Large activation memory in the high-resolution early layers
- No normalisation or shortcuts: depth beyond ~19 layers does not train well

**Question:** Three stacked 3×3 conv layers (C channels in and out, no pooling between) are compared with one 7×7 layer. Which statement is correct?

- A. The 7×7 layer has a larger receptive field
- B. The stack has 27C² weights vs 49C², and 3 non-linearities vs 1
- C. Both have the same number of weights
- D. The stack has fewer non-linearities

**Answer:** B. The stack has 27C² weights vs 49C², and 3 non-linearities vs 1 — Each $3\times3$ layer adds 2 to the receptive field, so three give $7\times7$, the same as the single layer. Weights: $3\cdot9C^2=27C^2$ against $49C^2$, so the big kernel needs 81% more. And the stack has a ReLU after each layer.

**Question:** Roughly what fraction of VGG-16's 138M parameters sit in the first fully connected layer (fc6)?

- A. About 10%
- B. About 25%
- C. About 50%
- D. About 75%

**Answer:** D. About 75% — fc6 maps $512\cdot7\cdot7=25{,}088$ inputs to 4096 outputs: $102.8$M parameters, 74.3% of the total. All 13 conv layers together have only 14.7M, yet they perform 99.2% of the multiply-adds.

**Key points**

- Only $3\times3$ convs (stride 1, pad 1) and $2\times2$ max-pools; widths 64→128→256→512→512; dense head 4096-4096-1000.
- $n$ stacked $3\times3$ layers see $(2n+1)^2$ pixels with $9nC^2$ weights instead of $(2n+1)^2C^2$, and with $n$ ReLUs instead of one.
- Configurations A–E: 11–19 weight layers, 133–144M parameters; D = VGG-16 (138.36M, 15.47 G multiply-adds), E = VGG-19.
- fc6 holds 74% of VGG-16's parameters; the convs do 99% of the compute. Halving the side while doubling the width keeps each layer's cost constant.
- ILSVRC-2014: 2nd in classification (7.3% top-5), 1st in localisation; scale jittering and dense evaluation were part of the recipe.
- Its frozen features are the standard backbone for perceptual and style losses.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
