# SSD

> SSD predicts classes and box offsets for 8732 default boxes on six feature maps (38² down to 1²) in one pass, trained with IoU > 0.5 matching and 3:1 hard negative mining; SSD300 reached 74.3 mAP on VOC2007 at 59 FPS.

Source: https://www.mlfoundry.in/learn/ssd  
Section: Deep Learning › Deep Learning for Computer Vision  
Level: core · 7 min read · updated 2026-10-15

**SSD:** (Single Shot MultiBox Detector; Liu, Anguelov, Erhan, Szegedy, Reed, Fu and Berg, ECCV 2016) is a one-stage object detector. A convolutional network predicts, at every location of several feature maps, class scores and box offsets for a fixed set of **default boxes** (anchors) of different scales and aspect ratios. There is no region-proposal stage and no per-region feature resampling.

Input: an image resized to $300\times300$ (SSD300) or $512\times512$ (SSD512). Output: a list of boxes, each with a class and a score. **Faster R-CNN** reached high accuracy with two stages: a region proposal network, then RoI pooling and a classifier for each of about 300 proposals. That second stage made it run at about 7 FPS. **YOLO** (v1) ran at 45 FPS with one pass, but it predicted from a single coarse $7\times7$ grid and lost accuracy, especially on small objects.

SSD keeps the single pass and adds two ideas. First, it predicts from **several feature maps**: high-resolution early maps for small objects, coarse late maps for large ones. Second, it uses anchor boxes with several **aspect ratios** at every location, as the RPN does, but here each anchor directly gets the final class scores.

## Architecture: six prediction maps

The base is VGG-16 (see **VGGNet**) pre-trained on ImageNet. fc6 and fc7 become convolutions (fc6 is a dilated $3\times3$ convolution, see **Dilated convolution**), pool5 becomes $3\times3$ with stride 1, and extra convolution layers shrink the map further. Predictions come from conv4_3 ($38\times38$), fc7 ($19\times19$), conv8_2 ($10\times10$), conv9_2 ($5\times5$), conv10_2 ($3\times3$) and conv11_2 ($1\times1$). conv4_3 has a much larger feature scale than the others, so it is L2-normalised with a learned per-channel scale.

Each map with $k$ default boxes per location gets one $3\times3$ convolution with $k(C+4)$ output channels: $C$ class scores (including background) and 4 offsets per box. For PASCAL VOC, $C=21$. With $k=4$ on conv4_3, conv10_2 and conv11_2 and $k=6$ elsewhere, the total is $38^2\cdot4+19^2\cdot6+10^2\cdot6+5^2\cdot6+3^2\cdot4+1\cdot4=8732$ boxes.

> Figure: SSD300. The VGG-16 trunk and the extra layers form a pyramid of six maps; each map feeds a 3×3 convolutional head that outputs, for each of its k default boxes per cell, 21 VOC scores and 4 offsets. All 8732 predictions are concatenated, low scores are dropped, and per-class NMS (IoU 0.45) keeps up to 200 detections. Inset: the six default boxes of one cell (two squares, then 2:1, 1:2, 3:1, 1:3).

## Default boxes

Map $k$ of $m$ gets a box scale $s_k$ (as a fraction of the image side) that grows linearly. Each location gets boxes with aspect ratios $a_r\in\{1,2,3,\tfrac12,\tfrac13\}$, plus a second square of scale $s_k'=\sqrt{s_k s_{k+1}}$. Maps with $k=4$ omit $3$ and $\tfrac13$. Box centres sit at the cell centres.

$$
s_k=s_{\min}+\frac{s_{\max}-s_{\min}}{m-1}(k-1),\qquad w_k^{a}=s_k\sqrt{a_r},\quad h_k^{a}=\frac{s_k}{\sqrt{a_r}}
$$

- $s_{\min},\,s_{\max}$: 0.2 and 0.9 in the paper, so $s_k = 0.2, 0.34, \dots, 0.9$ for $m=6$
- $a_r$: aspect ratio $w/h$; the box area $s_k^2$ is the same for every ratio
- $s_k'$: extra square box $\sqrt{s_k s_{k+1}}$ for $a_r=1$

The released VOC model deviates slightly: conv4_3 gets scale 0.1, so the box sizes are 30, 60, 111, 162, 213 and 264 px, with extra squares of 42, 82, 134, 186, 237 and 288 px. The figure below uses these sizes and clips boxes to the image.

## Matching and the loss

Each ground-truth box is matched to the default box with the highest IoU. Every other default box with IoU above 0.5 to some ground truth is also positive. The rest are negatives (background). An object therefore trains several neighbouring boxes, often on two adjacent maps.

$$
L=\frac{1}{N}\Big(L_{\text{conf}}+\alpha\,L_{\text{loc}}\Big),\qquad L_{\text{loc}}=\sum_{i\in\text{Pos}}\ \sum_{m\in\{cx,cy,w,h\}}\operatorname{smooth}_{L1}\big(l_i^m-\hat g_i^m\big)
$$

- $N$: number of positive default boxes; the loss is 0 if $N=0$
- $L_{\text{conf}}$: softmax cross-entropy over $C$ classes, summed over positives and the mined negatives
- $\alpha$: localisation weight, 1 in the paper
- $l_i$: predicted offsets of box $i$

$$
\hat g^{cx}=\frac{g^{cx}-d^{cx}}{d^{w}},\quad \hat g^{cy}=\frac{g^{cy}-d^{cy}}{d^{h}},\quad \hat g^{w}=\log\frac{g^{w}}{d^{w}},\quad \hat g^{h}=\log\frac{g^{h}}{d^{h}}
$$

- $d$: default box (centre, width, height)
- $g$: matched ground-truth box; the same parameterisation as **Faster R-CNN**

After matching, almost all of the 8732 boxes are negatives. SSD uses **hard negative mining**: it sorts the negatives by their confidence loss and keeps only the highest, so that negatives:positives is at most 3:1. The paper reports that this gives faster optimisation and more stable training.

**Configuration and results (Liu et al., 2016)**

- **Backbone:** VGG-16 pre-trained on ImageNet, fc6/fc7 converted to convolutions, plus extra layers conv8–conv11
- **Inputs and boxes:** SSD300: $300\times300$, 8732 default boxes. SSD512 adds a seventh map and uses 24,564 boxes.
- **Augmentation:** random crops whose IoU with the objects is at least 0.1, 0.3, 0.5, 0.7 or 0.9, plus flips and photometric distortion; the paper credits the crop sampling with +8.8 mAP
- **VOC2007 test (07+12 train):** SSD300 74.3 mAP, SSD512 76.8 mAP; Faster R-CNN (VGG-16) 73.2
- **Speed (Titan X):** SSD300 59 FPS; Faster R-CNN 7 FPS; YOLO v1 45 FPS at 63.4 mAP
- **COCO test-dev:** AP 23.2 (SSD300) and 26.8 (SSD512)

> Figure: One ground-truth box at the image centre against all 8732 default boxes. Default (100×100 px, 1:1, IoU 0.5): 16 positives, 8 on fc7 and 8 on conv8_2; the best is a 111 px conv8_2 box with IoU 0.797. Hard negative mining keeps 48 of 8716 negatives, so 99.27% of the boxes give no classification loss. At 30 px all 7 positives come from conv4_3 (best IoU 0.772). At 16 px no box clears 0.5: the best-match rule keeps one conv4_3 box with IoU 0.284. At 240 px the 13 positives sit on conv9_2, conv10_2 and conv11_2 (best IoU 0.976).

## Worked example: one 100 px object

Ground truth $g=(100,100,200,200)$ in a $300\times300$ image. For each map, take its default box with the highest IoU:

Best default box per map for g = (100, 100, 200, 200); boxes clipped to the image

| Map | Best default box (x₁, y₁, x₂, y₂) | IoU | Label |
| --- | --- | --- | --- |
| conv4_3 | (102.8, 102.8, 145.2, 145.2), 42 px square | 0.180 | negative |
| fc7 | (111.2, 111.2, 192.8, 192.8), inside g: 81.6² / 100² = 0.666 | 0.666 | positive |
| conv8_2 | (88.5, 88.5, 199.5, 199.5): 99.5² / (111² + 100² − 99.5²) | 0.797 | positive (best) |
| conv9_2 | (83.7, 113.2, 300.0, 206.8), 2:1 box | 0.403 | negative |

The best box has centre $(144,144)$ and side 111, so its targets are $\hat g^{cx}=\hat g^{cy}=(150-144)/111=0.054$ and $\hat g^{w}=\hat g^{h}=\log(100/111)=-0.104$. In total 16 boxes exceed IoU 0.5, so $N=16$ and the classification loss uses the 48 hardest of the 8716 negatives.

```python
import numpy as np

fmap  = [38, 19, 10, 5, 3, 1]
step  = [8, 16, 32, 64, 100, 300]
smin  = [30, 60, 111, 162, 213, 264]
smax  = [60, 111, 162, 213, 264, 315]
ratio = [[2], [2, 3], [2, 3], [2, 3], [2], [2]]

boxes = []
for f, st, s, S, ars in zip(fmap, step, smin, smax, ratio):
    wh = [(s, s), (np.sqrt(s * S),) * 2]
    wh += [(s * np.sqrt(a), s / np.sqrt(a)) for a in ars] + [(s / np.sqrt(a), s * np.sqrt(a)) for a in ars]
    for i in range(f):
        for j in range(f):
            cx, cy = (j + .5) * st, (i + .5) * st
            boxes += [(cx - w / 2, cy - h / 2, cx + w / 2, cy + h / 2) for w, h in wh]
D = np.clip(np.array(boxes), 0, 300)
print(len(D))                                            # 8732

def iou(D, g):
    lt, rb = np.maximum(D[:, :2], g[:2]), np.minimum(D[:, 2:], g[2:])
    inter = np.prod(np.clip(rb - lt, 0, None), axis=1)
    area = lambda b: (b[..., 2] - b[..., 0]) * (b[..., 3] - b[..., 1])
    return inter / (area(D) + area(g) - inter)

g = np.array([100., 100., 200., 200.])
o = iou(D, g)
pos = o > 0.5
pos[o.argmax()] = True                                   # every gt keeps its best box
n_pos = pos.sum()
print(n_pos, o.max().round(3), min(3 * n_pos, (~pos).sum()))   # 16 0.797 48
```

SSD300 default boxes and matching in numpy (VOC sizes; runs as is)

> **Small objects:** The finest map, conv4_3, has stride 8 and its smallest box is 30 px. A 16 px object reaches IoU 0.284 at best, so it gets one positive with a poor target, and conv4_3 features are shallow. SSD300 is weakest on small objects. SSD512, the "zoom-out" (expansion) augmentation added in the final version of the paper, and later feature pyramids (**RetinaNet**) all address this.

> **Same anchors, different job:** The default boxes are the anchors of the region proposal network in **Faster R-CNN**, but there they only score "object vs not" and propose regions for a second stage. In SSD they carry the final class scores and offsets. Removing the second stage gives the speed; the multi-scale maps and the aspect ratios recover most of the accuracy.

**Strengths**

- One forward pass with only convolutions: 59 FPS for SSD300, versus 7 FPS for Faster R-CNN with the same VGG-16 backbone
- Multi-scale maps and several aspect ratios per location, so object sizes are covered without an image pyramid
- Simple to deploy; SSDLite with a MobileNet backbone became a standard mobile detector

**Limitations**

- Weak on small objects: the finest map is stride 8 with shallow features
- Hand-designed scales and aspect ratios must suit the dataset
- Hard negative mining keeps only the top 3:1 negatives; **RetinaNet**'s focal loss trains on all of them instead

**Question:** An SSD300 image contains one object. After matching, 12 default boxes are positive. How many default boxes contribute to the classification loss?

- A. 12
- B. 36
- C. 48
- D. 8732

**Answer:** C. 48 — The 12 positives plus the $3\times12=36$ negatives with the highest confidence loss, 48 in total. The other 8684 boxes give no gradient in that iteration.

**Question:** Why does SSD attach predictors to conv4_3 as well as to the deep 1×1–10×10 maps?

- A. conv4_3 has more channels than the later maps
- B. Its 38×38 grid (stride 8) and small 30 px default boxes are needed to match small objects
- C. It is the only layer that was pre-trained on ImageNet
- D. Its features are already normalised, so it needs no head

**Answer:** B. Its 38×38 grid (stride 8) and small 30 px default boxes are needed to match small objects — Each map covers one range of box scales. Only the high-resolution conv4_3 map has default boxes small enough, and a dense enough grid, for objects of about 30 px. Coarse maps such as conv11_2 ($1\times1$) cover objects that fill most of the image.

**Key points**

- One-stage detector: a $3\times3$ conv head on six maps ($38^2,19^2,10^2,5^2,3^2,1^2$) predicts $C$ scores and 4 offsets for $k=4$ or 6 default boxes per cell, 8732 in total for SSD300.
- Default-box scales grow linearly from 0.2 to 0.9 of the image; aspect ratios $\{1,2,3,\tfrac12,\tfrac13\}$ plus an extra square $\sqrt{s_k s_{k+1}}$.
- Positives: each ground truth's best box plus every box with IoU > 0.5; offsets use the Faster R-CNN (cx, cy, log w, log h) encoding.
- Hard negative mining keeps negatives at most 3:1 to positives; the loss is (confidence + localisation) / N.
- SSD300: 74.3 mAP on VOC2007 at 59 FPS; weak on small objects, which later feature-pyramid detectors improve.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
