SSD

SSD predicts classes and box offsets for 8732 default boxes on six feature maps (38² down to 1²) in one pass, trained with IoU > 0.5 matching and 3:1 hard negative mining; SSD300 reached 74.3 mAP on VOC2007 at 59 FPS.

7 min read789 wordscore0 viewsupdated 2026-10-15

Definition

SSD (Single Shot MultiBox Detector; Liu, Anguelov, Erhan, Szegedy, Reed, Fu and Berg, ECCV 2016) is a one-stage object detector. A convolutional network predicts, at every location of several feature maps, class scores and box offsets for a fixed set of default boxes (anchors) of different scales and aspect ratios. There is no region-proposal stage and no per-region feature resampling.

Input: an image resized to 300×300300\times300 (SSD300) or 512×512512\times512 (SSD512). Output: a list of boxes, each with a class and a score. Faster R-CNN reached high accuracy with two stages: a region proposal network, then RoI pooling and a classifier for each of about 300 proposals. That second stage made it run at about 7 FPS. YOLO (v1) ran at 45 FPS with one pass, but it predicted from a single coarse 7×77\times7 grid and lost accuracy, especially on small objects.

SSD keeps the single pass and adds two ideas. First, it predicts from several feature maps: high-resolution early maps for small objects, coarse late maps for large ones. Second, it uses anchor boxes with several aspect ratios at every location, as the RPN does, but here each anchor directly gets the final class scores.

Architecture: six prediction maps

The base is VGG-16 (see VGGNet) pre-trained on ImageNet. fc6 and fc7 become convolutions (fc6 is a dilated 3×33\times3 convolution, see Dilated convolution), pool5 becomes 3×33\times3 with stride 1, and extra convolution layers shrink the map further. Predictions come from conv4_3 (38×3838\times38), fc7 (19×1919\times19), conv8_2 (10×1010\times10), conv9_2 (5×55\times5), conv10_2 (3×33\times3) and conv11_2 (1×11\times1). conv4_3 has a much larger feature scale than the others, so it is L2-normalised with a learned per-channel scale.

Each map with kk default boxes per location gets one 3×33\times3 convolution with k(C+4)k(C+4) output channels: CC class scores (including background) and 4 offsets per box. For PASCAL VOC, C=21C=21. With k=4k=4 on conv4_3, conv10_2 and conv11_2 and k=6k=6 elsewhere, the total is 382⋅4+192⋅6+102⋅6+52⋅6+32⋅4+1⋅4=873238^2\cdot4+19^2\cdot6+10^2\cdot6+5^2\cdot6+3^2\cdot4+1\cdot4=8732 boxes.

SSD300: one forward pass, predictions from six feature maps of decreasing resolutioncatdog300×300×3VGG-16to conv5_3fc6, fc7as convsconv4_338×38×512k=45776fc719×19×1024k=62166conv8_210×10×512k=6600conv9_25×5×256k=6150conv10_23×3×256k=436conv11_21×1×256k=443×3 convk(21+4) ch8732 default boxes × (21 class scores + 4 offsets)5776 + 2166 + 600 + 150 + 36 + 4 · VOC: 20 classes + backgrounddrop scores < 0.01 · per-class NMS, IoU 0.45keep top 200 detections per imagedefault boxes ofone cell (k = 6)1:1 s · √(s·s′) dashed2:1, 1:2 · 3:1, 1:3
fig.SSD300. The VGG-16 trunk and the extra layers form a pyramid of six maps; each map feeds a 3×3 convolutional head that outputs, for each of its k default boxes per cell, 21 VOC scores and 4 offsets. All 8732 predictions are concatenated, low scores are dropped, and per-class NMS (IoU 0.45) keeps up to 200 detections. Inset: the six default boxes of one cell (two squares, then 2:1, 1:2, 3:1, 1:3).

Default boxes

Map kk of mm gets a box scale sks_k (as a fraction of the image side) that grows linearly. Each location gets boxes with aspect ratios ar∈{1,2,3,12,13}a_r\in\{1,2,3,\tfrac12,\tfrac13\}, plus a second square of scale sk′=sksk+1s_k'=\sqrt{s_k s_{k+1}}. Maps with k=4k=4 omit 33 and 13\tfrac13. Box centres sit at the cell centres.

sk=smin⁡+smax⁡−smin⁡m−1(k−1),wka=skar,hka=skars_k=s_{\min}+\frac{s_{\max}-s_{\min}}{m-1}(k-1),\qquad w_k^{a}=s_k\sqrt{a_r},\quad h_k^{a}=\frac{s_k}{\sqrt{a_r}}
smin⁡, smax⁡s_{\min},\,s_{\max}
0.2 and 0.9 in the paper, so sk=0.2,0.34,…,0.9s_k = 0.2, 0.34, \dots, 0.9 for m=6m=6
ara_r
aspect ratio w/hw/h; the box area sk2s_k^2 is the same for every ratio
sk′s_k'
extra square box sksk+1\sqrt{s_k s_{k+1}} for ar=1a_r=1

The released VOC model deviates slightly: conv4_3 gets scale 0.1, so the box sizes are 30, 60, 111, 162, 213 and 264 px, with extra squares of 42, 82, 134, 186, 237 and 288 px. The figure below uses these sizes and clips boxes to the image.

Matching and the loss

Each ground-truth box is matched to the default box with the highest IoU. Every other default box with IoU above 0.5 to some ground truth is also positive. The rest are negatives (background). An object therefore trains several neighbouring boxes, often on two adjacent maps.

L=1N(Lconf+α Lloc),Lloc=∑i∈Pos ∑m∈{cx,cy,w,h}smooth⁡L1(lim−g^im)L=\frac{1}{N}\Big(L_{\text{conf}}+\alpha\,L_{\text{loc}}\Big),\qquad L_{\text{loc}}=\sum_{i\in\text{Pos}}\ \sum_{m\in\{cx,cy,w,h\}}\operatorname{smooth}_{L1}\big(l_i^m-\hat g_i^m\big)
NN
number of positive default boxes; the loss is 0 if N=0N=0
LconfL_{\text{conf}}
softmax cross-entropy over CC classes, summed over positives and the mined negatives
α\alpha
localisation weight, 1 in the paper
lil_i
predicted offsets of box ii
g^cx=gcx−dcxdw,g^cy=gcy−dcydh,g^w=log⁡gwdw,g^h=log⁡ghdh\hat g^{cx}=\frac{g^{cx}-d^{cx}}{d^{w}},\quad \hat g^{cy}=\frac{g^{cy}-d^{cy}}{d^{h}},\quad \hat g^{w}=\log\frac{g^{w}}{d^{w}},\quad \hat g^{h}=\log\frac{g^{h}}{d^{h}}
dd
default box (centre, width, height)
gg
matched ground-truth box; the same parameterisation as Faster R-CNN

After matching, almost all of the 8732 boxes are negatives. SSD uses hard negative mining: it sorts the negatives by their confidence loss and keeps only the highest, so that negatives:positives is at most 3:1. The paper reports that this gives faster optimisation and more stable training.

Configuration and results (Liu et al., 2016)

Backbone
VGG-16 pre-trained on ImageNet, fc6/fc7 converted to convolutions, plus extra layers conv8–conv11
Inputs and boxes
SSD300: 300×300300\times300, 8732 default boxes. SSD512 adds a seventh map and uses 24,564 boxes.
Augmentation
random crops whose IoU with the objects is at least 0.1, 0.3, 0.5, 0.7 or 0.9, plus flips and photometric distortion; the paper credits the crop sampling with +8.8 mAP
VOC2007 test (07+12 train)
SSD300 74.3 mAP, SSD512 76.8 mAP; Faster R-CNN (VGG-16) 73.2
Speed (Titan X)
SSD300 59 FPS; Faster R-CNN 7 FPS; YOLO v1 45 FPS at 63.4 mAP
COCO test-dev
AP 23.2 (SSD300) and 26.8 (SSD512)
SSD300 matching: 8732 default boxes vs one ground truthdashed = gt 100×100 px · grid: conv8_2, stride 32thick box = best match · thin = IoU > 0.50layermapboxess(px)best IoUposconv4_338²5776300fc719²2166608conv8_210²6001118conv9_25²1501620conv10_23²362130conv11_21²42640bar = best IoU per layer · tick = thresholdpositives N = 16 (2 layers)best: conv8_2, IoU 0.797each gt also keeps its best boxhard negative mining, neg:pos ≤ 3:116 pos + 48 hardest neg of 8716 neg99.27% of boxes give no classification lossregression target for the best boxΔcx +0.054 Δcy +0.054log(w/w_d) -0.104 log(h/h_d) -0.104
fig.One ground-truth box at the image centre against all 8732 default boxes. Default (100×100 px, 1:1, IoU 0.5): 16 positives, 8 on fc7 and 8 on conv8_2; the best is a 111 px conv8_2 box with IoU 0.797. Hard negative mining keeps 48 of 8716 negatives, so 99.27% of the boxes give no classification loss. At 30 px all 7 positives come from conv4_3 (best IoU 0.772). At 16 px no box clears 0.5: the best-match rule keeps one conv4_3 box with IoU 0.284. At 240 px the 13 positives sit on conv9_2, conv10_2 and conv11_2 (best IoU 0.976).

Worked example: one 100 px object

Ground truth g=(100,100,200,200)g=(100,100,200,200) in a 300×300300\times300 image. For each map, take its default box with the highest IoU:

MapBest default box (x₁, y₁, x₂, y₂)IoULabel
conv4_3(102.8, 102.8, 145.2, 145.2), 42 px square0.180negative
fc7(111.2, 111.2, 192.8, 192.8), inside g: 81.6² / 100² = 0.6660.666positive
conv8_2(88.5, 88.5, 199.5, 199.5): 99.5² / (111² + 100² − 99.5²)0.797positive (best)
conv9_2(83.7, 113.2, 300.0, 206.8), 2:1 box0.403negative
Best default box per map for g = (100, 100, 200, 200); boxes clipped to the image

The best box has centre (144,144)(144,144) and side 111, so its targets are g^cx=g^cy=(150−144)/111=0.054\hat g^{cx}=\hat g^{cy}=(150-144)/111=0.054 and g^w=g^h=log⁡(100/111)=−0.104\hat g^{w}=\hat g^{h}=\log(100/111)=-0.104. In total 16 boxes exceed IoU 0.5, so N=16N=16 and the classification loss uses the 48 hardest of the 8716 negatives.

python
1import numpy as np
2
3fmap = [38, 19, 10, 5, 3, 1]
4step = [8, 16, 32, 64, 100, 300]
5smin = [30, 60, 111, 162, 213, 264]
6smax = [60, 111, 162, 213, 264, 315]
7ratio = [[2], [2, 3], [2, 3], [2, 3], [2], [2]]
8
9boxes = []
10for f, st, s, S, ars in zip(fmap, step, smin, smax, ratio):
11 wh = [(s, s), (np.sqrt(s * S),) * 2]
12 wh += [(s * np.sqrt(a), s / np.sqrt(a)) for a in ars] + [(s / np.sqrt(a), s * np.sqrt(a)) for a in ars]
13 for i in range(f):
14 for j in range(f):
15 cx, cy = (j + .5) * st, (i + .5) * st
16 boxes += [(cx - w / 2, cy - h / 2, cx + w / 2, cy + h / 2) for w, h in wh]
17D = np.clip(np.array(boxes), 0, 300)
18print(len(D)) # 8732
19
20def iou(D, g):
21 lt, rb = np.maximum(D[:, :2], g[:2]), np.minimum(D[:, 2:], g[2:])
22 inter = np.prod(np.clip(rb - lt, 0, None), axis=1)
23 area = lambda b: (b[..., 2] - b[..., 0]) * (b[..., 3] - b[..., 1])
24 return inter / (area(D) + area(g) - inter)
25
26g = np.array([100., 100., 200., 200.])
27o = iou(D, g)
28pos = o > 0.5
29pos[o.argmax()] = True # every gt keeps its best box
30n_pos = pos.sum()
31print(n_pos, o.max().round(3), min(3 * n_pos, (~pos).sum())) # 16 0.797 48
SSD300 default boxes and matching in numpy (VOC sizes; runs as is)

Use when

  • +One forward pass with only convolutions: 59 FPS for SSD300, versus 7 FPS for Faster R-CNN with the same VGG-16 backbone
  • +Multi-scale maps and several aspect ratios per location, so object sizes are covered without an image pyramid
  • +Simple to deploy; SSDLite with a MobileNet backbone became a standard mobile detector

Avoid when

  • −Weak on small objects: the finest map is stride 8 with shallow features
  • −Hand-designed scales and aspect ratios must suit the dataset
  • −Hard negative mining keeps only the top 3:1 negatives; RetinaNet's focal loss trains on all of them instead

Check yourself

An SSD300 image contains one object. After matching, 12 default boxes are positive. How many default boxes contribute to the classification loss?

Check yourself

Why does SSD attach predictors to conv4_3 as well as to the deep 1×1–10×10 maps?

Remember

  • ✓One-stage detector: a 3×33\times3 conv head on six maps (382,192,102,52,32,1238^2,19^2,10^2,5^2,3^2,1^2) predicts CC scores and 4 offsets for k=4k=4 or 6 default boxes per cell, 8732 in total for SSD300.
  • ✓Default-box scales grow linearly from 0.2 to 0.9 of the image; aspect ratios {1,2,3,12,13}\{1,2,3,\tfrac12,\tfrac13\} plus an extra square sksk+1\sqrt{s_k s_{k+1}}.
  • ✓Positives: each ground truth's best box plus every box with IoU > 0.5; offsets use the Faster R-CNN (cx, cy, log w, log h) encoding.
  • ✓Hard negative mining keeps negatives at most 3:1 to positives; the loss is (confidence + localisation) / N.
  • ✓SSD300: 74.3 mAP on VOC2007 at 59 FPS; weak on small objects, which later feature-pyramid detectors improve.
← back to Deep Learning roadmap