YOLO

YOLO (Redmon et al., 2016) regresses a 7×7 grid to 2 boxes and 20 class probabilities per cell (a 7×7×30 tensor) in one pass, 63.4 VOC2007 mAP at 45 FPS; later versions added anchors, three scales and anchor-free heads.

7 min read657 wordscore0 viewsupdated 2026-10-15

Definition

YOLO (You Only Look Once; Redmon, Divvala, Girshick and Farhadi, "You Only Look Once: Unified, Real-Time Object Detection", CVPR 2016) frames detection as a single regression problem. One convolutional network reads the whole image once and outputs all boxes and class probabilities directly. There are no region proposals, no sliding windows and no per-region classifier.

Input: an image resized to 448×448448\times448. Output: boxes with class scores. In 2015 the accurate detectors were two-stage: R-CNN-style pipelines that propose regions, then classify each one (see Fast R-CNN and Faster R-CNN). Faster R-CNN with VGG-16 ran at about 7 FPS. YOLO traded some accuracy for a simple, end-to-end network that ran at 45 FPS; the paper reports more than twice the mAP of earlier real-time detectors on PASCAL VOC.

The grid

The image is divided into an S×SS\times S grid with S=7S=7. If an object's centre falls into a cell, that cell is responsible for detecting it. Each cell predicts B=2B=2 boxes, each with 5 numbers (x,y,w,h,c)(x,y,w,h,c), and one set of C=20C=20 conditional class probabilities Pr⁡(classi∣object)\Pr(\text{class}_i\mid\text{object}). The output is an S×S×(5B+C)=7×7×30S\times S\times(5B+C)=7\times7\times30 tensor: 1470 numbers and 98 boxes per image.

(x,y)(x,y) is the box centre as an offset inside its cell, in [0,1][0,1]. (w,h)(w,h) are relative to the whole image. The confidence target is c=Pr⁡(object)⋅IoUpredtruthc=\Pr(\text{object})\cdot\mathrm{IoU}^{\text{truth}}_{\text{pred}}: 0 for cells without an object, and the IoU between the predicted and the true box otherwise.

YOLO v1: one network, one pass, the whole image regressed to a 7×7×30 tensor448×448 input, S = 7dogcentre in cell (4, 3): it isresponsible for the dogCNN24 conv+ 2 FCleaky ReLU→ 14707×7×3030 = 2×5 + 2098 boxes per image(x, y) in the cell(w, h) of the imagecell (4, 3):xywhcxywhcbox 1box 2Pr(class | object), 20 VOC classesc = Pr(object)·IoU; both boxes share one class vectorscore = Pr(class|obj)·c98 boxes × 20 classesthreshold scoresdrop low-confidenceNMS (see Object detection)final detectionsv1 20167×7 grid, B = 2v2 2017k-means anchorsv3 20183 scales, 9 anchorsYOLOX 2021anchor-freeYOLOv8 2023anchor-free
fig.YOLO v1. The dog's centre lies in cell (row 4, col 3), so that cell must predict it. The network (24 convolutional layers and 2 fully connected layers) outputs 30 numbers per cell: two boxes (x, y, w, h, confidence) and 20 class probabilities shared by both boxes. At test time the class scores are multiplied by the box confidence, thresholded and passed to NMS. Bottom: the lineage of later versions.

Network and loss

The network has 24 convolutional layers followed by 2 fully connected layers, using 1×11\times1 reduction layers before 3×33\times3 convolutions in the style of GoogLeNet (see GoogLeNet / Inception). Fast YOLO uses 9 convolutional layers. The first 20 layers are pre-trained on ImageNet at 224×224224\times224, then the input is raised to 448×448448\times448 for detection. All layers use leaky ReLU with slope 0.1, except the last, which is linear.

Training uses sum-squared error on all outputs. Of the BB predictors in a responsible cell, the one whose current box has the highest IoU with the ground truth is made responsible (1ijobj=1\mathbb 1^{\text{obj}}_{ij}=1). This makes the two predictors specialise in different sizes and shapes.

L=λcoord∑i=0S2∑j=0B1ijobj[(xi−x^i)2+(yi−y^i)2+(wi−w^i)2+(hi−h^i)2]+∑i,j1ijobj(Ci−C^i)2+λnoobj∑i,j1ijnoobj(Ci−C^i)2+∑i1iobj∑c(pi(c)−p^i(c))2\begin{aligned}\mathcal L={}&\lambda_{\text{coord}}\sum_{i=0}^{S^2}\sum_{j=0}^{B}\mathbb 1^{\text{obj}}_{ij}\Big[(x_i-\hat x_i)^2+(y_i-\hat y_i)^2+\big(\sqrt{w_i}-\sqrt{\hat w_i}\big)^2+\big(\sqrt{h_i}-\sqrt{\hat h_i}\big)^2\Big]\\&+\sum_{i,j}\mathbb 1^{\text{obj}}_{ij}(C_i-\hat C_i)^2+\lambda_{\text{noobj}}\sum_{i,j}\mathbb 1^{\text{noobj}}_{ij}(C_i-\hat C_i)^2+\sum_{i}\mathbb 1^{\text{obj}}_{i}\sum_{c}\big(p_i(c)-\hat p_i(c)\big)^2\end{aligned}
1ijobj\mathbb 1^{\text{obj}}_{ij}
1 if predictor jj of cell ii is responsible for an object
λcoord=5\lambda_{\text{coord}}=5
up-weights box coordinates
λnoobj=0.5\lambda_{\text{noobj}}=0.5
down-weights the confidence of the many empty-cell predictors
w,h\sqrt{w},\sqrt{h}
an error of 0.05 in ww costs 6.7×6.7\times more on a box of width 0.1 than on one of width 0.8

At test time the class-specific score of each box is Pr⁡(classi∣object)⋅c=Pr⁡(classi)⋅IoU\Pr(\text{class}_i\mid\text{object})\cdot c=\Pr(\text{class}_i)\cdot\mathrm{IoU}. Low scores are dropped and non-maximum suppression removes duplicates; see Object detection for IoU and NMS.

Results and the lineage (only the published facts)

YOLO v1 (2016)
VOC2007 test: 63.4 mAP at 45 FPS; Fast YOLO 52.7 mAP at 155 FPS (Titan X). Fewer background false positives than Fast R-CNN, more localisation errors.
YOLOv2 (2017)
Redmon and Farhadi, "YOLO9000: Better, Faster, Stronger" (CVPR 2017). Batch norm, Darknet-19, anchor boxes from k-means on training boxes (distance 1−IoU1-\mathrm{IoU}, k=5k=5), centres bx=σ(tx)+cxb_x=\sigma(t_x)+c_x, sizes bw=pwetwb_w=p_we^{t_w}, a 13×1313\times13 grid at 416 px, multi-scale training. VOC2007: 76.8 mAP at 67 FPS (416 px).
YOLOv3 (2018)
Redmon and Farhadi, "YOLOv3: An Incremental Improvement" (tech report). Darknet-53, predictions at 3 scales (strides 32, 16, 8), 3 of 9 k-means anchors per scale, independent logistic class scores. At 416 px: 3⋅(132+262+522)=10,6473\cdot(13^2+26^2+52^2)=10{,}647 boxes.
Later
YOLOv4 (Bochkovskiy, Wang and Liao, 2020); YOLOX (Ge et al., 2021): anchor-free with a decoupled head; Ultralytics YOLOv5 (2020) and YOLOv8 (2023, anchor-free), released as software without a paper
YOLOv1, 448×448: 7×7 grid, B = 2, C = 20shaded = responsible cell · rose = 2+ centresoutput: 7×7×(5·2 + 20) = 7×7×301470 numbers, 98 boxes5 objects · scene: streetcells with an object: 4 of 491 object lost: cell already takenno-object conf terms: 94 of 98 (λ = 0.5)selected: object 2 (person), 45×179 pxcell (row 4, col 4) holds the centretargets: x, y in the cell; w, h of imagex = 0.620 y = 0.200√w = 0.316 √h = 0.632confidence target = IoU(pred, truth)class target: one-hot over 20, per cellc_x = (4 + 0.620) / 7 · 448 = 295.7 pxloss: λ_coord = 5 on x, y, √w, √hsum-squared error on all 30 outputsone class distribution per cell
modelscene
fig.Target assignment, computed. Default (street, v1, S = 7): the two people stand side by side, both centres fall in cell (4, 4), and since a v1 cell has a single class distribution only one of them can be a target: 1 of 5 objects is lost. The selected person gets x = 0.620, y = 0.200, √w = 0.316, √h = 0.632. Raise S to 13 and both have their own cells. The flock of 14 birds loses 3 at S = 7 and none at S = 13. Switch to YOLOv3: the car goes to stride 32, the other four objects to stride 16, all 14 birds to stride 8 with their own slots, out of 10,647 boxes.

Worked example: two people in one cell

A 448×448448\times448 image has two people with centres at (0.66,0.60)(0.66,0.60) and (0.70,0.62)(0.70,0.62) in normalised coordinates, sizes 0.10×0.400.10\times0.40 and 0.09×0.360.09\times0.36.

ObjectS = 7: cx·S, cy·Scelltargets x, y, √w, √hS = 13: cell
person 14.62, 4.20(4, 4)0.620, 0.200, 0.316, 0.632(7, 8)
person 24.90, 4.34(4, 4)0.900, 0.340, 0.300, 0.600(8, 9)
YOLO v1 targets; cell = (⌊cy·S⌋, ⌊cx·S⌋)

At S=7S=7 both centres land in cell (4,4)(4,4). The cell has two box predictors but only one class vector and, in practice, one target object, so one person goes unsupervised. Decoding person 1 back: cx=(4+0.62)/7⋅448=295.7c_x=(4+0.62)/7\cdot448=295.7 px and w=0.3162⋅448=44.8w=0.316^2\cdot448=44.8 px. At S=13S=13 the two people fall in different cells. This is the limitation the paper names: YOLO struggles with small objects in groups, such as flocks of birds.

python
1import numpy as np
2
3S, B, C, IMG = 7, 2, 20, 448
4
5def decode_v1(out, thresh=0.2):
6 """out: (S, S, B*5 + C) YOLOv1 output -> list of (x1, y1, x2, y2, score, cls) in pixels."""
7 dets = []
8 for row in range(S):
9 for col in range(S):
10 cell = out[row, col]
11 cls_prob = cell[B * 5:] # Pr(class | object), shared by both boxes
12 for b in range(B):
13 x, y, sw, sh, conf = cell[b * 5:(b + 1) * 5]
14 cx, cy = (col + x) / S * IMG, (row + y) / S * IMG # x, y are offsets inside the cell
15 w, h = sw ** 2 * IMG, sh ** 2 * IMG # the net predicts sqrt(w), sqrt(h)
16 scores = conf * cls_prob # class-specific confidence
17 k = scores.argmax()
18 if scores[k] > thresh:
19 dets.append((cx - w / 2, cy - h / 2, cx + w / 2, cy + h / 2, scores[k], k))
20 return dets # then per-class NMS, e.g. torchvision.ops.batched_nms
21
22out = np.zeros((S, S, B * 5 + C))
23out[4, 4, :5] = [0.62, 0.20, 0.316, 0.632, 0.9] # the person of the worked example
24out[4, 4, 5:10] = [0.50, 0.50, 0.200, 0.200, 0.1] # second predictor of the same cell, low confidence
25out[4, 4, 10 + 14] = 0.95 # class 14 = person in VOC order
26for d in decode_v1(out):
27 print(np.round(d[:4], 1), round(d[4], 3), d[5]) # [273.3 179.3 318. 358.3] 0.855 14
Decoding a YOLO v1 output tensor (numpy; runs as is). Real outputs come from a trained network; here one cell is filled by hand.

Where it's used

Real-time video

Surveillance, traffic and sports analytics, where 30+ FPS on one GPU matters more than the last few AP points

Edge and robotics

Small YOLO variants run on embedded GPUs and phones; the single fully convolutional head (from v2 on) exports easily to ONNX and TensorRT

Tracking front end

Per-frame YOLO detections feed trackers that associate boxes by IoU and motion (see Object tracking)

Check yourself

For S = 7, B = 2 and C = 20, what is the size of YOLO v1's output, and how many boxes does it predict?

Check yourself

Why does YOLO v1 regress √w and √h instead of w and h?

Remember

  • ✓One network, one pass: the image is split into a 7×7 grid; the cell holding an object's centre is responsible for it.
  • ✓Each cell predicts 2 boxes (x, y, √w, √h, confidence = Pr(object)·IoU) and 20 class probabilities, a 7×7×30 tensor and 98 boxes.
  • ✓Sum-squared loss with λ_coord = 5 and λ_noobj = 0.5; score = class probability × confidence, then NMS.
  • ✓63.4 mAP on VOC2007 at 45 FPS; weak on small grouped objects and on precise localisation.
  • ✓v2: k-means anchors and σ(t) + c offsets; v3: 3 scales, 9 anchors, 10,647 boxes at 416 px; YOLOX and YOLOv8 are anchor-free.
← back to Deep Learning roadmap