Intersection over Union (IoU) loss

1 − |A∩B|/|A∪B| trains masks and boxes on the metric they are judged by. Plain IoU loss has zero gradient when boxes do not overlap; GIoU, DIoU and CIoU add penalty terms that keep pulling the prediction toward the target.

13 min read993 wordsadvanced0 viewsupdated 2026-10-11

Definition

IoU loss is L=1−IoUL=1-\mathrm{IoU}, where the Jaccard index IoU(A,B)=∣A∩B∣/∣A∪B∣\mathrm{IoU}(A,B)=|A\cap B|/|A\cup B| measures the overlap between a predicted region AA and a target region BB. The region is a segmentation mask (soft Jaccard loss) or an axis-aligned bounding box (IoU, GIoU, DIoU and CIoU losses).

Detection and segmentation are scored by IoU. A detection counts as correct if its IoU with a ground-truth box exceeds a threshold (0.5 in PASCAL VOC, 0.5 to 0.95 averaged in COCO), and segmentation benchmarks report mean IoU per class. Classic losses do not measure this. An ℓ2\ell_2 or smooth-ℓ1\ell_1 loss on box coordinates treats each coordinate independently and depends on the box's absolute size: the same 2-pixel error is a disaster for a 10-pixel box and negligible for a 500-pixel one. IoU is scale invariant, and it couples all four coordinates through the overlap.

The index is Jaccard's (1901 and 1912) coefficient of community for plant species. As a training loss it appeared in two lines of work in 2016. For boxes, Yu et al.'s UnitBox used −ln⁡IoU-\ln\mathrm{IoU} for face detection. For masks, Rahman and Wang optimized a soft, differentiable IoU for segmentation. Rezatofighi et al. (CVPR 2019) pointed out that IoU is flat when boxes do not overlap and proposed Generalized IoU (GIoU). Zheng et al. (AAAI 2020) added the Distance-IoU (DIoU) and Complete-IoU (CIoU) losses. Berman, Rannen Triki and Blaschko (CVPR 2018) derived the Lovász-softmax loss, a convex surrogate for the per-class Jaccard loss in segmentation.

Soft Jaccard loss for masks

LJacc=1−∑ipigi+ε∑ipi+∑igi−∑ipigi+εD=2J1+J,J=D2−DL_{\text{Jacc}}=1-\frac{\sum_i p_ig_i+\varepsilon}{\sum_i p_i+\sum_i g_i-\sum_i p_ig_i+\varepsilon}\qquad D=\frac{2J}{1+J},\quad J=\frac{D}{2-D}
pi,  gip_i,\;g_i
predicted foreground probability and binary label of pixel ii
∑pigi\sum p_ig_i
soft intersection; the soft union is ∑p+∑g−∑pg\sum p+\sum g-\sum pg (the probabilistic ∣A∣+∣B∣−∣A∩B∣|A|+|B|-|A\cap B|)
J,  DJ,\;D
Jaccard (IoU) and Dice coefficients. With the same sums and ε=0\varepsilon=0 the relation is exact, for soft values too

Since D=2J/(1+J)D=2J/(1+J) is increasing, Dice and IoU rank masks identically: a better Dice is always a better IoU. The losses still differ in shape. 1−J1-J penalizes medium overlaps harder than 1−D1-D does: at J=0.5J=0.5 the Dice loss is only 1−0.667=0.3331-0.667=0.333. Their gradients weight pixels differently too. In a multi-class setting the soft Jaccard is computed per class channel and averaged, which yields a differentiable stand-in for mean IoU. Lovász-softmax takes a different route. The Jaccard loss is a submodular set function of the mispredicted pixels. Its Lovász extension, evaluated on sorted per-pixel errors, is a tight convex surrogate. Berman et al. report higher mean IoU than with cross-entropy training alone.

Box losses: IoU, GIoU, DIoU, CIoU

LIoU=1−IoULGIoU=1−IoU+∣C∖(A∪B)∣∣C∣LDIoU=1−IoU+ρ2(b,bgt)c2LCIoU=LDIoU+αv,v=4π2(arctan⁡wgthgt−arctan⁡wh)2,α=v(1−IoU)+v\begin{aligned}L_{\text{IoU}}&=1-\mathrm{IoU}\\ L_{\text{GIoU}}&=1-\mathrm{IoU}+\frac{|C\setminus(A\cup B)|}{|C|}\\ L_{\text{DIoU}}&=1-\mathrm{IoU}+\frac{\rho^2(\mathbf b,\mathbf b^{gt})}{c^2}\\ L_{\text{CIoU}}&=L_{\text{DIoU}}+\alpha v,\quad v=\frac{4}{\pi^2}\Big(\arctan\frac{w^{gt}}{h^{gt}}-\arctan\frac{w}{h}\Big)^2,\quad \alpha=\frac{v}{(1-\mathrm{IoU})+v}\end{aligned}
A,  BA,\;B
predicted and ground-truth boxes
CC
smallest axis-aligned box enclosing both AA and BB
ρ(b,bgt)\rho(\mathbf b,\mathbf b^{gt})
Euclidean distance between the two box centres
cc
length of the diagonal of CC, so ρ2/c2∈[0,1)\rho^2/c^2\in[0,1) is scale invariant
v,  αv,\;\alpha
aspect-ratio mismatch and its weight; α\alpha is treated as a constant when differentiating (as in the paper and in torchvision)

Why the extra terms. If A∩B=∅A\cap B=\varnothing, then IoU is 0 no matter how far apart the boxes are, so LIoU=1L_{\text{IoU}}=1 and its gradient is exactly zero. A prediction that starts disjoint from its target never moves. GIoU adds the share of the enclosing box that neither box covers. This share shrinks as the boxes approach, so it gives a gradient everywhere, and LGIoU∈[0,2)L_{\text{GIoU}}\in[0,2). GIoU has a blind spot, though. When one box contains the other, CC equals the larger box, so the penalty is 0 and GIoU reduces to IoU. The position then has no gradient until the boxes' edges meet. DIoU penalizes the centre distance directly, normalized by CC's diagonal, so it always pulls the centres together. CIoU also matches the aspect ratio. The weight α\alpha is small while overlap is poor (IoU matters first) and grows as IoU approaches 1.

At a glance

Task
bounding-box regression in detectors; mask segmentation (soft Jaccard, Lovász)
Range
LIoU∈[0,1]L_{\text{IoU}}\in[0,1]; LGIoU∈[0,2)L_{\text{GIoU}}\in[0,2); LDIoU∈[0,2)L_{\text{DIoU}}\in[0,2)
Scale invariant
yes: scaling both boxes by any factor leaves every loss unchanged
Gradient when disjoint
IoU: none. GIoU, DIoU, CIoU: nonzero
Box format
torchvision expects (x1,y1,x2,y2)(x_1,y_1,x_2,y_2) with x2≥x1x_2\ge x_1, y2≥y1y_2\ge y_1
Typical use
DETR: ℓ1\ell_1 + GIoU; YOLOv5/YOLOv8: CIoU; FCOS: IoU-type loss on the regressed distances
PyTorch
torchvision.ops.generalized_box_iou_loss, distance_box_iou_loss, complete_box_iou_loss, box_iou
target · prediction (drag) · enclosing box Closs as the box slides horizontally00.511.520246810predicted cx (target at 5)L_IoU = 0.8571L_GIoU = 1.0794L_DIoU = 0.9683L_CIoU = 0.9683IoU 0.143 · Dice 2J/(1+J) 0.250|C∖U|/|C| 0.222 · ρ²/c² 0.111 · αv 0.0000∇L_GIoU: ∂/∂cx 0.208 · ∂/∂cy 0.311 · ∂/∂w −0.052 · ∂/∂h −0.078I = 1.50, U = 10.50, |C| = 13.50 · press "descend" to run 300 GD steps (lr 0.3)
loss
presets
fig.The target is the 3×2 warm box centred at (5, 5). Drag the blue prediction or use the sliders. The right panel plots all four losses as the prediction slides horizontally: the IoU curve is flat at 1 wherever the boxes are apart, while the other three keep sloping toward the target. In "disjoint", L_IoU = 1 with gradient exactly 0, and "descend" never moves the box. GIoU reaches IoU 0.5 after 71 steps (lr 0.3), DIoU and CIoU after 108. In "inside target" GIoU = IoU = 0.800 and ∂/∂cx = ∂/∂cy = 0, while DIoU pulls the centre with ∂/∂cx = −0.123.

Worked example

Predicted box A=(1,1,5,4)A=(1,1,5,4) and target B=(2,2,6,6)B=(2,2,6,6) in (x1,y1,x2,y2)(x_1,y_1,x_2,y_2) format. Every value below was computed in Node from the figure's library. The gradients are w.r.t. the predicted centre (cx,cy)(c_x,c_y), with α\alpha held fixed.

From boxes to losses

  1. 1Areas ∣A∣=4⋅3=12|A|=4\cdot3=12, ∣B∣=16|B|=16. Intersection [2,5]×[2,4][2,5]\times[2,4]: ∣A∩B∣=6|A\cap B|=6. Union 12+16−6=2212+16-6=22, so IoU=6/22=0.2727\mathrm{IoU}=6/22=0.2727 and LIoU=0.7273L_{\text{IoU}}=0.7273 (UnitBox's −ln⁡IoU=1.2993-\ln\mathrm{IoU}=1.2993).
  2. 2Enclosing box C=(1,1,6,6)C=(1,1,6,6), ∣C∣=25|C|=25. Empty share (25−22)/25=0.12(25-22)/25=0.12, so LGIoU=0.8473L_{\text{GIoU}}=0.8473.
  3. 3Centres (3,2.5)(3,2.5) and (4,4)(4,4): ρ2=1+2.25=3.25\rho^2=1+2.25=3.25. Diagonal c2=52+52=50c^2=5^2+5^2=50, so ρ2/c2=0.065\rho^2/c^2=0.065 and LDIoU=0.7923L_{\text{DIoU}}=0.7923.
  4. 4Aspect term v=4π2(arctan⁡1−arctan⁡43)2=0.00816v=\tfrac{4}{\pi^2}(\arctan1-\arctan\tfrac43)^2=0.00816, α=0.00816/(0.7273+0.00816)=0.0111\alpha=0.00816/(0.7273+0.00816)=0.0111, so LCIoU=0.7924L_{\text{CIoU}}=0.7924.
  5. 5For comparison, the mask relation D=2J/(1+J)D=2J/(1+J) turns IoU 0.2727 into Dice 0.4286.
predictionIoULIoUL_{\text{IoU}}LGIoUL_{\text{GIoU}}LDIoUL_{\text{DIoU}}LCIoUL_{\text{CIoU}}∂LGIoU/∂(cx,cy)\partial L_{\text{GIoU}}/\partial(c_x,c_y)
(1, 1, 5, 4)0.27270.72730.84730.79230.7924(−0.212, −0.230)
(7, 0, 9, 2)011.52381.29411.2941(+0.068, −0.079)
(8, 0, 10, 2)011.58331.34001.3400(+0.052, −0.069)
The example pair, and two disjoint predictions for the same target. IoU loss cannot tell the disjoint ones apart

The two disjoint boxes are square, like the target, so v=0v=0 and CIoU equals DIoU. Both have IoU loss exactly 1 with zero gradient. GIoU and DIoU rank the closer one better, and gradient descent moves the centre toward smaller xx and larger yy, toward the target.

python
1import torch, torch.nn.functional as F
2from torchvision.ops import (box_iou, generalized_box_iou_loss,
3 distance_box_iou_loss, complete_box_iou_loss)
4
5pred = torch.tensor([[1., 1., 5., 4.], [7., 0., 9., 2.]], requires_grad=True) # (x1, y1, x2, y2)
6gt = torch.tensor([[2., 2., 6., 6.], [2., 2., 6., 6.]])
7
8print(box_iou(pred, gt).diagonal()) # [0.2727, 0.0000] (box_iou is N x M)
9print(generalized_box_iou_loss(pred, gt)) # [0.8473, 1.5238] (reduction="none")
10print(distance_box_iou_loss(pred, gt)) # [0.7923, 1.2941]
11print(complete_box_iou_loss(pred, gt)) # [0.7924, 1.2941]
12
13loss = generalized_box_iou_loss(pred, gt, reduction="mean")
14loss.backward() # disjoint box gets a gradient too
15
16# In a detector: an L1 term for fast early progress + an IoU-type term (DETR uses 5 * L1 + 2 * GIoU)
17def box_loss(pred, gt, w_l1=5.0, w_giou=2.0):
18 return w_l1 * F.l1_loss(pred, gt) + w_giou * generalized_box_iou_loss(pred, gt, reduction="mean")
19
20def soft_jaccard_loss(logits, target, eps=1.0, dims=(-2, -1)):
21 """Binary soft IoU for masks. logits, target: e.g. (N, H, W); target in {0, 1}."""
22 p = torch.sigmoid(logits)
23 inter = (p * target).sum(dim=dims)
24 union = p.sum(dim=dims) + target.sum(dim=dims) - inter
25 return (1 - (inter + eps) / (union + eps)).mean()
Box losses from torchvision, and a soft Jaccard loss for masks

Use when

  • +Optimizes the evaluation metric directly; scale invariant across small and large objects
  • +Couples the four box coordinates through the overlap instead of treating them independently
  • +GIoU, DIoU and CIoU give a gradient even for non-overlapping boxes
  • +Soft Jaccard and Lovász give differentiable stand-ins for mean IoU in segmentation

Avoid when

  • −Plain 1−IoU1-\mathrm{IoU} is flat (zero gradient) for disjoint boxes
  • −GIoU degenerates to IoU when one box contains the other
  • −Needs valid, non-degenerate boxes; sensitive to box-format mistakes
  • −Like Dice, mask IoU is non-decomposable, so gradients are noisy for tiny or empty masks

Check yourself

A predicted box does not overlap its target box at all. Which loss gives zero gradient for its coordinates?

Check yourself

A model reaches mean soft Dice 0.8 on a validation set of masks. What is the corresponding IoU of a single mask with Dice exactly 0.8?

Remember

  • ✓IoU =∣A∩B∣/∣A∪B∣=|A\cap B|/|A\cup B| is scale invariant and is the metric detection and segmentation are scored by; L=1−IoUL=1-\mathrm{IoU} trains on it directly.
  • ✓For masks use the soft Jaccard (sums of probabilities) or Lovász-softmax; D=2J/(1+J)D=2J/(1+J), so Dice and IoU rank masks identically.
  • ✓For boxes, 1−IoU1-\mathrm{IoU} has zero gradient when boxes are disjoint. GIoU adds the empty share of the enclosing box CC.
  • ✓DIoU adds ρ2/c2\rho^2/c^2 and pulls centres together even when one box contains the other; CIoU adds an aspect-ratio term αv\alpha v.
  • ✓In PyTorch: torchvision.ops.generalized_box_iou_loss / distance_box_iou_loss / complete_box_iou_loss on (x1,y1,x2,y2)(x_1,y_1,x_2,y_2) boxes, usually combined with an ℓ1\ell_1 term.
← back to Deep Learning roadmap