Definition
IoU loss is , where the Jaccard index measures the overlap between a predicted region and a target region . The region is a segmentation mask (soft Jaccard loss) or an axis-aligned bounding box (IoU, GIoU, DIoU and CIoU losses).
Detection and segmentation are scored by IoU. A detection counts as correct if its IoU with a ground-truth box exceeds a threshold (0.5 in PASCAL VOC, 0.5 to 0.95 averaged in COCO), and segmentation benchmarks report mean IoU per class. Classic losses do not measure this. An or smooth- loss on box coordinates treats each coordinate independently and depends on the box's absolute size: the same 2-pixel error is a disaster for a 10-pixel box and negligible for a 500-pixel one. IoU is scale invariant, and it couples all four coordinates through the overlap.
The index is Jaccard's (1901 and 1912) coefficient of community for plant species. As a training loss it appeared in two lines of work in 2016. For boxes, Yu et al.'s UnitBox used for face detection. For masks, Rahman and Wang optimized a soft, differentiable IoU for segmentation. Rezatofighi et al. (CVPR 2019) pointed out that IoU is flat when boxes do not overlap and proposed Generalized IoU (GIoU). Zheng et al. (AAAI 2020) added the Distance-IoU (DIoU) and Complete-IoU (CIoU) losses. Berman, Rannen Triki and Blaschko (CVPR 2018) derived the Lovász-softmax loss, a convex surrogate for the per-class Jaccard loss in segmentation.
Soft Jaccard loss for masks
- predicted foreground probability and binary label of pixel
- soft intersection; the soft union is (the probabilistic )
- Jaccard (IoU) and Dice coefficients. With the same sums and the relation is exact, for soft values too
Since is increasing, Dice and IoU rank masks identically: a better Dice is always a better IoU. The losses still differ in shape. penalizes medium overlaps harder than does: at the Dice loss is only . Their gradients weight pixels differently too. In a multi-class setting the soft Jaccard is computed per class channel and averaged, which yields a differentiable stand-in for mean IoU. Lovász-softmax takes a different route. The Jaccard loss is a submodular set function of the mispredicted pixels. Its Lovász extension, evaluated on sorted per-pixel errors, is a tight convex surrogate. Berman et al. report higher mean IoU than with cross-entropy training alone.
Box losses: IoU, GIoU, DIoU, CIoU
- predicted and ground-truth boxes
- smallest axis-aligned box enclosing both and
- Euclidean distance between the two box centres
- length of the diagonal of , so is scale invariant
- aspect-ratio mismatch and its weight; is treated as a constant when differentiating (as in the paper and in torchvision)
Why the extra terms. If , then IoU is 0 no matter how far apart the boxes are, so and its gradient is exactly zero. A prediction that starts disjoint from its target never moves. GIoU adds the share of the enclosing box that neither box covers. This share shrinks as the boxes approach, so it gives a gradient everywhere, and . GIoU has a blind spot, though. When one box contains the other, equals the larger box, so the penalty is 0 and GIoU reduces to IoU. The position then has no gradient until the boxes' edges meet. DIoU penalizes the centre distance directly, normalized by 's diagonal, so it always pulls the centres together. CIoU also matches the aspect ratio. The weight is small while overlap is poor (IoU matters first) and grows as IoU approaches 1.
At a glance
- Task
- bounding-box regression in detectors; mask segmentation (soft Jaccard, Lovász)
- Range
- ; ;
- Scale invariant
- yes: scaling both boxes by any factor leaves every loss unchanged
- Gradient when disjoint
- IoU: none. GIoU, DIoU, CIoU: nonzero
- Box format
- torchvision expects with ,
- Typical use
- DETR: + GIoU; YOLOv5/YOLOv8: CIoU; FCOS: IoU-type loss on the regressed distances
- PyTorch
torchvision.ops.generalized_box_iou_loss,distance_box_iou_loss,complete_box_iou_loss,box_iou
Worked example
Predicted box and target in format. Every value below was computed in Node from the figure's library. The gradients are w.r.t. the predicted centre , with held fixed.
From boxes to losses
- 1Areas , . Intersection : . Union , so and (UnitBox's ).
- 2Enclosing box , . Empty share , so .
- 3Centres and : . Diagonal , so and .
- 4Aspect term , , so .
- 5For comparison, the mask relation turns IoU 0.2727 into Dice 0.4286.
| prediction | IoU | |||||
|---|---|---|---|---|---|---|
| (1, 1, 5, 4) | 0.2727 | 0.7273 | 0.8473 | 0.7923 | 0.7924 | (−0.212, −0.230) |
| (7, 0, 9, 2) | 0 | 1 | 1.5238 | 1.2941 | 1.2941 | (+0.068, −0.079) |
| (8, 0, 10, 2) | 0 | 1 | 1.5833 | 1.3400 | 1.3400 | (+0.052, −0.069) |
The two disjoint boxes are square, like the target, so and CIoU equals DIoU. Both have IoU loss exactly 1 with zero gradient. GIoU and DIoU rank the closer one better, and gradient descent moves the centre toward smaller and larger , toward the target.
1import torch, torch.nn.functional as F2from torchvision.ops import (box_iou, generalized_box_iou_loss,3 distance_box_iou_loss, complete_box_iou_loss)45pred = torch.tensor([[1., 1., 5., 4.], [7., 0., 9., 2.]], requires_grad=True) # (x1, y1, x2, y2)6gt = torch.tensor([[2., 2., 6., 6.], [2., 2., 6., 6.]])78print(box_iou(pred, gt).diagonal()) # [0.2727, 0.0000] (box_iou is N x M)9print(generalized_box_iou_loss(pred, gt)) # [0.8473, 1.5238] (reduction="none")10print(distance_box_iou_loss(pred, gt)) # [0.7923, 1.2941]11print(complete_box_iou_loss(pred, gt)) # [0.7924, 1.2941]1213loss = generalized_box_iou_loss(pred, gt, reduction="mean")14loss.backward() # disjoint box gets a gradient too1516# In a detector: an L1 term for fast early progress + an IoU-type term (DETR uses 5 * L1 + 2 * GIoU)17def box_loss(pred, gt, w_l1=5.0, w_giou=2.0):18 return w_l1 * F.l1_loss(pred, gt) + w_giou * generalized_box_iou_loss(pred, gt, reduction="mean")1920def soft_jaccard_loss(logits, target, eps=1.0, dims=(-2, -1)):21 """Binary soft IoU for masks. logits, target: e.g. (N, H, W); target in {0, 1}."""22 p = torch.sigmoid(logits)23 inter = (p * target).sum(dim=dims)24 union = p.sum(dim=dims) + target.sum(dim=dims) - inter25 return (1 - (inter + eps) / (union + eps)).mean()Use when
- +Optimizes the evaluation metric directly; scale invariant across small and large objects
- +Couples the four box coordinates through the overlap instead of treating them independently
- +GIoU, DIoU and CIoU give a gradient even for non-overlapping boxes
- +Soft Jaccard and Lovász give differentiable stand-ins for mean IoU in segmentation
Avoid when
- −Plain is flat (zero gradient) for disjoint boxes
- −GIoU degenerates to IoU when one box contains the other
- −Needs valid, non-degenerate boxes; sensitive to box-format mistakes
- −Like Dice, mask IoU is non-decomposable, so gradients are noisy for tiny or empty masks
Check yourself
A predicted box does not overlap its target box at all. Which loss gives zero gradient for its coordinates?
Check yourself
A model reaches mean soft Dice 0.8 on a validation set of masks. What is the corresponding IoU of a single mask with Dice exactly 0.8?
Remember
- ✓IoU is scale invariant and is the metric detection and segmentation are scored by; trains on it directly.
- ✓For masks use the soft Jaccard (sums of probabilities) or Lovász-softmax; , so Dice and IoU rank masks identically.
- ✓For boxes, has zero gradient when boxes are disjoint. GIoU adds the empty share of the enclosing box .
- ✓DIoU adds and pulls centres together even when one box contains the other; CIoU adds an aspect-ratio term .
- ✓In PyTorch:
torchvision.ops.generalized_box_iou_loss/distance_box_iou_loss/complete_box_iou_losson boxes, usually combined with an term.