# Intersection over Union (IoU) loss

> 1 − |A∩B|/|A∪B| trains masks and boxes on the metric they are judged by. Plain IoU loss has zero gradient when boxes do not overlap; GIoU, DIoU and CIoU add penalty terms that keep pulling the prediction toward the target.

Source: https://www.mlfoundry.in/learn/intersection-over-union-iou-loss  
Section: Deep Learning › Loss Functions  
Level: advanced · 13 min read · updated 2026-10-11

**IoU loss:** is $L=1-\mathrm{IoU}$, where the **Jaccard index** $\mathrm{IoU}(A,B)=|A\cap B|/|A\cup B|$ measures the overlap between a predicted region $A$ and a target region $B$. The region is a segmentation mask (soft Jaccard loss) or an axis-aligned bounding box (IoU, GIoU, DIoU and CIoU losses).

Detection and segmentation are scored by IoU. A detection counts as correct if its IoU with a ground-truth box exceeds a threshold (0.5 in PASCAL VOC, 0.5 to 0.95 averaged in COCO), and segmentation benchmarks report mean IoU per class. Classic losses do not measure this. An $\ell_2$ or smooth-$\ell_1$ loss on box coordinates treats each coordinate independently and depends on the box's absolute size: the same 2-pixel error is a disaster for a 10-pixel box and negligible for a 500-pixel one. IoU is **scale invariant**, and it couples all four coordinates through the overlap.

The index is Jaccard's (1901 and 1912) coefficient of community for plant species. As a training loss it appeared in two lines of work in 2016. For **boxes**, Yu et al.'s UnitBox used $-\ln\mathrm{IoU}$ for face detection. For **masks**, Rahman and Wang optimized a soft, differentiable IoU for segmentation. Rezatofighi et al. (CVPR 2019) pointed out that IoU is flat when boxes do not overlap and proposed **Generalized IoU (GIoU)**. Zheng et al. (AAAI 2020) added the **Distance-IoU (DIoU)** and **Complete-IoU (CIoU)** losses. Berman, Rannen Triki and Blaschko (CVPR 2018) derived the **Lovász-softmax** loss, a convex surrogate for the per-class Jaccard loss in segmentation.

## Soft Jaccard loss for masks

$$
L_{\text{Jacc}}=1-\frac{\sum_i p_ig_i+\varepsilon}{\sum_i p_i+\sum_i g_i-\sum_i p_ig_i+\varepsilon}\qquad D=\frac{2J}{1+J},\quad J=\frac{D}{2-D}
$$

- $p_i,\;g_i$: predicted foreground probability and binary label of pixel $i$
- $\sum p_ig_i$: soft intersection; the soft union is $\sum p+\sum g-\sum pg$ (the probabilistic $|A|+|B|-|A\cap B|$)
- $J,\;D$: Jaccard (IoU) and Dice coefficients. With the same sums and $\varepsilon=0$ the relation is exact, for soft values too

Since $D=2J/(1+J)$ is increasing, Dice and IoU **rank masks identically**: a better Dice is always a better IoU. The losses still differ in shape. $1-J$ penalizes medium overlaps harder than $1-D$ does: at $J=0.5$ the Dice loss is only $1-0.667=0.333$. Their gradients weight pixels differently too. In a multi-class setting the soft Jaccard is computed per class channel and averaged, which yields a differentiable stand-in for mean IoU. **Lovász-softmax** takes a different route. The Jaccard loss is a submodular set function of the mispredicted pixels. Its Lovász extension, evaluated on sorted per-pixel errors, is a tight convex surrogate. Berman et al. report higher mean IoU than with cross-entropy training alone.

## Box losses: IoU, GIoU, DIoU, CIoU

$$
\begin{aligned}L_{\text{IoU}}&=1-\mathrm{IoU}\\ L_{\text{GIoU}}&=1-\mathrm{IoU}+\frac{|C\setminus(A\cup B)|}{|C|}\\ L_{\text{DIoU}}&=1-\mathrm{IoU}+\frac{\rho^2(\mathbf b,\mathbf b^{gt})}{c^2}\\ L_{\text{CIoU}}&=L_{\text{DIoU}}+\alpha v,\quad v=\frac{4}{\pi^2}\Big(\arctan\frac{w^{gt}}{h^{gt}}-\arctan\frac{w}{h}\Big)^2,\quad \alpha=\frac{v}{(1-\mathrm{IoU})+v}\end{aligned}
$$

- $A,\;B$: predicted and ground-truth boxes
- $C$: smallest axis-aligned box enclosing both $A$ and $B$
- $\rho(\mathbf b,\mathbf b^{gt})$: Euclidean distance between the two box centres
- $c$: length of the diagonal of $C$, so $\rho^2/c^2\in[0,1)$ is scale invariant
- $v,\;\alpha$: aspect-ratio mismatch and its weight; $\alpha$ is treated as a constant when differentiating (as in the paper and in torchvision)

**Why the extra terms.** If $A\cap B=\varnothing$, then IoU is 0 no matter how far apart the boxes are, so $L_{\text{IoU}}=1$ and its gradient is exactly zero. A prediction that starts disjoint from its target never moves. GIoU adds the share of the enclosing box that neither box covers. This share shrinks as the boxes approach, so it gives a gradient everywhere, and $L_{\text{GIoU}}\in[0,2)$. GIoU has a blind spot, though. When one box contains the other, $C$ equals the larger box, so the penalty is 0 and GIoU reduces to IoU. The position then has no gradient until the boxes' edges meet. DIoU penalizes the **centre distance** directly, normalized by $C$'s diagonal, so it always pulls the centres together. CIoU also matches the aspect ratio. The weight $\alpha$ is small while overlap is poor (IoU matters first) and grows as IoU approaches 1.

**At a glance**

- **Task:** bounding-box regression in detectors; mask segmentation (soft Jaccard, Lovász)
- **Range:** $L_{\text{IoU}}\in[0,1]$; $L_{\text{GIoU}}\in[0,2)$; $L_{\text{DIoU}}\in[0,2)$
- **Scale invariant:** yes: scaling both boxes by any factor leaves every loss unchanged
- **Gradient when disjoint:** IoU: none. GIoU, DIoU, CIoU: nonzero
- **Box format:** torchvision expects $(x_1,y_1,x_2,y_2)$ with $x_2\ge x_1$, $y_2\ge y_1$
- **Typical use:** DETR: $\ell_1$ + GIoU; YOLOv5/YOLOv8: CIoU; FCOS: IoU-type loss on the regressed distances
- **PyTorch:** `torchvision.ops.generalized_box_iou_loss`, `distance_box_iou_loss`, `complete_box_iou_loss`, `box_iou`

> Figure: The target is the 3×2 warm box centred at (5, 5). Drag the blue prediction or use the sliders. The right panel plots all four losses as the prediction slides horizontally: the IoU curve is flat at 1 wherever the boxes are apart, while the other three keep sloping toward the target. In "disjoint", L_IoU = 1 with gradient exactly 0, and "descend" never moves the box. GIoU reaches IoU 0.5 after 71 steps (lr 0.3), DIoU and CIoU after 108. In "inside target" GIoU = IoU = 0.800 and ∂/∂cx = ∂/∂cy = 0, while DIoU pulls the centre with ∂/∂cx = −0.123.

## Worked example

Predicted box $A=(1,1,5,4)$ and target $B=(2,2,6,6)$ in $(x_1,y_1,x_2,y_2)$ format. Every value below was computed in Node from the figure's library. The gradients are w.r.t. the predicted centre $(c_x,c_y)$, with $\alpha$ held fixed.

**From boxes to losses**

1. Areas $|A|=4\cdot3=12$, $|B|=16$. Intersection $[2,5]\times[2,4]$: $|A\cap B|=6$. Union $12+16-6=22$, so $\mathrm{IoU}=6/22=0.2727$ and $L_{\text{IoU}}=0.7273$ (UnitBox's $-\ln\mathrm{IoU}=1.2993$).
2. Enclosing box $C=(1,1,6,6)$, $|C|=25$. Empty share $(25-22)/25=0.12$, so $L_{\text{GIoU}}=0.8473$.
3. Centres $(3,2.5)$ and $(4,4)$: $\rho^2=1+2.25=3.25$. Diagonal $c^2=5^2+5^2=50$, so $\rho^2/c^2=0.065$ and $L_{\text{DIoU}}=0.7923$.
4. Aspect term $v=\tfrac{4}{\pi^2}(\arctan1-\arctan\tfrac43)^2=0.00816$, $\alpha=0.00816/(0.7273+0.00816)=0.0111$, so $L_{\text{CIoU}}=0.7924$.
5. For comparison, the mask relation $D=2J/(1+J)$ turns IoU 0.2727 into Dice 0.4286.

The example pair, and two disjoint predictions for the same target. IoU loss cannot tell the disjoint ones apart

| prediction | IoU | $L_{\text{IoU}}$ | $L_{\text{GIoU}}$ | $L_{\text{DIoU}}$ | $L_{\text{CIoU}}$ | $\partial L_{\text{GIoU}}/\partial(c_x,c_y)$ |
| --- | --- | --- | --- | --- | --- | --- |
| (1, 1, 5, 4) | 0.2727 | 0.7273 | 0.8473 | 0.7923 | 0.7924 | (−0.212, −0.230) |
| (7, 0, 9, 2) | 0 | 1 | 1.5238 | 1.2941 | 1.2941 | (+0.068, −0.079) |
| (8, 0, 10, 2) | 0 | 1 | 1.5833 | 1.3400 | 1.3400 | (+0.052, −0.069) |

The two disjoint boxes are square, like the target, so $v=0$ and CIoU equals DIoU. Both have IoU loss exactly 1 with zero gradient. GIoU and DIoU rank the closer one better, and gradient descent moves the centre toward smaller $x$ and larger $y$, toward the target.

```python
import torch, torch.nn.functional as F
from torchvision.ops import (box_iou, generalized_box_iou_loss,
                             distance_box_iou_loss, complete_box_iou_loss)

pred = torch.tensor([[1., 1., 5., 4.], [7., 0., 9., 2.]], requires_grad=True)  # (x1, y1, x2, y2)
gt   = torch.tensor([[2., 2., 6., 6.], [2., 2., 6., 6.]])

print(box_iou(pred, gt).diagonal())                    # [0.2727, 0.0000]  (box_iou is N x M)
print(generalized_box_iou_loss(pred, gt))              # [0.8473, 1.5238]  (reduction="none")
print(distance_box_iou_loss(pred, gt))                 # [0.7923, 1.2941]
print(complete_box_iou_loss(pred, gt))                 # [0.7924, 1.2941]

loss = generalized_box_iou_loss(pred, gt, reduction="mean")
loss.backward()                                        # disjoint box gets a gradient too

# In a detector: an L1 term for fast early progress + an IoU-type term (DETR uses 5 * L1 + 2 * GIoU)
def box_loss(pred, gt, w_l1=5.0, w_giou=2.0):
    return w_l1 * F.l1_loss(pred, gt) + w_giou * generalized_box_iou_loss(pred, gt, reduction="mean")

def soft_jaccard_loss(logits, target, eps=1.0, dims=(-2, -1)):
    """Binary soft IoU for masks. logits, target: e.g. (N, H, W); target in {0, 1}."""
    p = torch.sigmoid(logits)
    inter = (p * target).sum(dim=dims)
    union = p.sum(dim=dims) + target.sum(dim=dims) - inter
    return (1 - (inter + eps) / (union + eps)).mean()
```

Box losses from torchvision, and a soft Jaccard loss for masks

> **Degenerate and invalid boxes:** The losses assume $x_2\ge x_1$ and $y_2\ge y_1$. A raw regression head can output $x_2<x_1$, which gives a negative "area" and meaningless IoU values. Parameterize the outputs so validity holds by construction, for example a centre plus $w=e^{t_w}$, or distances $l,t,r,b\ge0$ through a softplus or exp as in FCOS. Also mind the box format: torchvision's losses want corners, and many datasets store $(c_x,c_y,w,h)$ or $(x,y,w,h)$; convert with `torchvision.ops.box_convert`.

> **IoU loss alone stalls on bad starts:** Early in training, and for anchors far from the object, many predictions do not overlap their target, and plain $1-\mathrm{IoU}$ gives them nothing to learn from. This is why detectors use GIoU/DIoU/CIoU, pair the IoU term with an $\ell_1$ term on coordinates, or apply the loss only to predictions already matched to an overlapping target.

> **Same ranking, different losses:** Dice and IoU are monotone transforms of each other, $D=2J/(1+J)$, so they agree on which mask is better. Their optima are the same, but their gradients are not. In mask segmentation the choice between soft Dice, soft Jaccard and Lovász matters less than how you average (per image or per batch, per class) and whether you add cross-entropy. For boxes, the choice between IoU variants matters because they behave differently when boxes do not overlap or one contains the other.

> **Which variant the paper results favour:** Zheng et al. report that DIoU and CIoU converge faster than GIoU in their simulation, which averages over many box configurations, and improve AP on YOLOv3, SSD and Faster R-CNN. On a single configuration the order can flip: from the figure's "disjoint" preset, GIoU reaches IoU 0.5 first. The robust advantage of DIoU is the "inside target" case, where GIoU has no positional gradient at all.

**Strengths**

- Optimizes the evaluation metric directly; scale invariant across small and large objects
- Couples the four box coordinates through the overlap instead of treating them independently
- GIoU, DIoU and CIoU give a gradient even for non-overlapping boxes
- Soft Jaccard and Lovász give differentiable stand-ins for mean IoU in segmentation

**Limitations**

- Plain $1-\mathrm{IoU}$ is flat (zero gradient) for disjoint boxes
- GIoU degenerates to IoU when one box contains the other
- Needs valid, non-degenerate boxes; sensitive to box-format mistakes
- Like Dice, mask IoU is non-decomposable, so gradients are noisy for tiny or empty masks

**Question:** A predicted box does not overlap its target box at all. Which loss gives zero gradient for its coordinates?

- A. 1 − IoU
- B. GIoU loss
- C. DIoU loss
- D. CIoU loss

**Answer:** A. 1 − IoU — With $A\cap B=\varnothing$, IoU is 0 for every nearby position and size, so $1-\mathrm{IoU}$ is constant at 1 and its gradient is 0. GIoU's empty-share term, DIoU's $\rho^2/c^2$ and CIoU's terms all change as the box moves, so they keep pulling it toward the target.

**Question:** A model reaches mean soft Dice 0.8 on a validation set of masks. What is the corresponding IoU of a single mask with Dice exactly 0.8?

- A. 0.667
- B. 0.8
- C. 0.889
- D. 0.64

**Answer:** A. 0.667 — $J=D/(2-D)=0.8/1.2=0.667$. Dice is always at least IoU (equal only at 0 and 1). The mapping is per mask, so a **mean** Dice of 0.8 does not convert exactly to a mean IoU.

**Key points**

- IoU $=|A\cap B|/|A\cup B|$ is scale invariant and is the metric detection and segmentation are scored by; $L=1-\mathrm{IoU}$ trains on it directly.
- For masks use the soft Jaccard (sums of probabilities) or Lovász-softmax; $D=2J/(1+J)$, so Dice and IoU rank masks identically.
- For boxes, $1-\mathrm{IoU}$ has zero gradient when boxes are disjoint. GIoU adds the empty share of the enclosing box $C$.
- DIoU adds $\rho^2/c^2$ and pulls centres together even when one box contains the other; CIoU adds an aspect-ratio term $\alpha v$.
- In PyTorch: `torchvision.ops.generalized_box_iou_loss` / `distance_box_iou_loss` / `complete_box_iou_loss` on $(x_1,y_1,x_2,y_2)$ boxes, usually combined with an $\ell_1$ term.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
