Definition
Dice loss is , where the soft Dice coefficient measures the overlap between predicted foreground probabilities and a binary ground-truth mask . It is 0 for a perfect mask and close to 1 when the prediction misses the object.
Most segmentation losses score each pixel on its own and then average. Dice loss scores the whole mask at once: it asks what fraction of the predicted and true foreground overlap. The sums count only foreground mass (predicted and true), so the thousands of easy background pixels never enter the numerator, and the loss means the same thing for a 12-pixel lesion as for a 12,000-pixel organ. That is why it became the standard loss for medical image segmentation, where the structure of interest often covers 1% of the volume or less.
The coefficient is an old similarity index. Dice (1945) and Sørensen (1948) introduced it independently to measure how often two species occur together. On binary sets it is , which is exactly the F1 score: . Milletari, Navab and Ahmadi (2016, V-Net) turned it into a training objective for 3-D prostate MRI segmentation by replacing the sets with soft probabilities. They argued it removes the need to re-weight foreground and background by hand. Sudre et al. (2017) proposed the generalized Dice loss for many classes, and nnU-Net (Isensee et al., 2021), a strong default across many medical segmentation benchmarks, trains with the sum of Dice and cross-entropy.
Formula and gradient
- predicted foreground probability of pixel , usually or a softmax channel
- ground-truth label of pixel (1 = foreground)
- soft intersection: probability mass placed on true foreground
- soft size of prediction plus size of the target
- smoothing constant (often 1 or ); keeps the ratio defined for empty masks and softens the gradient
The gradient has only two values per image. Every foreground pixel gets (raise ), and every background pixel gets (lower ). Which pixel it is does not matter, only which class and how well the mask fits overall. The loss is not a sum over pixels: each pixel's gradient depends on the whole image through and . The network sees logits, so the chain rule adds a factor .
Two common variants. V-Net squares the denominator, . For a binary , , so only the prediction term changes. The generalized Dice loss sums intersections and sizes over classes with weights , so rare classes count as much as common ones. The Tversky loss (Salehi et al., 2017) replaces the denominator with to trade false positives against false negatives. With it is Dice again.
At a glance
- Task
- binary or multi-class semantic segmentation (2-D or 3-D masks)
- Range
- ; 0 for a perfect mask
- Hard version
- of the thresholded mask:
- Imbalance
- true negatives never appear, so background size does not dilute the loss
- Decomposable
- no: a ratio of sums over the whole mask (per image or per batch)
- Relation to IoU
- and ; they rank masks identically
- PyTorch
- no built-in in
torch.nn; a few lines by hand, ormonai.losses.DiceLoss/DiceCELoss
Why it handles class imbalance
Mean BCE averages over all pixels, so its value scales with the share of pixels you get wrong. In the figure, predicting nothing on a 32-pixel object costs BCE 0.509. Shrink the object to radius 2 (12 pixels) and BCE for the empty prediction drops to about 0.12, so a model can keep the loss low by ignoring small structures. Dice is a ratio of foreground quantities: missing the object costs nearly 1 at any size (0.94 for 32 pixels, 0.99 for 208). The gradient shows the same effect. For a 1% object, more than 80% of BCE's gradient goes to background pixels.
The same normalization is also the weakness. When is small (a tiny object, or an image with no foreground), the denominator is small, so the gradient is large and changes quickly as the prediction moves. In the figure, max is for the 32-pixel good match and for the 4-pixel object. Dice training on small structures is known to be noisy for this reason.
The smoothing term ε
does two jobs. First, it makes the loss defined when both masks are empty: , so a correct empty prediction costs 0 instead of . Second, it decides how much an empty target forgives a few false positives. With the loss is . For that is 0.338 at and 0.998 at . A tiny treats any trace of probability on an empty image as a total failure. Large also pulls every Dice value toward 1 on small objects (0.577 → 0.357 loss for the 4-pixel case going from to 10), so choose it relative to typical object sizes.
Worked example: six pixels
Take a 1-D "image" with truth and prediction , . All numbers below were computed in Node from the figure's library.
| pixel | ||||||
|---|---|---|---|---|---|---|
| 1 | 0 | 0.1 | 0 | +0.1148 | +0.0103 | +0.0167 |
| 2 | 0 | 0.2 | 0 | +0.1148 | +0.0184 | +0.0333 |
| 3 | 1 | 0.9 | 0.9 | −0.1882 | −0.0169 | −0.0167 |
| 4 | 1 | 0.7 | 0.7 | −0.1882 | −0.0395 | −0.0500 |
| 5 | 1 | 0.4 | 0.4 | −0.1882 | −0.0452 | −0.1000 |
| 6 | 0 | 0.3 | 0 | +0.1148 | +0.0241 | +0.0500 |
Computing the loss
- 1Sums: , , , so .
- 2Soft Dice , loss . With it would be , loss .
- 3Gradient: . Foreground pixels get , background pixels .
- 4Mean BCE on the same pixels is , so Dice + BCE . The V-Net squared denominator gives and Dice (with ).
- 5Soft IoU is , and : the Dice above.
Pixel 5 (, true foreground) is the most wrong pixel. Measured w.r.t. , Dice treats it exactly like pixel 3 (). Only the sigmoid factor makes its logit gradient larger: against . BCE, by contrast, gives it , six times pixel 3's.
1import torch, torch.nn.functional as F23def soft_dice_loss(logits, target, eps=1.0, dims=(-2, -1)):4 """Binary soft Dice. logits, target: same shape, e.g. (N, H, W); target in {0, 1}."""5 p = torch.sigmoid(logits)6 inter = (p * target).sum(dim=dims)7 size = p.sum(dim=dims) + target.sum(dim=dims)8 return (1 - (2 * inter + eps) / (size + eps)).mean() # Dice per image, then mean910# The six-pixel example11p = torch.tensor([[0.1, 0.2, 0.9, 0.7, 0.4, 0.3]])12g = torch.tensor([[0., 0., 1., 1., 1., 0.]])13z = torch.logit(p).requires_grad_()14dice = soft_dice_loss(z, g, dims=-1)15bce = F.binary_cross_entropy_with_logits(z, g)16print(round(dice.item(), 4), round(bce.item(), 4)) # 0.2424 0.343917dice.backward()18print(z.grad) # tensor([[ 0.0103, 0.0184, -0.0169, -0.0395, -0.0452, 0.0241]])1920def multiclass_dice_loss(logits, labels, eps=1.0, include_background=False):21 """logits (N, C, H, W); labels (N, H, W) int64 class ids."""22 C = logits.shape[1]23 p = logits.softmax(dim=1)24 y = F.one_hot(labels, C).permute(0, 3, 1, 2).to(p.dtype)25 inter = (p * y).sum(dim=(2, 3)) # (N, C)26 size = p.sum(dim=(2, 3)) + y.sum(dim=(2, 3))27 dice = (2 * inter + eps) / (size + eps)28 if not include_background:29 dice = dice[:, 1:]30 return 1 - dice.mean()3132def dice_ce_loss(logits, labels, w_dice=1.0, w_ce=1.0):33 return w_dice * multiclass_dice_loss(logits, labels) + w_ce * F.cross_entropy(logits, labels)3435# Library alternative (labels as (N, 1, H, W)):36# from monai.losses import DiceCELoss37# loss_fn = DiceCELoss(to_onehot_y=True, softmax=True)Use when
- +Normalized by object size: small foreground regions are not drowned out by background
- +Directly targets the Dice/F1 overlap metric that segmentation is evaluated on
- +No class weights to tune for the foreground/background ratio
- +Easy to implement for any number of classes and dimensions
Avoid when
- −Non-decomposable: gradients depend on the whole mask, and are large and noisy when is small
- −Logit gradient saturates through ; confident misses learn slowly without CE
- −Ill-defined on empty masks; behaviour there is set by
- −Produces poorly calibrated probabilities
Check yourself
An image is 99% background. A model predicts background everywhere. Which statement is true?
Check yourself
In one image, two background pixels have and . How do their Dice gradients w.r.t. compare?
Remember
- ✓: one minus a soft F1 between predicted probabilities and the true mask.
- ✓True negatives never appear, so the loss is normalized by object size and a 1% foreground is not swamped by background.
- ✓The gradient has one value per class per image; through the sigmoid it is scaled by and saturates on confident mistakes.
- ✓ makes empty masks well-defined and sets how harshly false positives on empty images are punished.
- ✓Use Dice + cross-entropy in practice (nnU-Net's default); Dice and IoU are monotone in each other, .