Dice loss

One minus the soft Dice overlap 2Σpg/(Σp+Σg) between a predicted mask and the true one. It is normalized by object size, so small foreground regions are not swamped by background; in practice it is usually paired with cross-entropy.

11 min read1139 wordscore0 viewsupdated 2026-10-11

Definition

Dice loss is L=1−DSCL=1-\mathrm{DSC}, where the soft Dice coefficient DSC=2∑ipigi+ε∑ipi+∑igi+ε\mathrm{DSC}=\dfrac{2\sum_i p_ig_i+\varepsilon}{\sum_i p_i+\sum_i g_i+\varepsilon} measures the overlap between predicted foreground probabilities pi∈[0,1]p_i\in[0,1] and a binary ground-truth mask gi∈{0,1}g_i\in\{0,1\}. It is 0 for a perfect mask and close to 1 when the prediction misses the object.

Most segmentation losses score each pixel on its own and then average. Dice loss scores the whole mask at once: it asks what fraction of the predicted and true foreground overlap. The sums count only foreground mass (predicted and true), so the thousands of easy background pixels never enter the numerator, and the loss means the same thing for a 12-pixel lesion as for a 12,000-pixel organ. That is why it became the standard loss for medical image segmentation, where the structure of interest often covers 1% of the volume or less.

The coefficient is an old similarity index. Dice (1945) and Sørensen (1948) introduced it independently to measure how often two species occur together. On binary sets it is 2∣A∩B∣/(∣A∣+∣B∣)2|A\cap B|/(|A|+|B|), which is exactly the F1 score: 2TP/(2TP+FP+FN)2\mathrm{TP}/(2\mathrm{TP}+\mathrm{FP}+\mathrm{FN}). Milletari, Navab and Ahmadi (2016, V-Net) turned it into a training objective for 3-D prostate MRI segmentation by replacing the sets with soft probabilities. They argued it removes the need to re-weight foreground and background by hand. Sudre et al. (2017) proposed the generalized Dice loss for many classes, and nnU-Net (Isensee et al., 2021), a strong default across many medical segmentation benchmarks, trains with the sum of Dice and cross-entropy.

Formula and gradient

LDice=1−2∑ipigi+ε∑ipi+∑igi+ε∂L∂pi=− 2gi (S+ε)−(2I+ε)(S+ε)2L_{\text{Dice}}=1-\frac{2\sum_i p_i g_i+\varepsilon}{\sum_i p_i+\sum_i g_i+\varepsilon}\qquad\frac{\partial L}{\partial p_i}=-\,\frac{2g_i\,(S+\varepsilon)-(2I+\varepsilon)}{(S+\varepsilon)^2}
pip_i
predicted foreground probability of pixel ii, usually σ(zi)\sigma(z_i) or a softmax channel
gig_i
ground-truth label of pixel ii (1 = foreground)
I=∑ipigiI=\sum_i p_ig_i
soft intersection: probability mass placed on true foreground
S=∑ipi+∑igiS=\sum_i p_i+\sum_i g_i
soft size of prediction plus size of the target
ε\varepsilon
smoothing constant (often 1 or 10−510^{-5}); keeps the ratio defined for empty masks and softens the gradient

The gradient has only two values per image. Every foreground pixel gets ∂L/∂pi=−(2S+ε−2I)/(S+ε)2<0\partial L/\partial p_i=-(2S+\varepsilon-2I)/(S+\varepsilon)^2<0 (raise pp), and every background pixel gets +(2I+ε)/(S+ε)2>0+(2I+\varepsilon)/(S+\varepsilon)^2>0 (lower pp). Which pixel it is does not matter, only which class and how well the mask fits overall. The loss is not a sum over pixels: each pixel's gradient depends on the whole image through II and SS. The network sees logits, so the chain rule adds a factor ∂pi/∂zi=pi(1−pi)\partial p_i/\partial z_i=p_i(1-p_i).

Two common variants. V-Net squares the denominator, ∑ipi2+∑igi2\sum_i p_i^2+\sum_i g_i^2. For a binary gg, ∑gi2=∑gi\sum g_i^2=\sum g_i, so only the prediction term changes. The generalized Dice loss sums intersections and sizes over classes cc with weights wc=1/(∑igic)2w_c=1/(\sum_i g_{ic})^2, so rare classes count as much as common ones. The Tversky loss (Salehi et al., 2017) replaces the denominator with I+α FP+β FNI+\alpha\,\mathrm{FP}+\beta\,\mathrm{FN} to trade false positives against false negatives. With α=β=12\alpha=\beta=\tfrac12 it is Dice again.

At a glance

Task
binary or multi-class semantic segmentation (2-D or 3-D masks)
Range
[0,1][0,1]; 0 for a perfect mask
Hard version
1−F11-\mathrm{F1} of the thresholded mask: 2TP/(2TP+FP+FN)2\mathrm{TP}/(2\mathrm{TP}+\mathrm{FP}+\mathrm{FN})
Imbalance
true negatives never appear, so background size does not dilute the loss
Decomposable
no: a ratio of sums over the whole mask (per image or per batch)
Relation to IoU
D=2J/(1+J)D=2J/(1+J) and J=D/(2−D)J=D/(2-D); they rank masks identically
PyTorch
no built-in in torch.nn; a few lines by hand, or monai.losses.DiceLoss / DiceCELoss
prediction p = σ(z), ground truth outlined∂L/∂zᵢ for Dice32 of 400 pixels are foreground (8.0%)■ push p up · ■ push p down · max |g| 6.3e−3Σpg = 16.38 · Σp = 22.21 · Σg = 32 · ε = 1.000soft Dice = (2Σpg + ε)/(Σp + Σg + ε) = 0.6116 → Dice loss 0.3884 · BCE 0.1080∂L_Dice/∂p: foreground pixel −0.0251, background pixel 0.0111 (same for every pixel of a class)at p ≥ 0.5: pixel accuracy 95.5% · hard Dice 0.667current loss 0.3884 · share of |∂L/∂z| on foreground pixels 72.3%
gradient of
presets
fig.A 20×20 segmentation. The ground truth is the warm disc, and the prediction is a soft disc p = σ(k(r − d)) you can resize, shift and sharpen. The right grid shows ∂L/∂z per pixel for Dice, BCE or their sum. In the default state Dice loss is 0.388 and BCE 0.108. "predict nothing" misses a 32-pixel object, but pixel accuracy is still 92%; Dice loss jumps to 0.940. In "tiny object" (4 pixels, 1% of the image) only 16.9% of BCE's gradient falls on the object, against 42.0% for Dice. In "empty image", ε = 1 gives loss 0.338 for a near-empty prediction (Σp = 0.51), and ε = 0.001 gives 0.998.

Why it handles class imbalance

Mean BCE averages over all pixels, so its value scales with the share of pixels you get wrong. In the figure, predicting nothing on a 32-pixel object costs BCE 0.509. Shrink the object to radius 2 (12 pixels) and BCE for the empty prediction drops to about 0.12, so a model can keep the loss low by ignoring small structures. Dice is a ratio of foreground quantities: missing the object costs nearly 1 at any size (0.94 for 32 pixels, 0.99 for 208). The gradient shows the same effect. For a 1% object, more than 80% of BCE's gradient goes to background pixels.

The same normalization is also the weakness. When SS is small (a tiny object, or an image with no foreground), the denominator is small, so the gradient ∝1/(S+ε)\propto 1/(S+\varepsilon) is large and changes quickly as the prediction moves. In the figure, max ∣∂L/∂z∣|\partial L/\partial z| is 4.7×10−34.7\times10^{-3} for the 32-pixel good match and 2.7×10−22.7\times10^{-2} for the 4-pixel object. Dice training on small structures is known to be noisy for this reason.

The smoothing term ε

ε\varepsilon does two jobs. First, it makes the loss defined when both masks are empty: (0+ε)/(0+ε)=1(0+\varepsilon)/(0+\varepsilon)=1, so a correct empty prediction costs 0 instead of 0/00/0. Second, it decides how much an empty target forgives a few false positives. With g≡0g\equiv0 the loss is 1−ε/(∑p+ε)1-\varepsilon/(\sum p+\varepsilon). For ∑p=0.51\sum p=0.51 that is 0.338 at ε=1\varepsilon=1 and 0.998 at ε=10−3\varepsilon=10^{-3}. A tiny ε\varepsilon treats any trace of probability on an empty image as a total failure. Large ε\varepsilon also pulls every Dice value toward 1 on small objects (0.577 → 0.357 loss for the 4-pixel case going from ε=1\varepsilon=1 to 10), so choose it relative to typical object sizes.

Worked example: six pixels

Take a 1-D "image" with truth g=(0,0,1,1,1,0)g=(0,0,1,1,1,0) and prediction p=(0.1,0.2,0.9,0.7,0.4,0.3)p=(0.1,0.2,0.9,0.7,0.4,0.3), ε=1\varepsilon=1. All numbers below were computed in Node from the figure's library.

pixelgig_ipip_ipigip_ig_i∂LDice/∂pi\partial L_{\text{Dice}}/\partial p_i∂LDice/∂zi\partial L_{\text{Dice}}/\partial z_i∂LBCE/∂zi\partial L_{\text{BCE}}/\partial z_i
100.10+0.1148+0.0103+0.0167
200.20+0.1148+0.0184+0.0333
310.90.9−0.1882−0.0169−0.0167
410.70.7−0.1882−0.0395−0.0500
510.40.4−0.1882−0.0452−0.1000
600.30+0.1148+0.0241+0.0500
Per-pixel quantities. The Dice gradient w.r.t. p has one value per class; w.r.t. the logit z it is multiplied by p(1 − p)

Computing the loss

  1. 1Sums: I=0.9+0.7+0.4=2.0I=0.9+0.7+0.4=2.0, ∑p=2.6\sum p=2.6, ∑g=3\sum g=3, so S=5.6S=5.6.
  2. 2Soft Dice =(2⋅2.0+1)/(5.6+1)=5/6.6=0.7576=(2\cdot2.0+1)/(5.6+1)=5/6.6=0.7576, loss 0.24240.2424. With ε=0\varepsilon=0 it would be 4/5.6=0.71434/5.6=0.7143, loss 0.28570.2857.
  3. 3Gradient: (S+ε)2=43.56(S+\varepsilon)^2=43.56. Foreground pixels get −(2⋅6.6−5)/43.56=−0.1882-(2\cdot6.6-5)/43.56=-0.1882, background pixels 5/43.56=+0.11485/43.56=+0.1148.
  4. 4Mean BCE on the same pixels is 0.34390.3439, so Dice + BCE =0.5863=0.5863. The V-Net squared denominator gives ∑p2=1.6\sum p^2=1.6 and Dice 4/4.6=0.86964/4.6=0.8696 (with ε=0\varepsilon=0).
  5. 5Soft IoU is I/(S−I)=2/3.6=0.5556I/(S-I)=2/3.6=0.5556, and 2J/(1+J)=0.71432J/(1+J)=0.7143: the ε=0\varepsilon=0 Dice above.

Pixel 5 (p=0.4p=0.4, true foreground) is the most wrong pixel. Measured w.r.t. pp, Dice treats it exactly like pixel 3 (p=0.9p=0.9). Only the sigmoid factor p(1−p)p(1-p) makes its logit gradient larger: −0.0452-0.0452 against −0.0169-0.0169. BCE, by contrast, gives it (p−g)/n=−0.1(p-g)/n=-0.1, six times pixel 3's.

python
1import torch, torch.nn.functional as F
2
3def soft_dice_loss(logits, target, eps=1.0, dims=(-2, -1)):
4 """Binary soft Dice. logits, target: same shape, e.g. (N, H, W); target in {0, 1}."""
5 p = torch.sigmoid(logits)
6 inter = (p * target).sum(dim=dims)
7 size = p.sum(dim=dims) + target.sum(dim=dims)
8 return (1 - (2 * inter + eps) / (size + eps)).mean() # Dice per image, then mean
9
10# The six-pixel example
11p = torch.tensor([[0.1, 0.2, 0.9, 0.7, 0.4, 0.3]])
12g = torch.tensor([[0., 0., 1., 1., 1., 0.]])
13z = torch.logit(p).requires_grad_()
14dice = soft_dice_loss(z, g, dims=-1)
15bce = F.binary_cross_entropy_with_logits(z, g)
16print(round(dice.item(), 4), round(bce.item(), 4)) # 0.2424 0.3439
17dice.backward()
18print(z.grad) # tensor([[ 0.0103, 0.0184, -0.0169, -0.0395, -0.0452, 0.0241]])
19
20def multiclass_dice_loss(logits, labels, eps=1.0, include_background=False):
21 """logits (N, C, H, W); labels (N, H, W) int64 class ids."""
22 C = logits.shape[1]
23 p = logits.softmax(dim=1)
24 y = F.one_hot(labels, C).permute(0, 3, 1, 2).to(p.dtype)
25 inter = (p * y).sum(dim=(2, 3)) # (N, C)
26 size = p.sum(dim=(2, 3)) + y.sum(dim=(2, 3))
27 dice = (2 * inter + eps) / (size + eps)
28 if not include_background:
29 dice = dice[:, 1:]
30 return 1 - dice.mean()
31
32def dice_ce_loss(logits, labels, w_dice=1.0, w_ce=1.0):
33 return w_dice * multiclass_dice_loss(logits, labels) + w_ce * F.cross_entropy(logits, labels)
34
35# Library alternative (labels as (N, 1, H, W)):
36# from monai.losses import DiceCELoss
37# loss_fn = DiceCELoss(to_onehot_y=True, softmax=True)
Binary and multi-class soft Dice, and the Dice + CE combination

Use when

  • +Normalized by object size: small foreground regions are not drowned out by background
  • +Directly targets the Dice/F1 overlap metric that segmentation is evaluated on
  • +No class weights to tune for the foreground/background ratio
  • +Easy to implement for any number of classes and dimensions

Avoid when

  • −Non-decomposable: gradients depend on the whole mask, and are large and noisy when SS is small
  • −Logit gradient saturates through p(1−p)p(1-p); confident misses learn slowly without CE
  • −Ill-defined on empty masks; behaviour there is set by ε\varepsilon
  • −Produces poorly calibrated probabilities

Check yourself

An image is 99% background. A model predicts background everywhere. Which statement is true?

Check yourself

In one image, two background pixels have p=0.05p=0.05 and p=0.6p=0.6. How do their Dice gradients w.r.t. pp compare?

Remember

  • ✓L=1−2∑pg+ε∑p+∑g+εL=1-\dfrac{2\sum pg+\varepsilon}{\sum p+\sum g+\varepsilon}: one minus a soft F1 between predicted probabilities and the true mask.
  • ✓True negatives never appear, so the loss is normalized by object size and a 1% foreground is not swamped by background.
  • ✓The gradient has one value per class per image; through the sigmoid it is scaled by p(1−p)p(1-p) and saturates on confident mistakes.
  • ✓ε\varepsilon makes empty masks well-defined and sets how harshly false positives on empty images are punished.
  • ✓Use Dice + cross-entropy in practice (nnU-Net's default); Dice and IoU are monotone in each other, D=2J/(1+J)D=2J/(1+J).
← back to Deep Learning roadmap