# Dice loss

> One minus the soft Dice overlap 2Σpg/(Σp+Σg) between a predicted mask and the true one. It is normalized by object size, so small foreground regions are not swamped by background; in practice it is usually paired with cross-entropy.

Source: https://www.mlfoundry.in/learn/dice-loss  
Section: Deep Learning › Loss Functions  
Level: core · 11 min read · updated 2026-10-11

**Dice loss:** is $L=1-\mathrm{DSC}$, where the **soft Dice coefficient** $\mathrm{DSC}=\dfrac{2\sum_i p_ig_i+\varepsilon}{\sum_i p_i+\sum_i g_i+\varepsilon}$ measures the overlap between predicted foreground probabilities $p_i\in[0,1]$ and a binary ground-truth mask $g_i\in\{0,1\}$. It is 0 for a perfect mask and close to 1 when the prediction misses the object.

Most segmentation losses score each pixel on its own and then average. Dice loss scores the **whole mask at once**: it asks what fraction of the predicted and true foreground overlap. The sums count only foreground mass (predicted and true), so the thousands of easy background pixels never enter the numerator, and the loss means the same thing for a 12-pixel lesion as for a 12,000-pixel organ. That is why it became the standard loss for medical image segmentation, where the structure of interest often covers 1% of the volume or less.

The coefficient is an old similarity index. Dice (1945) and Sørensen (1948) introduced it independently to measure how often two species occur together. On binary sets it is $2|A\cap B|/(|A|+|B|)$, which is exactly the **F1 score**: $2\mathrm{TP}/(2\mathrm{TP}+\mathrm{FP}+\mathrm{FN})$. Milletari, Navab and Ahmadi (2016, **V-Net**) turned it into a training objective for 3-D prostate MRI segmentation by replacing the sets with soft probabilities. They argued it removes the need to re-weight foreground and background by hand. Sudre et al. (2017) proposed the **generalized Dice loss** for many classes, and nnU-Net (Isensee et al., 2021), a strong default across many medical segmentation benchmarks, trains with the **sum of Dice and cross-entropy**.

## Formula and gradient

$$
L_{\text{Dice}}=1-\frac{2\sum_i p_i g_i+\varepsilon}{\sum_i p_i+\sum_i g_i+\varepsilon}\qquad\frac{\partial L}{\partial p_i}=-\,\frac{2g_i\,(S+\varepsilon)-(2I+\varepsilon)}{(S+\varepsilon)^2}
$$

- $p_i$: predicted foreground probability of pixel $i$, usually $\sigma(z_i)$ or a softmax channel
- $g_i$: ground-truth label of pixel $i$ (1 = foreground)
- $I=\sum_i p_ig_i$: soft intersection: probability mass placed on true foreground
- $S=\sum_i p_i+\sum_i g_i$: soft size of prediction plus size of the target
- $\varepsilon$: smoothing constant (often 1 or $10^{-5}$); keeps the ratio defined for empty masks and softens the gradient

The gradient has only **two values per image**. Every foreground pixel gets $\partial L/\partial p_i=-(2S+\varepsilon-2I)/(S+\varepsilon)^2<0$ (raise $p$), and every background pixel gets $+(2I+\varepsilon)/(S+\varepsilon)^2>0$ (lower $p$). Which pixel it is does not matter, only which class and how well the mask fits overall. The loss is **not a sum over pixels**: each pixel's gradient depends on the whole image through $I$ and $S$. The network sees logits, so the chain rule adds a factor $\partial p_i/\partial z_i=p_i(1-p_i)$.

Two common variants. V-Net squares the denominator, $\sum_i p_i^2+\sum_i g_i^2$. For a binary $g$, $\sum g_i^2=\sum g_i$, so only the prediction term changes. The **generalized Dice loss** sums intersections and sizes over classes $c$ with weights $w_c=1/(\sum_i g_{ic})^2$, so rare classes count as much as common ones. The **Tversky loss** (Salehi et al., 2017) replaces the denominator with $I+\alpha\,\mathrm{FP}+\beta\,\mathrm{FN}$ to trade false positives against false negatives. With $\alpha=\beta=\tfrac12$ it is Dice again.

**At a glance**

- **Task:** binary or multi-class semantic segmentation (2-D or 3-D masks)
- **Range:** $[0,1]$; 0 for a perfect mask
- **Hard version:** $1-\mathrm{F1}$ of the thresholded mask: $2\mathrm{TP}/(2\mathrm{TP}+\mathrm{FP}+\mathrm{FN})$
- **Imbalance:** true negatives never appear, so background size does not dilute the loss
- **Decomposable:** no: a ratio of sums over the whole mask (per image or per batch)
- **Relation to IoU:** $D=2J/(1+J)$ and $J=D/(2-D)$; they rank masks identically
- **PyTorch:** no built-in in `torch.nn`; a few lines by hand, or `monai.losses.DiceLoss` / `DiceCELoss`

> Figure: A 20×20 segmentation. The ground truth is the warm disc, and the prediction is a soft disc p = σ(k(r − d)) you can resize, shift and sharpen. The right grid shows ∂L/∂z per pixel for Dice, BCE or their sum. In the default state Dice loss is 0.388 and BCE 0.108. "predict nothing" misses a 32-pixel object, but pixel accuracy is still 92%; Dice loss jumps to 0.940. In "tiny object" (4 pixels, 1% of the image) only 16.9% of BCE's gradient falls on the object, against 42.0% for Dice. In "empty image", ε = 1 gives loss 0.338 for a near-empty prediction (Σp = 0.51), and ε = 0.001 gives 0.998.

## Why it handles class imbalance

Mean BCE averages over **all** pixels, so its value scales with the share of pixels you get wrong. In the figure, predicting nothing on a 32-pixel object costs BCE 0.509. Shrink the object to radius 2 (12 pixels) and BCE for the empty prediction drops to about 0.12, so a model can keep the loss low by ignoring small structures. Dice is a ratio of foreground quantities: missing the object costs nearly 1 at any size (0.94 for 32 pixels, 0.99 for 208). The gradient shows the same effect. For a 1% object, more than 80% of BCE's gradient goes to background pixels.

The same normalization is also the weakness. When $S$ is small (a tiny object, or an image with no foreground), the denominator is small, so the gradient $\propto 1/(S+\varepsilon)$ is large and changes quickly as the prediction moves. In the figure, max $|\partial L/\partial z|$ is $4.7\times10^{-3}$ for the 32-pixel good match and $2.7\times10^{-2}$ for the 4-pixel object. Dice training on small structures is known to be noisy for this reason.

## The smoothing term ε

$\varepsilon$ does two jobs. First, it makes the loss defined when both masks are empty: $(0+\varepsilon)/(0+\varepsilon)=1$, so a correct empty prediction costs 0 instead of $0/0$. Second, it decides how much an empty target forgives a few false positives. With $g\equiv0$ the loss is $1-\varepsilon/(\sum p+\varepsilon)$. For $\sum p=0.51$ that is 0.338 at $\varepsilon=1$ and 0.998 at $\varepsilon=10^{-3}$. A tiny $\varepsilon$ treats any trace of probability on an empty image as a total failure. Large $\varepsilon$ also pulls every Dice value toward 1 on small objects (0.577 → 0.357 loss for the 4-pixel case going from $\varepsilon=1$ to 10), so choose it relative to typical object sizes.

## Worked example: six pixels

Take a 1-D "image" with truth $g=(0,0,1,1,1,0)$ and prediction $p=(0.1,0.2,0.9,0.7,0.4,0.3)$, $\varepsilon=1$. All numbers below were computed in Node from the figure's library.

Per-pixel quantities. The Dice gradient w.r.t. p has one value per class; w.r.t. the logit z it is multiplied by p(1 − p)

| pixel | $g_i$ | $p_i$ | $p_ig_i$ | $\partial L_{\text{Dice}}/\partial p_i$ | $\partial L_{\text{Dice}}/\partial z_i$ | $\partial L_{\text{BCE}}/\partial z_i$ |
| --- | --- | --- | --- | --- | --- | --- |
| 1 | 0 | 0.1 | 0 | +0.1148 | +0.0103 | +0.0167 |
| 2 | 0 | 0.2 | 0 | +0.1148 | +0.0184 | +0.0333 |
| 3 | 1 | 0.9 | 0.9 | −0.1882 | −0.0169 | −0.0167 |
| 4 | 1 | 0.7 | 0.7 | −0.1882 | −0.0395 | −0.0500 |
| 5 | 1 | 0.4 | 0.4 | −0.1882 | −0.0452 | −0.1000 |
| 6 | 0 | 0.3 | 0 | +0.1148 | +0.0241 | +0.0500 |

**Computing the loss**

1. Sums: $I=0.9+0.7+0.4=2.0$, $\sum p=2.6$, $\sum g=3$, so $S=5.6$.
2. Soft Dice $=(2\cdot2.0+1)/(5.6+1)=5/6.6=0.7576$, loss $0.2424$. With $\varepsilon=0$ it would be $4/5.6=0.7143$, loss $0.2857$.
3. Gradient: $(S+\varepsilon)^2=43.56$. Foreground pixels get $-(2\cdot6.6-5)/43.56=-0.1882$, background pixels $5/43.56=+0.1148$.
4. Mean BCE on the same pixels is $0.3439$, so Dice + BCE $=0.5863$. The V-Net squared denominator gives $\sum p^2=1.6$ and Dice $4/4.6=0.8696$ (with $\varepsilon=0$).
5. Soft IoU is $I/(S-I)=2/3.6=0.5556$, and $2J/(1+J)=0.7143$: the $\varepsilon=0$ Dice above.

Pixel 5 ($p=0.4$, true foreground) is the most wrong pixel. Measured w.r.t. $p$, Dice treats it exactly like pixel 3 ($p=0.9$). Only the sigmoid factor $p(1-p)$ makes its logit gradient larger: $-0.0452$ against $-0.0169$. BCE, by contrast, gives it $(p-g)/n=-0.1$, six times pixel 3's.

```python
import torch, torch.nn.functional as F

def soft_dice_loss(logits, target, eps=1.0, dims=(-2, -1)):
    """Binary soft Dice. logits, target: same shape, e.g. (N, H, W); target in {0, 1}."""
    p = torch.sigmoid(logits)
    inter = (p * target).sum(dim=dims)
    size = p.sum(dim=dims) + target.sum(dim=dims)
    return (1 - (2 * inter + eps) / (size + eps)).mean()   # Dice per image, then mean

# The six-pixel example
p = torch.tensor([[0.1, 0.2, 0.9, 0.7, 0.4, 0.3]])
g = torch.tensor([[0., 0., 1., 1., 1., 0.]])
z = torch.logit(p).requires_grad_()
dice = soft_dice_loss(z, g, dims=-1)
bce = F.binary_cross_entropy_with_logits(z, g)
print(round(dice.item(), 4), round(bce.item(), 4))      # 0.2424 0.3439
dice.backward()
print(z.grad)   # tensor([[ 0.0103,  0.0184, -0.0169, -0.0395, -0.0452,  0.0241]])

def multiclass_dice_loss(logits, labels, eps=1.0, include_background=False):
    """logits (N, C, H, W); labels (N, H, W) int64 class ids."""
    C = logits.shape[1]
    p = logits.softmax(dim=1)
    y = F.one_hot(labels, C).permute(0, 3, 1, 2).to(p.dtype)
    inter = (p * y).sum(dim=(2, 3))                       # (N, C)
    size = p.sum(dim=(2, 3)) + y.sum(dim=(2, 3))
    dice = (2 * inter + eps) / (size + eps)
    if not include_background:
        dice = dice[:, 1:]
    return 1 - dice.mean()

def dice_ce_loss(logits, labels, w_dice=1.0, w_ce=1.0):
    return w_dice * multiclass_dice_loss(logits, labels) + w_ce * F.cross_entropy(logits, labels)

# Library alternative (labels as (N, 1, H, W)):
# from monai.losses import DiceCELoss
# loss_fn = DiceCELoss(to_onehot_y=True, softmax=True)
```

Binary and multi-class soft Dice, and the Dice + CE combination

> **Dice alone can stall on confident mistakes:** The Dice gradient w.r.t. a logit carries the factor $p_i(1-p_i)$. A foreground pixel predicted with $p\approx0$ gets almost no push. In the figure's "predict nothing" preset, the rim of the missed object receives $|\partial L/\partial z|\approx9\times10^{-6}$ from Dice and $2.2\times10^{-3}$ from BCE, about 240 times more. BCE's logit gradient $p-g$ never saturates. Adding cross-entropy gives stable early training and a calibrated per-pixel signal, while Dice keeps small structures from being ignored. That is why **Dice + CE** is the usual default.

> **Per image, per batch, and empty masks:** Computing Dice per image and then averaging makes every image count equally. Images with no foreground then give a loss that depends almost entirely on $\varepsilon$. Summing $I$ and $S$ over the whole batch ("batch Dice", an option in nnU-Net) is smoother, but it lets large objects dominate. For multi-class problems, the background channel usually has a Dice near 1 and only adds noise, so it is often excluded, as in `include_background=False`.

> **A differentiable F1:** With hard 0/1 predictions, Dice is F1: $2\mathrm{TP}/(2\mathrm{TP}+\mathrm{FP}+\mathrm{FN})$. Soft Dice replaces TP by $\sum p g$, FP by $\sum p(1-g)$ and FN by $\sum(1-p)g$. Training with it optimizes a smooth stand-in for the metric most segmentation benchmarks report. True negatives never enter, which is exactly why background size does not matter.

> **Calibration:** Dice rewards overlap, not honest probabilities. Networks trained on Dice alone tend to produce over-confident, nearly binary outputs whose values are poor probability estimates. If you need uncertainty or will threshold at different operating points, keep a cross-entropy term or recalibrate.

**Strengths**

- Normalized by object size: small foreground regions are not drowned out by background
- Directly targets the Dice/F1 overlap metric that segmentation is evaluated on
- No class weights to tune for the foreground/background ratio
- Easy to implement for any number of classes and dimensions

**Limitations**

- Non-decomposable: gradients depend on the whole mask, and are large and noisy when $S$ is small
- Logit gradient saturates through $p(1-p)$; confident misses learn slowly without CE
- Ill-defined on empty masks; behaviour there is set by $\varepsilon$
- Produces poorly calibrated probabilities

**Question:** An image is 99% background. A model predicts background everywhere. Which statement is true?

- A. Pixel accuracy is about 99% and soft Dice loss is close to 1
- B. Both accuracy and Dice loss look good
- C. Dice loss is close to 0 because most pixels are right
- D. Dice loss is undefined because the prediction is empty

**Answer:** A. Pixel accuracy is about 99% and soft Dice loss is close to 1 — True negatives never enter Dice. With $I\approx0$ the loss is about $1-\varepsilon/(\sum g+\varepsilon)$, close to 1 for any object of more than a few pixels. Accuracy counts the background, so it stays near 99%. Only an **empty target** with an empty prediction makes the ratio depend on $\varepsilon$ alone.

**Question:** In one image, two background pixels have $p=0.05$ and $p=0.6$. How do their Dice gradients w.r.t. $p$ compare?

- A. They are equal
- B. The p = 0.6 pixel's gradient is 12 times larger
- C. The p = 0.05 pixel's gradient is larger
- D. Background pixels get zero gradient from Dice

**Answer:** A. They are equal — For a background pixel $\partial L/\partial p_i=(2I+\varepsilon)/(S+\varepsilon)^2$, the same for every background pixel in the image. Per-pixel differences appear only through the sigmoid factor $p(1-p)$ when you go back to the logits. This is unlike BCE, whose logit gradient $p-g$ grows with the error.

**Key points**

- $L=1-\dfrac{2\sum pg+\varepsilon}{\sum p+\sum g+\varepsilon}$: one minus a soft F1 between predicted probabilities and the true mask.
- True negatives never appear, so the loss is normalized by object size and a 1% foreground is not swamped by background.
- The gradient has one value per class per image; through the sigmoid it is scaled by $p(1-p)$ and saturates on confident mistakes.
- $\varepsilon$ makes empty masks well-defined and sets how harshly false positives on empty images are punished.
- Use **Dice + cross-entropy** in practice (nnU-Net's default); Dice and IoU are monotone in each other, $D=2J/(1+J)$.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
