# U-Net++

> U-Net++ (Zhou et al., 2018) replaces U-Net's skips with a grid of nested, densely connected convolution nodes, supervises its four full-resolution outputs, and can be pruned at inference to the sub-network of any depth.

Source: https://www.mlfoundry.in/learn/u-net-plus-plus  
Section: Deep Learning › Deep Learning for Computer Vision  
Level: advanced · 7 min read · updated 2026-10-15

**U-Net++:** is an encoder–decoder segmentation network in which the plain skip connections of **U-Net** are replaced by **nested, dense skip pathways**: a grid of convolution nodes $X^{i,j}$ that fuse every earlier node on the same level with an upsampled node from the level below. It was introduced by Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh and Jianming Liang in "UNet++: A Nested U-Net Architecture for Medical Image Segmentation" (DLMIA workshop, MICCAI 2018), with an extended journal version in IEEE Transactions on Medical Imaging (2020).

The task is the same as U-Net's: an image in, a per-pixel mask out, often from a few hundred labelled medical images. U-Net concatenates the encoder map at level $i$ directly with the decoder map at level $i$. The authors argue that this joins features that are semantically far apart: shallow encoder features are edges and textures, the decoder features have passed through the whole network.

A second problem is depth. The best U-Net depth differs between datasets and is found by training several networks. U-Net++ embeds U-Nets of depths 1 to 4 in one model that shares a single encoder.

## Nested skip pathways

Index nodes by level $i$ (downsampling step, top row $i=0$) and column $j$ (convolution node along the skip pathway). Column $j=0$ is the encoder. Every other node concatenates all earlier nodes of its row with the upsampled output of the node diagonally below:

$$
x^{i,j}=\begin{cases}\mathcal H\big(\mathcal D(x^{i-1,0})\big), & j=0\\[2pt] \mathcal H\Big(\big[\,[x^{i,k}]_{k=0}^{j-1},\ \mathcal U\big(x^{i+1,j-1}\big)\big]\Big), & j>0\end{cases}
$$

- $\mathcal H$: a convolution block (two $3\times3$ convolutions with activations); its input for $j=0$ is $\mathcal D$, $2\times2$ max-pooling of the node above
- $\mathcal U$: upsampling by 2, a $2\times2$ stride-2 transposed convolution in the reference code
- $[\cdot]$: channel concatenation; node $X^{i,j}$ receives $(j+1)$ maps of the level's width

U-Net is the special case that keeps only the outer edge of this grid: the encoder $X^{i,0}$ and the nodes with $i+j=4$, each fed by one skip. The inner nodes are new. They form a dense block along each row, in the spirit of **DenseNet**, so the encoder map is refined by a few convolutions before it meets the decoder.

> Figure: The U-Net++ grid with the paper's filter widths (32 to 512) at a 96×96 input. Blue nodes are the encoder, orange nodes the decoder that a depth-4 U-Net also has, green nodes the nested additions. Dashed arrows and arcs are the dense skips; diagonal arrows are 2×2 up-convolutions. Each X0,j for j = 1…4 has its own 1×1 sigmoid head. The violet cuts mark the pruned networks L1 to L3. Hover a node to see what it concatenates.

## Deep supervision and pruning

The four top-row nodes $X^{0,1},\dots,X^{0,4}$ are full-resolution outputs of U-Nets of depth 1 to 4. With **deep supervision**, each gets a $1\times1$ convolution and a sigmoid, and each output is trained against the same mask. The paper uses a combined binary cross-entropy and Dice loss on every head:

$$
\mathcal L(Y,\hat Y)=-\frac1N\sum_{b=1}^{N}\Big(\tfrac12\,Y_b\log\hat Y_b+\frac{2\,Y_b\hat Y_b}{Y_b+\hat Y_b}\Big)
$$

- $Y_b,\ \hat Y_b$: flattened ground-truth mask and predicted probabilities of image $b$; the products and sums run over pixels
- $N$: batch size; the loss is applied to each of the four heads

At test time there are two modes. **Accurate mode** averages the four outputs. **Fast mode** keeps one output $X^{0,d}$ and deletes every node with $i+j>d$. This is exact, not an approximation: by the node rule, $X^{0,d}$ depends only on nodes with $i+j\le d$. The deleted nodes still mattered during training, because their gradients flowed back into the shared nodes. The result is called U-Net++ $L^d$.

**Configuration and reported results (2018 paper)**

- **Widths:** 32, 64, 128, 256, 512 filters at levels 0 to 4; four 2× downsamplings
- **Parameters:** U-Net 7.76M, U-Net++ 9.04M (with or without deep supervision); the counts follow from the widths above
- **Tasks:** four medical segmentation tasks: cell nuclei (microscopy), colon polyps (colonoscopy), liver (CT), lung nodules (CT, LIDC-IDRI)
- **Baselines:** U-Net and a **wide U-Net** with more filters, to show that the gain is not just from parameter count
- **Result:** U-Net++ with deep supervision: average IoU gain of 3.9 points over U-Net and 3.4 over wide U-Net
- **Pruning:** U-Net++ $L^3$ cut inference time by 32.2% on average for a 0.6-point IoU drop

> **Matched parameters are not matched compute:** The wide U-Net control gives U-Net more filters per level so that the comparison is not about weight count. But weights and compute live in different places: U-Net++ adds its nodes in the high-resolution rows, where convolutions are cheap in parameters and expensive in multiply-accumulates. The figure below makes the gap concrete.

> Figure: Parameters and multiply-accumulates (MACs) of U-Net and U-Net++ at depths 1 to 4, counted layer by layer for a 96×96 RGB input. At the default, U-Net++ L4 has 15 nodes, 9.04M parameters and 4.19 GMACs, against 7.76M and 1.70 GMACs for U-Net: 17% more weights but 2.47× the compute. The extra cost sits in the top row, which holds 40.5% of all MACs and only 2.3% of the parameters. Click X0,4: it concatenates 160 channels and costs 63,584 parameters. Drag the depth to 3: U-Net++ L3 needs 2.21M parameters and 2.46 GMACs, 58.7% of the full model.

## Worked example: the cost of one nested node

Node $X^{0,2}$ at $96\times96$ concatenates $X^{0,0}$, $X^{0,1}$ (32 channels each) and $\mathcal U(X^{1,1})$, which maps 64 channels down to 32. A convolution with $c_{in}$ inputs, $c_{out}$ outputs and a $k\times k$ kernel has $c_{in}k^2c_{out}+c_{out}$ parameters and $c_{in}k^2c_{out}$ MACs per output pixel.

Node X0,2 of U-Net++ (96×96 input, 9,216 output pixels per layer)

| Layer | Channels | Parameters | MACs |
| --- | --- | --- | --- |
| up-conv 2×2 of X1,1 | 64 → 32 | 64·4·32 + 32 = 8,224 | 64·32·9,216 = 18.9M |
| 3×3 conv on the concat | 32 + 32 + 32 = 96 → 32 | 96·9·32 + 32 = 27,680 | 96·9·32·9,216 = 254.8M |
| 3×3 conv | 32 → 32 | 32·9·32 + 32 = 9,248 | 32·9·32·9,216 = 84.9M |
| **node total** |  | **45,152** | **358.6M** |

The node has under 0.5% of the model's 9.04M parameters but 8.6% of its 4.19 GMACs. Nested nodes at full resolution are cheap in memory for weights and expensive in compute and activations, the reverse of the deep levels, where levels 3 and 4 hold 74% of the parameters.

```python
import torch, torch.nn as nn, torch.nn.functional as F

def block(cin, cout):                                  # H: two 3x3 conv + ReLU
    return nn.Sequential(nn.Conv2d(cin, cout, 3, padding=1), nn.ReLU(inplace=True),
                         nn.Conv2d(cout, cout, 3, padding=1), nn.ReLU(inplace=True))

class UNetPP(nn.Module):
    def __init__(self, cin=3, f=(32, 64, 128, 256, 512), L=4):
        super().__init__()
        self.L, self.pool = L, nn.MaxPool2d(2)
        self.node, self.up = nn.ModuleDict(), nn.ModuleDict()
        for i in range(L + 1):
            self.node[f"{i}_0"] = block(cin if i == 0 else f[i - 1], f[i])
        for j in range(1, L + 1):
            for i in range(L + 1 - j):
                self.up[f"{i}_{j}"] = nn.ConvTranspose2d(f[i + 1], f[i], 2, stride=2)
                self.node[f"{i}_{j}"] = block((j + 1) * f[i], f[i])
        self.head = nn.ModuleList(nn.Conv2d(f[0], 1, 1) for _ in range(L))

    def forward(self, x, depth=None):                  # depth d < L runs only nodes with i + j <= d
        d, X = depth or self.L, {}
        for i in range(d + 1):
            X[i, 0] = self.node[f"{i}_0"](x if i == 0 else self.pool(X[i - 1, 0]))
        for j in range(1, d + 1):
            for i in range(d + 1 - j):
                skips = [X[i, k] for k in range(j)] + [self.up[f"{i}_{j}"](X[i + 1, j - 1])]
                X[i, j] = self.node[f"{i}_{j}"](torch.cat(skips, dim=1))
        return [self.head[j - 1](X[0, j]) for j in range(1, d + 1)]    # logits of heads 1..d

def bce_dice(logit, y, eps=1.0):
    p = torch.sigmoid(logit)
    dice = (2 * (p * y).sum() + eps) / (p.sum() + y.sum() + eps)
    return 0.5 * F.binary_cross_entropy_with_logits(logit, y) + (1 - dice)

model = UNetPP()
x, y = torch.randn(2, 3, 96, 96), (torch.rand(2, 1, 96, 96) > 0.7).float()
loss = sum(bce_dice(o, y) for o in model(x)) / 4                  # deep supervision on X0,1..X0,4
accurate = torch.stack([o.sigmoid() for o in model(x)]).mean(0)    # accurate mode
fast = model(x, depth=2)[-1].sigmoid()                             # fast mode: U-Net++ L2
```

A minimal U-Net++ in PyTorch with deep supervision and depth pruning (not run here: no PyTorch in the sandbox). Without batch norm its parameter count equals the 9.04M above plus three extra 1×1 heads.

> **Pruning needs deep supervision:** Without the auxiliary heads, $X^{0,1}$ to $X^{0,3}$ are never trained to produce a mask, so cutting the network at $L^2$ or $L^3$ gives a useless output. Train with all four heads if you want the option to prune later, then pick the depth on a validation set.

**Strengths**

- Encoder features pass through refining convolutions before they meet the decoder, narrowing the semantic gap the design targets
- One training run yields U-Nets of four depths; choose the speed/accuracy point afterwards
- Only about 17% more parameters than U-Net at the same widths

**Limitations**

- About 2.5× U-Net's compute at depth 4, and much more activation memory, because the extra nodes run at the highest resolutions
- The gains over a well-tuned U-Net are a few IoU points and vary by dataset
- Dense concatenation makes the top-row inputs wide (160 channels at X0,4)

**Question:** Why can U-Net++ drop every node with i + j > 2 at inference and still produce exactly the same output at X0,2?

- A. Because those nodes have zero weights after training
- B. Because X0,2 is computed only from nodes with i + j ≤ 2
- C. Because deep supervision makes all heads identical
- D. It cannot; the output changes slightly

**Answer:** B. Because X0,2 is computed only from nodes with i + j ≤ 2 — Each node uses earlier nodes of its row ($i+k<i+j$) and $X^{i+1,j-1}$ (same $i+j$), and the encoder nodes it needs have $i\le 2$. So the forward pass to $X^{0,2}$ never touches the deleted nodes. They only influenced training, through gradients into the shared nodes.

**Question:** Node X1,2 sits on level 1 (64 filters). How many channels enter its first 3×3 convolution?

- A. 64
- B. 128
- C. 192
- D. 256

**Answer:** C. 192 — It concatenates $X^{1,0}$ and $X^{1,1}$ (64 each) with the up-convolved $X^{2,1}$ (64 after the up-conv): $(j+1)\cdot64=3\cdot64=192$.

**Key points**

- Nodes $X^{i,j}=\mathcal H([x^{i,0},\dots,x^{i,j-1},\mathcal U(x^{i+1,j-1})])$: dense skips along each level, upsampling from below.
- U-Net is the outer edge of the grid; U-Net++ adds the inner nodes and nests U-Nets of depths 1 to 4 on one encoder.
- Deep supervision: a 1×1 sigmoid head on each of X0,1…X0,4, trained with BCE plus Dice.
- Inference: average the heads (accurate) or prune to L^d by keeping i + j ≤ d (fast, exact).
- Paper widths 32 to 512: 9.04M vs 7.76M parameters; about 2.5× U-Net's MACs, mostly in the full-resolution row.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
