# Mask R-CNN

> Mask R-CNN adds a small FCN branch to Faster R-CNN that predicts a 28×28 sigmoid mask per class for every RoI, and replaces RoIPool with RoIAlign, whose bilinear sampling removes the rounding that misaligned features with pixels.

Source: https://www.mlfoundry.in/learn/mask-r-cnn  
Section: Deep Learning › Deep Learning for Computer Vision  
Level: core · 8 min read · updated 2026-10-15

**Mask R-CNN:** is an instance segmentation model (Kaiming He, Georgia Gkioxari, Piotr Dollár and Ross Girshick, "Mask R-CNN", ICCV 2017, Marr Prize). It extends **Faster R-CNN** with a third head, parallel to classification and box regression, that outputs a binary mask for each region of interest. It also introduces **RoIAlign**, a feature-extraction layer without quantisation.

Instance segmentation asks for a class, a box and a pixel mask for every object, so two touching people get two separate masks. Semantic segmentation (see **Image segmentation** and **FCN**) labels pixels but cannot tell instances apart. Detectors separate instances but give only boxes.

Earlier systems such as DeepMask, MNC and FCIS proposed segments first and then classified them, or predicted position-sensitive score maps. Mask R-CNN does the reverse: detect first, then segment inside each box. Mask prediction is decoupled from classification.

## Architecture

The backbone and RPN are those of Faster R-CNN, usually with a ResNet-50/101 or ResNeXt-101 backbone and a **Feature Pyramid Network** (FPN). Each RoI is pooled from the pyramid level that matches its size. The box head sees 7×7 features. The mask head sees 14×14 features and is a small fully convolutional network: four 3×3 convs, a 2× deconvolution and a 1×1 conv to $K$ channels at 28×28.

> Figure: Mask R-CNN with an FPN head. RoIAlign extracts 7 × 7 × 256 features for the box head (two fc-1024 layers, then K+1 classes and 4K box offsets) and 14 × 14 × 256 features for the mask head. Four 3 × 3 convs, a stride-2 deconvolution and a 1 × 1 conv give a 28 × 28 × K stack, one mask per class (K = 80 on COCO). Only the mask of the class in question is used: the ground-truth class in training, the predicted class at test time.

## The mask loss: one sigmoid mask per class

$$
L=L_{\text{cls}}+L_{\text{box}}+L_{\text{mask}},\qquad L_{\text{mask}}=-\frac{1}{m^2}\sum_{(i,j)}\Big[y_{ij}\log\sigma\big(z^{k}_{ij}\big)+\big(1-y_{ij}\big)\log\big(1-\sigma(z^{k}_{ij})\big)\Big]
$$

- $z^{k}$: the $m\times m$ logits of class $k$, the RoI's ground-truth class; the other $K-1$ masks get no loss
- $y_{ij}$: target: the ground-truth mask cropped to the RoI and resized to m × m (m = 28)
- $L_{\text{cls}},\ L_{\text{box}}$: as in **Fast R-CNN**; $L_{\text{mask}}$ is defined only on positive RoIs (IoU ≥ 0.5 with a ground-truth box)

Because each class has its own sigmoid mask, classes do not compete per pixel; the classification head decides the class. A per-pixel softmax over $K$ classes couples the two tasks, and the paper reports it costs 5.5 mask AP (24.8 vs 30.3 with ResNet-50-C4).

The mask is predicted at $m\times m$ with convolutions, not with fc layers, so it keeps the RoI's spatial layout. That only works if the RoI features line up with the image. A shift of a few pixels hardly changes a box score, but it does change a mask.

## RoIAlign: sample, don't round

**RoIPool** rounds twice: the RoI coordinate $x/16$ to a whole cell, and each bin boundary to a whole cell. Either can move the pooled region by several pixels in the image. **RoIAlign** keeps the continuous coordinates. It divides the RoI into exact bins, samples 4 regularly spaced points per bin (2×2), reads each point by bilinear interpolation of the 4 nearest feature cells, and aggregates them by max or average. The paper notes the result is not sensitive to the exact sample positions or how many there are, as long as nothing is quantised.

$$
f(u,v)=\sum_{a\in\{\lfloor u'\rfloor,\lfloor u'\rfloor+1\}}\ \sum_{b\in\{\lfloor v'\rfloor,\lfloor v'\rfloor+1\}} \big(1-|u'-a|\big)\big(1-|v'-b|\big)\,F_{b,a},\qquad u'=u-\tfrac12,\ v'=v-\tfrac12
$$

- $(u,v)$: a sample point in feature-map units, e.g. $x/16$; no rounding anywhere
- $F_{b,a}$: feature value of cell $(a,b)$, whose centre is at $(a+\tfrac12,b+\tfrac12)$; the half-pixel offset is torchvision's `aligned=True`
- $\partial f/\partial F$: the bilinear weights, so gradients reach the 4 neighbouring cells of every sample

> Figure: RoIPool vs RoIAlign on one channel of a stride-16 map, using torchvision's rounding rules for RoIPool. At the default RoI (x1 = 32 px) RoIPool's box has moved 0, −6, +24 and +14 px from the requested box. 24 px is 9.3 mask pixels at 28 × 28 for this 72-px RoI. Its bin (0, 0) returns 0.630, and RoIAlign's returns 0.395. Slide the RoI from +13 to +14 px: RoIPool's bin (0, 0) jumps from 0.63 to 0.81 as the rounded left edge flips from −7 to +8 px, while RoIAlign moves smoothly from 0.494 to 0.508. Over the 32-px sweep RoIPool changes only twice (at +14 and +30).

## Worked example: one RoIAlign bin

The RoI from the figure at its default position: $(x_1,y_1)=(32,22)$ px, size 72×76, pooled to 2×2. In feature units it starts at $(2.0, 1.375)$ and each bin is $2.25\times2.375$ cells. Bin (0, 0) samples at $u\in\{2.5625, 3.6875\}$ and $v\in\{1.96875, 3.15625\}$.

The sample at (u, v) = (3.6875, 3.15625): u′ = 3.1875, v′ = 2.65625, so the fractions are 0.1875 and 0.65625 (computed in Python)

| Neighbour cell (a, b) | Value F | Weight | Contribution |
| --- | --- | --- | --- |
| (3, 2) | 0.30 | 0.8125 × 0.34375 = 0.2793 | 0.0838 |
| (4, 2) | 0.48 | 0.1875 × 0.34375 = 0.0645 | 0.0309 |
| (3, 3) | 0.38 | 0.8125 × 0.65625 = 0.5332 | 0.2026 |
| (4, 3) | 0.63 | 0.1875 × 0.65625 = 0.1230 | 0.0775 |
| Sum |  | 1.0000 | 0.3949 |

The other three samples give 0.160, 0.278 and 0.200, so the bin outputs 0.395 (max) or 0.258 (average). RoIPool rounds the same RoI to cells x 2–7, y 1–6. Its bin (0, 0) covers cells x 2–4, y 1–3 and returns 0.63 from cell (4, 3), whose centre (4.5, 3.5) lies outside the true bin, which ends at u = 4.25.

**Reported results (COCO)**

- **Mask AP, test-dev:** 35.7 with ResNet-101-FPN and 37.1 with ResNeXt-101-FPN, single model, vs 33.6 for FCIS+++, the COCO 2016 winner
- **Box AP, test-dev:** 38.2 (ResNet-101-FPN) and 39.8 (ResNeXt-101-FPN), above Faster R-CNN with the same backbone, because of RoIAlign and multi-task training
- **RoIAlign ablation:** relative mask-AP gains of 10–50%. With a stride-32 ResNet-50-C5 backbone, +7.3 mask AP and +10.5 AP₇₅.
- **Speed and training:** about 5 fps (195 ms per image on a Tesla M40) for ResNet-101-FPN. Trains in 1–2 days on one 8-GPU machine, with images of short side 800.
- **Inference:** the mask branch runs on the top 100 detections only. Each 28 × 28 float mask is resized to its box and thresholded at 0.5.

## Keypoints with the same head

For human pose, each of the $K=17$ COCO keypoints becomes a one-hot $m\times m$ mask with a single foreground pixel. The head predicts 17 maps at 56×56 and uses a softmax cross-entropy over the $m^2$ positions of each map, so it has to pick one location. A ResNet-50-FPN model reaches 62.7 keypoint AP, above the 61.8 of the COCO 2016 keypoint winner, and still runs at about 5 fps. See **Pose estimation**.

```python
import torch
from torchvision.io import read_image
from torchvision.models.detection import maskrcnn_resnet50_fpn, MaskRCNN_ResNet50_FPN_Weights

weights = MaskRCNN_ResNet50_FPN_Weights.COCO_V1
model = maskrcnn_resnet50_fpn(weights=weights).eval()
img = weights.transforms()(read_image("people.jpg"))

with torch.no_grad():
    out = model([img])[0]            # boxes [N,4], labels [N], scores [N], masks [N,1,H,W]

keep = out["scores"] > 0.7
masks = out["masks"][keep, 0] > 0.5  # already pasted into the image: soft 28x28 masks resized to each box
names = weights.meta["categories"]
for lab, m in zip(out["labels"][keep], masks):
    print(names[lab], int(m.sum()), "pixels")
```

Instance masks from torchvision's COCO-trained Mask R-CNN (ResNet-50-FPN); weights download on first use (not run here)

> **Why decoupling works:** The box head already decides what the object is. The mask head only has to answer "which pixels of this box belong to a class-$k$ object", a binary question that a per-class sigmoid answers directly. Asking it to also choose among 80 classes per pixel is harder and gave 5.5 AP less.

> **28 × 28 limits boundary detail:** A 300-px-wide object gets a mask of 28 × 28 values, about 11 px per mask cell, so thin structures and boundaries are blurred after upsampling. Later work refines boundaries (for example PointRend), and query-based models such as **Mask2Former** predict masks at higher resolution.

**Question:** An RoI's ground-truth class is 'dog' and COCO has K = 80 classes. Which mask outputs receive gradient from L_mask?

- A. All 80 masks, through a per-pixel softmax
- B. Only the dog mask, with a per-pixel sigmoid and binary cross-entropy
- C. The masks of the top-5 predicted classes
- D. None: the mask head is trained separately after the detector

**Answer:** B. Only the dog mask, with a per-pixel sigmoid and binary cross-entropy — L_mask is defined only on the m × m mask of the ground-truth class k. The other 79 masks get no loss for this RoI, so there is no competition between classes at each pixel.

**Question:** Why does RoIAlign matter more for masks than for box classification?

- A. Masks are predicted at pixel level, so a few pixels of feature misalignment move the mask; class scores barely change
- B. RoIAlign is faster than RoIPool
- C. RoIPool cannot handle 80 classes
- D. Masks use sigmoid, which requires bilinear inputs

**Answer:** A. Masks are predicted at pixel level, so a few pixels of feature misalignment move the mask; class scores barely change — RoIPool's rounding can move the pooled region by up to about a cell (16 px at stride 16). That is several mask pixels at 28 × 28, while classification is fairly translation-invariant. The paper's ablations show the largest RoIAlign gains at strict IoU (AP₇₅) and with large strides.

**Key points**

- Mask R-CNN = Faster R-CNN + an FCN mask head that predicts K masks of 28 × 28 per RoI. Only the mask of the RoI's class is used.
- $L=L_{\text{cls}}+L_{\text{box}}+L_{\text{mask}}$; $L_{\text{mask}}$ is the average per-pixel binary cross-entropy on positive RoIs. A softmax across classes costs 5.5 AP.
- RoIAlign removes both roundings of RoIPool and reads 4 bilinear samples per bin. Its output changes smoothly with the box.
- COCO test-dev: 37.1 mask AP and 39.8 box AP with ResNeXt-101-FPN, at about 5 fps.
- The same head predicts 17 keypoints as one-hot 56 × 56 masks with a spatial softmax: 62.7 keypoint AP.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
