Mask2Former (Cheng et al., CVPR 2022) predicts N (class, mask) pairs, each mask a query–pixel dot product, with masked cross-attention limited to the previous mask; one model reached 57.8 PQ, 50.1 AP and 57.7 mIoU.
8 min read897 wordsadvanced0 viewsupdated 2026-10-15
Definition
Mask2Former (Masked-attention Mask Transformer; Cheng, Misra, Schwing, Kirillov and Girdhar, "Masked-attention Mask Transformer for Universal Image Segmentation", CVPR 2022) is a segmentation model that represents any output as a set of N binary masks, each with a class label. Semantic, instance and panoptic segmentation then share one architecture and one loss. Its key change is masked attention: each query's cross-attention is restricted to the foreground of the mask it predicted at the previous layer.
The three segmentation tasks used to need different designs (see Image segmentation). Semantic segmentation was per-pixel classification, as in FCN and DeepLab: a K-way softmax at every pixel. Instance segmentation was detect-then-segment, as in Mask R-CNN. Panoptic segmentation combined the two. A per-pixel classifier cannot separate two touching dogs, and a box-based model handles amorphous "stuff" such as sky or road badly.
Mask classification handles all three. Predict N pairs (pi,mi), where pi is a distribution over K classes plus "no object" ∅ and mi∈[0,1]H×W is a mask. A thing class gets one pair per instance, and a stuff class one pair per region. MaskFormer (Cheng, Schwing and Kirillov, NeurIPS 2021) introduced this with a DETR-style decoder. Mask2Former keeps the formulation and fixes what made MaskFormer slow to train and weak on small objects.
Architecture
Three parts. A backbone (ResNet or Swin Transformer) gives features at strides 4 to 32. A pixel decoder, by default 6 multi-scale deformable-attention layers as in Deformable DETR, refines the 1/32, 1/16 and 1/8 maps and upsamples them with a lateral connection to per-pixel embeddings F∈RC×4H×4W. A transformer decoder of 9 layers updates N=100 query features. Successive layers read the 1/32, 1/16 and 1/8 maps in round robin, three times over. After every layer, each query predicts its class with a linear layer and its mask with a dot product:
mi[h,w]=σ(εi⊤F[:,h,w]),εi=MLP(qi)∈RC
qi
output feature of query i after the current decoder layer
εi
mask embedding of query i, from a small MLP; C=256
F
per-pixel embeddings at 1/4 resolution from the pixel decoder, shared by all queries
fig.Mask2Former. The pixel decoder (multi-scale deformable attention) refines 1/8–1/32 features and builds 1/4 per-pixel embeddings F. Nine decoder layers update 100 queries, reading 1/32, 1/16 and 1/8 features in turn. After each layer every query predicts a class and a mask σ(MLP(q)·F). The mask, resized and thresholded at 0.5, limits where that query may look in the next layer's cross-attention. The same 100 (class, mask) pairs are merged into a semantic, instance or panoptic output.
Masked attention
In a DETR-style decoder, each query's cross-attention spans the whole feature map. MaskFormer needed long training partly because a query has to learn to ignore the background around its object. Mask2Former simply forbids that background. Each query attends only inside the mask it predicted at the previous layer:
the N×C query features after layer l; Ql=fQ(Xl−1)
Kl,Vl
keys and values from the image features of the scale assigned to layer l
Ml−1
attention mask from the previous layer's mask prediction, resized to layer l's resolution; M0 comes from the learnable query features before the first layer
The masks therefore refine themselves: a better mask focuses the next layer's attention, which gives a better query, which gives a better mask. If a query's mask is empty, the official code lets that query attend everywhere, so it is never left with an empty softmax. Three smaller changes help as well. Each layer runs cross-attention before self-attention. Query features are learned and supervised directly. Dropout is removed.
cross-attention
fig.A toy dog query refined over decoder layers, with no learned weights: unit 4-d pixel embeddings, masks from σ(γ(q·F − τ)), and the update q ← unit(q + attention read-out). The layer-0 mask is poor in both modes (IoU 0.313). At the defaults (layer 2, grass ≈ dog 0.5), masked attention looks at 74 of 384 pixels and puts 88.9% of its weight on the dog, so mask IoU reaches 0.803. Standard attention puts 28.3% there, and IoU falls to 0.202. By layer 5 it is 99.4% / 0.984 against 9.9% / 0.034. Raise the grass similarity to 0.6 and the first mask holds more grass (32 pixels) than dog (26). Masked attention then locks in the error and ends at IoU 0.034.
Training and inference
Training follows DETR. Predictions are Hungarian-matched to ground-truth segments with a class and mask cost, and unmatched predictions are trained toward ∅ (weight 0.1). The loss is L=λclsLcls+λceLBCE+λdiceLDice, with λcls=2 and λce=λdice=5 (see Dice loss), applied after every decoder layer. The mask terms are not computed on all pixels. They use K=12,544 sampled points (112×112): uniform points for the matching cost, and PointRend-style importance sampling for the loss. That cuts training memory about threefold, from 18 GB to 6 GB per image.
At inference the N pairs are merged according to the task. Semantic: drop ∅ and take argmaxc∑ipi(c)mi[h,w]. Instance: each query is one instance, scored by its class probability times its mean mask probability inside the mask. Panoptic: assign each pixel to argmaxipi(ci)mi[h,w], then drop low-confidence queries and those left with little visible area.
Results reported by Cheng et al. (2022)
Headline (Swin-L based)
57.8 PQ on COCO panoptic, 50.1 AP on COCO instance, 57.7 mIoU on ADE20K semantic
ResNet-50, COCO
51.9 PQ panoptic, 43.7 AP instance (50 epochs)
Defaults
100 queries, 9 decoder layers (3 rounds over 1/32, 1/16, 1/8), 6-layer MSDeformAttn pixel decoder, C = 256
Optimisation
AdamW, lr 1e-4, weight decay 0.05, backbone lr × 0.1; mask loss on 12,544 points per mask
Worked example: semantic inference from mask classification
Three queries, four pixels A–D. Class probabilities over (dog, cat, grass, ∅): p1=(0.90,0.05,0.02,0.03), p2=(0.10,0.80,0.05,0.05), p3=(0.02,0.03,0.90,0.05). Masks on A–D: m1=(0.95,0.30,0.10,0.05), m2=(0.20,0.85,0.05,0.02), m3=(0.05,0.30,0.90,0.60). The score of class c at a pixel is ∑ipi(c)mi:
Pixel
dog
cat
grass
Label
A
0.90·0.95 + 0.10·0.20 + 0.02·0.05 = 0.876
0.209
0.074
dog
B
0.361
0.05·0.30 + 0.80·0.85 + 0.03·0.30 = 0.704
0.3185
cat
C
0.113
0.072
0.8145
grass
D
0.059
0.0365
0.02·0.05 + 0.05·0.02 + 0.90·0.60 = 0.542
grass
Σ_i p_i(c)·m_i per pixel (computed with numpy); ∅ is dropped before merging
At pixel B the dog query is confident about its class (0.90) but its mask there is low (0.30). The cat query wins with 0.80⋅0.85. The class and the location come from different factors, and both must be high. For panoptic output, the same pixels go to argmaxipi(ci)mi: A→query 1 (0.855), B→query 2 (0.680), C and D→query 3 (0.810, 0.540).
19print(semseg.shape, seg.shape, [model.config.id2label[i] for i in seg.unique().tolist()][:5])
Mask2Former with Hugging Face transformers: semantic output by the Σ p·m rule, and by the built-in post-processor (not run here: needs torch and transformers)
Check yourself
How does a Mask2Former query produce its mask?
Check yourself
What exactly does masked attention change compared with DETR/MaskFormer cross-attention?
Remember
✓Mask classification: N (class, mask) pairs, masks mi=σ(MLP(qi)⊤F) on 1/4-resolution per-pixel embeddings; one architecture for semantic, instance and panoptic.
✓Masked attention: softmax(Ml−1+QK⊤)V with −∞ outside the previous layer's mask (threshold 0.5), so the masks refine themselves.
✓Pixel decoder = 6 multi-scale deformable-attention layers; 9 decoder layers read 1/32, 1/16 and 1/8 features in round robin; cross-attention before self-attention.
✓Hungarian matching; loss 2·CE + 5·BCE + 5·Dice, with mask terms on 12,544 sampled points (about 3× less memory).