Deformable DETR

Deformable DETR (Zhu et al., ICLR 2021) replaces DETR's dense attention with multi-scale deformable attention: each query samples K = 4 points per head on each of 4 feature levels, giving 43.8 COCO AP in 50 epochs versus 42.0 in 500 for DETR.

8 min read876 wordsadvanced0 viewsupdated 2026-10-15

Definition

Deformable DETR (Zhu, Su, Lu, Li, Wang and Dai, "Deformable DETR: Deformable Transformers for End-to-End Object Detection", ICLR 2021) is a DETR-style set-prediction detector in which every attention module over image features is replaced by multi-scale deformable attention. A query no longer compares itself with every pixel. It predicts a few sampling offsets around a reference point, reads the features there by bilinear interpolation, and averages them with weights it also predicts.

DETR showed that detection can be end to end, with Hungarian matching and no anchors or NMS. It had two problems. It needs 500 epochs on COCO, and its small-object AP is weak (20.5 AP_S). Both come from dense attention. At initialisation each query spreads its attention almost evenly over all NkN_k pixels, and learning to focus on a few takes many epochs. The encoder cost is O(NqNkC)O(N_q N_k C) with Nq=Nk=HWN_q=N_k=HW, quadratic in the number of pixels, so DETR can only afford the stride-32 map. High-resolution, multi-scale features are exactly what small objects need.

The fix borrows from Deformable convolution: sample a small set of learned, input-dependent locations instead of a fixed or exhaustive set. Deformable attention keeps the transformer's multi-head form and applies this idea to it.

Multi-scale deformable attention

Take query qq with feature zqz_q and a normalised reference point p^q∈[0,1]2\hat p_q\in[0,1]^2. In the encoder, every pixel of every level is a query and its reference point is its own position. In the decoder, p^q\hat p_q is predicted from the object query by a linear layer and a sigmoid. With feature maps {xl}l=1L\{x^l\}_{l=1}^{L}:

MSDeformAttn⁡(zq,p^q,{xl})=∑m=1MWm[∑l=1L∑k=1KAmlqk⋅Wm′ xl(ϕl(p^q)+Δpmlqk)],∑l=1L∑k=1KAmlqk=1\operatorname{MSDeformAttn}\big(z_q,\hat p_q,\{x^l\}\big)=\sum_{m=1}^{M}W_m\Big[\sum_{l=1}^{L}\sum_{k=1}^{K}A_{mlqk}\cdot W'_m\,x^l\big(\phi_l(\hat p_q)+\Delta p_{mlqk}\big)\Big],\qquad \sum_{l=1}^{L}\sum_{k=1}^{K}A_{mlqk}=1
M, L, KM,\,L,\,K
heads, feature levels, sampling points per head per level; M=8M=8, L=4L=4, K=4K=4
Δpmlqk\Delta p_{mlqk}
2-D sampling offset in level-ll pixels, from a linear layer on zqz_q (M ⁣⋅ ⁣L ⁣⋅ ⁣K ⁣⋅ ⁣2=256M\!\cdot\!L\!\cdot\!K\!\cdot\!2=256 outputs); fractional, so xl(⋅)x^l(\cdot) is read by bilinear interpolation
AmlqkA_{mlqk}
attention weight from another linear layer on zqz_q, softmax-normalised over the L⋅K=16L\cdot K=16 points of head mm; there is no query–key dot product
ϕl\phi_l
rescales the normalised p^q\hat p_q to the coordinates of level ll, so one query samples the same image region on every scale
Wm, Wm′W_m,\,W'_m
output and value projections of head mm, as in standard Multi-head attention

Each query reads M⋅L⋅K=128M\cdot L\cdot K=128 values in total, whatever the image size. The paper gives the cost as O(2NqC2+min⁡(HWC2, NqKC2))O(2N_qC^2+\min(HWC^2,\,N_qKC^2)) for one level. In the encoder (Nq=HWN_q=HW) this is O(HWC2)O(HWC^2), linear in the number of pixels. In the decoder's cross-attention it does not depend on HWHW at all. The decoder's self-attention among the object queries stays dense: it is cheap, and it lets queries suppress duplicates.

Deformable DETR: multi-scale features, and each query attends to only L·K = 16 points per headResNet-50 stagesC3 /8512C4 /161024C5 /322048C6 /644 maps × 2561×1 conv + GN+ level embed e_l+ sine pos. enc.encoder × 6MS-deform self-attnquery = every pixelp̂ = its own positiondecoder × 6self-attn: 300 queriesMS-deform cross-attnp̂ = σ(Linear(query))headsclass+ box Δfrom p̂C6 = 3×3 s2 conv on C5≈ 22k tokens at 800×1333dense attn: queries onlyone multi-scale deformable attention layer, for query q (8 heads; head m shown)query z_q256-d, ref p̂_qLinear → ΔpM·L·K·2 = 256Linear → Asoftmax over 16offsets & weights come from the query, not from q·kC3C4C5C6K = 4 points on each of L = 4 levelsbilinear sample of W′_m x_l at p̂ + Δp / (W_l, H_l)(red = same normalised p̂ on every level)weighted sumh_m = Σ_l Σ_k A_mlk ·W′_m x_l(p̂ + Δp_mlk)out = Σ_m W_m h_mcost per query:M·L·K = 128 samplesdense: N_k ≈ 22k keysper head per query
fig.Deformable DETR. C3–C5 of a ResNet-50 (strides 8, 16, 32) and an extra stride-64 map from a 3×3 stride-2 conv on C5 are projected to 256 channels. The encoder and decoder both use multi-scale deformable attention. Below, one query uses two linear layers to predict 256 offset coordinates and 128 attention weights, samples 4 points on each of 4 levels for each of 8 heads, and sums them. No query–key products are computed.

The rest of the detector

The levels share one transformer, so a learned level embedding ele_l is added to the positional encoding to say which scale a token comes from. Sampling points of one query span all levels, so information already moves across scales. The paper found that adding an FPN gave no further gain. The decoder uses N=300N=300 queries instead of 100, and focal loss (weight 2) for classification. Each box is predicted relative to the query's reference point: b^=σ(Δb+σ−1(p^q))\hat b=\sigma\big(\Delta b+\sigma^{-1}(\hat p_q)\big) for the centre. Hungarian matching and the L1 + GIoU box loss are kept from DETR.

Two variants improve accuracy. Iterative bounding-box refinement: each decoder layer refines the box of the previous layer and uses it as the new reference, so later layers sample inside the current box estimate. Two-stage: the encoder output first acts as a dense proposal generator, and the top-scoring proposals become the decoder's object queries.

head 1/8 · 4 levels × K = 4 → 16 sampled keys (all heads: 128)C3 · stride 8 · 32×32ΣA = 0.250C4 · stride 16 · 16×16ΣA = 0.250C5 · stride 32 · 8×8ΣA = 0.250C6 · stride 64 · 4×4ΣA = 0.250demo 256×256 input · red = reference point p̂ (same normalised spot on every level) · dot area ∝ AC3 x(p) 1.06 0.98 0.87 0.73A 0.063 0.063 0.063 0.063C4 x(p) 0.97 0.72 0.45 0.26A 0.063 0.063 0.063 0.063C5 x(p) 0.71 0.30 0.13 0.10A 0.063 0.063 0.063 0.063C6 x(p) 0.28 0.12 0.00 0.00A 0.063 0.063 0.063 0.063head output = Σ_l Σ_k A·x_l(p̂ + Δp/W_l) = 0.4795init: head 1 points at 0°, point k at k+1 level pixels, A uniform = 1/16encoder layer at 800×1333: 22,223 tokens (100×167 + 50×84 + 25×42 + 13×21)dense3.95e9 pairsdeformable2.84e6 pairsquery–key pairs over 8 heads (log scale) · dense / deformable = 1389× · dense fp32 attention maps: 15.8 GBdeformable cost is linear in the token count; dense cost is quadratic
offsetslevels
fig.One query at the red reference point, sampling a synthetic one-channel feature map on 4 levels of a demo 256×256 input. With the official initialisation, head 1 points along 0° with point k at k + 1 level pixels, and every A = 1/16. The same offsets reach 8× further in image space on C6 than on C3, and the last two C6 points fall off the map and read 0 (zero padding). The head output is 0.4795. Lower panel: at 800×1333 the 4 levels hold 22,223 tokens. Dense attention needs 3.95e9 query–key pairs per layer over 8 heads (15.8 GB of fp32 weights); deformable attention needs 2.84e6, 1389× fewer. With C5 only it is 1,050 tokens and 263× fewer.

COCO val2017, ResNet-50 (Zhu et al., 2021)

DETR, 500 epochs
42.0 AP, AP_S 20.5; 41M params, 86 GFLOPs
DETR-DC5, 500 / 50 epochs
43.3 AP at 500 epochs; only 35.3 AP when trained for 50
Deformable DETR, 50 epochs
43.8 AP, AP_S 26.4; 40M params, 173 GFLOPs
+ iterative box refinement
45.4 AP
++ two-stage
46.2 AP, AP_S 28.8
Defaults
M=8M=8 heads, L=4L=4 levels, K=4K=4 points, 6 + 6 layers, d=256d=256, 300 queries

So in a tenth of the epochs it beats DETR overall, by 5.9 AP on small objects. FLOPs are higher than DETR's, because it processes the stride-8 map. They are lower than DETR-DC5's 187 G, which works at stride 16 only.

Worked example: one head, one level, K = 2

Sampling C5 (25×38, from an 800×1216 image) for a query at p̂ = (0.40, 0.56)

  1. 1Scale the reference point to level pixels: (0.40⋅38, 0.56⋅25)=(15.2, 14.0)(0.40\cdot38,\ 0.56\cdot25)=(15.2,\ 14.0).
  2. 2Point 1, offset Δp=(1.3, −0.6)\Delta p=(1.3,\,-0.6): location (16.5, 13.4)(16.5,\ 13.4), normalised (0.4342, 0.536)(0.4342,\ 0.536). Cell centres sit at i+0.5i+0.5, so the grid index is (u,v)=(16.0, 12.9)(u,v)=(16.0,\ 12.9). That is exactly column 16, 0.90.9 of the way from row 12 to row 13. With x12,16=0.2x_{12,16}=0.2 and x13,16=0.7x_{13,16}=0.7: 0.1⋅0.2+0.9⋅0.7=0.650.1\cdot0.2+0.9\cdot0.7=0.65.
  3. 3Point 2, offset (−2.2, 0.5)(-2.2,\,0.5): location (13.0, 14.5)(13.0,\ 14.5), index (12.5, 14.0)(12.5,\ 14.0), halfway between columns 12 and 13 on row 14. With values 0.40.4 and 0.20.2: 0.5⋅0.4+0.5⋅0.2=0.300.5\cdot0.4+0.5\cdot0.2=0.30.
  4. 4Attention logits (1.0, 0.0)(1.0,\ 0.0) from the linear layer: softmax gives A=(0.7311, 0.2689)A=(0.7311,\ 0.2689).
  5. 5Head output before WmW_m: 0.7311⋅0.65+0.2689⋅0.30=0.55590.7311\cdot0.65+0.2689\cdot0.30=0.5559. Gradients reach Δp\Delta p through the bilinear weights and reach AA directly, so both are learned from the detection loss.
FeaturesTokensDense pairs (N²)Deformable (16 per query)Ratio
C5 only (DETR)25×42 = 1,0501,102,50016,80066×
dilated C5, stride 16 (DETR-DC5)50×84 = 4,20017,640,00067,200262.5×
C3–C5 + C6 (Deformable DETR)22,223493,861,729355,5681,389×
Encoder query–key pairs per head per layer at 800×1333 (stride-k maps rounded up)
python
1import torch
2import torch.nn.functional as F
3
4def ms_deform_attn(value, shapes, loc, attn):
5 """value: (B, S, M, D) all levels flattened, S = sum(H_l * W_l)
6 shapes: [(H_l, W_l)] * L; loc: (B, Q, M, L, K, 2) normalised (x, y) in [0, 1]
7 attn: (B, Q, M, L, K), softmax-ed over L*K. Returns (B, Q, M*D)."""
8 B, S, M, D = value.shape
9 _, Q, _, L, K, _ = loc.shape
10 grids = 2 * loc - 1 # grid_sample wants [-1, 1]
11 sampled = []
12 for l, v in enumerate(value.split([h * w for h, w in shapes], dim=1)):
13 h, w = shapes[l]
14 v = v.flatten(2).transpose(1, 2).reshape(B * M, D, h, w)
15 g = grids[:, :, :, l].transpose(1, 2).flatten(0, 1) # (B*M, Q, K, 2)
16 sampled.append(F.grid_sample(v, g, mode="bilinear", padding_mode="zeros", align_corners=False))
17 a = attn.transpose(1, 2).reshape(B * M, 1, Q, L * K)
18 out = (torch.stack(sampled, dim=-2).flatten(-2) * a).sum(-1) # (B*M, D, Q)
19 return out.view(B, M * D, Q).transpose(1, 2)
20
21shapes = [(100, 167), (50, 84), (25, 42), (13, 21)] # 800x1333 input, strides 8..64
22S, M, D, L, K, Q = sum(h * w for h, w in shapes), 8, 32, 4, 4, 300
23z = torch.randn(1, Q, 256) # decoder queries
24ref = torch.rand(1, Q, 1, 1, 1, 2) # reference points
25offsets = torch.nn.Linear(256, M * L * K * 2)(z).view(1, Q, M, L, K, 2)
26norm = torch.tensor([[w, h] for h, w in shapes], dtype=torch.float).view(1, 1, 1, L, 1, 2)
27loc = ref + offsets / norm # offsets are in level pixels
28attn = torch.nn.Linear(256, M * L * K)(z).view(1, Q, M, L * K).softmax(-1).view(1, Q, M, L, K)
29out = ms_deform_attn(torch.randn(1, S, M, D), shapes, loc, attn)
30print(S, out.shape) # 22223 torch.Size([1, 300, 256])
Pure-PyTorch multi-scale deformable attention core, following the reference implementation in the official repo (not run here; training uses the repo's CUDA kernel)

Check yourself

In deformable attention, how are the attention weights A_mlqk computed?

Check yourself

Why can Deformable DETR use the stride-8 map when DETR could not?

Remember

  • ✓Each query samples K=4K=4 points per head on each of L=4L=4 levels (strides 8–64) around its reference point: 128 bilinear reads, weights and offsets both predicted from the query.
  • ✓Encoder cost becomes linear in HWHW, so multi-scale, high-resolution features fit, and no FPN is needed.
  • ✓43.8 COCO AP in 50 epochs vs DETR's 42.0 in 500; AP_S 26.4 vs 20.5; 46.2 AP with box refinement and two stages.
  • ✓300 queries, focal loss, boxes relative to reference points; Hungarian matching and L1 + GIoU are kept from DETR.
  • ✓The same multi-scale deformable attention became a standard pixel decoder and encoder block in later detectors and segmenters, such as Mask2Former.
← back to Deep Learning roadmap