# DETR

> DETR (Carion et al., 2020) treats detection as set prediction: a CNN + transformer turns 100 learned queries into 100 boxes, Hungarian-matched one-to-one to the ground truth, so it needs no anchors or NMS; 42.0 COCO AP after 500 epochs.

Source: https://www.mlfoundry.in/learn/detr  
Section: Deep Learning › Deep Learning for Computer Vision  
Level: advanced · 8 min read · updated 2026-10-15

**DETR:** (DEtection TRansformer; Carion, Massa, Synnaeve, Usunier, Kirillov and Zagoruyko, "End-to-End Object Detection with Transformers", ECCV 2020) is an object detector that outputs a fixed-size **set** of $N=100$ predictions in one pass. A CNN backbone feeds a transformer encoder–decoder, and a bipartite (Hungarian) matching assigns each ground-truth object to exactly one prediction during training. Every other prediction is trained to say "no object" ($\varnothing$).

Input: an image. Output: a set of (class, box) pairs, with no fixed count and no order. Earlier detectors (**Faster R-CNN**, **RetinaNet**, **YOLO**) reach this set indirectly. They place thousands of anchors or proposals, label each one by an IoU threshold, regress offsets, and then remove duplicates with non-maximum suppression (NMS). The anchor sizes, IoU thresholds and NMS threshold are hand-tuned, and NMS is not part of the trained model.

DETR removes all of these steps. The model predicts the set directly, and the loss is defined on sets. That needs two things: a loss that does not depend on the order of the predictions, and an architecture whose outputs can talk to each other, so that two slots do not both claim the same object.

## Architecture

A **ResNet** turns a $3\times H\times W$ image into its C5 map, $2048\times\frac{H}{32}\times\frac{W}{32}$. A $1\times1$ conv reduces this to $d=256$ channels, and the map is flattened into $HW/32^2$ tokens: 950 for an $800\times1216$ input. A 6-layer transformer encoder (8 heads, FFN width 2048) applies **Self-attention** over all tokens, with fixed 2-D sine positional encodings added to queries and keys in every layer.

The decoder (6 layers) takes $N=100$ **object queries**, learned embeddings that act as positional codes for output slots. In each layer the queries self-attend, so slots can coordinate, and then read the image through **Cross-attention** to the encoder memory. All 100 slots are decoded in parallel, not autoregressively. A shared head turns each output into class probabilities over $K$ classes plus $\varnothing$ (linear + softmax) and a normalised box $(c_x,c_y,w,h)\in[0,1]^4$ (3-layer MLP + sigmoid).

> Figure: DETR end to end. The backbone and 1×1 conv give 950 tokens of width 256 for an 800×1216 image; the encoder relates them globally; the decoder turns 100 learned queries into 100 slots in parallel; one shared head gives each slot a class (or ∅) and a box. During training a Hungarian matching assigns every ground-truth object to one slot. The duplicate cat in slot 4 is assigned ∅, so the model learns to suppress duplicates itself.

## Hungarian matching and the set loss

Pad the ground-truth set $y$ with $\varnothing$ up to $N$ elements. Find the permutation of the predictions $\hat y$ with the lowest total matching cost. This is a linear assignment problem, solved exactly by the Hungarian algorithm (in practice `scipy.optimize.linear_sum_assignment`, $O(N^3)$):

$$
\hat\sigma=\arg\min_{\sigma\in\mathfrak S_N}\sum_{i=1}^{N}\mathcal L_{\text{match}}\big(y_i,\hat y_{\sigma(i)}\big),\qquad \mathcal L_{\text{match}}=-\mathbb 1_{\{c_i\neq\varnothing\}}\,\hat p_{\sigma(i)}(c_i)+\mathbb 1_{\{c_i\neq\varnothing\}}\,\mathcal L_{\text{box}}\big(b_i,\hat b_{\sigma(i)}\big)
$$

- $\mathfrak S_N$: all permutations of the $N$ prediction slots
- $\hat p_{\sigma(i)}(c_i)$: predicted probability of the true class $c_i$; the probability, not its log, is used in the cost
- $\mathcal L_{\text{box}}$: $\lambda_{L1}\lVert b_i-\hat b_{\sigma(i)}\rVert_1+\lambda_{\text{iou}}\,\mathcal L_{\text{GIoU}}$, with $\lambda_{L1}=5$, $\lambda_{\text{iou}}=2$ and $\mathcal L_{\text{GIoU}}=1-\text{GIoU}$

With the matching fixed, the **Hungarian loss** is a normal detection loss on the matched pairs. Every slot gets cross-entropy on its target class. Slots matched to $\varnothing$ get no box loss, and their class term is down-weighted by 10 against class imbalance (most of the 100 slots are empty):

$$
\mathcal L_{\text{Hungarian}}(y,\hat y)=\sum_{i=1}^{N}\Big[-\log\hat p_{\hat\sigma(i)}(c_i)+\mathbb 1_{\{c_i\neq\varnothing\}}\,\mathcal L_{\text{box}}\big(b_i,\hat b_{\hat\sigma(i)}\big)\Big]
$$

- $\hat\sigma$: the optimal assignment from the matching step; no gradient flows through it
- $\text{GIoU}$: generalised IoU, $\text{IoU}-|C\setminus(A\cup B)|/|C|$ with $C$ the smallest enclosing box (see **Intersection over Union (IoU) loss**); scale-invariant, unlike L1

The same loss is applied after **every** decoder layer, through the shared head (auxiliary losses). One-to-one matching is what removes NMS: a second slot on the same object is a $\varnothing$ target, so the decoder learns to keep only one. At inference you keep the slots whose top class is not $\varnothing$, or whose score passes a threshold.

> Figure: Three objects, eight of the 100 query slots. Each cell is DETR's matching cost −λ_cls·p(c) + λ_L1·L1 − λ_GIoU·GIoU. At the default weights (1, 5, 2), the minimum over all 336 one-to-one assignments is −6.952: g1←q1, g2←q3, g3←q5, and the other five slots get ∅. The summed loss is 2.769. Raise λ_cls to 5.5 and g2 switches to q4, the confident but loose box (p = 0.83, IoU 0.541). The loss rises to 3.852. Switch to anchor-style IoU ≥ 0.5 assignment and 6 slots become positives for 3 objects, the duplicates that NMS would otherwise have to remove.

**Configuration and COCO val results (Carion et al., 2020)**

- **Model:** ResNet-50 + 6 encoder + 6 decoder layers, $d=256$, 8 heads, $N=100$ queries; 41M parameters, 86 GFLOPs, 28 FPS on a V100
- **Training:** AdamW, lr $10^{-4}$ (transformer) and $10^{-5}$ (backbone); 300 epochs take 3 days on 16 V100s at batch 64; the 500-epoch schedule (lr drop at 400) adds 1.5 AP
- **DETR R50, 500 ep:** 42.0 AP; AP_S 20.5, AP_L 61.1
- **Faster R-CNN-FPN+ R50:** 42.0 AP; AP_S 26.6, AP_L 53.4 (a strengthened baseline with GIoU loss and a longer schedule)
- **Larger DETR:** DC5-R50 43.3 AP (dilated C5, stride 16); R101 43.5 AP

So DETR matches a well-tuned Faster R-CNN overall, but in a different way. It is much better on large objects (+7.7 AP_L), because the encoder reasons globally. It is much worse on small ones (−6.1 AP_S), because it sees only a stride-32 map with no feature pyramid. Adding an FPN is not easy: encoder self-attention costs $O((HW)^2d)$, and going from stride 32 to stride 16 already multiplies the tokens by 4 and the attention cost by 16.

## Worked example: matching two objects

Ground truth: cat $A=(0.30,0.50,0.20,0.40)$ and dog $B=(0.70,0.50,0.20,0.40)$ in $(c_x,c_y,w,h)$. Three slots: $q_1=(0.32,0.50,0.20,0.40)$ with $p(\text{cat})=0.90$; $q_2=(0.50,0.50,0.40,0.40)$ with $p(\text{cat})=0.60$, $p(\text{dog})=0.30$, a wide box over both animals; $q_3=(0.68,0.52,0.24,0.40)$ with $p(\text{dog})=0.80$. Cost $=-p+5\,\text{L1}-2\,\text{GIoU}$:

Matching costs (computed in Python); IoU of q1 with A = 0.072 / 0.088

| Pair | p(c) | L1 | GIoU | Cost |
| --- | --- | --- | --- | --- |
| q1 → A (cat) | 0.90 | 0.02 | 0.8182 | −0.90 + 0.10 − 1.6364 = −2.4364 |
| q2 → A (cat) | 0.60 | 0.40 | 0.2000 | −0.60 + 2.00 − 0.40 = 1.0000 |
| q2 → B (dog) | 0.30 | 0.40 | 0.2000 | −0.30 + 2.00 − 0.40 = 1.3000 |
| q3 → B (dog) | 0.80 | 0.08 | 0.7521 | −0.80 + 0.40 − 1.5041 = −1.9041 |
| q1 → B, q3 → A | 0.05, 0.10 | 0.38, 0.44 | −0.3103, −0.3016 | 2.4707, 2.7032 |

Of the 6 one-to-one assignments, $\{q_1\to A,\ q_3\to B\}$ costs $-2.4364-1.9041=-4.3405$; the next best, $\{q_1\to A,\ q_2\to B\}$, costs $-1.1364$. So $q_2$ becomes a $\varnothing$ target, and its loss term is $-0.1\log\hat p_2(\varnothing)$. The negative GIoU of the disjoint pairs ($-0.31$) still says how far apart the boxes are, which plain IoU (0 for both) cannot.

```python
import torch
from PIL import Image
from scipy.optimize import linear_sum_assignment
from torchvision.ops import box_convert, generalized_box_iou
from transformers import DetrForObjectDetection, DetrImageProcessor

proc = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50").eval()
image = Image.open("street.jpg").convert("RGB")
with torch.no_grad():
    out = model(**proc(images=image, return_tensors="pt"))
print(out.logits.shape, out.pred_boxes.shape)          # (1, 100, 92) = 91 COCO ids + no-object, (1, 100, 4)
det = proc.post_process_object_detection(out, threshold=0.9, target_sizes=[image.size[::-1]])[0]
for s, l, b in zip(det["scores"], det["labels"], det["boxes"]):   # no NMS anywhere
    print(model.config.id2label[l.item()], f"{s:.2f}", [round(v) for v in b.tolist()])

def hungarian_match(logits, boxes, tgt_labels, tgt_boxes, w_cls=1.0, w_l1=5.0, w_giou=2.0):
    """logits (N, K+1), boxes (N, 4) cxcywh in [0, 1]; returns (pred_idx, gt_idx)."""
    prob = logits.softmax(-1)
    xyxy = lambda b: box_convert(b, "cxcywh", "xyxy")
    cost = (-w_cls * prob[:, tgt_labels]                         # (N, M)
            + w_l1 * torch.cdist(boxes, tgt_boxes, p=1)
            - w_giou * generalized_box_iou(xyxy(boxes), xyxy(tgt_boxes)))
    return linear_sum_assignment(cost.detach().cpu().numpy())
```

Pretrained DETR inference with Hugging Face, plus DETR's matcher (not run here: needs torch, transformers and scipy)

> **Slow convergence:** DETR needs 300–500 epochs on COCO, where Faster R-CNN needs about 12–36. At initialisation every query's cross-attention is spread almost evenly over all 950 tokens. Learning to focus on a few relevant locations takes a long time. The matching can also change between epochs, so a slot's target is not stable early on. **Deformable DETR** fixes the attention side: it samples a few points per query and converges in 50 epochs. Later variants (DN-DETR, DINO) add noised ground-truth boxes as extra queries to stabilise the matching.

**Strengths**

- No anchors, no proposal stage, no NMS: the loss is defined on the final output set
- Global reasoning in the encoder gives strong large-object AP (61.1 AP_L vs 53.4 for the R50 Faster R-CNN baseline)
- The same query decoder extends to panoptic segmentation with a mask head, and to later set-prediction models such as **Mask2Former**

**Limitations**

- Very slow training: 500 epochs for 42.0 AP
- Weak on small objects (20.5 AP_S): a single stride-32 map, and attention cost is quadratic in the number of tokens
- N must exceed the number of objects in any image; 100 suits COCO but caps dense scenes

**Question:** An image has 3 objects and DETR uses N = 100 queries. During training, how many slots get a box loss and how many are trained to predict ∅?

- A. 100 box losses, 0 ∅
- B. 3 box losses, 97 ∅
- C. Every slot with IoU ≥ 0.5 to an object gets a box loss
- D. 3 box losses, the other 97 get no loss at all

**Answer:** B. 3 box losses, 97 ∅ — The Hungarian matching pairs each of the 3 objects with exactly one slot. Those 3 get class and box losses. The 97 others are matched to the padded $\varnothing$ entries and get only a class loss for $\varnothing$, down-weighted by 0.1. IoU thresholds are the anchor-style rule DETR replaces.

**Question:** Why can DETR skip non-maximum suppression?

- A. Its boxes are always more accurate than anchor-based ones
- B. One-to-one matching makes a duplicate slot a ∅ target, and decoder self-attention lets slots coordinate so only one claims each object
- C. The softmax over 92 classes removes duplicates
- D. It runs NMS inside the encoder

**Answer:** B. One-to-one matching makes a duplicate slot a ∅ target, and decoder self-attention lets slots coordinate so only one claims each object — With one-to-many assignment, several anchors are trained to fire on the same object, so duplicates are expected and NMS removes them afterwards. In DETR the loss itself penalises a second detection of the same object, and the slots can see each other through self-attention. The paper finds NMS helps the first decoder layer's outputs but not those of the last layer.

**Key points**

- Detection as set prediction: CNN → 950 tokens (stride 32, $d=256$) → 6-layer encoder → 6-layer decoder over $N=100$ learned queries → class (incl. $\varnothing$) + box per slot.
- Hungarian matching with cost $-\hat p(c)+5\,\text{L1}-2\,\text{GIoU}$ assigns each object to one slot. The loss is CE + $5\,\text{L1}+2(1-\text{GIoU})$, with $\varnothing$ weighted 0.1, applied at every decoder layer.
- No anchors and no NMS: duplicates are trained to predict ∅.
- R50, 500 epochs: 42.0 AP, the same as a strong Faster R-CNN, with +7.7 AP_L but −6.1 AP_S.
- Main weaknesses: slow convergence and small objects. Deformable DETR addresses both with sparse multi-scale attention.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
