Definition
DETR (DEtection TRansformer; Carion, Massa, Synnaeve, Usunier, Kirillov and Zagoruyko, "End-to-End Object Detection with Transformers", ECCV 2020) is an object detector that outputs a fixed-size set of predictions in one pass. A CNN backbone feeds a transformer encoder–decoder, and a bipartite (Hungarian) matching assigns each ground-truth object to exactly one prediction during training. Every other prediction is trained to say "no object" ().
Input: an image. Output: a set of (class, box) pairs, with no fixed count and no order. Earlier detectors (Faster R-CNN, RetinaNet, YOLO) reach this set indirectly. They place thousands of anchors or proposals, label each one by an IoU threshold, regress offsets, and then remove duplicates with non-maximum suppression (NMS). The anchor sizes, IoU thresholds and NMS threshold are hand-tuned, and NMS is not part of the trained model.
DETR removes all of these steps. The model predicts the set directly, and the loss is defined on sets. That needs two things: a loss that does not depend on the order of the predictions, and an architecture whose outputs can talk to each other, so that two slots do not both claim the same object.
Architecture
A ResNet turns a image into its C5 map, . A conv reduces this to channels, and the map is flattened into tokens: 950 for an input. A 6-layer transformer encoder (8 heads, FFN width 2048) applies Self-attention over all tokens, with fixed 2-D sine positional encodings added to queries and keys in every layer.
The decoder (6 layers) takes object queries, learned embeddings that act as positional codes for output slots. In each layer the queries self-attend, so slots can coordinate, and then read the image through Cross-attention to the encoder memory. All 100 slots are decoded in parallel, not autoregressively. A shared head turns each output into class probabilities over classes plus (linear + softmax) and a normalised box (3-layer MLP + sigmoid).
Hungarian matching and the set loss
Pad the ground-truth set with up to elements. Find the permutation of the predictions with the lowest total matching cost. This is a linear assignment problem, solved exactly by the Hungarian algorithm (in practice scipy.optimize.linear_sum_assignment, ):
- all permutations of the prediction slots
- predicted probability of the true class ; the probability, not its log, is used in the cost
- , with , and
With the matching fixed, the Hungarian loss is a normal detection loss on the matched pairs. Every slot gets cross-entropy on its target class. Slots matched to get no box loss, and their class term is down-weighted by 10 against class imbalance (most of the 100 slots are empty):
- the optimal assignment from the matching step; no gradient flows through it
- generalised IoU, with the smallest enclosing box (see Intersection over Union (IoU) loss); scale-invariant, unlike L1
The same loss is applied after every decoder layer, through the shared head (auxiliary losses). One-to-one matching is what removes NMS: a second slot on the same object is a target, so the decoder learns to keep only one. At inference you keep the slots whose top class is not , or whose score passes a threshold.
Configuration and COCO val results (Carion et al., 2020)
- Model
- ResNet-50 + 6 encoder + 6 decoder layers, , 8 heads, queries; 41M parameters, 86 GFLOPs, 28 FPS on a V100
- Training
- AdamW, lr (transformer) and (backbone); 300 epochs take 3 days on 16 V100s at batch 64; the 500-epoch schedule (lr drop at 400) adds 1.5 AP
- DETR R50, 500 ep
- 42.0 AP; AP_S 20.5, AP_L 61.1
- Faster R-CNN-FPN+ R50
- 42.0 AP; AP_S 26.6, AP_L 53.4 (a strengthened baseline with GIoU loss and a longer schedule)
- Larger DETR
- DC5-R50 43.3 AP (dilated C5, stride 16); R101 43.5 AP
So DETR matches a well-tuned Faster R-CNN overall, but in a different way. It is much better on large objects (+7.7 AP_L), because the encoder reasons globally. It is much worse on small ones (−6.1 AP_S), because it sees only a stride-32 map with no feature pyramid. Adding an FPN is not easy: encoder self-attention costs , and going from stride 32 to stride 16 already multiplies the tokens by 4 and the attention cost by 16.
Worked example: matching two objects
Ground truth: cat and dog in . Three slots: with ; with , , a wide box over both animals; with . Cost :
| Pair | p(c) | L1 | GIoU | Cost |
|---|---|---|---|---|
| q1 → A (cat) | 0.90 | 0.02 | 0.8182 | −0.90 + 0.10 − 1.6364 = −2.4364 |
| q2 → A (cat) | 0.60 | 0.40 | 0.2000 | −0.60 + 2.00 − 0.40 = 1.0000 |
| q2 → B (dog) | 0.30 | 0.40 | 0.2000 | −0.30 + 2.00 − 0.40 = 1.3000 |
| q3 → B (dog) | 0.80 | 0.08 | 0.7521 | −0.80 + 0.40 − 1.5041 = −1.9041 |
| q1 → B, q3 → A | 0.05, 0.10 | 0.38, 0.44 | −0.3103, −0.3016 | 2.4707, 2.7032 |
Of the 6 one-to-one assignments, costs ; the next best, , costs . So becomes a target, and its loss term is . The negative GIoU of the disjoint pairs () still says how far apart the boxes are, which plain IoU (0 for both) cannot.
1import torch2from PIL import Image3from scipy.optimize import linear_sum_assignment4from torchvision.ops import box_convert, generalized_box_iou5from transformers import DetrForObjectDetection, DetrImageProcessor67proc = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")8model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50").eval()9image = Image.open("street.jpg").convert("RGB")10with torch.no_grad():11 out = model(**proc(images=image, return_tensors="pt"))12print(out.logits.shape, out.pred_boxes.shape) # (1, 100, 92) = 91 COCO ids + no-object, (1, 100, 4)13det = proc.post_process_object_detection(out, threshold=0.9, target_sizes=[image.size[::-1]])[0]14for s, l, b in zip(det["scores"], det["labels"], det["boxes"]): # no NMS anywhere15 print(model.config.id2label[l.item()], f"{s:.2f}", [round(v) for v in b.tolist()])1617def hungarian_match(logits, boxes, tgt_labels, tgt_boxes, w_cls=1.0, w_l1=5.0, w_giou=2.0):18 """logits (N, K+1), boxes (N, 4) cxcywh in [0, 1]; returns (pred_idx, gt_idx)."""19 prob = logits.softmax(-1)20 xyxy = lambda b: box_convert(b, "cxcywh", "xyxy")21 cost = (-w_cls * prob[:, tgt_labels] # (N, M)22 + w_l1 * torch.cdist(boxes, tgt_boxes, p=1)23 - w_giou * generalized_box_iou(xyxy(boxes), xyxy(tgt_boxes)))24 return linear_sum_assignment(cost.detach().cpu().numpy())Use when
- +No anchors, no proposal stage, no NMS: the loss is defined on the final output set
- +Global reasoning in the encoder gives strong large-object AP (61.1 AP_L vs 53.4 for the R50 Faster R-CNN baseline)
- +The same query decoder extends to panoptic segmentation with a mask head, and to later set-prediction models such as Mask2Former
Avoid when
- −Very slow training: 500 epochs for 42.0 AP
- −Weak on small objects (20.5 AP_S): a single stride-32 map, and attention cost is quadratic in the number of tokens
- −N must exceed the number of objects in any image; 100 suits COCO but caps dense scenes
Check yourself
An image has 3 objects and DETR uses N = 100 queries. During training, how many slots get a box loss and how many are trained to predict ∅?
Check yourself
Why can DETR skip non-maximum suppression?
Remember
- ✓Detection as set prediction: CNN → 950 tokens (stride 32, ) → 6-layer encoder → 6-layer decoder over learned queries → class (incl. ) + box per slot.
- ✓Hungarian matching with cost assigns each object to one slot. The loss is CE + , with weighted 0.1, applied at every decoder layer.
- ✓No anchors and no NMS: duplicates are trained to predict ∅.
- ✓R50, 500 epochs: 42.0 AP, the same as a strong Faster R-CNN, with +7.7 AP_L but −6.1 AP_S.
- ✓Main weaknesses: slow convergence and small objects. Deformable DETR addresses both with sparse multi-scale attention.