CutMix

Cut a random box out of one training image, paste in the same region of another, and mix the labels by the pasted area. Every pixel stays real, no pixels are wasted as in Cutout, and the label stays honest about what is visible.

10 min read764 wordscore0 viewsupdated 2026-10-11

Definition

CutMix is a data augmentation for images that replaces a rectangular region of image xax_a with the same region of image xbx_b and sets the target to λya+(1−λ)yb\lambda y_a+(1-\lambda)y_b, where 1−λ1-\lambda is the fraction of the image area taken from xbx_b.

Two earlier ideas each fixed half of a problem. Cutout (DeVries and Taylor, 2017) erases a random square, forcing a classifier to use the whole object instead of one discriminative part. But the erased pixels carry no information, and the label still claims 100% of a class whose evidence may have been cut away. Mixup (Zhang et al., 2018) uses every pixel and mixes the labels, but a pixel-wise blend of two photos is not a locally plausible image, which confuses tasks that depend on local detail, such as localization.

Yun, Han, Oh, Chun, Choe and Yoo (2019), CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features (ICCV), combined them: cut like Cutout, but fill the hole with a patch of another training image, and mix the labels in proportion to area. On ImageNet with ResNet-50 they reported top-1 error 21.40%, against 23.68% for the baseline, 22.93% for Cutout and 22.58% for Mixup, and better weakly supervised localization and detection transfer. CutMix and Mixup are now used together in most image-classification recipes, from DeiT (Touvron et al., 2021) to the torchvision reference models.

Formula

x~=M⊙xa+(1−M)⊙xb,y~=λya+(1−λ) yb,λ=1−rw rhWH\tilde x=M\odot x_a+(\mathbf 1-M)\odot x_b,\qquad \tilde y=\lambda y_a+(1-\lambda)\,y_b,\qquad \lambda=1-\frac{r_w\,r_h}{W H}
M∈{0,1}W×HM\in\{0,1\}^{W\times H}
binary mask: 0 inside the box, 1 elsewhere (shared across channels)
⊙\odot
element-wise product
rw,rhr_w,r_h
the box size after clipping to the image
λ\lambda
the area fraction kept from xax_a; also its label weight

Sampling the box (Yun et al., 2019; torchvision v2.CutMix)

  1. 1Draw λ∼Beta(α,α)\lambda\sim\mathrm{Beta}(\alpha,\alpha); the paper and torchvision default to α=1\alpha=1, i.e. λ∼U(0,1)\lambda\sim U(0,1).
  2. 2Box size rw=W1−λr_w=W\sqrt{1-\lambda}, rh=H1−λr_h=H\sqrt{1-\lambda}: same aspect ratio as the image and area (1−λ)WH(1-\lambda)WH.
  3. 3Box centre (cx,cy)(c_x,c_y) uniform over the image. Clip the box to the image borders.
  4. 4Clipping shrinks the box, so recompute λ=1−rwrh/(WH)\lambda=1-r_wr_h/(WH) from the actual box. The label follows the pixels, not the sampled number.
  5. 5Paste the box from the partner image (a shuffled or rolled copy of the batch) and mix the labels with the corrected λ\lambda.

At a glance

Kind
label-aware, region-level data augmentation for images (also used on spectrograms)
Hyperparameter
α\alpha (default 1), plus the probability of applying it
Label
area ratio λ\lambda after clipping; soft-target cross-entropy
Pixels
all real, from one of the two images; only the box edges are artificial
Cost
one slice copy per batch
Compared with
Cutout (box erased, label unchanged), Mixup (all pixels blended, label blended)
A: "sun"B: "stripes"mixuplabel (0.750, 0.250)real pixels 0.0%A 0.0% · B 0.0%cutoutlabel (1.000, 0.000)real pixels 75.0%A 75.0% · B 0.0%CutMixlabel (0.750, 0.250)real pixels 100.0%A 75.0% · B 25.0%λ = 0.75 → box side 20·√(1−λ) ≈ 10px, centre (10, 10) → [5, 15) × [5, 15) = 100 pxλ′ = 1 − 100/400 = 0.7500cutout: label says 100% A, but only 75.0% of the pixels show Adrag on the CutMix image to move the box · ■ A ■ B
same λ and box for all three
fig.Two procedural 20×20 images, a sun scene (A) and stripes (B), combined with the same λ and box. At λ = 0.75 and centre (10, 10) the box is [5, 15) × [5, 15), 100 pixels, so λ′ = 1 − 100/400 = 0.75. Mixup has 0% untouched pixels: every one is a blend. Cutout keeps 75% of the pixels but still labels the image (1, 0). CutMix keeps 100% real pixels, 75% from A and 25% from B, exactly matching its label (0.75, 0.25). Drag the box to the corner at (18, 18): it is clipped to 7 × 7 = 49 pixels and the label becomes (0.8775, 0.1225). The sample button draws λ and the centre as the paper does; integer box sizes also move λ′ slightly.

Why it works: local realism and honest labels

Each method changes the input and the label. The difference is whether the two stay consistent and whether the input still looks like natural images locally.

methodinputuntouched real pixelspixels showing Alabel (A, B)
Cutoutbox filled with the mean colour75%75%(1, 0)
Mixup0.75A+0.25B0.75A+0.25B everywhere0%blended(0.75, 0.25)
CutMixbox replaced by B's pixels100%75%(0.75, 0.25)
Same λ = 0.75 and box as the figure's default (20×20 images), values from the figure's code

Local realism. Apart from the edges of the box, every 3×3 neighbourhood in a CutMix image comes from a real image, so convolutional filters see the statistics they will see at test time. Honest labels. The label weight equals the share of visible evidence. No wasted pixels. The region Cutout spends on a grey square now teaches the second class. And because the network must recognize a class from a partial view and from a pasted patch in an unusual place, it learns to spread its attention over the whole object, which is why CutMix improved class-activation-map localization in the paper.

Worked example: a clipped box on a 32×32 image

λ = 0.75 drawn, CIFAR-sized image, checked in Node with the torchvision box rule

  1. 1Half sizes ⌊121−0.75⋅32⌋=⌊8⌋=8\lfloor\tfrac12\sqrt{1-0.75}\cdot32\rfloor=\lfloor8\rfloor=8, so the box is 16×1616\times16.
  2. 2Centre (16,16)(16,16): box [8,24)×[8,24)[8,24)\times[8,24), area 256, λ′=1−256/1024=0.75\lambda'=1-256/1024=0.75. Nothing changes.
  3. 3Centre (28,28)(28,28): the box [20,36)[20,36) is clipped to [20,32)[20,32) in both axes, area 12×12=14412\times12=144, so λ′=1−144/1024=0.859\lambda'=1-144/1024=0.859.
  4. 4Label (0.859, 0.141)(0.859,\,0.141). If the model predicts p(a)=0.6p(a)=0.6, p(b)=0.3p(b)=0.3, the loss is −0.859ln⁡0.6−0.141ln⁡0.3=0.6083-0.859\ln0.6-0.141\ln0.3=0.6083.
  5. 5Using the uncorrected λ=0.75\lambda=0.75 would claim 25% of class bb when only 14% of the pixels show it.
python
1import math, torch, torch.nn.functional as F
2from torchvision.transforms import v2
3
4def cutmix(x, y, num_classes, alpha=1.0):
5 """x: (N, C, H, W), y: (N,) int labels -> mixed images, soft labels."""
6 N, _, H, W = x.shape
7 lam = torch.distributions.Beta(alpha, alpha).sample().item()
8 cx, cy = torch.randint(W, ()).item(), torch.randint(H, ()).item()
9 rw, rh = int(0.5 * math.sqrt(1 - lam) * W), int(0.5 * math.sqrt(1 - lam) * H)
10 x1, x2 = max(cx - rw, 0), min(cx + rw, W)
11 y1, y2 = max(cy - rh, 0), min(cy + rh, H)
12 idx = torch.randperm(N)
13 x = x.clone()
14 x[:, :, y1:y2, x1:x2] = x[idx, :, y1:y2, x1:x2]
15 lam = 1 - (x2 - x1) * (y2 - y1) / (W * H) # area actually pasted
16 y1h = F.one_hot(y, num_classes).float()
17 return x, lam * y1h + (1 - lam) * y1h[idx]
18
19# torchvision: batch-level transforms (apply after the DataLoader or in collate_fn)
20NUM_CLASSES = 1000
21cutmix_t = v2.CutMix(alpha=1.0, num_classes=NUM_CLASSES)
22mixup_t = v2.MixUp(alpha=0.2, num_classes=NUM_CLASSES)
23cutmix_or_mixup = v2.RandomChoice([cutmix_t, mixup_t]) # one of the two per batch
24
25images = torch.rand(16, 3, 224, 224)
26labels = torch.randint(0, NUM_CLASSES, (16,))
27images, soft = cutmix_or_mixup(images, labels) # soft: (16, 1000)
28# loss = F.cross_entropy(model(images), soft)
29
30# timm's equivalent (DeiT-style recipe): one object, switch between the two per batch
31# from timm.data import Mixup
32# mix = Mixup(mixup_alpha=0.8, cutmix_alpha=1.0, switch_prob=0.5, label_smoothing=0.1, num_classes=1000)
33# loss with timm.loss.SoftTargetCrossEntropy()
CutMix by hand, and the torchvision v2 transforms with a random choice between CutMix and MixUp

Use when

  • +Every pixel is real and informative; no grey squares as in Cutout
  • +Label weights match the visible area, so targets stay honest
  • +Improves classification and localization; locally realistic inputs suit CNNs
  • +One hyperparameter and almost no compute

Avoid when

  • −Box edges create artificial boundaries the model may learn to detect
  • −The pasted patch may contain only background, so the label can still over-state what is visible
  • −Image-specific: needs a spatial layout to cut
  • −Needs soft-label losses and longer schedules to pay off

Check yourself

On a 224×224 image, CutMix pastes a 112×112 box from the partner image that lies fully inside the image. What is the label weight λ of the original image?

Check yourself

Which statement contrasts Cutout, Mixup and CutMix correctly?

Remember

  • ✓x~=M⊙xa+(1−M)⊙xb\tilde x=M\odot x_a+(1-M)\odot x_b: paste a box from xbx_b into xax_a; y~=λya+(1−λ)yb\tilde y=\lambda y_a+(1-\lambda)y_b with λ\lambda the area kept (Yun et al., 2019).
  • ✓Box side 1−λ\sqrt{1-\lambda} times the image side, centre uniform, then clip and recompute λ\lambda from the actual area.
  • ✓Fixes Cutout's wasted pixels and dishonest label, and Mixup's unrealistic blended pixels.
  • ✓ResNet-50 on ImageNet: 21.40% top-1 error versus 23.68% baseline, 22.93% Cutout and 22.58% Mixup in the paper; better localization too.
  • ✓torchvision.transforms.v2.CutMix on batches; v2.RandomChoice([CutMix, MixUp]) or timm's Mixup(switch_prob=0.5) alternate the two.
← back to Deep Learning roadmap