Triplet loss

max(0, d(a,p) − d(a,n) + m): the anchor must be closer to its positive than to any negative by a margin. Only relative distances matter, most triplets are already satisfied, so which negatives you mine (hard, semi-hard) decides whether it learns.

12 min read820 wordscore0 viewsupdated 2026-10-11

Definition

Triplet loss is a loss on an anchor aa, a positive pp of the same class and a negative nn of a different class: L=max⁡(0, d(a,p)−d(a,n)+m)L=\max\big(0,\ d(a,p)-d(a,n)+m\big), where dd is a distance between embeddings and m>0m>0 a margin. It is zero once the negative is at least mm farther from the anchor than the positive is.

Contrastive loss asks every similar pair to be near 0 and every dissimilar pair beyond a fixed mm. That is a strong demand: some identities are naturally spread out (many poses, lighting), others are tight. The triplet loss only asks for an ordering: for each anchor, its own class should come before other classes, with a gap. That is exactly what nearest-neighbour retrieval and verification need, and it leaves the network free to choose the absolute scale of each class.

Relative-comparison constraints appear in metric learning before deep nets: Schultz and Joachims (2003) learned a metric from "A is closer to B than to C" statements, and the large-margin nearest neighbour method of Weinberger, Blitzer and Saul (2005) used the same hinge on triplets for a Mahalanobis metric. The deep version became famous with FaceNet (Schroff, Kalenichenko and Philbin, CVPR 2015). FaceNet mapped faces to 128-dimensional L2-normalised embeddings with squared distances and margin α=0.2\alpha=0.2, reached 99.63% accuracy on LFW, and showed that how triplets are chosen matters as much as the loss itself.

Formula and gradient

L(a,p,n)=max⁡(0, d(a,p)−d(a,n)+m),d(u,v)=∥u−v∥2  (PyTorch)or∥u−v∥22  (FaceNet)L(a,p,n)=\max\big(0,\ d(a,p)-d(a,n)+m\big),\qquad d(u,v)=\lVert u-v\rVert_2\ \ \text{(PyTorch)}\quad\text{or}\quad \lVert u-v\rVert_2^2\ \ \text{(FaceNet)}
a,p,na,p,n
embeddings f(xa),f(xp),f(xn)f(x_a),f(x_p),f(x_n) from one shared network; pp has the anchor's label, nn does not
mm
the margin (α\alpha in FaceNet): the gap required between the negative and positive distances
dd
Euclidean distance, or its square; the margin's scale depends on which
if L>0:∂L∂p=−a−p∥a−p∥,∂L∂n=a−n∥a−n∥,∂L∂a=a−p∥a−p∥−a−n∥a−n∥\text{if }L>0:\quad \frac{\partial L}{\partial p}=-\frac{a-p}{\lVert a-p\rVert},\qquad \frac{\partial L}{\partial n}=\frac{a-n}{\lVert a-n\rVert},\qquad \frac{\partial L}{\partial a}=\frac{a-p}{\lVert a-p\rVert}-\frac{a-n}{\lVert a-n\rVert}
−∂L/∂p-\partial L/\partial p
gradient descent moves pp toward aa along a unit vector
−∂L/∂n-\partial L/\partial n
moves nn away from aa along a unit vector
−∂L/∂a-\partial L/\partial a
moves the anchor toward pp and away from nn at once

With plain distances, every active triplet applies unit-strength forces, regardless of how badly it is violated. With FaceNet's squared distances the forces are 2(a−p)2(a-p) and 2(a−n)2(a-n), so they scale with distance. In both cases a satisfied triplet, d(a,n)≥d(a,p)+md(a,n)\ge d(a,p)+m, gives exactly zero gradient.

At a glance

Input
triplets (anchor, positive, negative), usually formed online inside a batch
Constrains
relative distances d(a,p)+m≤d(a,n)d(a,p)+m\le d(a,n), not absolute ones
Range
[0,∞)[0,\infty); a collapsed embedding (f≡f\equiv const) has loss exactly mm
Hard negative
d(a,n)<d(a,p)d(a,n)<d(a,p): the negative is closer than the positive
Semi-hard
d(a,p)≤d(a,n)<d(a,p)+md(a,p)\le d(a,n)<d(a,p)+m: correctly ordered but inside the margin
Easy
d(a,n)≥d(a,p)+md(a,n)\ge d(a,p)+m: zero loss, zero gradient
PyTorch
nn.TripletMarginLoss(margin=1.0, p=2.0, swap=False), unsquared dd; nn.TripletMarginWithDistanceLoss for a custom dd
drag any point · arrows = −∂L/∂x12345678PAd = ‖·‖ · d(A,P) = 0.949 · m = 0.80Nd(A,N)typemax(0, d_ap − d_an + m)▸10.806hard0.942▸21.082semi-hard0.667▸31.442semi-hard0.306▸41.556semi-hard0.193▸52.419easy0.000▸62.608easy0.000▸72.220easy0.000▸81.700semi-hard0.049● hard · ● semi-hard · ● easy (zero loss)mining: all 8 (mean) → L = 0.2697 · triplets with nonzero loss used: 5/8∂L/∂A = (-0.560, -0.246) · ‖∂L/∂A‖ = 0.612
negatives
distance
fig.Anchor A, positive P and eight negatives. The rose disc has radius d(A,P) (hard negatives inside), the dashed orange circle has radius d(A,P) + m (semi-hard negatives in the band). At the start, with plain distances and m = 0.8, d(A,P) = 0.949: negative 1 is hard (loss 0.942), negatives 2, 3, 4 and 8 are semi-hard, and 5, 6, 7 are easy with zero loss. Averaging all 8 triplets, as reduction='mean' does, gives L = 0.2697 although only 5 triplets are active. The hardest negative alone gives 0.9425; FaceNet's semi-hard rule picks negative 2 (d = 1.082) and gives 0.667. Switch to squared distances: d(A,P) becomes 0.900 and only negatives 1 and 2 are still inside the margin.

Mining: which negatives to use

The number of possible triplets grows cubically with the dataset, and after a little training almost all of them are easy and contribute nothing. Averaging over all of them dilutes the signal (the figure's mean of 0.2697 over 8 triplets, only 5 nonzero). So triplet training is mostly about mining: picking triplets that still violate the margin. Doing it online, within each mini-batch using the current embeddings, is the standard approach, because offline mining over the whole set goes stale as the network changes.

Common mining strategies (within a batch of P identities × K images)

  1. 1Batch-all: every valid triplet in the batch, averaged over the ones with L>0L>0 only, so easy triplets do not dilute the mean.
  2. 2Hardest (batch-hard): for each anchor, the farthest positive and the closest negative (Hermans, Beyer and Leibe, 2017). Strong signal, but sensitive to label noise and outliers.
  3. 3Semi-hard (FaceNet): for each anchor–positive pair, a negative with d(a,p)<d(a,n)<d(a,p)+md(a,p)<d(a,n)<d(a,p)+m, i.e. one that is correctly ordered but still inside the margin. FaceNet used batches of about 1,800 faces with around 40 per identity so that such negatives exist.
  4. 4Distance-weighted or random among violators: sample negatives in proportion to how informative they are rather than always taking the extreme one.

Worked example: three negatives, two distances

Anchor a=(0,0)a=(0,0), positive p=(0.6,0.8)p=(0.6,0.8) so ∥a−p∥=1.0\lVert a-p\rVert=1.0. Negatives n1=(0.5,0)n_1=(0.5,0), n2=(1.2,0)n_2=(1.2,0), n3=(0,−2)n_3=(0,-2) at distances 0.50.5, 1.21.2 and 2.02.0.

negative‖a−n‖type (plain, m=0.5)L plain‖a−n‖²L squared (α=0.2)
n1n_10.5hard1.0−0.5+0.5=1.01.0-0.5+0.5=1.00.251−0.25+0.2=0.951-0.25+0.2=0.95
n2n_21.2semi-hard1.0−1.2+0.5=0.31.0-1.2+0.5=0.31.44max⁡(0,−0.24)=0\max(0,-0.24)=0
n3n_32.0easy04.000
Per-triplet loss with plain distances (m = 0.5) and with FaceNet's squared distances (α = 0.2), checked in Node

Gradient of the semi-hard triplet (a, p, n₂), plain distances

  1. 1Unit vectors: (a−p)/∥a−p∥=(−0.6,−0.8)(a-p)/\lVert a-p\rVert=(-0.6,-0.8) and (a−n2)/∥a−n2∥=(−1,0)(a-n_2)/\lVert a-n_2\rVert=(-1,0).
  2. 2∂L/∂p=(0.6,0.8)\partial L/\partial p=(0.6,0.8), so descent moves pp by −η(0.6,0.8)-\eta(0.6,0.8): straight toward aa.
  3. 3∂L/∂n2=(−1,0)\partial L/\partial n_2=(-1,0), so descent moves n2n_2 along +x+x: straight away from aa.
  4. 4∂L/∂a=(−0.6,−0.8)−(−1,0)=(0.4,−0.8)\partial L/\partial a=(-0.6,-0.8)-(-1,0)=(0.4,-0.8): the anchor moves toward pp and away from n2n_2.
  5. 5Averaged with reduction='mean' over all three triplets the loss is (1.0+0.3+0)/3=0.4333(1.0+0.3+0)/3=0.4333. Note how n2n_2 is semi-hard under one distance and easy under the other: a margin is only meaningful for a specific dd and embedding scale.
python
1import torch, torch.nn as nn, torch.nn.functional as F
2
3a = torch.tensor([[0.0, 0.0]] * 3, requires_grad=True)
4p = torch.tensor([[0.6, 0.8]] * 3)
5n = torch.tensor([[0.5, 0.0], [1.2, 0.0], [0.0, -2.0]])
6
7loss_fn = nn.TripletMarginLoss(margin=0.5, p=2.0) # max(0, ||a-p|| - ||a-n|| + m), mean over batch
8print(loss_fn(a, p, n)) # ~0.4333 (1.0 + 0.3 + 0) / 3; eps=1e-6 shifts it slightly
9
10# Batch-hard mining (Hermans et al., 2017): batch of P identities x K images
11def batch_hard_triplet(emb, labels, margin=0.2):
12 emb = F.normalize(emb, dim=1) # put embeddings on the unit sphere
13 d = torch.cdist(emb, emb) # (B, B) pairwise distances
14 same = labels[:, None] == labels[None, :]
15 eye = torch.eye(len(labels), dtype=torch.bool)
16 hardest_pos = d.masked_fill(~same | eye, 0).amax(dim=1) # farthest positive
17 hardest_neg = d.masked_fill(same, float("inf")).amin(dim=1) # closest negative
18 return F.relu(hardest_pos - hardest_neg + margin).mean()
19
20emb = torch.randn(32, 128, requires_grad=True) # e.g. 8 identities x 4 images
21labels = torch.arange(8).repeat_interleave(4)
22batch_hard_triplet(emb, labels).backward()
nn.TripletMarginLoss, plus batch-hard online mining

Use when

  • +Only relative order matters: classes may have different spreads, which suits faces and products
  • +Directly optimises what nearest-neighbour retrieval and verification measure
  • +Works with open sets of identities; new classes need no retraining
  • +Simple loss with clear geometry: unit-strength pull on the positive, push on the negative

Avoid when

  • −Most triplets are easy and give zero gradient; needs online mining and carefully built batches (P × K)
  • −Hardest-negative mining is unstable and can collapse the embedding
  • −Margin depends on the distance (plain vs squared) and the embedding scale
  • −One negative per term: slower to converge than many-negative losses like InfoNCE

Check yourself

With squared distances and α = 0.2, d(a,p)² = 1.0. Which negative is semi-hard?

Check yourself

Every embedding collapses to the same point. What is the triplet loss, and why is this a trap?

Remember

  • ✓L=max⁡(0, d(a,p)−d(a,n)+m)L=\max(0,\,d(a,p)-d(a,n)+m): the negative must be mm farther than the positive (FaceNet, 2015).
  • ✓It constrains only relative distances, which is what retrieval and verification need.
  • ✓Hard: d(a,n) < d(a,p). Semi-hard: inside the margin but correctly ordered. Easy: zero loss.
  • ✓Mining decides success: online, within P × K batches; semi-hard or batch-hard; avoid averaging in all the zeros.
  • ✓Normalise embeddings and remember that nn.TripletMarginLoss uses unsquared distances while FaceNet used squared ones.
← back to Deep Learning roadmap