Definition
Face recognition is the task of deciding who a face belongs to by comparing learned embeddings. A network maps an aligned face crop to a vector on the unit sphere so that images of the same person are close in angle and images of different people are far apart. Key papers: DeepFace (Taigman et al., CVPR 2014), FaceNet (Schroff, Kalenichenko and Philbin, CVPR 2015), SphereFace (Liu et al., CVPR 2017), CosFace (Wang et al., CVPR 2018) and ArcFace (Deng, Guo, Xue and Zafeiriou, CVPR 2019).
There are two tasks. Verification (1:1) asks whether two faces show the same person: phone unlock, border gates. Identification (1:N) searches a gallery of enrolled people for the best match, and in the open-set case must also reject people who are not enrolled. Both reduce to a score and a threshold. The network never sees the test identities during training, so recognition is a metric-learning problem, not a fixed classifier.
The input is the output of Face detection: a crop warped by a five-landmark similarity transform, so the eyes and mouth sit at fixed positions. Earlier systems used hand-crafted descriptors such as LBP, or Eigenfaces (PCA of pixel intensities). DeepFace, with 3D alignment and a deep CNN, reached 97.35% on LFW in 2014, close to the human level reported for that benchmark.
Metric learning with triplets: FaceNet
FaceNet trained a 128-d embedding directly with the Triplet loss with : an anchor must be closer to a positive (same person) than to a negative by a margin. Most random triplets are already satisfied, so FaceNet mined semi-hard negatives online in large batches, training on 100 to 200 million images of about 8 million identities. It reached 99.63% on LFW. Triplet training is sensitive to mining and needs very large data, which pushed the field toward classification losses.
Margin softmax: SphereFace, CosFace, ArcFace
Train a softmax classifier over the training identities, but remove everything except the angle. Normalise the embedding and each class weight , so the logit is ; multiply by a scale ; and make the true class harder to win by a margin inside :
- angle between the normalised embedding and the normalised class centre :
- scale (the feature radius), 64 for CosFace and ArcFace; SphereFace kept the unnormalised feature norm here
- margin: multiplicative on the angle (SphereFace), additive on the cosine (CosFace), additive on the angle (ArcFace)
Plain softmax separates two classes at the bisector . With ArcFace, class 1 wins only when , so the two decision boundaries are radians apart everywhere on the sphere: a constant geodesic margin. CosFace's margin is constant in cosine, not in angle; SphereFace's grows with the angle and needed a softmax-blending schedule to train. ArcFace also defines a combined form covering all three. In code, is used only while , because beyond that it is no longer decreasing in ; insightface falls back to .
After training, the classifier weights (512 × 85K ≈ 44M numbers for MS1MV2) are discarded. Only the embedding network is deployed, and adding a new person means storing one embedding.
Benchmarks and reference results
- LFW
- Labeled Faces in the Wild (Huang et al., 2007): 13,233 images of 5,749 people; 6,000 verification pairs in 10 folds, mean accuracy; saturated above 99.8%
- IJB-B / IJB-C
- NIST benchmarks of templates (sets of stills and video frames) with large pose and quality variation; reported as TAR at FAR 1e-4 or 1e-5
- FaceNet (2015)
- 128-d, triplet loss, 99.63% on LFW
- ArcFace (2019)
- ResNet-100, 512-d, s = 64, m = 0.5, trained on MS1MV2 (a cleaned MS-Celeb-1M, 5.8M images, 85K identities): 99.83% on LFW
- With RetinaFace alignment
- ArcFace verification on IJB-C reached TAR 89.59% at FAR 1e-6 (RetinaFace paper)
Verification metrics: TAR at FAR
- threshold fixed so that the false accept rate on impostor pairs equals a target such as
- true accept rate at that threshold; LFW accuracy hides this, because its 3,000 impostor pairs cannot measure a FAR below 1/3000
Worked example: what the margin does to one sample
An embedding has () to its own class and cosines 0.4 and 0.2 to two others; .
| Loss | ψ(θ_y) | target logit | p(y) | −log p(y) |
|---|---|---|---|---|
| softmax (normalised) | cos 60° = 0.5000 | 32.00 | 0.99834 | 0.0017 |
| CosFace, m = 0.35 | 0.5 − 0.35 = 0.1500 | 9.60 | 1.1e-7 | 16.00 |
| ArcFace, m = 0.5 | cos(60° + 28.65°) = 0.0236 | 1.51 | 3.5e-11 | 24.09 |
Normalised softmax treats this sample as solved, because 0.5 beats 0.4. Both margin losses say it is badly classified: the sample is pushed toward its centre until beats every other cosine, which is what makes embeddings compact enough for thresholds at FAR 1e-6.
1import numpy as np23def arcface_logits(x, W, y, s=64.0, m=0.5):4 """x: (B, 512) embeddings, W: (512, C) class centres, y: (B,) labels -> (B, C) logits."""5 x = x / np.linalg.norm(x, axis=1, keepdims=True)6 W = W / np.linalg.norm(W, axis=0, keepdims=True)7 cos = np.clip(x @ W, -1.0, 1.0)8 th = np.arccos(cos[np.arange(len(y)), y])9 target = np.where(th + m <= np.pi, np.cos(th + m), np.cos(th) - m * np.sin(m)) # insightface fallback10 out = cos.copy()11 out[np.arange(len(y)), y] = target12 return s * out # feed to ordinary softmax cross-entropy1314def tar_at_far(genuine, impostor, far=1e-3):15 """Threshold = the (1 - far) quantile of impostor scores; TAR = share of genuine pairs above it."""16 tau = np.quantile(impostor, 1 - far)17 return tau, (genuine > tau).mean()1819rng = np.random.default_rng(0)20x, W = rng.normal(size=(4, 512)), rng.normal(size=(512, 10))21print(arcface_logits(x, W, np.array([0, 3, 5, 9])).shape) # (4, 10)2223genuine = np.clip(rng.normal(0.45, 0.13, 5_000), -1, 1) # cosine of same-person pairs24impostor = np.clip(rng.normal(0.02, 0.08, 500_000), -1, 1) # different people25for far in (1e-2, 1e-3, 1e-4):26 tau, tar = tar_at_far(genuine, impostor, far)27 print(f"FAR {far:.0e}: tau = {tau:.3f}, TAR = {tar:.4f}")28# FAR 1e-02: tau = 0.207, TAR = 0.968829# FAR 1e-03: tau = 0.269, TAR = 0.919430# FAR 1e-04: tau = 0.318, TAR = 0.8542Check yourself
ArcFace normalises both x and W_j, so every logit is a cosine in [−1, 1]. Why multiply by s = 64?
Check yourself
With ArcFace margin m, where is the decision boundary between two classes whose centres are Δ apart, for an embedding lying between them?
Remember
- ✓Verification (1:1) and identification (1:N) both threshold the cosine between L2-normalised embeddings of aligned 112×112 faces.
- ✓FaceNet: 128-d embedding trained with the triplet loss () and semi-hard mining; 99.63% on LFW.
- ✓Margin softmax normalises features and weights and makes the true class harder: SphereFace, CosFace, ArcFace with , .
- ✓ArcFace's boundaries are m radians apart everywhere (a geodesic margin); the classifier head is dropped at deployment.
- ✓Report TAR at a fixed FAR (IJB-B/C), not LFW accuracy; error rates differ across demographic groups, and data collection raises privacy law issues.