# Face recognition

> Face recognition maps an aligned face to a unit embedding and thresholds cosine similarity; FaceNet trained it with a triplet loss, ArcFace with softmax plus an additive angular margin cos(θ + 0.5) at scale 64 (99.83% on LFW).

Source: https://www.mlfoundry.in/learn/face-recognition  
Section: Deep Learning › Deep Learning for Computer Vision  
Level: core · 8 min read · updated 2026-10-15

**Face recognition:** is the task of deciding who a face belongs to by comparing learned **embeddings**. A network maps an aligned face crop to a vector $f(x)$ on the unit sphere so that images of the same person are close in angle and images of different people are far apart. Key papers: DeepFace (Taigman et al., CVPR 2014), FaceNet (Schroff, Kalenichenko and Philbin, CVPR 2015), SphereFace (Liu et al., CVPR 2017), CosFace (Wang et al., CVPR 2018) and ArcFace (Deng, Guo, Xue and Zafeiriou, CVPR 2019).

There are two tasks. **Verification** (1:1) asks whether two faces show the same person: phone unlock, border gates. **Identification** (1:N) searches a gallery of $N$ enrolled people for the best match, and in the open-set case must also reject people who are not enrolled. Both reduce to a score $\cos(f_1, f_2)$ and a threshold. The network never sees the test identities during training, so recognition is a metric-learning problem, not a fixed classifier.

The input is the output of **Face detection**: a $112\times112$ crop warped by a five-landmark similarity transform, so the eyes and mouth sit at fixed positions. Earlier systems used hand-crafted descriptors such as LBP, or Eigenfaces (PCA of pixel intensities). DeepFace, with 3D alignment and a deep CNN, reached 97.35% on LFW in 2014, close to the human level reported for that benchmark.

> Figure: Training (top): an aligned 112×112 face passes through a ResNet-100 and a BN–dropout–FC–BN head to a 512-d embedding, L2-normalised. The ArcFace head holds one unit vector per training identity (85K for MS1MV2); it adds the margin 0.5 to the angle of the true class, scales by 64 and applies softmax cross-entropy. Deployment (bottom): the head is discarded. Two embeddings are compared by their dot product against a threshold τ chosen for a target false accept rate, or one embedding is searched against an enrolled gallery.

## Metric learning with triplets: FaceNet

FaceNet trained a 128-d embedding directly with the **Triplet loss** $\max(0,\ \lVert f(a)-f(p)\rVert^2-\lVert f(a)-f(n)\rVert^2+\alpha)$ with $\alpha = 0.2$: an anchor must be closer to a positive (same person) than to a negative by a margin. Most random triplets are already satisfied, so FaceNet mined semi-hard negatives online in large batches, training on 100 to 200 million images of about 8 million identities. It reached 99.63% on LFW. Triplet training is sensitive to mining and needs very large data, which pushed the field toward classification losses.

## Margin softmax: SphereFace, CosFace, ArcFace

Train a softmax classifier over the $C$ training identities, but remove everything except the angle. Normalise the embedding $x$ and each class weight $W_j$, so the logit is $\cos\theta_j$; multiply by a scale $s$; and make the true class $y$ harder to win by a margin inside $\psi$:

$$
\mathcal L=-\log\frac{e^{\,s\,\psi(\theta_y)}}{e^{\,s\,\psi(\theta_y)}+\sum_{j\ne y}e^{\,s\cos\theta_j}},\qquad \psi(\theta)=\begin{cases}\cos(m\theta) & \text{SphereFace},\ m=4\\ \cos\theta-m & \text{CosFace},\ m=0.35\\ \cos(\theta+m) & \text{ArcFace},\ m=0.5\end{cases}
$$

- $\theta_j$: angle between the normalised embedding $x$ and the normalised class centre $W_j$: $\cos\theta_j=W_j^\top x$
- $s$: scale (the feature radius), 64 for CosFace and ArcFace; SphereFace kept the unnormalised feature norm here
- $m$: margin: multiplicative on the angle (SphereFace), additive on the cosine (CosFace), additive on the angle (ArcFace)

Plain softmax separates two classes at the bisector $\theta_1=\theta_2$. With ArcFace, class 1 wins only when $\theta_1+m<\theta_2$, so the two decision boundaries are $m$ radians apart everywhere on the sphere: a constant **geodesic** margin. CosFace's margin is constant in cosine, not in angle; SphereFace's grows with the angle and needed a softmax-blending schedule to train. ArcFace also defines a combined form $\cos(m_1\theta+m_2)-m_3$ covering all three. In code, $\cos(\theta+m)$ is used only while $\theta+m\le\pi$, because beyond that it is no longer decreasing in $\theta$; insightface falls back to $\cos\theta-m\sin m$.

After training, the classifier weights (512 × 85K ≈ 44M numbers for MS1MV2) are discarded. Only the embedding network is deployed, and adding a new person means storing one embedding.

**Benchmarks and reference results**

- **LFW:** Labeled Faces in the Wild (Huang et al., 2007): 13,233 images of 5,749 people; 6,000 verification pairs in 10 folds, mean accuracy; saturated above 99.8%
- **IJB-B / IJB-C:** NIST benchmarks of templates (sets of stills and video frames) with large pose and quality variation; reported as TAR at FAR 1e-4 or 1e-5
- **FaceNet (2015):** 128-d, triplet loss, 99.63% on LFW
- **ArcFace (2019):** ResNet-100, 512-d, s = 64, m = 0.5, trained on MS1MV2 (a cleaned MS-Celeb-1M, 5.8M images, 85K identities): 99.83% on LFW
- **With RetinaFace alignment:** ArcFace verification on IJB-C reached TAR 89.59% at FAR 1e-6 (RetinaFace paper)

## Verification metrics: TAR at FAR

$$
\text{same}\iff f_1^\top f_2\ge\tau,\qquad \mathrm{FAR}(\tau)=\Pr\big[f_1^\top f_2\ge\tau\mid\text{impostor}\big],\qquad \mathrm{TAR}(\tau)=\Pr\big[f_1^\top f_2\ge\tau\mid\text{genuine}\big]
$$

- $\tau$: threshold fixed so that the false accept rate on impostor pairs equals a target such as $10^{-4}$
- $\mathrm{TAR}$: true accept rate at that threshold; LFW accuracy hides this, because its 3,000 impostor pairs cannot measure a FAR below 1/3000

> Figure: Two class centres 90° apart and an embedding x of class 1 between them. At the default (ArcFace, m = 0.5, s = 64, θ1 = 30°) the target logit is 64·cos(58.65°) = 33.30 against 32.00 for W2, so p = 0.786 and the loss is 0.241, while plain softmax already gives 6.7e-11. Class 1 wins only for θ1 < 30.68°, leaving a 28.65° gap, exactly m. CosFace m = 0.35 gives almost the same boundary here (30.67°); SphereFace m = 4 moves it to 18.0° (gap 54.0°). At θ1 = 45° ArcFace's loss is 27.24 against softmax's ln 2 = 0.693. Lower s to 16 at 30° and the loss rises to 0.544.

## Worked example: what the margin does to one sample

An embedding has $\cos\theta_y = 0.5$ ($\theta_y = 60°$) to its own class and cosines 0.4 and 0.2 to two others; $s = 64$.

Logits are s·ψ(θ_y), 64 × 0.4 = 25.6 and 64 × 0.2 = 12.8

| Loss | ψ(θ_y) | target logit | p(y) | −log p(y) |
| --- | --- | --- | --- | --- |
| softmax (normalised) | cos 60° = 0.5000 | 32.00 | 0.99834 | 0.0017 |
| CosFace, m = 0.35 | 0.5 − 0.35 = 0.1500 | 9.60 | 1.1e-7 | 16.00 |
| ArcFace, m = 0.5 | cos(60° + 28.65°) = 0.0236 | 1.51 | 3.5e-11 | 24.09 |

Normalised softmax treats this sample as solved, because 0.5 beats 0.4. Both margin losses say it is badly classified: the sample is pushed toward its centre until $\cos(\theta_y+m)$ beats every other cosine, which is what makes embeddings compact enough for thresholds at FAR 1e-6.

```python
import numpy as np

def arcface_logits(x, W, y, s=64.0, m=0.5):
    """x: (B, 512) embeddings, W: (512, C) class centres, y: (B,) labels -> (B, C) logits."""
    x = x / np.linalg.norm(x, axis=1, keepdims=True)
    W = W / np.linalg.norm(W, axis=0, keepdims=True)
    cos = np.clip(x @ W, -1.0, 1.0)
    th = np.arccos(cos[np.arange(len(y)), y])
    target = np.where(th + m <= np.pi, np.cos(th + m), np.cos(th) - m * np.sin(m))  # insightface fallback
    out = cos.copy()
    out[np.arange(len(y)), y] = target
    return s * out                                   # feed to ordinary softmax cross-entropy

def tar_at_far(genuine, impostor, far=1e-3):
    """Threshold = the (1 - far) quantile of impostor scores; TAR = share of genuine pairs above it."""
    tau = np.quantile(impostor, 1 - far)
    return tau, (genuine > tau).mean()

rng = np.random.default_rng(0)
x, W = rng.normal(size=(4, 512)), rng.normal(size=(512, 10))
print(arcface_logits(x, W, np.array([0, 3, 5, 9])).shape)          # (4, 10)

genuine = np.clip(rng.normal(0.45, 0.13, 5_000), -1, 1)            # cosine of same-person pairs
impostor = np.clip(rng.normal(0.02, 0.08, 500_000), -1, 1)         # different people
for far in (1e-2, 1e-3, 1e-4):
    tau, tar = tar_at_far(genuine, impostor, far)
    print(f"FAR {far:.0e}: tau = {tau:.3f}, TAR = {tar:.4f}")
# FAR 1e-02: tau = 0.207, TAR = 0.9688
# FAR 1e-03: tau = 0.269, TAR = 0.9194
# FAR 1e-04: tau = 0.318, TAR = 0.8542
```

ArcFace logits and TAR at a fixed FAR in NumPy (runs as is). The scores are synthetic Gaussians standing in for real genuine and impostor pairs.

> **Thresholds do not transfer:** A threshold tuned for FAR 1e-4 on one population, camera or image quality gives a different FAR on another. Identification multiplies the problem: with a gallery of $N$ people each search makes $N$ comparisons, so the expected number of false matches per search is about $N\cdot\mathrm{FAR}$, which is 1 for $N=10^6$ at FAR $10^{-6}$. Large galleries need far stricter thresholds than 1:1 verification.

> **Bias and privacy:** NIST's FRVT demographic study (NISTIR 8280, Grother, Ngan and Hanaoka, 2019) found that, for many algorithms, false positive rates varied by factors of 10 to over 100 between demographic groups at a fixed threshold, so a single global threshold does not give equal error rates. Training sets scraped from the web without consent, such as MS-Celeb-1M (withdrawn by Microsoft in 2019), raise legal and ethical problems. The EU AI Act bans building face-recognition databases by untargeted scraping of facial images from the internet or CCTV and tightly restricts real-time remote biometric identification in public spaces.

**Question:** ArcFace normalises both x and W_j, so every logit is a cosine in [−1, 1]. Why multiply by s = 64?

- A. To speed up training
- B. Without it, even a perfect sample cannot get a high softmax probability over tens of thousands of classes
- C. To make the margin larger in degrees
- D. It converts radians to degrees

**Answer:** B. Without it, even a perfect sample cannot get a high softmax probability over tens of thousands of classes — With $s = 1$, a sample at $\cos\theta_y = 1$ and cosine 0 to 85,000 other classes gets $p = e/(e + 84{,}999)\approx 3.2\times10^{-5}$, so the loss stays near 10 and gradients never vanish for good samples. At $s = 64$ the same sample gets $p \approx 1$. The scale sets the radius of the hypersphere the logits live on.

**Question:** With ArcFace margin m, where is the decision boundary between two classes whose centres are Δ apart, for an embedding lying between them?

- A. At Δ/2 from each centre
- B. At (Δ − m)/2 from each centre, leaving a gap of m between the two boundaries
- C. At Δ/(m + 1)
- D. At Δ − m from W1

**Answer:** B. At (Δ − m)/2 from each centre, leaving a gap of m between the two boundaries — Class 1 needs $\theta_1 + m < \theta_2 = \Delta - \theta_1$, so $\theta_1 < (\Delta - m)/2$, and symmetrically for class 2. The gap is $\Delta - 2\cdot(\Delta - m)/2 = m$ for any $\Delta$; Δ/(m + 1) is SphereFace's boundary.

**Key points**

- Verification (1:1) and identification (1:N) both threshold the cosine between L2-normalised embeddings of aligned 112×112 faces.
- FaceNet: 128-d embedding trained with the triplet loss ($\alpha = 0.2$) and semi-hard mining; 99.63% on LFW.
- Margin softmax normalises features and weights and makes the true class harder: $\cos(m\theta)$ SphereFace, $\cos\theta - m$ CosFace, $\cos(\theta + m)$ ArcFace with $s = 64$, $m = 0.5$.
- ArcFace's boundaries are m radians apart everywhere (a geodesic margin); the classifier head is dropped at deployment.
- Report TAR at a fixed FAR (IJB-B/C), not LFW accuracy; error rates differ across demographic groups, and data collection raises privacy law issues.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
