Definition
Deepfake detection is the task of deciding whether an image, video or audio clip was synthesised or manipulated by a generative model, usually by a classifier that outputs . The reference benchmark for face video is FaceForensics++ (Rössler, Cozzolino, Verdoliva, Riess, Thies and Nießner, ICCV 2019). Detection is complemented by provenance: signed metadata under the C2PA standard that records how content was made.
Face swaps, reenactment, voice clones and fully generated images are used for fraud, non-consensual imagery and disinformation. Platforms, newsrooms and identity-verification services need a triage signal at upload time. The realistic deliverable is a calibrated score plus evidence for a human reviewer, not an automatic verdict.
Two families of evidence exist. Passive detection inspects the content itself and works on anything, but must guess. Active provenance relies on the creator: a signed C2PA manifest or an embedded watermark (see Watermarking generated content). It is reliable when present and says nothing when absent.
What detectors look for
| Cue | Why it exists | Examples |
|---|---|---|
| Blending boundary | a swapped face is pasted into a target frame, leaving a seam in colour, noise and blur statistics | Face X-ray (Li et al., CVPR 2020); self-blended images (Shiohara and Yamasaki, CVPR 2022) |
| Frequency artefacts | transposed-convolution and upsampling layers leave periodic peaks in the image spectrum | Durall et al. (CVPR 2020); Frank et al. (ICML 2020) |
| Physiological / temporal | early generators produced rare blinks, inconsistent head pose and lip–audio mismatch across frames | eye blinking (Li, Chang and Lyu, 2018); head-pose inconsistency (Yang, Li and Lyu, 2019) |
| Learned features | a CNN or ViT learns whatever separates real from fake in the training set | XceptionNet in FaceForensics++; CLIP features with a linear probe (Ojha et al., CVPR 2023) |
Hand-designed cues are interpretable but get fixed by the next generator: newer face-swap models soon blinked normally. Learned detectors are stronger in-distribution and weaker outside it, because they latch onto the artefacts of the generators they saw.
Scores, thresholds and base rates
- aligned face crop from frame t
- the k most suspicious frames; averaging all frames dilutes a short manipulated segment
- target false-positive rate on real content, e.g. 1%
- share of fakes in the traffic being screened (prevalence)
- detection rate on fakes and false-alarm rate on real content at threshold τ
Benchmarks are often balanced () and report accuracy or AUC. Real uploads are mostly genuine, so the false-positive rate dominates. Choose from the FPR you can afford, measured on real content from the deployment domain, and re-measure it after every change of codec or source.
Benchmarks and results
- FaceForensics++
- 1,000 YouTube videos manipulated with Deepfakes, Face2Face, FaceSwap and NeuralTextures; H.264 at raw, high quality (c23) and low quality (c40)
- FF++ finding
- XceptionNet on face crops was the strongest detector; all detectors lost accuracy at c40, and a user study found humans worse than the learned detectors
- DFDC
- Deepfake Detection Challenge (Meta and partners, 2020): over 100,000 clips with paid actors; the top entry reached about 65% accuracy on the held-out black-box set
- Generalisation
- Wang et al. (CVPR 2020): a classifier trained on ProGAN images with blur and JPEG augmentation transfers to many other CNN generators; transfer to diffusion images is much weaker
- C2PA
- Coalition for Content Provenance and Authenticity (founded 2021 by Adobe, Arm, BBC, Intel, Microsoft and Truepic): signed manifests of assertions, hard-bound to the asset by a cryptographic hash, shown to users as Content Credentials
Worked example: base rates
| Share of fakes π | Fakes caught | Real items flagged | Precision |
|---|---|---|---|
| 50% | 487,500 | 17,500 | 96.5% |
| 10% | 97,500 | 31,500 | 75.6% |
| 1% | 9,750 | 34,650 | 22.0% |
| 0.1% | 975 | 34,965 | 2.7% |
The detector is identical in every row. At 0.1% prevalence, 35 of every 36 flags are genuine content. That is why production systems use detectors to rank a review queue, raise thresholds for automated action, and combine them with provenance and account-level signals.
1import numpy as np23def threshold_at_fpr(real_scores, target_fpr=0.01):4 """Pick the threshold from held-out REAL videos so that target_fpr of them are flagged."""5 return float(np.quantile(real_scores, 1 - target_fpr))67def video_score(frame_probs, top=0.5):8 """Average the most suspicious half of the frames: one manipulated segment is enough."""9 p = np.sort(np.asarray(frame_probs))[::-1]10 return float(p[: max(1, int(len(p) * top))].mean())1112def precision_at_prevalence(tpr, fpr, prevalence):13 return tpr * prevalence / (tpr * prevalence + fpr * (1 - prevalence))1415rng = np.random.default_rng(0)16real_val = rng.beta(2, 8, size=5000) # stand-in for detector scores on real validation videos17tau = threshold_at_fpr(real_val, 0.01)18clip = [0.12, 0.18, 0.91, 0.88, 0.35, 0.82, 0.15, 0.10] # face swap in the middle of the clip19s = video_score(clip)20print(round(tau, 3), round(s, 3), s > tau, round(float(np.mean(clip)), 3))21for prev in (0.5, 0.01, 0.001):22 print(prev, round(precision_at_prevalence(0.90, 0.01, prev), 3))The mean over all eight frames, 0.439, would fall below and miss the swap. The top-half mean, 0.74, catches it.
Evaluating a detector before deployment
- 1Split by identity and by source video, never by frame: frames of one video in both train and test inflate accuracy.
- 2Hold out whole generators and whole datasets (for example train on FaceForensics++, test on Celeb-DF and DFDC).
- 3Apply the deployment transformations to the test set: platform re-encoding, resizing, cropping, screenshots.
- 4Calibrate on real content from the target domain and report TPR at that FPR, not only AUC.
- 5Break results down by skin tone, age, gender and video quality; false positives are rarely uniform.
Use when
- +Passive detection works on any content, including content from uncooperative generators
- +Cheap enough to score every upload and rank a human review queue
- +Artefact cues (seams, flagged frames, heatmaps) give reviewers evidence, not just a number
Avoid when
- −Generalises poorly to unseen generators and compression; needs constant retraining
- −False positives on real content at scale, especially for under-represented faces and low-quality video
- −Evadable by adversarial perturbation or by simply re-encoding; an arms race
Check yourself
A detector has 99% TPR and 1% FPR. On a platform where 1 in 10,000 videos is a deepfake, roughly what fraction of flagged videos are actually fake?
Check yourself
An uploaded image has no C2PA manifest and no detectable watermark. What can you conclude?
Remember
- ✓Detectors exploit blending seams, spectral upsampling artefacts, temporal glitches, or learned CNN/ViT features on aligned face crops.
- ✓FaceForensics++ (Rössler et al., 2019): 1,000 videos, 4 manipulation methods, raw/c23/c40; XceptionNet led, compression hurt every detector.
- ✓Generalisation is the core weakness: new generators, re-encoding and adversarial noise push accuracy toward chance (DFDC winner about 65%).
- ✓Calibrate τ on real content from the deployment domain; precision = TPR·π / (TPR·π + FPR·(1 − π)) collapses at low prevalence.
- ✓Pair detection with C2PA provenance and watermarks: a valid credential is strong evidence, a missing one is none; route flags to humans.