Deepfake detection

Deepfake detectors score faces for blending seams, spectral artefacts and temporal glitches, trained on sets like FaceForensics++, but fail on unseen generators and compression; pair them with C2PA provenance.

8 min read626 wordsadvanced0 viewsupdated 2026-10-21

Definition

Deepfake detection is the task of deciding whether an image, video or audio clip was synthesised or manipulated by a generative model, usually by a classifier that outputs p(fake∣x)p(\text{fake}\mid x). The reference benchmark for face video is FaceForensics++ (Rössler, Cozzolino, Verdoliva, Riess, Thies and Nießner, ICCV 2019). Detection is complemented by provenance: signed metadata under the C2PA standard that records how content was made.

Face swaps, reenactment, voice clones and fully generated images are used for fraud, non-consensual imagery and disinformation. Platforms, newsrooms and identity-verification services need a triage signal at upload time. The realistic deliverable is a calibrated score plus evidence for a human reviewer, not an automatic verdict.

Two families of evidence exist. Passive detection inspects the content itself and works on anything, but must guess. Active provenance relies on the creator: a signed C2PA manifest or an embedded watermark (see Watermarking generated content). It is reliable when present and says nothing when absent.

What detectors look for

CueWhy it existsExamples
Blending boundarya swapped face is pasted into a target frame, leaving a seam in colour, noise and blur statisticsFace X-ray (Li et al., CVPR 2020); self-blended images (Shiohara and Yamasaki, CVPR 2022)
Frequency artefactstransposed-convolution and upsampling layers leave periodic peaks in the image spectrumDurall et al. (CVPR 2020); Frank et al. (ICML 2020)
Physiological / temporalearly generators produced rare blinks, inconsistent head pose and lip–audio mismatch across frameseye blinking (Li, Chang and Lyu, 2018); head-pose inconsistency (Yang, Li and Lyu, 2019)
Learned featuresa CNN or ViT learns whatever separates real from fake in the training setXceptionNet in FaceForensics++; CLIP features with a linear probe (Ojha et al., CVPR 2023)
Cue families and representative methods

Hand-designed cues are interpretable but get fixed by the next generator: newer face-swap models soon blinked normally. Learned detectors are stronger in-distribution and weaker outside it, because they latch onto the artefacts of the generators they saw.

triage: check provenance first, then detect, then send evidence to a humanuploadedvideo / imageprovenance (who made it?)C2PA manifest?signed claims + hashvalid: show Content Credentialsissuer, tool, edits; bound by hashmissing / stripped / invalidno conclusion either waywatermark checkonly finds marks a cooperating generator embeddeddetection (does it look manipulated?)sampleframes,align facesdetector on face crops· blending boundary· frequency artefacts· blinks, lips, head pose· learned CNN / ViT (Xception in FF++)p(fake) per frame.91.88.35.82mean 0.74 > τ = 0.5output: score + evidence (manifest, flagged frames, heatmap) → human reviewfails on: unseen generators · re-compression (FF++ c23 → c40) · adversarial noiselow base rate: at 1% fakes, even 97.5% TPR with 3.5% FPR gives 22% precisiona "real" verdict is not proof: absence of artefacts or of a manifest proves nothing
fig.A practical triage pipeline. The provenance lane checks for a C2PA manifest: a valid signature and matching hash show who or what made the asset, while a missing or stripped manifest proves nothing. The detection lane samples frames, aligns faces and scores each crop. Here the four most suspicious frames score 0.91, 0.88, 0.35 and 0.82, mean 0.74, above τ = 0.5, so the clip goes to a reviewer with the flagged frames. With a 1% share of fakes, a detector with 97.5% TPR and 3.5% FPR is right only 22% of the times it says "fake".

Scores, thresholds and base rates

s(video)=1k∑t∈top-kp(fake∣xt),flag if s>τ,τ=quantile1−α(s on real validation videos)s(\text{video})=\frac{1}{k}\sum_{t\in\text{top-}k}p(\text{fake}\mid x_t),\qquad \text{flag if } s>\tau,\quad \tau=\text{quantile}_{1-\alpha}\big(s\text{ on real validation videos}\big)
xtx_t
aligned face crop from frame t
top-k\text{top-}k
the k most suspicious frames; averaging all frames dilutes a short manipulated segment
α\alpha
target false-positive rate on real content, e.g. 1%
precision=TPR⋅πTPR⋅π+FPR⋅(1−π)\text{precision}=\frac{\mathrm{TPR}\cdot\pi}{\mathrm{TPR}\cdot\pi+\mathrm{FPR}\cdot(1-\pi)}
π\pi
share of fakes in the traffic being screened (prevalence)
TPR, FPR\mathrm{TPR},\,\mathrm{FPR}
detection rate on fakes and false-alarm rate on real content at threshold τ

Benchmarks are often balanced (π=0.5\pi=0.5) and report accuracy or AUC. Real uploads are mostly genuine, so the false-positive rate dominates. Choose τ\tau from the FPR you can afford, measured on real content from the deployment domain, and re-measure it after every change of codec or source.

Benchmarks and results

FaceForensics++
1,000 YouTube videos manipulated with Deepfakes, Face2Face, FaceSwap and NeuralTextures; H.264 at raw, high quality (c23) and low quality (c40)
FF++ finding
XceptionNet on face crops was the strongest detector; all detectors lost accuracy at c40, and a user study found humans worse than the learned detectors
DFDC
Deepfake Detection Challenge (Meta and partners, 2020): over 100,000 clips with paid actors; the top entry reached about 65% accuracy on the held-out black-box set
Generalisation
Wang et al. (CVPR 2020): a classifier trained on ProGAN images with blur and JPEG augmentation transfers to many other CNN generators; transfer to diffusion images is much weaker
C2PA
Coalition for Content Provenance and Authenticity (founded 2021 by Adobe, Arm, BBC, Intel, Microsoft and Truepic): signed manifests of assertions, hard-bound to the asset by a cryptographic hash, shown to users as Content Credentials
linear detector trained on real vs A only · test set compression 0.0flagged fake ↗spectral (upsampling) artefact →blending-boundary artefact →realgen A, seengen B, unseenROC on the test set (dots: operating point)false-positive ratetrue-positive rateAUC A 0.996AUC B 0.696chance 0.5threshold: 5% FPR on clean real validationtest FPR 3.5% · TPR A 97.5% · B 23.0%stream with 1.0% fakes (half A, half B):precision of a "fake" flag = 14.8%the seen generator is easy; the unseen one barely crosses the line the detector learned
train on
fig.Two artefact features, 200 test images per class, and a linear (Fisher LDA) detector whose threshold gives 5% FPR on 1,000 clean real validation images. Trained on generator A only: AUC 0.996 on A but 0.696 on the unseen generator B, TPR 97.5% vs 23.0%, test FPR 3.5%. In a stream with 1% fakes (half A, half B) only 14.8% of "fake" flags are right. Compression 1.0 removes 70% of the artefact signal: AUC on A falls to 0.673 and FPR on real images rises to 12.5%. Training on A + B lifts AUC on B to 0.845 and costs A (0.965); precision at 1% stays 13.8%.

Worked example: base rates

Share of fakes πFakes caughtReal items flaggedPrecision
50%487,50017,50096.5%
10%97,50031,50075.6%
1%9,75034,65022.0%
0.1%97534,9652.7%
Detector with TPR 97.5% and FPR 3.5%, screening 1,000,000 uploads (computed in Python)

The detector is identical in every row. At 0.1% prevalence, 35 of every 36 flags are genuine content. That is why production systems use detectors to rank a review queue, raise thresholds for automated action, and combine them with provenance and account-level signals.

python
1import numpy as np
2
3def threshold_at_fpr(real_scores, target_fpr=0.01):
4 """Pick the threshold from held-out REAL videos so that target_fpr of them are flagged."""
5 return float(np.quantile(real_scores, 1 - target_fpr))
6
7def video_score(frame_probs, top=0.5):
8 """Average the most suspicious half of the frames: one manipulated segment is enough."""
9 p = np.sort(np.asarray(frame_probs))[::-1]
10 return float(p[: max(1, int(len(p) * top))].mean())
11
12def precision_at_prevalence(tpr, fpr, prevalence):
13 return tpr * prevalence / (tpr * prevalence + fpr * (1 - prevalence))
14
15rng = np.random.default_rng(0)
16real_val = rng.beta(2, 8, size=5000) # stand-in for detector scores on real validation videos
17tau = threshold_at_fpr(real_val, 0.01)
18clip = [0.12, 0.18, 0.91, 0.88, 0.35, 0.82, 0.15, 0.10] # face swap in the middle of the clip
19s = video_score(clip)
20print(round(tau, 3), round(s, 3), s > tau, round(float(np.mean(clip)), 3))
21for prev in (0.5, 0.01, 0.001):
22 print(prev, round(precision_at_prevalence(0.90, 0.01, prev), 3))
Calibrating a threshold on real validation scores, aggregating frame scores and checking precision (runs here with stand-in scores; prints 0.551 0.74 True 0.439, then precision 0.989, 0.476 and 0.083)

The mean over all eight frames, 0.439, would fall below τ=0.551\tau=0.551 and miss the swap. The top-half mean, 0.74, catches it.

Evaluating a detector before deployment

  1. 1Split by identity and by source video, never by frame: frames of one video in both train and test inflate accuracy.
  2. 2Hold out whole generators and whole datasets (for example train on FaceForensics++, test on Celeb-DF and DFDC).
  3. 3Apply the deployment transformations to the test set: platform re-encoding, resizing, cropping, screenshots.
  4. 4Calibrate τ\tau on real content from the target domain and report TPR at that FPR, not only AUC.
  5. 5Break results down by skin tone, age, gender and video quality; false positives are rarely uniform.

Use when

  • +Passive detection works on any content, including content from uncooperative generators
  • +Cheap enough to score every upload and rank a human review queue
  • +Artefact cues (seams, flagged frames, heatmaps) give reviewers evidence, not just a number

Avoid when

  • −Generalises poorly to unseen generators and compression; needs constant retraining
  • −False positives on real content at scale, especially for under-represented faces and low-quality video
  • −Evadable by adversarial perturbation or by simply re-encoding; an arms race

Check yourself

A detector has 99% TPR and 1% FPR. On a platform where 1 in 10,000 videos is a deepfake, roughly what fraction of flagged videos are actually fake?

Check yourself

An uploaded image has no C2PA manifest and no detectable watermark. What can you conclude?

Remember

  • ✓Detectors exploit blending seams, spectral upsampling artefacts, temporal glitches, or learned CNN/ViT features on aligned face crops.
  • ✓FaceForensics++ (Rössler et al., 2019): 1,000 videos, 4 manipulation methods, raw/c23/c40; XceptionNet led, compression hurt every detector.
  • ✓Generalisation is the core weakness: new generators, re-encoding and adversarial noise push accuracy toward chance (DFDC winner about 65%).
  • ✓Calibrate τ on real content from the deployment domain; precision = TPR·π / (TPR·π + FPR·(1 − π)) collapses at low prevalence.
  • ✓Pair detection with C2PA provenance and watermarks: a valid credential is strong evidence, a missing one is none; route flags to humans.
← back to Generative AI roadmap