# Deepfake detection

> Deepfake detectors score faces for blending seams, spectral artefacts and temporal glitches, trained on sets like FaceForensics++, but fail on unseen generators and compression; pair them with C2PA provenance.

Source: https://www.mlfoundry.in/learn/deepfake-detection  
Section: Generative AI › Safety, Alignment and Responsible AI  
Level: advanced · 8 min read · updated 2026-10-21

**Deepfake detection:** is the task of deciding whether an image, video or audio clip was synthesised or manipulated by a generative model, usually by a classifier that outputs $p(\text{fake}\mid x)$. The reference benchmark for face video is **FaceForensics++** (Rössler, Cozzolino, Verdoliva, Riess, Thies and Nießner, ICCV 2019). Detection is complemented by **provenance**: signed metadata under the C2PA standard that records how content was made.

Face swaps, reenactment, voice clones and fully generated images are used for fraud, non-consensual imagery and disinformation. Platforms, newsrooms and identity-verification services need a triage signal at upload time. The realistic deliverable is a calibrated score plus evidence for a human reviewer, not an automatic verdict.

Two families of evidence exist. **Passive detection** inspects the content itself and works on anything, but must guess. **Active provenance** relies on the creator: a signed C2PA manifest or an embedded watermark (see **Watermarking generated content**). It is reliable when present and says nothing when absent.

## What detectors look for

Cue families and representative methods

| Cue | Why it exists | Examples |
| --- | --- | --- |
| Blending boundary | a swapped face is pasted into a target frame, leaving a seam in colour, noise and blur statistics | Face X-ray (Li et al., CVPR 2020); self-blended images (Shiohara and Yamasaki, CVPR 2022) |
| Frequency artefacts | transposed-convolution and upsampling layers leave periodic peaks in the image spectrum | Durall et al. (CVPR 2020); Frank et al. (ICML 2020) |
| Physiological / temporal | early generators produced rare blinks, inconsistent head pose and lip–audio mismatch across frames | eye blinking (Li, Chang and Lyu, 2018); head-pose inconsistency (Yang, Li and Lyu, 2019) |
| Learned features | a CNN or ViT learns whatever separates real from fake in the training set | XceptionNet in FaceForensics++; CLIP features with a linear probe (Ojha et al., CVPR 2023) |

Hand-designed cues are interpretable but get fixed by the next generator: newer face-swap models soon blinked normally. Learned detectors are stronger in-distribution and weaker outside it, because they latch onto the artefacts of the generators they saw.

> Figure: A practical triage pipeline. The provenance lane checks for a C2PA manifest: a valid signature and matching hash show who or what made the asset, while a missing or stripped manifest proves nothing. The detection lane samples frames, aligns faces and scores each crop. Here the four most suspicious frames score 0.91, 0.88, 0.35 and 0.82, mean 0.74, above τ = 0.5, so the clip goes to a reviewer with the flagged frames. With a 1% share of fakes, a detector with 97.5% TPR and 3.5% FPR is right only 22% of the times it says "fake".

## Scores, thresholds and base rates

$$
s(\text{video})=\frac{1}{k}\sum_{t\in\text{top-}k}p(\text{fake}\mid x_t),\qquad \text{flag if } s>\tau,\quad \tau=\text{quantile}_{1-\alpha}\big(s\text{ on real validation videos}\big)
$$

- $x_t$: aligned face crop from frame t
- $\text{top-}k$: the k most suspicious frames; averaging all frames dilutes a short manipulated segment
- $\alpha$: target false-positive rate on real content, e.g. 1%

$$
\text{precision}=\frac{\mathrm{TPR}\cdot\pi}{\mathrm{TPR}\cdot\pi+\mathrm{FPR}\cdot(1-\pi)}
$$

- $\pi$: share of fakes in the traffic being screened (prevalence)
- $\mathrm{TPR},\,\mathrm{FPR}$: detection rate on fakes and false-alarm rate on real content at threshold τ

Benchmarks are often balanced ($\pi=0.5$) and report accuracy or AUC. Real uploads are mostly genuine, so the false-positive rate dominates. Choose $\tau$ from the FPR you can afford, measured on real content from the deployment domain, and re-measure it after every change of codec or source.

**Benchmarks and results**

- **FaceForensics++:** 1,000 YouTube videos manipulated with Deepfakes, Face2Face, FaceSwap and NeuralTextures; H.264 at raw, high quality (c23) and low quality (c40)
- **FF++ finding:** XceptionNet on face crops was the strongest detector; all detectors lost accuracy at c40, and a user study found humans worse than the learned detectors
- **DFDC:** Deepfake Detection Challenge (Meta and partners, 2020): over 100,000 clips with paid actors; the top entry reached about 65% accuracy on the held-out black-box set
- **Generalisation:** Wang et al. (CVPR 2020): a classifier trained on ProGAN images with blur and JPEG augmentation transfers to many other CNN generators; transfer to diffusion images is much weaker
- **C2PA:** Coalition for Content Provenance and Authenticity (founded 2021 by Adobe, Arm, BBC, Intel, Microsoft and Truepic): signed manifests of assertions, hard-bound to the asset by a cryptographic hash, shown to users as Content Credentials

> Figure: Two artefact features, 200 test images per class, and a linear (Fisher LDA) detector whose threshold gives 5% FPR on 1,000 clean real validation images. Trained on generator A only: AUC 0.996 on A but 0.696 on the unseen generator B, TPR 97.5% vs 23.0%, test FPR 3.5%. In a stream with 1% fakes (half A, half B) only 14.8% of "fake" flags are right. Compression 1.0 removes 70% of the artefact signal: AUC on A falls to 0.673 and FPR on real images rises to 12.5%. Training on A + B lifts AUC on B to 0.845 and costs A (0.965); precision at 1% stays 13.8%.

## Worked example: base rates

Detector with TPR 97.5% and FPR 3.5%, screening 1,000,000 uploads (computed in Python)

| Share of fakes π | Fakes caught | Real items flagged | Precision |
| --- | --- | --- | --- |
| 50% | 487,500 | 17,500 | 96.5% |
| 10% | 97,500 | 31,500 | 75.6% |
| 1% | 9,750 | 34,650 | 22.0% |
| 0.1% | 975 | 34,965 | 2.7% |

The detector is identical in every row. At 0.1% prevalence, 35 of every 36 flags are genuine content. That is why production systems use detectors to rank a review queue, raise thresholds for automated action, and combine them with provenance and account-level signals.

```python
import numpy as np

def threshold_at_fpr(real_scores, target_fpr=0.01):
    """Pick the threshold from held-out REAL videos so that target_fpr of them are flagged."""
    return float(np.quantile(real_scores, 1 - target_fpr))

def video_score(frame_probs, top=0.5):
    """Average the most suspicious half of the frames: one manipulated segment is enough."""
    p = np.sort(np.asarray(frame_probs))[::-1]
    return float(p[: max(1, int(len(p) * top))].mean())

def precision_at_prevalence(tpr, fpr, prevalence):
    return tpr * prevalence / (tpr * prevalence + fpr * (1 - prevalence))

rng = np.random.default_rng(0)
real_val = rng.beta(2, 8, size=5000)          # stand-in for detector scores on real validation videos
tau = threshold_at_fpr(real_val, 0.01)
clip = [0.12, 0.18, 0.91, 0.88, 0.35, 0.82, 0.15, 0.10]   # face swap in the middle of the clip
s = video_score(clip)
print(round(tau, 3), round(s, 3), s > tau, round(float(np.mean(clip)), 3))
for prev in (0.5, 0.01, 0.001):
    print(prev, round(precision_at_prevalence(0.90, 0.01, prev), 3))
```

Calibrating a threshold on real validation scores, aggregating frame scores and checking precision (runs here with stand-in scores; prints 0.551 0.74 True 0.439, then precision 0.989, 0.476 and 0.083)

The mean over all eight frames, 0.439, would fall below $\tau=0.551$ and miss the swap. The top-half mean, 0.74, catches it.

## Evaluating a detector before deployment

1. Split by identity and by source video, never by frame: frames of one video in both train and test inflate accuracy.
2. Hold out whole generators and whole datasets (for example train on FaceForensics++, test on Celeb-DF and DFDC).
3. Apply the deployment transformations to the test set: platform re-encoding, resizing, cropping, screenshots.
4. Calibrate $\tau$ on real content from the target domain and report TPR at that FPR, not only AUC.
5. Break results down by skin tone, age, gender and video quality; false positives are rarely uniform.

> **In-distribution accuracy is not a deployment number:** A detector trained and tested on the same FaceForensics++ methods can exceed 95% accuracy, then drop toward chance on a new generator, a new compression pipeline or a platform's re-encoding. Evaluate cross-dataset (train on FF++, test on Celeb-DF or DFDC), cross-generator (GANs to diffusion) and after realistic transformations: resizing, JPEG, screenshots, re-uploads. Detectors are also vulnerable to small adversarial perturbations, so an informed attacker can usually evade a known model.

> **Provenance proves origin; its absence proves nothing:** A valid C2PA manifest is cryptographic evidence of who signed the claims about an asset and of the edits recorded. But metadata is easily stripped by a screenshot or re-encode, and most content has none. So the logic is one-sided: a valid credential from a trusted issuer is strong positive evidence, while "no manifest" and "no watermark" carry no information. Watermarks embedded in pixels or audio survive more transformations; see **Watermarking generated content**.

**Strengths**

- Passive detection works on any content, including content from uncooperative generators
- Cheap enough to score every upload and rank a human review queue
- Artefact cues (seams, flagged frames, heatmaps) give reviewers evidence, not just a number

**Limitations**

- Generalises poorly to unseen generators and compression; needs constant retraining
- False positives on real content at scale, especially for under-represented faces and low-quality video
- Evadable by adversarial perturbation or by simply re-encoding; an arms race

**Question:** A detector has 99% TPR and 1% FPR. On a platform where 1 in 10,000 videos is a deepfake, roughly what fraction of flagged videos are actually fake?

- A. 99%
- B. 50%
- C. About 1%
- D. 0.01%

**Answer:** C. About 1% — Per million videos: 100 fakes, 99 caught; 999,900 real, about 9,999 flagged. Precision $=99/(99+9{,}999)\approx0.98\%$. A low base rate turns a good-looking detector into a review queue of mostly genuine videos.

**Question:** An uploaded image has no C2PA manifest and no detectable watermark. What can you conclude?

- A. It is authentic
- B. It is generated
- C. Nothing from these two checks: metadata is easily stripped and most content carries none
- D. It was edited

**Answer:** C. Nothing from these two checks: metadata is easily stripped and most content carries none — Provenance is opt-in and fragile under screenshots and re-encoding. Its presence and validity are informative, its absence is not. Only passive detection and context (source, account history) can add evidence here.

**Key points**

- Detectors exploit blending seams, spectral upsampling artefacts, temporal glitches, or learned CNN/ViT features on aligned face crops.
- FaceForensics++ (Rössler et al., 2019): 1,000 videos, 4 manipulation methods, raw/c23/c40; XceptionNet led, compression hurt every detector.
- Generalisation is the core weakness: new generators, re-encoding and adversarial noise push accuracy toward chance (DFDC winner about 65%).
- Calibrate τ on real content from the deployment domain; precision = TPR·π / (TPR·π + FPR·(1 − π)) collapses at low prevalence.
- Pair detection with C2PA provenance and watermarks: a valid credential is strong evidence, a missing one is none; route flags to humans.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
