# FID and Inception Score

> IS = exp E KL(p(y|x) ‖ p(y)) scores generated images alone; FID is the Fréchet distance between Gaussians fitted to real and generated Inception features. Both depend on sample size and preprocessing.

Source: https://www.mlfoundry.in/learn/fid-and-inception-score  
Section: Generative AI › Evaluation of Generative Models  
Level: core · 8 min read · updated 2026-10-21

**FID and Inception Score:** are sample-based quality metrics for image generators that score images with a frozen ImageNet **Inception-v3** network. The **Inception Score (IS)** (Salimans et al., "Improved Techniques for Training GANs", NeurIPS 2016) uses its class probabilities on generated images only. The **Fréchet Inception Distance (FID)** (Heusel et al., "GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium", NeurIPS 2017) compares the distribution of its 2048-d pool features on generated images with that on real images.

Image generators such as **Generative adversarial network (GAN)** and **Diffusion models** give no usable likelihood, or one that tracks perceived quality poorly. Teams still need one number per checkpoint to pick a sampler, a guidance scale or a training run. IS and FID provide it from a few thousand samples and a fixed network, with no human raters.

In practice FID is the headline metric of image-generation papers and training dashboards. IS appears mainly in older GAN work and in class-conditional ImageNet results. For text-to-image models FID measures realism only. Prompt alignment needs a separate metric such as **CLIP score**.

## Inception Score

$$
\mathrm{IS}=\exp\!\Big(\mathbb E_{x\sim p_g}\,\mathrm{KL}\big(p(y\mid x)\,\big\|\,p(y)\big)\Big),\qquad p(y)=\mathbb E_{x\sim p_g}\,p(y\mid x)
$$

- $p(y\mid x)$: Inception class softmax for one generated image
- $p(y)$: marginal label distribution over all generated images
- $\mathrm{IS}$: between 1 and the number of classes; higher is better

The KL term is large when each image is classified confidently (low entropy of $p(y\mid x)$, read as "sharp") and the labels are spread out (high entropy of $p(y)$, read as "diverse"). Equivalently, $\log\mathrm{IS}$ is the mutual information between image and label under the generator. It never touches real images, so it cannot tell whether the samples look like the training data, and it is blind to diversity **within** a class.

## Fréchet Inception Distance

$$
\mathrm{FID}=\big\|\mu_r-\mu_g\big\|_2^2+\mathrm{Tr}\Big(\Sigma_r+\Sigma_g-2\big(\Sigma_r\Sigma_g\big)^{1/2}\Big)
$$

- $\mu_r,\Sigma_r$: mean and covariance of the 2048-d pool features of real images
- $\mu_g,\Sigma_g$: the same for generated images
- $(\Sigma_r\Sigma_g)^{1/2}$: matrix square root; lower FID is better, 0 when the two Gaussians are equal

This is the squared 2-Wasserstein (Fréchet) distance between $\mathcal N(\mu_r,\Sigma_r)$ and $\mathcal N(\mu_g,\Sigma_g)$. Only the first two moments of the features count. The mean term grows when generated features are shifted, the trace term when their spread or correlation differ. Dropping modes changes both.

> Figure: Both metrics run images through the same frozen Inception-v3 after resizing to 299×299. FID fits one Gaussian per set to the 2048-d pool features. In the toy 2-D case, μg = (1, 0) and Σg = diag(4, 1) against a standard normal give a mean term of 1 and a trace term of 1 + 1 + 4 + 1 − 2(2 + 1) = 1, so FID = 2. IS uses the class softmax of generated images only: three confident images on three different classes give a uniform marginal and IS = exp(0.704) = 2.02, out of a maximum of 3.

## Sample-size bias and KID

FID is computed from sample estimates of $\mu$ and $\Sigma$, and the estimator is biased upward: two sets of samples from the **same** distribution give FID > 0, and the gap shrinks with $N$. Chong and Forsyth (CVPR 2020) showed the bias is roughly linear in $1/N$ and that it depends on the generator, so a ranking at $N=5{,}000$ can differ from one at $N=50{,}000$. They proposed extrapolating to $\mathrm{FID}_\infty$. IS is biased too.

$$
\mathrm{KID}=\frac{1}{m(m-1)}\sum_{i\ne j}\big[k(x_i,x_j)+k(y_i,y_j)\big]-\frac{2}{m^2}\sum_{i,j}k(x_i,y_j),\qquad k(x,y)=\Big(\tfrac{1}{d}x^\top y+1\Big)^3
$$

- $x_i,\,y_j$: Inception features of $m$ real and $m$ generated images, $d=2048$
- $k$: cubic polynomial kernel; it matches moments up to the third
- $\mathrm{KID}$: unbiased estimate of squared MMD; its expectation is 0 for identical distributions, so single estimates can be negative

The **Kernel Inception Distance** (Bińkowski et al., "Demystifying MMD GANs", ICLR 2018) uses the same features but has an unbiased estimator and makes no Gaussian assumption. It is the better choice when only a few thousand images are available, for example in few-shot or per-subject evaluation.

**Conventions and known results**

- **IS protocol:** Salimans et al.: 50,000 samples, split into 10 groups, reported as mean ± std over the groups; real CIFAR-10 training images score 11.24 ± 0.12
- **FID protocol:** pool3 features (2048-d) of Inception-v3; the authors recommend at least 10,000 samples, and 50,000 generated images against the training set is the common GAN convention
- **Text-to-image:** zero-shot FID on 30,000 MS-COCO validation captions (FID-30K); Imagen (Saharia et al., 2022) reported 7.27
- **Class-conditional:** BigGAN at 128×128 on ImageNet: IS 166.5 and FID 7.4 (Brock et al., ICLR 2019), up from the previous best of 52.52 and 18.65
- **Resizing:** Parmar et al. (CVPR 2022, clean-fid): different resize filters and JPEG compression of the same images change FID enough to reorder models
- **Class leakage:** Kynkäänniemi et al. (ICLR 2023): FID can be lowered by matching the ImageNet class histogram of the real set, without better images

> Figure: Inception features replaced by 2-D points: real data has four Gaussian modes (σ = 0.45), and the IS classifier is the Bayes posterior of those modes. At the default (N = 200, generator = real distribution) the population FID is 0.000, but the sample FID is 0.015 and KID is −0.193. The FID-vs-N inset falls from 0.103 at N = 25 to 0.006 at N = 1,000: that is pure estimator bias. Keep 1 mode: IS drops to 1.026 and FID rises to 5.427 (population 6.003). Keep 2 modes: IS 1.980, FID 2.449. Within-mode collapse (σ×0.3) raises IS to 3.960, above real data's 3.914, while FID moves only to 0.022: a single Gaussian over four modes barely sees it. Shift by 0.8: FID 0.625, of which 0.612 is the mean term.

## Worked example

IS on three generated images, 3 classes (natural logs; computed in Python)

| Generated set | p(y\|x) per image | p(y) | KL per image | IS |
| --- | --- | --- | --- | --- |
| sharp, diverse | (.9,.05,.05) (.05,.9,.05) (.05,.05,.9) | (.333, .333, .333) | 0.704 each | exp(0.704) = 2.022 |
| one blurry image | (.9,.05,.05) (.05,.9,.05) (.34,.33,.33) | (.430, .427, .143) | 0.505, 0.512, 0.111 | exp(0.376) = 1.456 |
| collapsed | (.9,.05,.05) three times | (.9, .05, .05) | 0, 0, 0 | exp(0) = 1.000 |

The collapsed set is three confident images of one class: each one is sharp, yet IS is at its minimum, because $p(y\mid x)=p(y)$. For FID, take the diagram's toy Gaussians: $\|\mu_r-\mu_g\|^2=1$, $\mathrm{Tr}(\Sigma_r)+\mathrm{Tr}(\Sigma_g)=2+5=7$, and $(\Sigma_r\Sigma_g)^{1/2}=\mathrm{diag}(2,1)$ with trace 3, so FID $=1+7-6=2$.

```python
import numpy as np

def psd_sqrt(a):
    w, v = np.linalg.eigh(a)
    return (v * np.sqrt(np.clip(w, 0, None))) @ v.T

def fid(feat_r, feat_g):
    """feat_*: (n, d) pool features of real and generated images."""
    mu_r, mu_g = feat_r.mean(0), feat_g.mean(0)
    s_r, s_g = np.cov(feat_r, rowvar=False), np.cov(feat_g, rowvar=False)
    r = psd_sqrt(s_r)                           # Tr((S_r S_g)^1/2) = Tr((r S_g r)^1/2), r = S_r^1/2
    tr_sqrt = np.trace(psd_sqrt(r @ s_g @ r))
    return float(((mu_r - mu_g) ** 2).sum() + np.trace(s_r) + np.trace(s_g) - 2 * tr_sqrt)

def kid(feat_r, feat_g):
    """Unbiased MMD^2 with the cubic polynomial kernel of Binkowski et al. (2018)."""
    d, m = feat_r.shape[1], len(feat_r)
    k = lambda a, b: (a @ b.T / d + 1) ** 3
    kxx, kyy, kxy = k(feat_r, feat_r), k(feat_g, feat_g), k(feat_r, feat_g)
    off = lambda K: (K.sum() - np.trace(K)) / (m * (m - 1))
    return float(off(kxx) + off(kyy) - 2 * kxy.mean())

rng = np.random.default_rng(0)
d = 64                                          # stand-in for 2048-d Inception features
real = rng.normal(size=(5000, d))
same = rng.normal(size=(5000, d))               # identical distribution
worse = 1.3 * rng.normal(size=(5000, d)) + 0.1  # wider and shifted
for n in (500, 5000):
    print(n, round(fid(real[:n], same[:n]), 3), round(fid(real[:n], worse[:n]), 3),
          round(kid(real[:n], same[:n]), 4), round(kid(real[:n], worse[:n]), 4))
```

FID and KID from feature matrices in numpy (run here on synthetic 64-d features). Prints 500 4.541 12.233 0.0009 0.0522 and 5000 0.466 6.96 0.0002 0.0542: for identical distributions FID falls from 4.54 to 0.47 as N grows, while KID stays near 0. The 'worse' set has population FID 6.40.

For real evaluations use a maintained implementation that fixes the network weights and resizing, such as `pytorch-fid`, `clean-fid` or `torchmetrics`, and compute the reference statistics once with the same tool.

> **Numbers from different pipelines are not comparable:** FID depends on $N$, the reference set (train or validation, how many images), the Inception weights (the original TensorFlow graph or a PyTorch port), the resize filter and the image format. Changing any of these can move FID by more than the gap between two papers. Report all of them, compute baselines yourself with the same code, and never compare a 10k-sample FID with a 50k-sample one.

> **FID is an ImageNet-feature distance:** Inception features encode what helps ImageNet classification. Generators can improve FID by matching class statistics rather than image quality, and the features say little about faces, text rendering, medical images or prompt fidelity. Pair FID with precision and recall (Kynkäänniemi et al., 2019) to separate fidelity from coverage, with human preference studies (**Human evaluation and arena-style ranking**), and consider DINOv2 or CLIP feature spaces. CMMD (Jayasumana et al., CVPR 2024) uses CLIP embeddings with an MMD distance.

**Strengths**

- FID: one number that responds to both fidelity and mode coverage, and is compared against real data
- Cheap: one forward pass per image through a fixed network, no raters
- KID: unbiased, so usable with a few thousand images

**Limitations**

- IS ignores real data and within-class diversity, and is capped by the 1,000 ImageNet classes
- FID assumes Gaussian features, is biased at small N and depends on preprocessing
- Neither measures prompt alignment, and both can be gamed by matching ImageNet statistics

**Question:** A generator memorises one perfect image per ImageNet class and outputs only those 1,000 images. What happens to IS and FID?

- A. Both look bad
- B. IS is near its maximum; FID is worse than for a model with real within-class variety
- C. IS is low; FID is near 0
- D. Both look excellent

**Answer:** B. IS is near its maximum; FID is worse than for a model with real within-class variety — Each image is classified confidently and the labels are uniform, so $p(y\mid x)$ is peaked and $p(y)$ is flat: IS approaches 1,000. FID compares feature covariances with real data, and 1,000 points have far less spread within each class than real images, so the trace term penalises the lack of variety.

**Question:** You compute FID between two disjoint halves of the same real test set, 2,000 images each, and get a clearly positive value. Why?

- A. The test set is corrupted
- B. The sample-mean and sample-covariance estimates make FID biased upward at finite N
- C. FID is not symmetric
- D. Inception-v3 is non-deterministic

**Answer:** B. The sample-mean and sample-covariance estimates make FID biased upward at finite N — The population FID between identical distributions is 0, but the plug-in estimator is not unbiased. With 2048-d features and only 2,000 images the covariance estimates are noisy, and the noise adds a positive offset that shrinks roughly like $1/N$. KID's unbiased estimator would be close to 0.

**Key points**

- IS $=\exp\mathbb E_x\,\mathrm{KL}(p(y\mid x)\,\|\,p(y))$ on generated images only: high for confident, varied labels, between 1 and the class count.
- FID $=\|\mu_r-\mu_g\|^2+\mathrm{Tr}(\Sigma_r+\Sigma_g-2(\Sigma_r\Sigma_g)^{1/2})$ on 2048-d Inception pool features; lower is better.
- Both are biased by sample size: report N (50k is the GAN convention, FID-30K on COCO for text-to-image) and never mix N.
- KID (cubic-kernel MMD) is unbiased and Gaussian-free, so it suits small evaluation sets.
- Same weights, resizing and reference statistics or the numbers are not comparable; add precision/recall, CLIP score and human studies.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
