FID and Inception Score

IS = exp E KL(p(y|x) ‖ p(y)) scores generated images alone; FID is the Fréchet distance between Gaussians fitted to real and generated Inception features. Both depend on sample size and preprocessing.

8 min read710 wordscore0 viewsupdated 2026-10-21

Definition

FID and Inception Score are sample-based quality metrics for image generators that score images with a frozen ImageNet Inception-v3 network. The Inception Score (IS) (Salimans et al., "Improved Techniques for Training GANs", NeurIPS 2016) uses its class probabilities on generated images only. The Fréchet Inception Distance (FID) (Heusel et al., "GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium", NeurIPS 2017) compares the distribution of its 2048-d pool features on generated images with that on real images.

Image generators such as Generative adversarial network (GAN) and Diffusion models give no usable likelihood, or one that tracks perceived quality poorly. Teams still need one number per checkpoint to pick a sampler, a guidance scale or a training run. IS and FID provide it from a few thousand samples and a fixed network, with no human raters.

In practice FID is the headline metric of image-generation papers and training dashboards. IS appears mainly in older GAN work and in class-conditional ImageNet results. For text-to-image models FID measures realism only. Prompt alignment needs a separate metric such as CLIP score.

Inception Score

IS=exp⁡ ⁣(Ex∼pg KL(p(y∣x) ∥ p(y))),p(y)=Ex∼pg p(y∣x)\mathrm{IS}=\exp\!\Big(\mathbb E_{x\sim p_g}\,\mathrm{KL}\big(p(y\mid x)\,\big\|\,p(y)\big)\Big),\qquad p(y)=\mathbb E_{x\sim p_g}\,p(y\mid x)
p(y∣x)p(y\mid x)
Inception class softmax for one generated image
p(y)p(y)
marginal label distribution over all generated images
IS\mathrm{IS}
between 1 and the number of classes; higher is better

The KL term is large when each image is classified confidently (low entropy of p(y∣x)p(y\mid x), read as "sharp") and the labels are spread out (high entropy of p(y)p(y), read as "diverse"). Equivalently, log⁡IS\log\mathrm{IS} is the mutual information between image and label under the generator. It never touches real images, so it cannot tell whether the samples look like the training data, and it is blind to diversity within a class.

Fréchet Inception Distance

FID=∥μr−μg∥22+Tr(Σr+Σg−2(ΣrΣg)1/2)\mathrm{FID}=\big\|\mu_r-\mu_g\big\|_2^2+\mathrm{Tr}\Big(\Sigma_r+\Sigma_g-2\big(\Sigma_r\Sigma_g\big)^{1/2}\Big)
μr,Σr\mu_r,\Sigma_r
mean and covariance of the 2048-d pool features of real images
μg,Σg\mu_g,\Sigma_g
the same for generated images
(ΣrΣg)1/2(\Sigma_r\Sigma_g)^{1/2}
matrix square root; lower FID is better, 0 when the two Gaussians are equal

This is the squared 2-Wasserstein (Fréchet) distance between N(μr,Σr)\mathcal N(\mu_r,\Sigma_r) and N(μg,Σg)\mathcal N(\mu_g,\Sigma_g). Only the first two moments of the features count. The mean term grows when generated features are shifted, the trace term when their spread or correlation differ. Dropping modes changes both.

one frozen network, two scores: only FID compares against real imagesreal imagese.g. 50k train setgenerated imagese.g. 50k samplesresize to 299×299filter changes FIDInception-v3, ImageNet weightsfrozen; same weights for everyonepool3 features: 2048-d per imagereal: μr ∈ ℝ²⁰⁴⁸, Σr ∈ ℝ²⁰⁴⁸ˣ²⁰⁴⁸generated: μg, Σg (one Gaussian per set)FID = ‖μr − μg‖² + Tr(Σr + Σg − 2(ΣrΣg)^½)toy 2-D: μr = (0, 0), Σr = Iμg = (1, 0), Σg = diag(4, 1)mean 1 + trace 7 − 2(2 + 1) = 1FID = 2 · lower is better, 0 = sameclass softmax p(y|x): generated onlyimage 1.9.05.05image 2.05.9.05image 3.05.05.9sharp: each p(y|x) peaked · diverse: p(y) = (⅓, ⅓, ⅓)IS = exp(𝔼ₓ KL(p(y|x) ‖ p(y)))IS = exp(0.704) = 2.02 · range 1 … 3a published number is only comparable with the same N, Inception weights, resizing and reference setthe sample FID is biased upward at small N; KID (unbiased MMD, cubic kernel) is not
fig.Both metrics run images through the same frozen Inception-v3 after resizing to 299×299. FID fits one Gaussian per set to the 2048-d pool features. In the toy 2-D case, μg = (1, 0) and Σg = diag(4, 1) against a standard normal give a mean term of 1 and a trace term of 1 + 1 + 4 + 1 − 2(2 + 1) = 1, so FID = 2. IS uses the class softmax of generated images only: three confident images on three different classes give a uniform marginal and IS = exp(0.704) = 2.02, out of a maximum of 3.

Sample-size bias and KID

FID is computed from sample estimates of μ\mu and Σ\Sigma, and the estimator is biased upward: two sets of samples from the same distribution give FID > 0, and the gap shrinks with NN. Chong and Forsyth (CVPR 2020) showed the bias is roughly linear in 1/N1/N and that it depends on the generator, so a ranking at N=5,000N=5{,}000 can differ from one at N=50,000N=50{,}000. They proposed extrapolating to FID∞\mathrm{FID}_\infty. IS is biased too.

KID=1m(m−1)∑i≠j[k(xi,xj)+k(yi,yj)]−2m2∑i,jk(xi,yj),k(x,y)=(1dx⊤y+1)3\mathrm{KID}=\frac{1}{m(m-1)}\sum_{i\ne j}\big[k(x_i,x_j)+k(y_i,y_j)\big]-\frac{2}{m^2}\sum_{i,j}k(x_i,y_j),\qquad k(x,y)=\Big(\tfrac{1}{d}x^\top y+1\Big)^3
xi, yjx_i,\,y_j
Inception features of mm real and mm generated images, d=2048d=2048
kk
cubic polynomial kernel; it matches moments up to the third
KID\mathrm{KID}
unbiased estimate of squared MMD; its expectation is 0 for identical distributions, so single estimates can be negative

The Kernel Inception Distance (Bińkowski et al., "Demystifying MMD GANs", ICLR 2018) uses the same features but has an unbiased estimator and makes no Gaussian assumption. It is the better choice when only a few thousand images are available, for example in few-shot or per-subject evaluation.

Conventions and known results

IS protocol
Salimans et al.: 50,000 samples, split into 10 groups, reported as mean ± std over the groups; real CIFAR-10 training images score 11.24 ± 0.12
FID protocol
pool3 features (2048-d) of Inception-v3; the authors recommend at least 10,000 samples, and 50,000 generated images against the training set is the common GAN convention
Text-to-image
zero-shot FID on 30,000 MS-COCO validation captions (FID-30K); Imagen (Saharia et al., 2022) reported 7.27
Class-conditional
BigGAN at 128×128 on ImageNet: IS 166.5 and FID 7.4 (Brock et al., ICLR 2019), up from the previous best of 52.52 and 18.65
Resizing
Parmar et al. (CVPR 2022, clean-fid): different resize filters and JPEG compression of the same images change FID enough to reorder models
Class leakage
Kynkäänniemi et al. (ICLR 2023): FID can be lowered by matching the ImageNet class histogram of the real set, without better images
N = 200 real + 200 generated · 2-D features · IS classifier: Bayes posteriorclass 1class 2class 3class 4realgenerated (4 modes, σ×1.0, shift 0.0)FID, sample estimate0.015 = 0.002 + 0.013‖μr − μg‖² + Tr(Σr + Σg − 2(ΣrΣg)^½)FID, exact population: 0.000KID (unbiased MMD²): -0.193IS generated 3.940 · real 3.914 · max 4marginal p(y) of generated samples1: 0.242: 0.303: 0.254: 0.20sample FID vs N (dashed: population)0.13025501002005001000same distribution: population FID = 0, the sample estimate is positive and shrinks with N
presets
fig.Inception features replaced by 2-D points: real data has four Gaussian modes (σ = 0.45), and the IS classifier is the Bayes posterior of those modes. At the default (N = 200, generator = real distribution) the population FID is 0.000, but the sample FID is 0.015 and KID is −0.193. The FID-vs-N inset falls from 0.103 at N = 25 to 0.006 at N = 1,000: that is pure estimator bias. Keep 1 mode: IS drops to 1.026 and FID rises to 5.427 (population 6.003). Keep 2 modes: IS 1.980, FID 2.449. Within-mode collapse (σ×0.3) raises IS to 3.960, above real data's 3.914, while FID moves only to 0.022: a single Gaussian over four modes barely sees it. Shift by 0.8: FID 0.625, of which 0.612 is the mean term.

Worked example

Generated setp(y|x) per imagep(y)KL per imageIS
sharp, diverse(.9,.05,.05) (.05,.9,.05) (.05,.05,.9)(.333, .333, .333)0.704 eachexp(0.704) = 2.022
one blurry image(.9,.05,.05) (.05,.9,.05) (.34,.33,.33)(.430, .427, .143)0.505, 0.512, 0.111exp(0.376) = 1.456
collapsed(.9,.05,.05) three times(.9, .05, .05)0, 0, 0exp(0) = 1.000
IS on three generated images, 3 classes (natural logs; computed in Python)

The collapsed set is three confident images of one class: each one is sharp, yet IS is at its minimum, because p(y∣x)=p(y)p(y\mid x)=p(y). For FID, take the diagram's toy Gaussians: ∥μr−μg∥2=1\|\mu_r-\mu_g\|^2=1, Tr(Σr)+Tr(Σg)=2+5=7\mathrm{Tr}(\Sigma_r)+\mathrm{Tr}(\Sigma_g)=2+5=7, and (ΣrΣg)1/2=diag(2,1)(\Sigma_r\Sigma_g)^{1/2}=\mathrm{diag}(2,1) with trace 3, so FID =1+7−6=2=1+7-6=2.

python
1import numpy as np
2
3def psd_sqrt(a):
4 w, v = np.linalg.eigh(a)
5 return (v * np.sqrt(np.clip(w, 0, None))) @ v.T
6
7def fid(feat_r, feat_g):
8 """feat_*: (n, d) pool features of real and generated images."""
9 mu_r, mu_g = feat_r.mean(0), feat_g.mean(0)
10 s_r, s_g = np.cov(feat_r, rowvar=False), np.cov(feat_g, rowvar=False)
11 r = psd_sqrt(s_r) # Tr((S_r S_g)^1/2) = Tr((r S_g r)^1/2), r = S_r^1/2
12 tr_sqrt = np.trace(psd_sqrt(r @ s_g @ r))
13 return float(((mu_r - mu_g) ** 2).sum() + np.trace(s_r) + np.trace(s_g) - 2 * tr_sqrt)
14
15def kid(feat_r, feat_g):
16 """Unbiased MMD^2 with the cubic polynomial kernel of Binkowski et al. (2018)."""
17 d, m = feat_r.shape[1], len(feat_r)
18 k = lambda a, b: (a @ b.T / d + 1) ** 3
19 kxx, kyy, kxy = k(feat_r, feat_r), k(feat_g, feat_g), k(feat_r, feat_g)
20 off = lambda K: (K.sum() - np.trace(K)) / (m * (m - 1))
21 return float(off(kxx) + off(kyy) - 2 * kxy.mean())
22
23rng = np.random.default_rng(0)
24d = 64 # stand-in for 2048-d Inception features
25real = rng.normal(size=(5000, d))
26same = rng.normal(size=(5000, d)) # identical distribution
27worse = 1.3 * rng.normal(size=(5000, d)) + 0.1 # wider and shifted
28for n in (500, 5000):
29 print(n, round(fid(real[:n], same[:n]), 3), round(fid(real[:n], worse[:n]), 3),
30 round(kid(real[:n], same[:n]), 4), round(kid(real[:n], worse[:n]), 4))
FID and KID from feature matrices in numpy (run here on synthetic 64-d features). Prints 500 4.541 12.233 0.0009 0.0522 and 5000 0.466 6.96 0.0002 0.0542: for identical distributions FID falls from 4.54 to 0.47 as N grows, while KID stays near 0. The 'worse' set has population FID 6.40.

For real evaluations use a maintained implementation that fixes the network weights and resizing, such as pytorch-fid, clean-fid or torchmetrics, and compute the reference statistics once with the same tool.

Use when

  • +FID: one number that responds to both fidelity and mode coverage, and is compared against real data
  • +Cheap: one forward pass per image through a fixed network, no raters
  • +KID: unbiased, so usable with a few thousand images

Avoid when

  • −IS ignores real data and within-class diversity, and is capped by the 1,000 ImageNet classes
  • −FID assumes Gaussian features, is biased at small N and depends on preprocessing
  • −Neither measures prompt alignment, and both can be gamed by matching ImageNet statistics

Check yourself

A generator memorises one perfect image per ImageNet class and outputs only those 1,000 images. What happens to IS and FID?

Check yourself

You compute FID between two disjoint halves of the same real test set, 2,000 images each, and get a clearly positive value. Why?

Remember

  • ✓IS =exp⁡Ex KL(p(y∣x) ∥ p(y))=\exp\mathbb E_x\,\mathrm{KL}(p(y\mid x)\,\|\,p(y)) on generated images only: high for confident, varied labels, between 1 and the class count.
  • ✓FID =∥μr−μg∥2+Tr(Σr+Σg−2(ΣrΣg)1/2)=\|\mu_r-\mu_g\|^2+\mathrm{Tr}(\Sigma_r+\Sigma_g-2(\Sigma_r\Sigma_g)^{1/2}) on 2048-d Inception pool features; lower is better.
  • ✓Both are biased by sample size: report N (50k is the GAN convention, FID-30K on COCO for text-to-image) and never mix N.
  • ✓KID (cubic-kernel MMD) is unbiased and Gaussian-free, so it suits small evaluation sets.
  • ✓Same weights, resizing and reference statistics or the numbers are not comparable; add precision/recall, CLIP score and human studies.
← back to Generative AI roadmap