# Denoising diffusion probabilistic models (DDPM)

> DDPM (Ho, Jain and Abbeel, 2020) noises data over T = 1000 Gaussian steps with a linear β schedule and trains a U-Net to predict the added noise with a plain MSE. Sampling runs the learned Gaussian reverse chain from pure noise, 1000 network calls per sample; it reached FID 3.17 on CIFAR-10.

Source: https://www.mlfoundry.in/learn/denoising-diffusion-probabilistic-models-ddpm  
Section: Deep Learning › Generative Deep Learning Algorithms  
Level: advanced · 15 min read · updated 2026-10-12

**Denoising diffusion probabilistic model (DDPM):** is the diffusion model of Ho, Jain and Abbeel, "Denoising Diffusion Probabilistic Models" (NeurIPS 2020). It is a latent-variable model $p_\theta(x_0)=\int p(x_T)\prod_{t=1}^{T}p_\theta(x_{t-1}\mid x_t)\,dx_{1:T}$ with a fixed Gaussian forward chain $q(x_t\mid x_{t-1})$. Each reverse step is a Gaussian whose mean is computed from a network $\varepsilon_\theta(x_t,t)$ that predicts the noise in $x_t$, and whose variance is fixed. The network is trained with the unweighted noise-prediction loss $L_{\text{simple}}$. Samples are drawn by **ancestral sampling**: start at $x_T\sim\mathcal N(0,I)$ and draw $x_{t-1}\sim p_\theta(x_{t-1}\mid x_t)$ for $t=T,\dots,1$.

Diffusion probabilistic models had been proposed by Sohl-Dickstein et al. in 2015 (see **Diffusion models**) but produced weak samples. Jonathan Ho, Ajay Jain and Pieter Abbeel made three choices that turned them into a state-of-the-art image generator. They fixed the reverse variances instead of learning them. They parameterised the reverse mean through a noise prediction $\varepsilon_\theta$, which turns each term of the variational bound into a weighted denoising score-matching loss. And they dropped the weights. On CIFAR-10 this gave an Inception score of 9.46 and FID 3.17, better than the GANs of the time, and LSUN 256×256 samples comparable to ProgressiveGAN. The same paper pointed out the connection to Song and Ermon's score networks and Langevin dynamics.

## Forward chain and its posterior

$$
q(x_t\mid x_{t-1})=\mathcal N\big(\sqrt{\alpha_t}\,x_{t-1},\,\beta_tI\big),\quad \alpha_t=1-\beta_t,\qquad q(x_t\mid x_0)=\mathcal N\big(\sqrt{\bar\alpha_t}\,x_0,\,(1-\bar\alpha_t)I\big),\quad \bar\alpha_t=\prod_{s=1}^t\alpha_s
$$

- $\beta_t$: linear from $\beta_1=10^{-4}$ to $\beta_T=0.02$ with $T=1000$; small enough that each reverse step is close to Gaussian, large enough that $\bar\alpha_T=4.04\times10^{-5}$
- $x_0$: an image with pixel values scaled linearly to $[-1,1]$
- $q(x_t\mid x_0)$: marginal in closed form: $x_t=\sqrt{\bar\alpha_t}\,x_0+\sqrt{1-\bar\alpha_t}\,\varepsilon$, $\varepsilon\sim\mathcal N(0,I)$

The true reverse step $q(x_{t-1}\mid x_t)$ is intractable. Conditioned on $x_0$ it is not: by Bayes' rule and the Gaussian forward kernels it is a Gaussian with known mean and variance.

$$
q(x_{t-1}\mid x_t,x_0)=\mathcal N\big(\tilde\mu_t(x_t,x_0),\,\tilde\beta_tI\big),\qquad \tilde\mu_t=\frac{\sqrt{\bar\alpha_{t-1}}\,\beta_t}{1-\bar\alpha_t}\,x_0+\frac{\sqrt{\alpha_t}\,(1-\bar\alpha_{t-1})}{1-\bar\alpha_t}\,x_t,\qquad \tilde\beta_t=\frac{1-\bar\alpha_{t-1}}{1-\bar\alpha_t}\,\beta_t
$$

- $\tilde\mu_t$: a weighted average of the clean sample and the current noisy one; for large $t$ almost all weight is on $x_t$ (0.9941 against 0.0031 at $t=500$)
- $\tilde\beta_t$: posterior variance; slightly below $\beta_t$, and much smaller only for the first few steps ($\tilde\beta_1=0$)

## The ε-parameterisation and L_simple

The model is $p_\theta(x_{t-1}\mid x_t)=\mathcal N(\mu_\theta(x_t,t),\sigma_t^2I)$. The variational bound splits into a KL term per step, $L_{t-1}=D_{\mathrm{KL}}\big(q(x_{t-1}\mid x_t,x_0)\,\|\,p_\theta(x_{t-1}\mid x_t)\big)$. Between two Gaussians with the same fixed variance, this KL is a squared distance between the means. Instead of predicting $\tilde\mu_t$ directly, write $x_0$ in terms of $x_t$ and the noise, and let the network predict the noise:

$$
\mu_\theta(x_t,t)=\frac{1}{\sqrt{\alpha_t}}\Big(x_t-\frac{\beta_t}{\sqrt{1-\bar\alpha_t}}\,\varepsilon_\theta(x_t,t)\Big),\qquad L_{t-1}=\mathbb E\Big[\frac{\beta_t^2}{2\sigma_t^2\,\alpha_t(1-\bar\alpha_t)}\big\|\varepsilon-\varepsilon_\theta(x_t,t)\big\|^2\Big]+C
$$

- $\mu_\theta$: exactly $\tilde\mu_t(x_t,\hat x_0)$ with $\hat x_0=(x_t-\sqrt{1-\bar\alpha_t}\,\varepsilon_\theta)/\sqrt{\bar\alpha_t}$ substituted for the unknown $x_0$
- $\sigma_t^2$: fixed reverse variance, $\beta_t$ or $\tilde\beta_t$; the paper found both gave similar sample quality
- $C$: a constant independent of $\theta$

$$
L_{\text{simple}}(\theta)=\mathbb E_{t\sim\mathcal U\{1,\dots,T\},\ x_0,\ \varepsilon\sim\mathcal N(0,I)}\Big[\big\|\varepsilon-\varepsilon_\theta\big(\sqrt{\bar\alpha_t}\,x_0+\sqrt{1-\bar\alpha_t}\,\varepsilon,\ t\big)\big\|^2\Big]
$$

- $L_{\text{simple}}$: the bound with the per-step weight set to 1; this down-weights the small-$t$ terms, where denoising is easy, relative to the true bound
- $t=1$: the discrete decoder term $L_0$ of the bound is approximated by the same MSE at $t=1$

Why the weights matter, and how ε-, $x_0$- and v-prediction relate, is covered in **Diffusion noise-prediction loss**. The paper also tried predicting $\tilde\mu_t$ directly. That works only with the true bound and gives worse samples. ε-prediction resembles Langevin dynamics, because $\varepsilon_\theta$ is a scaled score: $\varepsilon_\theta(x_t,t)\approx-\sqrt{1-\bar\alpha_t}\,\nabla_{x_t}\log q(x_t)$ (see **Score-based generative models**).

**The paper's two algorithms**

1. **Training** (Algorithm 1). Repeat: take $x_0$ from the data, $t\sim\mathcal U\{1,\dots,T\}$, $\varepsilon\sim\mathcal N(0,I)$; take a gradient step on $\|\varepsilon-\varepsilon_\theta(\sqrt{\bar\alpha_t}x_0+\sqrt{1-\bar\alpha_t}\varepsilon,t)\|^2$.
2. **Sampling** (Algorithm 2). Draw $x_T\sim\mathcal N(0,I)$. For $t=T,\dots,1$: $z\sim\mathcal N(0,I)$ if $t>1$, else $z=0$, and $x_{t-1}=\frac{1}{\sqrt{\alpha_t}}\big(x_t-\frac{1-\alpha_t}{\sqrt{1-\bar\alpha_t}}\varepsilon_\theta(x_t,t)\big)+\sigma_tz$. Return $x_0$.
3. Sampling needs $T=1000$ sequential network evaluations per batch. **Denoising diffusion implicit models (DDIM)** reuses the same trained $\varepsilon_\theta$ with far fewer steps.

## Architecture and setup

**Setup in the paper (Ho et al. 2020)**

- **Schedule:** $T=1000$, $\beta_t$ linear from $10^{-4}$ to $0.02$; $\sigma_t^2=\beta_t$ or $\tilde\beta_t$ (similar results)
- **Network:** U-Net based on an unmasked PixelCNN++: Wide ResNet blocks, group normalisation, self-attention at the 16×16 resolution, and the Transformer's sinusoidal embedding of $t$ added in every residual block; parameters shared across $t$
- **Size:** 35.7M parameters for CIFAR-10, 114M for LSUN and CelebA-HQ (256×256), about 256M for a larger LSUN Bedroom model
- **Optimisation:** Adam, learning rate $2\times10^{-4}$ ($2\times10^{-5}$ at 256×256), EMA of weights with decay 0.9999; dropout 0.1 and random horizontal flips on CIFAR-10; 800K steps at batch 128 for CIFAR-10 on a TPU v3-8
- **Loss:** $L_{\text{simple}}$; the true variational bound gives better likelihoods but worse samples

CIFAR-10 unconditional results (Ho et al. 2020, Tables 1 and 2); NLL is an upper bound on the test set

| parameterisation and objective | IS | FID | NLL (bits/dim) |
| --- | --- | --- | --- |
| predict μ̃, true bound L, fixed isotropic Σ | 8.06 | 13.22 |  |
| predict ε, true bound L, fixed isotropic Σ | 7.67 | 13.51 | ≤ 3.70 |
| predict ε, L_simple | 9.46 | 3.17 | ≤ 3.75 |

On LSUN 256×256 the paper reports FID 7.89 for Church and 6.36 for Bedroom (4.90 with the larger model). The likelihoods are worse than those of autoregressive models. The authors explain this through a rate–distortion analysis: most of the lossless codelength describes imperceptible image details, which $L_{\text{simple}}$ deliberately down-weights. Running the chain as progressive decoding shows large-scale features appearing first and fine details last.

## The cosine schedule (Nichol & Dhariwal, 2021)

$$
\bar\alpha_t=\frac{f(t)}{f(0)},\qquad f(t)=\cos^2\!\Big(\frac{t/T+s}{1+s}\cdot\frac{\pi}{2}\Big),\qquad s=0.008,\qquad \beta_t=\min\!\Big(1-\frac{\bar\alpha_t}{\bar\alpha_{t-1}},\ 0.999\Big)
$$

- $f$: makes $\bar\alpha_t$ fall slowly at both ends and roughly linearly in the middle
- $s$: small offset so that $\beta_t$ is not too small near $t=0$
- $0.999$: clip that prevents a singularity at $t=T$, where $\bar\alpha_T\to0$

In "Improved Denoising Diffusion Probabilistic Models" (ICML 2021), Alex Nichol and Prafulla Dhariwal noticed that with the linear schedule the last part of the forward chain is almost pure noise and contributes little. The SNR $\bar\alpha_t/(1-\bar\alpha_t)$ drops below 1 at $t=260$ for the linear schedule and at $t=497$ for the cosine one. They also learned the reverse variance as an interpolation $\Sigma_\theta=\exp(v\log\beta_t+(1-v)\log\tilde\beta_t)$, trained with a hybrid loss $L_{\text{simple}}+\lambda L_{\text{vlb}}$ with $\lambda=0.001$. That improved log-likelihoods and allowed sampling with far fewer steps.

> Figure: One-dimensional data q(x₀) = 0.3·N(−1, 0.12²) + 0.7·N(0.9, 0.2²) (mean 0.330, std 0.889). Its noised marginals are mixtures too, so the optimal ε*(x_t, t) = E[ε | x_t] is computed exactly and stands in for a perfectly trained network. The shading is q(x_t), normalised per column. 800 chains run Algorithm 2 for all 1000 steps, and 24 of them are drawn (rose if they end below 0). With the linear schedule and σ² = β the 800 samples have mean 0.302, std 0.888 and 69.1% above zero (true 70%). σ² = β̃ gives 0.303, 0.885 and 69.3%. The cosine schedule gives 0.321, 0.886 and 70.0%. Move the cursor to see the posterior-mean predictions x̂₀ of the drawn chains. At t = 750 (linear) they all lie between 0.25 and 0.41, close to the data mean: the noise carries almost no information about x₀ yet. At t = 500 they span −0.22 to 0.66, and by t = 250 the chains have committed to a mode (−1.00 to 0.95). With the cosine schedule they already span −0.21 to 0.74 at t = 750. The readouts show that the update coefficients barely change with t: 1/√α_t goes from 1.000 to 1.010, and the posterior mean puts 0.9941 of its weight on x_t at t = 500.

## Worked example: one reverse step

Linear schedule quantities at selected t (computed in Node); cx₀ and cx_t are the posterior-mean coefficients of x₀ and x_t

| t | β_t | ᾱ_t | β̃_t | 1/√α_t | β_t/√(1−ᾱ_t) | cx₀ | cx_t |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 2 | 1.199 × 10⁻⁴ | 0.9998 | 5.45 × 10⁻⁵ | 1.0001 | 0.00809 | 0.5453 | 0.4547 |
| 10 | 2.793 × 10⁻⁴ | 0.9981 | 2.382 × 10⁻⁴ | 1.0001 | 0.00642 | 0.1473 | 0.8527 |
| 100 | 0.002072 | 0.8970 | 0.002035 | 1.0010 | 0.00646 | 0.01908 | 0.9809 |
| 500 | 0.01004 | 0.07859 | 0.01003 | 1.0051 | 0.01046 | 0.003070 | 0.9941 |
| 1000 | 0.02000 | 4.04 × 10⁻⁵ | 0.02000 | 1.0102 | 0.02000 | 1.28 × 10⁻⁴ | 0.9899 |

**x_t = 0.5 at t = 100, on the figure's mixture, with the exact ε***

1. $\bar\alpha_{100}=0.89702$, so $\sqrt{\bar\alpha_t}=0.94711$ and $\sqrt{1-\bar\alpha_t}=0.32091$. The exact noise prediction is $\varepsilon^*=-0.81396$.
2. Denoised estimate: $\hat x_0=(0.5+0.32091\cdot0.81396)/0.94711=0.80372$, pulled toward the larger mode at $0.9$.
3. Posterior mean at $\hat x_0$: $\tilde\mu=0.019076\cdot0.80372+0.98090\cdot0.5=0.505780$.
4. Algorithm 2's mean: $\frac{1}{\sqrt{\alpha_t}}(x_t-\frac{\beta_t}{\sqrt{1-\bar\alpha_t}}\varepsilon^*)=1.00104\cdot(0.5+0.0064569\cdot0.81396)=0.505780$. The two agree, as the derivation says.
5. Add noise: $\sigma_t=\sqrt{\beta_t}=0.04552$ or $\sqrt{\tilde\beta_t}=0.04511$, so $x_{99}=0.50578+0.0455\,z$. The step moves the mean by only 0.0058; the noise term is eight times larger. That is why 1000 steps are needed.

```python
import math, torch, torch.nn as nn, torch.nn.functional as F

class DDPM:
    def __init__(self, T=1000, schedule="linear"):
        self.T = T
        if schedule == "linear":
            beta = torch.linspace(1e-4, 0.02, T, dtype=torch.float64)
        else:                                            # cosine (Nichol & Dhariwal 2021), s = 0.008
            s = 0.008
            f = lambda t: torch.cos((t / T + s) / (1 + s) * math.pi / 2) ** 2
            t = torch.arange(T + 1, dtype=torch.float64)
            beta = (1 - f(t[1:]) / f(t[:-1])).clamp(max=0.999)
        self.beta = beta.float()                         # index i ↔ step t = i + 1
        self.alpha = 1 - self.beta
        self.abar = torch.cumprod(beta.new_ones(T) - beta, 0).float()
        self.abar_prev = F.pad(self.abar[:-1], (1, 0), value=1.0)
        self.beta_tilde = self.beta * (1 - self.abar_prev) / (1 - self.abar)

    def q_sample(self, x0, t, eps):                      # x_t = √ᾱ_t x0 + √(1-ᾱ_t) ε
        a = self.abar[t - 1].view(-1, 1, 1, 1)
        return a.sqrt() * x0 + (1 - a).sqrt() * eps

    def loss(self, model, x0):                           # L_simple
        t = torch.randint(1, self.T + 1, (x0.size(0),))
        eps = torch.randn_like(x0)
        return F.mse_loss(model(self.q_sample(x0, t, eps), t), eps)

    def posterior_mean(self, x0, xt, t):                 # μ̃_t(x_t, x0) of q(x_{t-1} | x_t, x0)
        i = t - 1
        c0 = self.abar_prev[i].sqrt() * self.beta[i] / (1 - self.abar[i])
        ct = self.alpha[i].sqrt() * (1 - self.abar_prev[i]) / (1 - self.abar[i])
        return c0 * x0 + ct * xt

    @torch.no_grad()
    def sample(self, model, shape, var="beta"):          # Algorithm 2 (ancestral sampling)
        x = torch.randn(shape)
        for t in range(self.T, 0, -1):
            i = t - 1
            eps = model(x, torch.full((shape[0],), t))
            mean = (x - self.beta[i] / (1 - self.abar[i]).sqrt() * eps) / self.alpha[i].sqrt()
            sigma = (self.beta[i] if var == "beta" else self.beta_tilde[i]).sqrt()
            x = mean + sigma * torch.randn_like(x) if t > 1 else mean
        return x

def timestep_embedding(t, dim):                          # sinusoidal, as in the Transformer
    half = dim // 2
    freqs = torch.exp(-math.log(10000) * torch.arange(half) / half)
    a = t.float()[:, None] * freqs[None]
    return torch.cat([a.sin(), a.cos()], 1)

class TinyDenoiser(nn.Module):                           # stand-in for the U-Net: conv blocks + t embedding
    def __init__(self, ch=3, h=32):
        super().__init__()
        self.temb = nn.Sequential(nn.Linear(h, h), nn.SiLU(), nn.Linear(h, h))
        self.inp = nn.Conv2d(ch, h, 3, padding=1)
        self.mid = nn.Sequential(nn.GroupNorm(8, h), nn.SiLU(), nn.Conv2d(h, h, 3, padding=1))
        self.out = nn.Conv2d(h, ch, 3, padding=1)
        self.h = h
    def forward(self, x, t):
        e = self.temb(timestep_embedding(t, self.h))[:, :, None, None]
        return self.out(self.mid(self.inp(x) + e))

d = DDPM()
model = TinyDenoiser()
x0 = torch.rand(8, 3, 32, 32) * 2 - 1                    # images scaled to [-1, 1]
loss = d.loss(model, x0); loss.backward()
print(loss.shape, f"{d.abar[-1].item():.2e}")            # torch.Size([]) 4.04e-05

# Algorithm 2's mean is the posterior mean evaluated at x̂0 = (x_t - √(1-ᾱ_t) ε)/√ᾱ_t
t = torch.tensor([100]); xt = torch.randn(1, 3, 32, 32); eps = torch.randn_like(xt)
a = d.abar[t - 1]
x0_hat = (xt - (1 - a).sqrt() * eps) / a.sqrt()
alg2 = (xt - d.beta[t - 1] / (1 - a).sqrt() * eps) / d.alpha[t - 1].sqrt()
print(torch.allclose(alg2, d.posterior_mean(x0_hat, xt, t), atol=1e-5))   # True

x = d.sample(model.eval(), (2, 3, 32, 32))               # 1000 network calls
print(x.shape)                                           # torch.Size([2, 3, 32, 32])
c = DDPM(schedule="cosine")
print(f"{c.abar[499].item():.4f} {d.abar[499].item():.4f}")  # 0.4938 0.0786  (ᾱ_500: cosine keeps more signal)
```

DDPM in PyTorch: schedules, q_sample, L_simple, the posterior mean, and Algorithm 2 (a tiny conv net stands in for the U-Net)

> **Off-by-one in the schedule arrays:** The paper indexes steps $t=1,\dots,T$, with $\bar\alpha_0=1$. Code stores $\beta$ in a 0-based array, so step $t$ is index $t-1$, and $\bar\alpha_{t-1}$ needs a padded array (`abar_prev` above). Mixing the two conventions shifts every coefficient by one step. That barely matters at large $t$ but is badly wrong at $t=1,2$, where $\tilde\beta_t/\beta_t$ goes from 0 to 0.45. Also add no noise at the final step ($z=0$ at $t=1$).

> **Clip the predicted image, not the sample:** Early in sampling, $\hat x_0=(x_t-\sqrt{1-\bar\alpha_t}\varepsilon_\theta)/\sqrt{\bar\alpha_t}$ divides by a tiny $\sqrt{\bar\alpha_t}$ (0.0064 at $t=1000$) and can be far outside $[-1,1]$. The official implementation computes the mean as $\tilde\mu_t(x_t,\hat x_0)$ with $\hat x_0$ clipped to the data range. That is equivalent to Algorithm 2 when no clipping happens and more stable when it does. Clipping $x_t$ itself is wrong, because $x_t$ legitimately has variance close to 1 plus the signal.

> **β or β̃: two ends of the same range:** The paper notes that $\sigma_t^2=\beta_t$ is optimal when $x_0\sim\mathcal N(0,I)$ and $\sigma_t^2=\tilde\beta_t$ when $x_0$ is a single fixed point: they correspond to the upper and lower bounds on the reverse-process entropy for data with coordinate-wise unit variance. They differ noticeably only for the first few steps, which is why both work with 1000 steps (69.1% against 69.3% of samples above zero in the figure). With fewer steps the gap widens, and that is where learned variances (Nichol and Dhariwal) or the deterministic **Denoising diffusion implicit models (DDIM)** help.

**Strengths**

- Simple, stable training: one MSE, no adversary, no learned variances needed
- High sample quality and full mode coverage (CIFAR-10 FID 3.17 in 2020)
- Principled: a variational bound gives log-likelihood estimates and connects to score matching
- The trained ε_θ is reused by faster samplers (DDIM, ODE solvers) without retraining

**Limitations**

- Sampling is slow: 1000 sequential network evaluations per sample
- Likelihoods are worse than those of autoregressive models when trained with L_simple
- Fixed variances and the linear schedule are suboptimal for few-step sampling
- Pixel-space U-Nets are expensive at high resolution

**Question:** In DDPM, the network output ε_θ(x_t, t) enters the reverse step through which mean?

- A. $\mu_\theta=\sqrt{\bar\alpha_t}\,\varepsilon_\theta$
- B. $\mu_\theta=\frac{1}{\sqrt{\alpha_t}}\big(x_t-\frac{\beta_t}{\sqrt{1-\bar\alpha_t}}\varepsilon_\theta\big)$
- C. $\mu_\theta=x_t-\varepsilon_\theta$
- D. $\mu_\theta=\frac{x_t-\sqrt{1-\bar\alpha_t}\,\varepsilon_\theta}{\sqrt{\bar\alpha_t}}$

**Answer:** B. $\mu_\theta=\frac{1}{\sqrt{\alpha_t}}\big(x_t-\frac{\beta_t}{\sqrt{1-\bar\alpha_t}}\varepsilon_\theta\big)$ — It is the posterior mean $\tilde\mu_t(x_t,\hat x_0)$ with $\hat x_0$ written through the predicted noise. The last option is $\hat x_0$ itself, the estimate of the clean sample, not the mean of the next step. In the worked example both forms of the mean give 0.505780.

**Question:** Ho et al. trained with L_simple rather than the true variational bound. What did they observe?

- A. Better FID (3.17 against 13.51) but a slightly worse likelihood bound (3.75 against 3.70 bits/dim)
- B. Better FID and a better likelihood
- C. Unstable training that needed learned variances
- D. Identical results; the two losses are the same

**Answer:** A. Better FID (3.17 against 13.51) but a slightly worse likelihood bound (3.75 against 3.70 bits/dim) — $L_{\text{simple}}$ drops the per-step weight $\beta_t^2/(2\sigma_t^2\alpha_t(1-\bar\alpha_t))$, which is large for small $t$. The model then spends more capacity on the harder, noisier steps that matter for perceptual quality, and less on the imperceptible details that dominate the likelihood.

**Key points**

- DDPM: $q(x_t\mid x_0)=\mathcal N(\sqrt{\bar\alpha_t}x_0,(1-\bar\alpha_t)I)$, $T=1000$, linear $\beta$ from $10^{-4}$ to $0.02$; train with $L_{\text{simple}}=\mathbb E\|\varepsilon-\varepsilon_\theta(x_t,t)\|^2$.
- Sampling: $x_{t-1}=\frac{1}{\sqrt{\alpha_t}}\big(x_t-\frac{\beta_t}{\sqrt{1-\bar\alpha_t}}\varepsilon_\theta(x_t,t)\big)+\sigma_tz$ with $\sigma_t^2=\beta_t$ or $\tilde\beta_t$; 1000 network calls.
- The reverse mean equals the posterior mean $\tilde\mu_t(x_t,\hat x_0)$ at the network's denoised estimate $\hat x_0$.
- CIFAR-10: FID 3.17 and IS 9.46 with L_simple, against FID 13.51 with the true bound; U-Net with 35.7M parameters.
- Cosine schedule (Nichol & Dhariwal 2021): $\bar\alpha_t=f(t)/f(0)$, $f=\cos^2(\frac{t/T+s}{1+s}\frac\pi2)$, $s=0.008$, keeps SNR ≥ 1 for about half the chain.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
