Denoising diffusion implicit models (DDIM)

DDIM (Song, Meng and Ermon, ICLR 2021) reuses a DDPM-trained network with non-Markovian processes that keep DDPM's marginals. Its η = 0 member is deterministic, skips steps (10–50× faster) and maps noise to images invertibly, which enables encoding and interpolation.

14 min read1075 wordsadvanced0 viewsupdated 2026-10-12

Definition

Denoising diffusion implicit model (DDIM) is a sampler, and the generative process behind it, introduced by Jiaming Song, Chenlin Meng and Stefano Ermon, "Denoising Diffusion Implicit Models" (ICLR 2021). It uses a network εθ(xt,t)\varepsilon_\theta(x_t,t) trained exactly as in Denoising diffusion probabilistic models (DDPM). Each step first predicts the clean sample x^0\hat x_0, then re-noises it to the next, lower noise level along the predicted noise direction, plus a fresh noise term of size σt\sigma_t. A parameter η∈[0,1]\eta\in[0,1] scales σt\sigma_t. η=1\eta=1 recovers DDPM-style ancestral sampling, and η=0\eta=0 gives a deterministic map from xTx_T to x0x_0: an implicit generative model. Steps can skip along any increasing sub-sequence of noise levels.

DDPM needs one network evaluation for each of its T=1000T=1000 steps, because its reverse process inverts a Markov chain one small step at a time. The paper estimates about 20 hours to sample 50,000 images of size 32×32 from a DDPM on an Nvidia 2080 Ti GPU, against less than a minute for a GAN. Song, Meng and Ermon noticed that the DDPM loss depends only on the marginals q(xt∣x0)q(x_t\mid x_0), not on the joint forward process. Any forward process with the same marginals has a variational objective that LsimpleL_{\text{simple}} also minimises, so a trained DDPM network already serves all of them. They built a family of such processes, indexed by the noise level σ\sigma, whose reverse steps can be taken with big jumps. On CIFAR-10, 50 deterministic steps reach FID 4.67, against 3.17 for DDPM's full 1000 steps. That is a 20× reduction in network calls without retraining.

A non-Markovian forward process with the same marginals

qσ(xt−1∣xt,x0)=N ⁣(αˉt−1 x0+1−αˉt−1−σt2⋅xt−αˉt x01−αˉt, σt2I),qσ(xT∣x0)=N(αˉT x0,(1−αˉT)I)q_\sigma(x_{t-1}\mid x_t,x_0)=\mathcal N\!\Big(\sqrt{\bar\alpha_{t-1}}\,x_0+\sqrt{1-\bar\alpha_{t-1}-\sigma_t^2}\cdot\frac{x_t-\sqrt{\bar\alpha_t}\,x_0}{\sqrt{1-\bar\alpha_t}},\ \sigma_t^2I\Big),\qquad q_\sigma(x_T\mid x_0)=\mathcal N\big(\sqrt{\bar\alpha_T}\,x_0,(1-\bar\alpha_T)I\big)
αˉt\bar\alpha_t
the DDPM cumulative product; the DDIM paper writes it αt\alpha_t
xt−αˉtx01−αˉt\frac{x_t-\sqrt{\bar\alpha_t}x_0}{\sqrt{1-\bar\alpha_t}}
the noise ε\varepsilon that produced xtx_t from x0x_0
σt\sigma_t
a free choice for each tt; the mean is constructed so that qσ(xt∣x0)=N(αˉtx0,(1−αˉt)I)q_\sigma(x_t\mid x_0)=\mathcal N(\sqrt{\bar\alpha_t}x_0,(1-\bar\alpha_t)I) for every tt, whatever σ\sigma is

The forward process qσ(xt∣xt−1,x0)q_\sigma(x_t\mid x_{t-1},x_0) that follows from Bayes' rule depends on x0x_0, so it is not Markovian. That does not matter, because only the marginals enter the loss. The paper's Theorem 1 states that the variational objective JσJ_\sigma of each member equals a weighted noise-prediction loss LγL_\gamma plus a constant. If the network does not share parameters across tt, the optimum of LsimpleL_{\text{simple}} is the optimum of every JσJ_\sigma. In practice the shared-parameter DDPM network is simply reused.

The sampling step

The generative process replaces the unknown x0x_0 by the network's prediction and the noise by εθ\varepsilon_\theta:

xt−1=αˉt−1(xt−1−αˉt εθ(xt,t)αˉt)⏟predicted x0+1−αˉt−1−σt2  εθ(xt,t)⏟direction pointing to xt+σt z⏟random noise,z∼N(0,I)x_{t-1}=\sqrt{\bar\alpha_{t-1}}\underbrace{\Big(\frac{x_t-\sqrt{1-\bar\alpha_t}\,\varepsilon_\theta(x_t,t)}{\sqrt{\bar\alpha_t}}\Big)}_{\text{predicted }x_0}+\underbrace{\sqrt{1-\bar\alpha_{t-1}-\sigma_t^2}\;\varepsilon_\theta(x_t,t)}_{\text{direction pointing to }x_t}+\underbrace{\sigma_t\,z}_{\text{random noise}},\qquad z\sim\mathcal N(0,I)
predicted x0\text{predicted }x_0
x^0\hat x_0, the posterior-mean estimate of the clean sample
direction\text{direction}
re-uses the predicted noise to place the sample at the right noise level for step t−1t-1
σtz\sigma_t z
fresh noise; the deterministic part and the fresh part together have variance 1−αˉt−11-\bar\alpha_{t-1}
σt(η)=η 1−αˉt−11−αˉt 1−αˉtαˉt−1⟹η=1: σt2=β~t  (DDPM),η=0: σt=0  (DDIM)\sigma_t(\eta)=\eta\,\sqrt{\frac{1-\bar\alpha_{t-1}}{1-\bar\alpha_t}}\,\sqrt{1-\frac{\bar\alpha_t}{\bar\alpha_{t-1}}}\qquad\Longrightarrow\qquad \eta=1:\ \sigma_t^2=\tilde\beta_t\ \ (\text{DDPM}),\qquad \eta=0:\ \sigma_t=0\ \ (\text{DDIM})
η\eta
interpolates between DDPM-style ancestral sampling and the deterministic sampler
η=1\eta=1
σt2\sigma_t^2 equals the DDPM posterior variance β~t\tilde\beta_t for adjacent steps, and the forward process becomes Markovian again
σ^t\hat\sigma_t
the paper also tests σ^t=1−αˉt/αˉt−1\hat\sigma_t=\sqrt{1-\bar\alpha_t/\bar\alpha_{t-1}}, DDPM's larger variance σt2=βt\sigma_t^2=\beta_t; it is the best choice at 1000 steps but very poor with few steps

Skipping steps

Nothing in the derivation requires the steps to be adjacent. Pick an increasing sub-sequence τ1<⋯<τS\tau_1<\dots<\tau_S of {1,…,T}\{1,\dots,T\} with τS=T\tau_S=T, and define the forward process only on those indices. Its marginals are still q(xτi∣x0)=N(αˉτix0,(1−αˉτi)I)q(x_{\tau_i}\mid x_0)=\mathcal N(\sqrt{\bar\alpha_{\tau_i}}x_0,(1-\bar\alpha_{\tau_i})I), so the same network applies. Sampling replaces t−1t-1 by τi−1\tau_{i-1} and tt by τi\tau_i in the update, giving SS network calls instead of TT. The paper uses evenly spaced (linear) or quadratically spaced sub-sequences. Song et al. report that DDIM produces high-quality samples 10× to 50× faster in wall-clock time than DDPM.

The deterministic sampler degrades far more gracefully than the stochastic one as SS shrinks. With few steps each update is large, and fresh noise of size σ\sigma that the next few steps cannot remove ends up in the sample. With η=0\eta=0 the only error is the discretisation error of a smooth trajectory.

The ODE behind η = 0

xˉ=xαˉ,λ=1−αˉαˉ:xˉt−1=xˉt+(λt−1−λt) εθ(xt,t)⟷dxˉdλ=εθ(xˉλ2+1, t(λ))\bar x=\frac{x}{\sqrt{\bar\alpha}},\qquad \lambda=\sqrt{\frac{1-\bar\alpha}{\bar\alpha}}:\qquad \bar x_{t-1}=\bar x_t+(\lambda_{t-1}-\lambda_t)\,\varepsilon_\theta(x_t,t)\quad\longleftrightarrow\quad \frac{d\bar x}{d\lambda}=\varepsilon_\theta\Big(\frac{\bar x}{\sqrt{\lambda^2+1}},\ t(\lambda)\Big)
xˉ\bar x
the sample rescaled so that it reads "data plus noise of std λ\lambda": xˉ=x0+λε\bar x=x_0+\lambda\varepsilon
λ\lambda
noise-to-signal ratio, 1/SNR1/\sqrt{\mathrm{SNR}}; the paper writes σ\sigma
left\text{left}
the η=0\eta=0 update divided by αˉt−1\sqrt{\bar\alpha_{t-1}}: exactly one Euler step in λ\lambda
right\text{right}
the ODE it discretises; it is the probability-flow ODE of the variance-exploding SDE of Song et al. (see Score-based generative models)

Because DDIM takes Euler steps in λ\lambda rather than in tt, its error is smaller where εθ\varepsilon_\theta changes little with λ\lambda. DPM-Solver (Lu et al., NeurIPS 2022) shows that DDIM is its first-order case and adds higher-order steps, reaching good quality in about 10–20 evaluations. An ODE also defines an invertible map, which gives DDIM two abilities DDPM lacks.

Inversion, consistency and interpolation

Inversion. Running the η=0\eta=0 update upward, xτi+1=αˉτi+1 x^0(xτi)+1−αˉτi+1 εθ(xτi,τi)x_{\tau_{i+1}}=\sqrt{\bar\alpha_{\tau_{i+1}}}\,\hat x_0(x_{\tau_i})+\sqrt{1-\bar\alpha_{\tau_{i+1}}}\,\varepsilon_\theta(x_{\tau_i},\tau_i), encodes a real image x0x_0 into a latent xTx_T that decodes back to it. The paper shows the reconstruction error falls as SS grows. Inversion is the basis of most diffusion image-editing methods: invert, change the condition, decode. Consistency. With η=0\eta=0 the same xTx_T decoded with different SS gives images with the same high-level content, so xTx_T acts as a latent code. Interpolation. Spherical interpolation between two latents, xT(u)=sin⁡((1−u)θ)sin⁡θxT(0)+sin⁡(uθ)sin⁡θxT(1)x_T(u)=\frac{\sin((1-u)\theta)}{\sin\theta}x_T^{(0)}+\frac{\sin(u\theta)}{\sin\theta}x_T^{(1)}, decodes to a smooth semantic morph. With η>0\eta>0 fresh noise at every step makes both properties fail.

samplerS = 10S = 20S = 50S = 100S = 1000
η = 0 (DDIM)13.366.844.674.164.04
η = 1 (σ² = β̃)41.0718.368.015.784.73
σ̂ (DDPM, σ² = β)367.43133.3732.729.993.17
CIFAR-10 FID (lower is better) for the same DDPM-trained model, by number of sampling steps S (Song et al. 2021, Table 1)
x_s = √ᾱ_s · x̂₀ + √(1 − ᾱ_s − σ²) · ε* + σz, σ = η·√((1−ᾱ_s)/(1−ᾱ_t))·√(1 − ᾱ_t/ᾱ_s)200 samples · 12 paths, one vertex per step · ◯ x_Tsteps S, NFE10sub-sequence τ1000 → 900 → … → 100 → 0η0.0σ at first step0.000final samples (S = 10, η = 0.0):within 3σ of the moons 68.5%mean dist / σ 2.30 (data 0.88)‖x₀ − x₀(ODE, S=1000)‖ 0.174deterministic: x_T fixes x₀x̂₀ = (x_t − √(1−ᾱ_t)·ε*)/√ᾱ_tη = 1: DDPM (σ² = β̃ when S = T)η = 0: ODE, no noise after x_Tsame trained ε_θ for every S, ηε* is exact for this 20-Gaussian two-moons mixture (std 0.06): a perfectly trained DDPM networklinear β schedule, T = 1000; evenly spaced τ; the band shows where the data lives
mode
steps S

Inversion and interpolation need the deterministic sampler, so they always use η = 0.

fig.Two moons built from 20 Gaussians (std σ = 0.06), with the exact noise predictor of the mixture standing in for a perfectly trained DDPM network (linear β, T = 1000). Sample mode: 200 fixed noise vectors x_T are decoded with S evenly spaced steps, and 12 trajectories show one vertex per step. With η = 0 and S = 10 (the default) 68.5% of the samples are within 3σ of the moons, at a mean distance of 2.30σ (the data: 0.88σ). They lie on average 0.174 from where the 1000-step ODE puts the same x_T. With S = 20 the numbers are 91.0% and 0.086, with S = 50 100% and 0.029. S = 1 is a single jump to x̂₀ = E[x₀ | x_T]: every sample collapses near the centre (0% near the data). Raising η keeps the quality similar at each S (η = 1, S = 50: 99.5%) but the samples no longer follow x_T: with η = 1 they lie 1.1 to 1.3 from the ODE solution. Invert mode: 16 data points are encoded with the upward η = 0 update and decoded again. The mean reconstruction error is 0.109 (1.82σ) at S = 20, 0.031 at S = 100 and 0.0032 at S = 1000. Interpolate mode: three slerp paths of 41 points between pairs of x_T are decoded with η = 0. Nearby latents decode to nearby samples, so each path traces a continuous curve along a moon; at S = 50 one path jumps where it crosses from one moon's basin into the other's.

Worked example: one step at three values of η

ησ√(1−ᾱ_s−σ²)√ᾱ_sdeterministic part of x_s
000.68990.72390.3411
0.50.33130.60510.72390.3328
10.66260.19200.72390.2923
Step from t = 500 to s = 250 on the 1-D mixture 0.3·N(−1, 0.12²) + 0.7·N(0.9, 0.2²) at x_t = 0.2, with the exact ε* = 0.0980 and x̂₀ = 0.3778 (computed in Node)

Reading the table, then a full deterministic run

  1. 1αˉ500=0.07859\bar\alpha_{500}=0.07859 and αˉ250=0.52409\bar\alpha_{250}=0.52409. Predicted clean sample: x^0=(0.2−0.92141⋅0.0980)/0.07859=0.3778\hat x_0=(0.2-\sqrt{0.92141}\cdot0.0980)/\sqrt{0.07859}=0.3778.
  2. 2η=0\eta=0: x250=0.7239⋅0.3778+0.6899⋅0.0980=0.3411x_{250}=0.7239\cdot0.3778+0.6899\cdot0.0980=0.3411, with no randomness. The noise part has variance 0.68992=0.476=1−αˉ2500.6899^2=0.476=1-\bar\alpha_{250}, the full noise variance at s=250s=250, all of it carried over from ε∗\varepsilon^*.
  3. 3η=1\eta=1: σ=0.6626\sigma=0.6626, so σ2=0.439\sigma^2=0.439 of the 0.4760.476 noise variance is fresh noise and only 0.19202=0.0370.1920^2=0.037 is carried over from ε∗\varepsilon^*. A 250-step jump with η=1\eta=1 throws away almost all the information in xtx_t about its own noise.
  4. 4Deterministic decoding of xT=0.5x_T=0.5 with SS steps gives x0=0.3325x_0=0.3325 for S=1S=1, 0.45870.4587 for S=2S=2, 0.86540.8654 for S=5S=5, 0.90120.9012 for S=10S=10, 0.92170.9217 for S=50S=50 and 0.92890.9289 for S=1000S=1000. With S=1S=1 the result is E[x0∣xT]\mathbb E[x_0\mid x_T], close to the data mean 0.330.33. From about 10 steps on the trajectory has chosen the mode at 0.90.9 and only refines.
python
1import torch
2
3T = 1000
4abar = torch.cumprod(1 - torch.linspace(1e-4, 0.02, T, dtype=torch.float64), 0)
5A = lambda t: abar[t - 1] if t > 0 else torch.tensor(1.0, dtype=torch.float64) # ᾱ_0 = 1
6
7def ddim_step(eps_model, x, t, s, eta=0.0):
8 """One generalised step from t to s < t (eta = 0: DDIM, eta = 1: DDPM-like)."""
9 at, as_ = A(t), A(s)
10 e = eps_model(x, t)
11 x0 = (x - (1 - at).sqrt() * e) / at.sqrt() # predicted x0
12 sig = eta * ((1 - as_) / (1 - at) * (1 - at / as_)).sqrt()
13 return as_.sqrt() * x0 + (1 - as_ - sig**2).sqrt() * e + sig * torch.randn_like(x)
14
15def ddim_sample(eps_model, xT, S=50, eta=0.0):
16 taus = torch.linspace(0, T, S + 1).round().long().tolist() # [0, 20, 40, ..., 1000] for S = 50
17 x = xT
18 for i in range(S, 0, -1):
19 x = ddim_step(eps_model, x, taus[i], taus[i - 1], eta)
20 return x
21
22def ddim_invert(eps_model, x0, S=50):
23 """Run the eta = 0 update upward (x0 -> x_T), using ε at the current, less noisy step."""
24 taus = torch.linspace(0, T, S + 1).round().long().tolist()
25 taus[0] = 1 # ε is not defined at t = 0
26 x = x0
27 for i in range(S):
28 t, s = taus[i], taus[i + 1]
29 at, as_ = A(t), A(s)
30 e = eps_model(x, t)
31 x = as_.sqrt() * (x - (1 - at).sqrt() * e) / at.sqrt() + (1 - as_).sqrt() * e
32 return x
33
34# A model we can write down exactly: data x0 ~ N(m, s² I) gives ε*(x_t, t) = E[ε | x_t] in closed form.
35m, s2 = torch.tensor([0.5, -1.0], dtype=torch.float64), 0.2**2
36def eps_star(x, t):
37 a = A(t)
38 return (1 - a).sqrt() * (x - a.sqrt() * m) / (a * s2 + 1 - a)
39
40torch.manual_seed(0)
41xT = torch.randn(4096, 2, dtype=torch.float64)
42x20, x1000 = ddim_sample(eps_star, xT, S=20), ddim_sample(eps_star, xT, S=1000)
43print(x20.shape) # torch.Size([4096, 2])
44print(x20.std(0).tolist(), x1000.std(0).tolist()) # ≈ [0.140, 0.140] and ≈ [0.198, 0.199]; true std 0.2
45print(torch.equal(x20, ddim_sample(eps_star, xT, S=20))) # True: eta = 0 is deterministic given x_T
46z = ddim_invert(eps_star, x1000[:8], S=1000)
47rec = ddim_sample(eps_star, z, S=1000)
48print((z - xT[:8]).abs().max().item() < 0.05, (rec - x1000[:8]).abs().max().item() < 0.01) # True True
DDIM sampling with any η and sub-sequence, and DDIM inversion, checked against an exact noise predictor for Gaussian data

Use when

  • +No retraining: reuses any DDPM-trained noise predictor
  • +10–50× fewer network calls for similar quality; FID 4.67 at 50 steps on CIFAR-10
  • +Deterministic (η = 0): the same x_T always gives the same image, at any step count
  • +Invertible: encodes real images to latents for reconstruction, editing and interpolation

Avoid when

  • −At the full 1000 steps it is slightly worse than DDPM with σ² = β (FID 4.04 against 3.17)
  • −First-order: below about 10 steps quality drops quickly; higher-order solvers do better
  • −Inversion is approximate and degrades with few steps and strong guidance
  • −Deterministic sampling cannot correct its own errors with noise, unlike stochastic samplers

Check yourself

Why can DDIM use a network trained with DDPM's L_simple, without retraining?

Check yourself

A DDIM step goes from t = 500 to s = 250 with η = 1. Compared with η = 0, what changes?

Remember

  • ✓DDIM step: xs=αˉs x^0+1−αˉs−σ2 εθ(xt,t)+σzx_s=\sqrt{\bar\alpha_s}\,\hat x_0+\sqrt{1-\bar\alpha_s-\sigma^2}\,\varepsilon_\theta(x_t,t)+\sigma z with x^0=(xt−1−αˉtεθ)/αˉt\hat x_0=(x_t-\sqrt{1-\bar\alpha_t}\varepsilon_\theta)/\sqrt{\bar\alpha_t} and any s<ts<t.
  • ✓σ=η1−αˉs1−αˉt1−αˉtαˉs\sigma=\eta\sqrt{\frac{1-\bar\alpha_s}{1-\bar\alpha_t}}\sqrt{1-\frac{\bar\alpha_t}{\bar\alpha_s}}: η=1\eta=1 is DDPM-like (σ2=β~t\sigma^2=\tilde\beta_t), η=0\eta=0 is deterministic.
  • ✓Same marginals as DDPM, non-Markovian forward process, so the DDPM-trained network is reused unchanged on any step sub-sequence.
  • ✓η = 0 is an Euler discretisation of the probability-flow ODE: it gives inversion x₀ → x_T, consistent latents, and slerp interpolation.
  • ✓CIFAR-10 FID: 13.36 / 6.84 / 4.67 / 4.16 at 10 / 20 / 50 / 100 steps (η = 0), against 3.17 for DDPM at 1000 steps; 10–50× faster.
← back to Deep Learning roadmap