Definition
Flow matching (FM) is a training objective for continuous normalizing flows from Lipman, Chen, Ben-Hamu, Nickel and Le, "Flow Matching for Generative Modeling" (ICLR 2023). A network defines an ODE that carries a simple source at to the data distribution at . Instead of maximising likelihood through the ODE, FM regresses on a target vector field that is known to generate a chosen probability path . Its practical form, conditional flow matching (CFM), uses per-sample paths from noise to a data point , whose velocities are available in closed form. It has the same gradient as the intractable marginal objective.
A continuous normalizing flow (Chen et al., Neural ODEs, NeurIPS 2018) is attractive: any vector field gives an invertible map, and the change of variables needs no special architecture (contrast the coupling layers of Normalizing flows, RealNVP and Glow). Maximum-likelihood training, though, has to solve the ODE forward and backward for every minibatch, which made CNFs slow and hard to scale. Diffusion models scale well, but their training is tied to one family of noising processes and their sampling paths are curved. Lipman et al. showed that a CNF can be trained the way a diffusion model is: pick a path of distributions, write down a regression target, and train on random triples with no ODE solve. Diffusion paths are one choice. The straight optimal-transport (OT) path is another, and it trained faster and sampled with fewer function evaluations. Two concurrent papers reached closely related objectives: Rectified flow (Liu, Gong and Liu) and stochastic interpolants (Albergo and Vanden-Eijnden), both ICLR 2023.
Flows, paths and the flow-matching loss
- the flow: where a source point is at time
- the time-dependent vector field (the network )
- push-forward: the distribution of for
- continuity equation; a field generates a path when the pair satisfies it
- a conditional path: and concentrated around the data point
- a field that generates the conditional path
- the marginal path and field; is the posterior average of the conditional fields, and it generates (Theorem 1), with
is the right objective, but it cannot be computed, because is an integral over the whole data distribution. The key result is that the conditional version can be computed and gives the same gradients.
- sample a data point, sample on its conditional path, regress on its conditional velocity
- Theorem 2 (assuming ); the two losses differ by a constant, so they share their minimiser
The proof is one line. Expand both squares. The terms are identical, since both average over the same . The cross terms agree as well: , because is the posterior mean of . What remains, , does not involve . It is the variance of the conditional target given , and it is the floor below which the CFM loss cannot fall.
Gaussian conditional paths and the OT path
- every conditional path starts at the source
- and ends in a small Gaussian around the data point
- the conditional flow; with samples the path
- Theorem 3: the unique field of this affine flow; primes denote
Substituting gives the loss that is actually implemented, . Any with the boundary values above is allowed. The paper studies two families.
- a straight line from to (almost) , travelled at constant speed
- the regression target; it does not depend on
- the same field written as a function of position
- the conditional flow is the optimal-transport displacement map between and ; the marginal flow is not, in general, optimal transport
- the variance-preserving diffusion of Denoising diffusion probabilistic models (DDPM) and score SDEs, with time reversed so that data sits at
- the variance-exploding path of noise-conditional Score-based generative models
- noise schedule, e.g. linear from 0.1 to 20
With a diffusion path, FM trains a network whose ODE is the diffusion model's probability-flow ODE, but with a different and more stable regression target than denoising score matching (see Diffusion noise-prediction loss). For any Gaussian path written , the marginal velocity and the score are affinely related: . A trained velocity model therefore also gives a score model, and the reverse holds. Diffusion is one special case of flow matching. Flow matching itself is not tied to Gaussian noise or to a stochastic process.
Setup in the paper
- Images
- CIFAR-10 and ImageNet at , and ; the U-Net of Dhariwal and Nichol (2021) with minimal changes
- Paths compared
- FM with the OT path, FM with the VP diffusion path, and diffusion baselines (DDPM, score matching, ScoreFlow) trained on the same architecture
- Sampling
- dopri5 adaptive ODE solver, absolute and relative tolerance ; low-NFE comparisons with fixed-step solvers
- Likelihood
- exact NLL in bits per dimension via the instantaneous change of variables along the learned ODE
- Toy 2-D
- an MLP with 5 layers of 512 units
- Other uses
- super-resolution and class-conditional ImageNet; later the training objective of Stable Diffusion 3 (rectified-flow form) and of many audio, video and molecule generators
| model | CIFAR-10 | ImageNet 32×32 | ImageNet 64×64 |
|---|---|---|---|
| DDPM | 3.12 / 7.48 / 274 | 3.54 / 6.99 / 262 | 3.32 / 17.36 / 264 |
| Score matching | 3.16 / 19.94 / 242 | 3.56 / 5.68 / 178 | 3.40 / 19.74 / 441 |
| ScoreFlow | 3.09 / 20.78 / 428 | 3.55 / 14.14 / 195 | 3.36 / 24.95 / 601 |
| FM, VP diffusion path | 3.10 / 8.06 / 183 | 3.54 / 6.37 / 193 | 3.33 / 16.88 / 187 |
| FM, OT path | 2.99 / 6.35 / 142 | 3.53 / 5.02 / 122 | 3.31 / 14.45 / 138 |
With the same network, the OT path gave the best likelihood and FID on all three datasets and needed the fewest function evaluations. FM with the diffusion path already beat the score-matching baselines, so part of the gain comes from the objective and part from the path. On ImageNet FM-OT reached 2.90 bits/dim and FID 20.9.
Worked example: one CFM target, and the marginal field in 1-D
Take the OT path with , a noise sample , a data point and . Then, in one dimension, take data with , so that , and evaluate the marginal field at , .
| x₁ | posterior weight p(x₁ | xₜ = 0.25) | conditional velocity (x₁ − x)/(1 − t) |
|---|---|---|
| +1 | 0.7311 | 1.5 |
| −1 | 0.2689 | −2.5 |
| marginal uₜ(0.25) | 0.4242 |
From a training triple to the loss floor
- 1Position: .
- 2Target: . The field form gives the same: .
- 3If the network outputs , this sample contributes to .
- 41-D posterior: given , , so the weights are and , i.e. and . The marginal field is .
- 5The best network outputs at this point, yet the per-sample targets are or . The residual loss is , the conditional variance. A CFM loss that plateaus well above zero is normal.
- 6Check against the score: , and with , .
1import math, torch, torch.nn as nn23class VelocityMLP(nn.Module):4 def __init__(self, d=2, h=256):5 super().__init__()6 self.net = nn.Sequential(nn.Linear(d + 1, h), nn.SiLU(), nn.Linear(h, h), nn.SiLU(),7 nn.Linear(h, h), nn.SiLU(), nn.Linear(h, d))89 def forward(self, x, t): # x (B, d), t (B,)10 return self.net(torch.cat([x, t[:, None]], -1))1112def sample_data(n): # 8 Gaussians on a circle of radius 2.513 k = torch.randint(0, 8, (n,))14 ang = 2 * math.pi * k / 815 mu = 2.5 * torch.stack([ang.cos(), ang.sin()], -1)16 return mu + 0.15 * torch.randn(n, 2)1718def cfm_loss(model, x1, sigma_min=1e-4):19 x0 = torch.randn_like(x1) # source sample, p_0 = N(0, I)20 t = torch.rand(x1.size(0)) # t ~ U[0, 1]21 xt = (1 - (1 - sigma_min) * t[:, None]) * x0 + t[:, None] * x1 # psi_t(x0), OT path22 target = x1 - (1 - sigma_min) * x0 # d/dt psi_t(x0)23 return ((model(xt, t) - target) ** 2).sum(-1).mean()2425@torch.no_grad()26def sample(model, n, steps=8): # Euler on dx/dt = v_theta(x, t), t: 0 -> 127 x = torch.randn(n, 2)28 for i in range(steps):29 t = torch.full((n,), i / steps)30 x = x + model(x, t) / steps31 return x3233torch.manual_seed(0)34model = VelocityMLP()35opt = torch.optim.Adam(model.parameters(), lr=1e-3)36for it in range(3000): # no ODE solve anywhere in training37 loss = cfm_loss(model, sample_data(512))38 opt.zero_grad(); loss.backward(); opt.step()39x = sample(model, 1000, steps=8)40print(x.shape) # torch.Size([1000, 2])41print(round(loss.item(), 2)) # 4.69: stays well above 0, the conditional target is noisy42print(round(x.norm(dim=-1).mean().item(), 1)) # 2.4: samples sit near the circle of radius 2.5Use when
- +Simulation-free training: one network evaluation per sample, a plain regression loss
- +Any conditional path is allowed; the OT path gives a constant, -independent target and fewer sampling steps
- +Exact likelihood and deterministic, invertible encoding through the learned ODE
- +Contains diffusion as a special case, so tools such as guidance and latent training transfer
Avoid when
- −Sampling still needs an ODE solve (tens to hundreds of NFE without distillation)
- −Marginal paths stay curved with independent coupling; few-step sampling needs reflow, minibatch OT or distillation
- −The loss has a high noise floor, so it is a poor progress signal
- −The time convention and the boundary differ between papers and codebases
Check yourself
Why can flow matching train on per-sample conditional velocities u_t(x | x₁) instead of the marginal field u_t(x)?
Check yourself
With the OT path, σ_min = 0.01, x₀ = (0.5, −1), x₁ = (2, 1) and t = 0.25, what does the network regress on?
Remember
- ✓Conditional flow matching: has the same gradient as regressing the marginal field , so training needs no ODE simulation.
- ✓OT path: with target ; sample by integrating from noise () to data ().
- ✓Gaussian paths have field ; diffusion (VP/VE) paths are special cases, and the velocity is affine in the score.
- ✓The loss floor is the conditional variance of the target; a high plateau is expected.
- ✓Lipman et al. (Table 1): FM-OT 2.99 bits/dim and FID 6.35 on CIFAR-10 with 142 NFE, best of all compared methods.