# Rectified flow

> Rectified flow learns an ODE dZₜ = v(Zₜ, t)dt by regressing v on X₁ − X₀ along straight lines Xₜ = tX₁ + (1 − t)X₀. Reflow, retraining on the model's own (noise, sample) pairs, makes the paths straighter, until one Euler step, or a distilled one-step model, generates good samples.

Source: https://www.mlfoundry.in/learn/rectified-flow  
Section: Deep Learning › Generative Deep Learning Algorithms  
Level: advanced · 14 min read · updated 2026-10-12

**Rectified flow:** is a method for learning a transport map between two distributions $\pi_0$ and $\pi_1$ from samples, introduced by Xingchao Liu, Chengyue Gong and Qiang Liu, "Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow" (ICLR 2023). Given any coupling $(X_0,X_1)$ (for generation, independent noise $X_0\sim\mathcal N(0,I)$ and data $X_1$), it fits a velocity field $v(x,t)$ to the direction $X_1-X_0$ of the straight line $X_t=tX_1+(1-t)X_0$. The ODE $dZ_t=v(Z_t,t)\,dt$ then transports $\pi_0$ to $\pi_1$ along non-crossing paths. Applying the procedure again to the coupling the ODE produces, **reflow**, makes the paths straighter, so the ODE can be solved accurately in very few steps.

ODE samplers for **Diffusion models** follow curved trajectories, so they need dozens to hundreds of network evaluations. One Euler step is exact only along a straight path travelled at constant speed. Liu, Gong and Liu asked how to learn such paths directly. Straight lines between independently drawn noise and data cross each other, and an ODE cannot cross itself, so no flow can follow them exactly. The best ODE in the least-squares sense averages the crossing directions. Its trajectories untangle the lines, and the coupling it induces has lower transport cost than the independent one. Repeat the step, and the lines of the new coupling cross less, so the next flow is straighter. The same objective, with straight paths, appeared concurrently as the OT path of **Flow matching** (Lipman et al., ICLR 2023). Rectified flow adds the reflow procedure and the straightness analysis. It also covers transfer between two arbitrary distributions, such as cats to wild animals, not only noise to data.

## The rectified flow of a coupling

$$
\min_v\int_0^1\mathbb E\Big[\big\|(X_1-X_0)-v(X_t,t)\big\|^2\Big]dt,\qquad X_t=tX_1+(1-t)X_0,\qquad v^X(x,t)=\mathbb E\big[X_1-X_0\mid X_t=x\big]
$$

- $(X_0,X_1)$: any coupling of $\pi_0$ and $\pi_1$; for generation $X_0\sim\mathcal N(0,I)$ independent of data $X_1$
- $X_t$: linear interpolation; $\dot X_t=X_1-X_0$ is constant along each line
- $v^X$: the minimiser: the average direction of all lines passing through $x$ at time $t$

$$
dZ_t=v^X(Z_t,t)\,dt,\ \ Z_0\sim\pi_0\qquad\Longrightarrow\qquad \mathrm{Law}(Z_t)=\mathrm{Law}(X_t)\ \ \forall t,\qquad \mathbb E\,c(Z_1-Z_0)\le\mathbb E\,c(X_1-X_0)\ \text{ for convex }c
$$

- $Z=\mathrm{Rectflow}(X_0,X_1)$: the **rectified flow**; $(Z_0,Z_1)$ is a new, deterministic coupling of the same two distributions
- $\mathrm{Law}(Z_t)=\mathrm{Law}(X_t)$: marginal preservation: the flow reproduces every intermediate marginal of the interpolation, so $Z_1\sim\pi_1$
- $c$: any convex cost, e.g. $\|\cdot\|^2$; rectification never increases it, and it is not tied to one particular cost

Paths of $Z$ do not cross: an ODE with a well-behaved field has unique solutions, so two trajectories cannot occupy the same point at the same time. Where lines of $X$ cross, $v^X$ averages their directions, and the trajectories of $Z$ rewire around the crossing. That is why the induced coupling is cheaper. Paired points that were far apart are swapped for nearer ones.

## Reflow and straightness

$$
Z^{k+1}=\mathrm{Rectflow}\big(Z^k_0,Z^k_1\big),\quad Z^0=(X_0,X_1),\qquad S(Z)=\int_0^1\mathbb E\big\|(Z_1-Z_0)-\dot Z_t\big\|^2dt,\qquad \min_{k\le K}S(Z^k)\le\frac{\mathbb E\|X_1-X_0\|^2}{K}
$$

- $Z^k$: the $k$-rectified flow; $Z^1$ is trained on the independent coupling, $Z^2$ on pairs $(Z_0,Z_1)$ obtained by simulating $Z^1$
- $S(Z)$: straightness: zero exactly when every path is a straight line travelled at constant speed
- $1/K$: the paper's bound: straightness decays at rate $O(1/K)$ in the number of reflows

If $S(Z)=0$, then $Z_t=tZ_1+(1-t)Z_0$, and a single Euler step $Z_1=Z_0+v(Z_0,0)$ is exact. In practice one reflow already gets close. The remaining gap is closed by **distillation**: fit a one-step model to the deterministic pairs with $\mathbb E\|(Z^k_1-Z^k_0)-v(Z^k_0,0)\|^2$. Distilling directly from the 1-rectified flow is much harder, because its pairs come from curved paths. Each reflow costs a full round of simulation, $Z_1=\mathrm{ODE}(Z_0)$ on many noise draws, plus retraining. Errors also accumulate: every round fits an imperfect $v$, so the samples drift slightly from $\pi_1$.

$$
X_t=\alpha_tX_1+\beta_tX_0,\quad \text{target }\dot\alpha_tX_1+\dot\beta_tX_0\qquad\qquad \pi_{\rm ln}(t;m,s)=\frac{1}{s\sqrt{2\pi}}\,\frac{1}{t(1-t)}\exp\!\Big(-\frac{(\mathrm{logit}\,t-m)^2}{2s^2}\Big)
$$

- $\alpha_t,\beta_t$: the paper's nonlinear extension; the VP probability-flow ODE of diffusion is one such choice, and straight lines ($\alpha_t=t,\ \beta_t=1-t$) are the canonical one
- $\pi_{\rm ln}$: logit-normal density over $t$ used by Stable Diffusion 3 to sample training times; it puts more weight on intermediate $t$, where the velocity target is hardest
- $m,s$: location and scale; $m=0,\ s=1$ was the best setting overall in Esser et al. (2024)

**Setup and follow-ups**

- **Images:** CIFAR-10 with the DDPM++ U-Net of Song et al. (2021); also LSUN, CelebA-HQ and AFHQ at $256^2$, and image-to-image transfer
- **Solvers:** RK45 adaptive (for the full-quality numbers) or fixed-step Euler with $N$ steps
- **Reflow data:** pairs $(Z_0,Z_1)$ generated by simulating the previous flow; distillation regresses $Z_1-Z_0$ at $t=0$
- **InstaFlow:** Liu, Zhang, Ma, Peng and Liu (ICLR 2024): text-conditioned reflow of Stable Diffusion 1.5 and distillation to one step; FID 23.3 on MS COCO 2017-5k (previous best one-step 37.2) and 13.1 on MS COCO 2014-30k at 0.09 s per image, for 199 A100 GPU days of training
- **Stable Diffusion 3:** Esser et al., "Scaling Rectified Flow Transformers for High-Resolution Image Synthesis" (ICML 2024): rectified-flow objective with logit-normal $t$ sampling, an MM-DiT backbone, and data at $t=0$, noise at $t=1$ ($z_t=(1-t)x_0+t\epsilon$)

CIFAR-10, DDPM++ architecture (Liu et al. 2023, Table 1a); one-step numbers without distillation, distilled in parentheses

| model | NFE | IS ↑ | FID ↓ | recall ↑ |
| --- | --- | --- | --- | --- |
| 1-rectified flow (+distill) | 1 | 1.13 (9.08) | 378 (6.18) | 0.0 (0.45) |
| 2-rectified flow (+distill) | 1 | 8.08 (9.01) | 12.21 (4.85) | 0.34 (0.50) |
| 3-rectified flow (+distill) | 1 | 8.47 (8.79) | 8.15 (5.21) | 0.41 (0.51) |
| VP ODE (+distill) | 1 | 1.20 (8.73) | 451 (16.23) | 0.0 (0.29) |
| 1-rectified flow, RK45 | 127 | 9.60 | 2.58 | 0.57 |
| 2-rectified flow, RK45 | 110 | 9.24 | 3.36 | 0.54 |
| 3-rectified flow, RK45 | 104 | 9.01 | 3.96 | 0.53 |
| VP SDE (2000 steps) | 2000 | 9.58 | 2.55 | 0.58 |

The table shows both effects. One Euler step of the 1-rectified flow is useless (FID 378): at $t=0$ its field points every sample at the data mean. After one reflow the same single step gives 12.21, and after distillation 4.85. Reflow costs some quality at full NFE (2.58 → 3.36 → 3.96 with RK45), because each round refits an approximate field. Two or three rectifications are the useful range. The adaptive solver also needs fewer steps on the straighter flows (127 → 110 → 104 NFE).

> Figure: N(0, I) to four Gaussians at (±2, ±2) with std 0.2. The 1-rectified field E[X₁ − X₀ | Xₜ = x] is exact in closed form. The 2-rectified field is a kernel regression (width 0.12) on 800 reflow pairs (Z₀, Z₁), with Z₁ obtained by solving the 1-flow with RK4. It stands in for a second trained network. For 160 test points, the independent pairs have transport cost E‖X₁ − X₀‖² = 10.23, and 251 of the 1225 pairs of drawn chords cross. The 1-rectified coupling costs 3.158, only 5 chord pairs cross, but its paths are curved (straightness S = 2.628). One Euler step sends every sample to the data mean (0% on a mode, mean error 2.780 against the 16-step solution). Two steps give 37.5%, four 96.3%. After one reflow S drops to 0.012, and a single Euler step already puts 72.5% of samples on a mode, with mean error 0.208. The 2-rectified flow ends slightly worse at 16 steps (95.0% against 98.8%) and its cost is 3.217, not lower. That is the fitted field's error, the same effect as the paper's FID rising from 2.58 to 3.36 after reflow. The theorem guarantees that cost does not increase only for an exact fit.

## Worked example: two points on a line

Let $X_0\sim\mathcal N(0,1)$ and $X_1=\pm2$ with probability $\tfrac12$ each, independent. In one dimension any ODE flow preserves order, so the 1-rectified coupling is already the monotone (optimal) one, $Z_1=2\,\mathrm{sign}(Z_0)$. Its paths are still curved. Reflow on that coupling gives straight lines that never cross.

Couplings of N(0, 1) and ±2 (computed in Node)

| coupling | E(X₁ − X₀)² | velocity at t = 0 | one Euler step from x₀ = 0.8 |
| --- | --- | --- | --- |
| independent (lines only) | 5 | — | — |
| 1-rectified | 1.8085 | E[X₁] − x = −0.8 | 0 (the mean) |
| 2-rectified | 1.8085 | 2 sign(x) − x = 1.2 | 2 (exact) |

**Computing the fields**

1. Independent cost: $\mathbb E(X_1-X_0)^2=\mathbb EX_1^2+\mathbb EX_0^2=4+1=5$.
2. 1-rectified field at $t=0.5,\ x=0.5$: given $X_1=\pm2$, $X_t\sim\mathcal N(\pm1,\,0.25)$, so the posterior weights are $\propto e^{-0.5}$ and $e^{-4.5}$, i.e. $0.98201$ and $0.01799$.
3. Along the line through $x$ with endpoint $X_1$, the direction is $X_1-X_0=(X_1-x)/(1-t)$: $3$ for $+2$ and $-5$ for $-2$. So $v(0.5,0.5)=0.98201\cdot3+0.01799\cdot(-5)=2.8561$.
4. At $t=0$, $X_t=X_0$ carries no information about $X_1$: $v(x,0)=\mathbb E[X_1]-x=-x$. One Euler step maps every $x_0$ to $0$, the mean of $\pi_1$, which is not in its support.
5. Monotone coupling cost: $\mathbb E(2\,\mathrm{sign}Z_0-Z_0)^2=4-4\,\mathbb E|Z_0|+1=5-4\sqrt{2/\pi}=1.8085$. Rectification lowered the cost from 5 to 1.8085.
6. Reflow: the lines from $z_0$ to $2\,\mathrm{sign}(z_0)$ keep their order for $t<1$, so they never cross and $v^{2}(x,t)$ equals the line's slope. $S=0$, and one step $0.8+1.2=2$ is exact.

```python
import torch, torch.nn as nn

class Velocity(nn.Module):
    def __init__(self, d=2, h=256):
        super().__init__()
        self.net = nn.Sequential(nn.Linear(d + 1, h), nn.SiLU(), nn.Linear(h, h), nn.SiLU(),
                                 nn.Linear(h, h), nn.SiLU(), nn.Linear(h, d))

    def forward(self, x, t):                           # x (B, d), t (B,)
        return self.net(torch.cat([x, t[:, None]], -1))

def rf_loss(v, x0, x1):
    t = torch.rand(x0.size(0))
    xt = t[:, None] * x1 + (1 - t[:, None]) * x0       # X_t = t X1 + (1 - t) X0
    return ((v(xt, t) - (x1 - x0)) ** 2).sum(-1).mean()

@torch.no_grad()
def euler(v, x, steps):                                # dZ_t = v(Z_t, t) dt from t = 0 to 1
    for i in range(steps):
        x = x + v(x, torch.full((x.size(0),), i / steps)) / steps
    return x

def train(v, pair_fn, iters=2000, bs=512):
    opt = torch.optim.Adam(v.parameters(), lr=1e-3)
    for _ in range(iters):
        x0, x1 = pair_fn(bs)
        loss = rf_loss(v, x0, x1)
        opt.zero_grad(); loss.backward(); opt.step()
    return v

modes = torch.tensor([[2., 2.], [-2., 2.], [-2., -2.], [2., -2.]])
data = lambda n: modes[torch.randint(0, 4, (n,))] + 0.2 * torch.randn(n, 2)

torch.manual_seed(0)
# 1-rectified flow: independent coupling (X0, X1) ~ pi_0 x pi_1
v1 = train(Velocity(), lambda n: (torch.randn(n, 2), data(n)))

# Reflow: simulate the 1-flow once to get a deterministic coupling (Z0, Z1), then refit
z0 = torch.randn(20000, 2)
z1 = euler(v1, z0, 100)
def reflow_pairs(n):
    i = torch.randint(0, z0.size(0), (n,))
    return z0[i], z1[i]
v2 = train(Velocity(), reflow_pairs)

x0 = torch.randn(2000, 2)
for name, v in [("1-rectified", v1), ("2-rectified", v2)]:
    one, ref = euler(v, x0, 1), euler(v, x0, 100)
    print(name, one.shape, round((one - ref).norm(dim=-1).mean().item(), 2))
# 1-rectified torch.Size([2000, 2]) 2.81  one Euler step collapses toward the data mean
# 2-rectified torch.Size([2000, 2]) 0.17  near-straight paths: one step is close to 100 steps
```

Rectified flow, one reflow, and the one-step error on a 2-D toy

> **Reflow needs fresh, deterministic pairs:** The reflow coupling must come from **simulating** the previous flow: draw $Z_0\sim\pi_0$, solve the ODE accurately, and keep the pair $(Z_0,Z_1)$. Re-pairing data with random noise is just the 1-rectified objective again. Sloppy simulation, such as too few solver steps, bakes solver error into the new targets. Generate enough pairs to cover $\pi_0$. A finite set of pairs is memorised, and with too few the model overfits to them.

> **Straight means cheap to integrate:** Euler's local error is proportional to $\|\ddot Z_t\|$, the curvature in time. Straightness $S(Z)$ measures exactly the part that matters for sampling. A straight path travelled at constant speed makes every solver exact, whatever its step size. That is why the field of the 2-rectified flow, not just its samples, matters for one-step generation. It is also why distillation from a 2-rectified flow is easy: the student only has to reproduce a nearly affine map.

> **Relation to flow matching and diffusion:** The 1-rectified flow objective is conditional **Flow matching** with $\sigma_{\min}=0$ (Lipman et al.'s OT path) and is close to the stochastic interpolants of Albergo and Vanden-Eijnden. Choosing $\alpha_t,\beta_t$ as a diffusion schedule recovers the probability-flow ODE of **Score-based generative models**. Stable Diffusion 3 compared these formulations at scale and kept the straight interpolation with logit-normal time sampling. For few-step generation without reflow, compare **Consistency models**, which learn the map to the end of the trajectory directly, and **Denoising diffusion implicit models (DDIM)**.

**Strengths**

- A simple least-squares objective; any pair of distributions (noise→data or domain→domain)
- Marginals are preserved and every convex transport cost decreases under rectification
- Reflow straightens paths, so 1–2 Euler steps work, and distillation to one step becomes easy
- Scales: InstaFlow (one-step text-to-image) and Stable Diffusion 3 use it

**Limitations**

- Each reflow needs a large simulated dataset and a full retraining
- Approximation error accumulates over rounds; full-NFE quality drops slightly (FID 2.58 → 3.36 → 3.96)
- The 1-rectified flow itself is curved; one-step sampling without reflow fails (FID 378)
- Straightness is achieved for the fitted coupling, not for the optimal-transport coupling of a given cost

**Question:** A 1-rectified flow from N(0, I) to data is sampled with one Euler step. What does it produce?

- A. Exact samples, because training used straight lines
- B. Every sample lands at the data mean E[X₁]
- C. Samples from the data distribution plus Gaussian noise
- D. The nearest data point to each x₀

**Answer:** B. Every sample lands at the data mean E[X₁] — At $t=0$, $X_t=X_0$ is independent of $X_1$, so $v(x,0)=\mathbb E[X_1-X_0\mid X_0=x]=\mathbb E[X_1]-x$ and $x_0+v(x_0,0)=\mathbb E[X_1]$. The training lines are straight, but the learned trajectories are curved. Hence FID 378 at one step in Table 1, and 0% on a mode in the figure.

**Question:** What does reflow change?

- A. The interpolation, from linear to a diffusion schedule
- B. The coupling: the new flow is trained on pairs (Z₀, Z₁) produced by simulating the previous flow
- C. The loss, from L2 to LPIPS
- D. Nothing in training; it is a different ODE solver

**Answer:** B. The coupling: the new flow is trained on pairs (Z₀, Z₁) produced by simulating the previous flow — $Z^{k+1}=\mathrm{Rectflow}(Z^k_0,Z^k_1)$ uses the same straight-line objective on a deterministic coupling whose lines cross less. Straightness then satisfies $\min_{k\le K}S(Z^k)\le\mathbb E\|X_1-X_0\|^2/K$.

**Key points**

- Rectified flow: $\min_v\int_0^1\mathbb E\|(X_1-X_0)-v(X_t,t)\|^2dt$ with $X_t=tX_1+(1-t)X_0$; sample with $dZ_t=v(Z_t,t)dt$ from $Z_0\sim\mathcal N(0,I)$.
- Reflow: retrain on simulated pairs $(Z_0,Z_1)$; paths become straighter ($\min_{k\le K}S(Z^k)\le\mathbb E\|X_1-X_0\|^2/K$), so one Euler step or a distilled one-step model works.
- Rectification preserves all marginals and does not increase any convex transport cost; its trajectories never cross.
- CIFAR-10 (Table 1): one step FID 378 for 1-RF, 12.21 after one reflow, 4.85 after reflow plus distillation; 2.58 with RK45 at 127 NFE.
- Stable Diffusion 3 trains a rectified-flow transformer with logit-normal timestep sampling; InstaFlow reflows and distils Stable Diffusion into one step.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
