Definition
UMAP (Uniform Manifold Approximation and Projection) is a nonlinear dimensionality-reduction method. It describes the data as a weighted graph in which each point is joined to its nearest neighbours with a membership strength , then finds low-dimensional coordinates whose own graph of similarities matches as closely as possible under a fuzzy cross-entropy loss.
At a glance
- Task
- Unsupervised dimensionality reduction — mostly 2-D/3-D visualisation, also 5–50-D features for clustering
- Input → output
- points in (any metric) → points , usually
- Learns
- a coordinate per training point (non-parametric); new points are placed by a short extra optimisation
- Training
- approximate kNN by NN-descent (empirically about ) + SGD over the graph edges, for epochs
- Prediction
transform: find the new point's neighbours, start at their weighted mean, refine for a few epochs- Hyperparameters
n_neighbors(local vs global),min_dist(how tightly points pack),n_components,metric,n_epochs- Assumes
- the data lie near a low-dimensional manifold, and are roughly uniformly spread over it (each point gets its own distance scale to make this true)
The problem
PCA finds the best linear projection. When data lie on a curved surface — a rolled-up sheet, images of a rotating object, cell types along a developmental trajectory — a linear projection folds distant parts of the surface on top of each other. Linear latent-factor models (ICA, NMF, factor analysis) share that limit.
t-SNE fixed the folding by keeping only neighbourhoods: it matches neighbour probabilities in the input and the embedding. But it is slow ( per iteration with Barnes–Hut, and hard to scale past a few hundred thousand points), it cannot embed new points, it only works well in 2–3 output dimensions, and the distances between its clusters carry almost no information. UMAP keeps t-SNE's local view but builds it on a sparse graph that is cheap to optimise, so it is much faster, it can map new points, it works in any output dimension, and it usually keeps more of the large-scale arrangement.
The idea
Think of each point as looking around at its nearest neighbours with its own ruler. In a dense region the ruler is short; in a sparse region it is long. Measured with that local ruler, every point has neighbours at "unit" distance, as if the data were spread uniformly over the manifold. Each point then says how strongly it belongs with each neighbour — a fuzzy edge between 0 and 1. Two points' opinions about the same edge are merged with a fuzzy OR.
The result is one weighted graph — a skeleton of the data. UMAP then drops the points into 2-D at random and runs a spring simulation: edges with high weight pull their endpoints together, and every pair that is not strongly connected pushes apart. When the forces balance, points that were neighbours in the input are neighbours in the picture.
Vocabulary
- k (n_neighbors)
- size of each point's neighbourhood. In umap-learn the count includes the point itself, so
n_neighbors=4keeps 3 other points - ρᵢ (rho)
- distance from point to its nearest neighbour. Subtracting it guarantees every point is fully connected () to at least one neighbour — the local connectivity assumption
- σᵢ (sigma)
- point 's local length scale, found by binary search so its memberships add up to
- Membership vⱼ|ᵢ
- how strongly thinks is its neighbour: for the nearest, decaying exponentially beyond it, for non-neighbours
- Fuzzy union
- symmetrisation by probabilistic OR, : the edge exists if either endpoint thinks so
- Fuzzy simplicial set
- the topological name for this weighted graph (UMAP only uses its edges, the 1-simplices)
- min_dist
- the smallest distance two points are allowed to have in the embedding before the similarity curve stops rewarding closeness
- a, b
- the two constants of the low-dimensional similarity curve , fitted from
min_dist - Negative sample
- a randomly chosen point that is pushed away from an edge endpoint; this is how repulsion stays cheap
The math
Step 1 — local scales. For each point take its nearest neighbours at distances . Set to the smallest of them and choose so that
- input-space distance (Euclidean here; any metric works)
- distance from to its nearest neighbour
- local length scale of point i, solved by binary search (the left side increases with σ)
- target total membership — a fixed 'effective number of neighbours'
This is the analogue of t-SNE's perplexity calibration, but without normalising to a probability distribution and with a target of instead of bits. Because the nearest neighbour always contributes , the remaining neighbours share the remaining .
- directed membership of in 's neighbourhood (0 if is not among 's neighbours)
- symmetric edge weight of the input graph: the fuzzy union (probabilistic OR) of the two directions
The union is : the probability that at least one of two independent events happens. It is either direction, so edges are only ever strengthened by symmetrisation.
Step 2 — the low-dimensional similarity. In the embedding UMAP uses a smooth, heavy-tailed curve of the distance :
- similarity of points i and j in the embedding, in (0, 1]
- found once by least-squares fitting to the target curve (with \texttt{spread} = 1)
- distance below which points count as fully similar
With this is exactly t-SNE's Student-t kernel. min_dist changes and : a larger value makes stay near 1 for longer, so neighbours no longer gain anything by sitting on top of each other and the clusters loosen.
| min_dist | a | b | q(0.1) | q(0.5) | q(1) | q(2) |
|---|---|---|---|---|---|---|
| 0 | 1.9328 | 0.7905 | 0.952 | 0.608 | 0.341 | 0.147 |
| 0.1 (default) | 1.5769 | 0.8951 | 0.975 | 0.687 | 0.388 | 0.155 |
| 0.5 | 0.5830 | 1.3342 | 0.999 | 0.916 | 0.632 | 0.212 |
| 1.0 | 0.1150 | 1.9292 | 1.000 | 0.992 | 0.897 | 0.375 |
Step 3 — the loss. Treat each pair as a Bernoulli "edge present" variable: in the input, in the embedding. UMAP minimises the cross-entropy between the two fuzzy sets:
- the embedding coordinates (the only variables)
- large when an input edge is strong but the embedded points are far apart
- large when two unconnected points sit close together in the embedding
The terms do not depend on , so only matters. t-SNE's KL divergence has only the first term; the explicit repulsion term is what lets UMAP keep gaps between clusters without normalising over all pairs.
Deriving the gradient. Write , so . Then has derivative with respect to , and has derivative . Multiplying by :
- direction of the force; a positive coefficient pulls i toward j when we step against the gradient
- a small constant (0.001) that stops the repulsion blowing up at zero distance
Each term is a spring along the line between two points. Gradient descent moves , with every per-pair term clipped to and the step size decaying linearly to 0.
Step 4 — making it fast. The attractive sum has only about nonzero terms (the graph edges), but the repulsive sum runs over all pairs. umap-learn uses SGD with two sampling tricks. It visits each edge with frequency proportional to and applies the attractive term with weight 1. For each such visit it draws a few random points (negative_sample_rate = 5) and applies the repulsive term to as if . Each epoch therefore costs instead of .
Worked example: twelve points in 3-D
Three groups of four points. In the input, group B is the nearest to A (centroid gap 3.15), C is farther from B (5.56) and farthest from A (6.04). We use n_neighbors , so each point keeps 3 neighbours and its memberships must sum to .
| Point | Group | x₁ | x₂ | x₃ |
|---|---|---|---|---|
| 0 | A | 0.0 | 0.0 | 0.0 |
| 1 | A | 0.6 | 0.2 | 0.1 |
| 2 | A | 0.1 | 0.7 | 0.3 |
| 3 | A | 0.5 | 0.6 | −0.2 |
| 4 | B | 3.0 | 0.4 | 0.5 |
| 5 | B | 3.5 | 0.1 | 0.9 |
| 6 | B | 3.2 | 0.9 | 1.1 |
| 7 | B | 3.8 | 0.7 | 0.4 |
| 8 | C | 1.5 | 4.5 | 3.5 |
| 9 | C | 2.1 | 4.9 | 3.8 |
| 10 | C | 1.4 | 5.2 | 4.1 |
| 11 | C | 2.0 | 4.4 | 4.4 |
Point 0's fuzzy neighbourhood
- 1Neighbours. Distances from point 0: to 1 is , to 2 is , to 3 is ; every B or C point is at least 3.07 away. The three nearest are 1, 2, 3.
- 2ρ. , so the shifted distances are .
- 3σ. Solve . Binary search gives .
- 4Memberships. , , . They sum to , as required.
- 5Union with the other direction. Point 2 computes its own view, , so . Likewise , and because .
| Point | Neighbours | Distances | ρ | σ | Memberships v |
|---|---|---|---|---|---|
| 0 | 1, 2, 3 | 0.640, 0.768, 0.806 | 0.640 | 0.2106 | 1, 0.545, 0.455 |
| 1 | 3, 0, 2 | 0.510, 0.640, 0.735 | 0.510 | 0.2499 | 1, 0.593, 0.407 |
| 2 | 3, 1, 0 | 0.648, 0.735, 0.768 | 0.648 | 0.1478 | 1, 0.556, 0.444 |
| 3 | 1, 2, 0 | 0.510, 0.648, 0.806 | 0.510 | 0.2985 | 1, 0.629, 0.371 |
| 4 | 5, 6, 7 | 0.707, 0.806, 0.860 | 0.707 | 0.1790 | 1, 0.575, 0.425 |
| 5 | 4, 7, 6 | 0.707, 0.837, 0.877 | 0.707 | 0.2150 | 1, 0.547, 0.453 |
| 6 | 4, 5, 7 | 0.806, 0.877, 0.943 | 0.806 | 0.1450 | 1, 0.612, 0.388 |
| 7 | 5, 4, 6 | 0.837, 0.860, 0.943 | 0.837 | 0.0789 | 1, 0.742, 0.258 |
| 8 | 9, 10, 11 | 0.781, 0.927, 1.034 | 0.781 | 0.2810 | 1, 0.594, 0.406 |
| 9 | 8, 11, 10 | 0.781, 0.787, 0.819 | 0.781 | 0.0251 | 1, 0.776, 0.224 |
| 10 | 9, 8, 11 | 0.819, 0.927, 1.044 | 0.819 | 0.2306 | 1, 0.624, 0.376 |
| 11 | 9, 8, 10 | 0.787, 1.034, 1.044 | 0.787 | 0.3633 | 1, 0.507, 0.493 |
Notice point 9: its three neighbours are almost equidistant (0.781, 0.787, 0.819), so a tiny is needed to bring the total down to 2. Point 11's neighbours are spread out, so it gets a large . Each point's ruler adapts to its surroundings.
Laying the graph out in 2-D (min_dist = 0.1, so a = 1.5769, b = 0.8951)
- 1Initialise. Place the 12 points uniformly at random in (seeded). Points 0 and 2 start at and , a distance apart, so — far below .
- 2One force. For pair (0, 2) the attractive coefficient is and the repulsive one is . The net times gives the gradient : the step moves point 0 toward point 2.
- 3A force with no edge. Points 0 and 8 have and start apart. They only feel the repulsive coefficient, , which pushes them apart.
- 4Iterate. Sum these forces over all 66 pairs, step with , and repeat for 200 epochs. The loss falls from 53.27 to 6.38.
- 5Result. The groups collapse into three tight clumps with centroids A , B and C . Points 0 and 2 end up 0.448 apart, so .
- 6A new point . Its neighbours are 4, 5 and 6 (distances 0.374, 0.490, 0.574), giving , and memberships . Its starting position is the membership-weighted mean of , which is , right inside group B. A few more epochs with the training points held fixed refine it.
The three clumps are cleanly separated (trustworthiness 1.0), but look at the gaps between them. A–B is 8.19 and B–C is 8.51, nearly equal, while in the input B–C is 1.8× A–B. With the graph has no edges between groups, so only repulsion acts between them. Their placement is fixed by the random start, not by the data.
The algorithm, step by step
UMAP(X, k, min_dist, d)
- 1Neighbours. Find each point's approximate nearest neighbours with NN-descent: start from random-projection-tree candidates, then repeatedly ask "are my neighbours' neighbours closer?".
- 2Calibrate. For each point set to its nearest-neighbour distance and binary-search so that .
- 3Graph. Directed memberships , then fuzzy union , a sparse symmetric matrix with about nonzeros.
- 4Curve. Fit so that matches the
min_disttarget curve. - 5Initialise with the leading eigenvectors of the graph's normalised Laplacian (spectral embedding), rescaled, plus a little noise; random init is the fallback.
- 6Optimise. For epochs ( for small data, 200 for large): for each edge, with frequency , pull and together; for each pull, push away from 5 random points. Clip each update to and decay the learning rate linearly.
- 7Transform new data (optional): neighbours in the training set → memberships → weighted-mean start → a few epochs with the training embedding frozen.
Complexity
- kNN graph
- NN-descent is about distance evaluations in practice; exact search would be
- Calibration
- times about 64 binary-search iterations
- Spectral init
- a sparse eigen-solve on the graph Laplacian, roughly linear in
- Optimisation
- , linear in ; t-SNE with Barnes–Hut is per iteration
- Memory
- for the sparse graph, plus for the embedding
Hyperparameters and tuning
What each knob does
- n_neighbors (15)
- Sets the target and which edges exist. Small (5–10) builds a graph of tight local patches that may fall into disconnected pieces. Large (50–200) adds weak long-range bridges that pull the global layout into shape but blur fine structure.
- min_dist (0.1)
- Changes . Near 0, keeps rising as points get closer, so neighbours pile up into dense clumps, which helps clustering. At 0.5–1, already at moderate distances, so points spread out evenly, which helps when you want to see the structure inside a cluster.
- spread (1.0)
- Scale of the target curve;
min_distmust be ≤spread. Rarely changed. - n_components (2)
- Output dimension. Unlike t-SNE, UMAP stays cheap at 10–50 dimensions, which is useful as preprocessing for HDBSCAN.
- metric
- Any distance: euclidean, cosine (text and embeddings), manhattan, hamming/jaccard (binary data), or a custom numba function.
- n_epochs, learning_rate
- More epochs give a more converged layout. The default learning rate of 1.0 is rarely changed.
- negative_sample_rate (5)
- Number of repulsive samples per attractive one. Higher values give more separation and slower runs.
- init
- 'spectral' (default), 'pca', or 'random'. Spectral init is the main source of UMAP's global structure.
- set_op_mix_ratio (1.0)
- Interpolates between fuzzy union (1) and fuzzy intersection (0). Intersection keeps only edges that both endpoints agree on.
- densmap
- Adds a term that preserves local density, so sparse clusters look sparse instead of being equalised.
| n_neighbors | Edges | Edges between groups | Gap A–B | Gap B–C | Gap A–C | Final loss |
|---|---|---|---|---|---|---|
| 3 | 15 | 0 | 7.67 | 7.66 | 15.00 | 8.00 |
| 4 | 18 | 0 | 8.19 | 8.51 | 13.49 | 6.38 |
| 5 | 29 | 11 | 8.74 | 8.93 | 12.97 | 4.30 |
| 6 | 38 | 20 | 6.10 | 7.80 | 13.90 | 3.92 |
| 8 | 50 | 32 | 2.30 | 5.47 | 7.74 | 10.36 |
At the 11 bridge edges are almost weightless (the largest is 0.015), so the layout barely changes. At every A point has edges to every B point (weights up to 0.22), and the embedding finally reproduces the input's ordering A–B < B–C < A–C. The loss is not comparable across : a graph with more weak edges simply has more terms. On the same example with , raising min_dist from 0 to 0.1, 0.5 and 1.0 grows the mean within-group distance from 0.198 to 0.265, 0.565 and 0.980.
In code
1import numpy as np23X = np.array([[0.0, 0.0, 0.0], [0.6, 0.2, 0.1], [0.1, 0.7, 0.3], [0.5, 0.6, -0.2], # A4 [3.0, 0.4, 0.5], [3.5, 0.1, 0.9], [3.2, 0.9, 1.1], [3.8, 0.7, 0.4], # B5 [1.5, 4.5, 3.5], [2.1, 4.9, 3.8], [1.4, 5.2, 4.1], [2.0, 4.4, 4.4]]) # C67def fuzzy_graph(X, k):8 """k counts the point itself (umap-learn convention): keep k-1 neighbours."""9 n = len(X)10 D = np.sqrt(((X[:, None] - X[None]) ** 2).sum(-1))11 V = np.zeros((n, n)); rho = np.zeros(n); sigma = np.zeros(n)12 for i in range(n):13 nb = [j for j in np.argsort(D[i], kind="stable") if j != i][: k - 1]14 d = D[i, nb]15 rho[i] = d.min() # distance to the nearest neighbour16 lo, hi, s = 0.0, np.inf, 1.0 # binary search: sum of memberships = log2 k17 for _ in range(64):18 total = np.exp(-np.maximum(0, d - rho[i]) / s).sum()19 if abs(total - np.log2(k)) < 1e-5:20 break21 if total > np.log2(k):22 hi = s; s = (lo + hi) / 223 else:24 lo = s; s = s * 2 if hi == np.inf else (lo + hi) / 225 sigma[i] = max(s, 1e-3 * D[i][D[i] > 0].mean())26 V[i, nb] = np.exp(-np.maximum(0, d - rho[i]) / sigma[i])27 W = V + V.T - V * V.T # fuzzy union (probabilistic OR)28 return W, rho, sigma, V2930def mulberry32(seed): # same seeded generator as the figure31 s = seed32 def r():33 nonlocal s34 s = (s + 0x6D2B79F5) & 0xFFFFFFFF35 t = ((s ^ (s >> 15)) * (1 | s)) & 0xFFFFFFFF36 t = (t + (((t ^ (t >> 7)) * (61 | t)) & 0xFFFFFFFF)) & 0xFFFFFFFF ^ t37 return ((t ^ (t >> 14)) & 0xFFFFFFFF) / 2**3238 return r3940def layout(W, a, b, epochs=200, seed=7):41 n = len(W); r = mulberry32(seed)42 Y = np.array([[r() * 10 - 5, r() * 10 - 5] for _ in range(n)])43 iu, ju = np.triu_indices(n, 1)44 w = W[iu, ju]45 for e in range(epochs):46 diff = Y[iu] - Y[ju]47 s = (diff ** 2).sum(1) # squared low-dim distances48 att = 2 * a * b * w * s ** (b - 1) / (1 + a * s ** b)49 rep = 2 * b * (1 - w) / ((0.001 + s) * (1 + a * s ** b))50 g = np.clip((att - rep)[:, None] * diff, -4, 4)51 G = np.zeros_like(Y)52 np.add.at(G, iu, g); np.add.at(G, ju, -g)53 q = np.clip(1 / (1 + a * s ** b), 1e-12, 1 - 1e-12)54 loss = -(w * np.log(q) + (1 - w) * np.log(1 - q)).sum()55 Y = np.round(Y - (1 - e / epochs) * G, 4) # linearly decaying step; rounding = bit-reproducible56 return Y, loss5758W, rho, sigma, V = fuzzy_graph(X, k=4)59print("rho_0 = %.3f sigma_0 = %.4f" % (rho[0], sigma[0]))60print("v(.|0) =", V[0, 1:4].round(3), " w_02 = %.3f" % W[0, 2])61print("edges:", int((W > 0).sum() // 2))62Y, loss = layout(W, a=1.5769, b=0.8951) # min_dist = 0.163print("final loss %.2f" % loss)64for g in range(3):65 print("group", "ABC"[g], "centroid", Y[4 * g: 4 * g + 4].mean(0).round(2))1import numpy as np, umap2from umap.umap_ import fuzzy_simplicial_set, find_ab_params3from sklearn.manifold import trustworthiness45# X: the 12 x 3 array from the table above6W, sigma, rho = fuzzy_simplicial_set(X, n_neighbors=4, random_state=0, metric="euclidean")7W = W.toarray()8print(rho[:4].round(3), sigma[0].round(4)) # [0.64 0.51 0.648 0.51] 0.21069print(W[0, 1:4].round(3), int((W > 0).sum() // 2)) # [1. 0.747 0.657] 1810print(np.round(find_ab_params(1.0, 0.1), 4)) # [1.5769 0.8951]1112reducer = umap.UMAP(n_neighbors=4, min_dist=0.1, random_state=42)13Y = reducer.fit_transform(X)14print(trustworthiness(X, Y, n_neighbors=3)) # 1.015for g in range(3):16 print("ABC"[g], Y[4*g:4*g+4].mean(0).round(2)) # A [-9.34 -15.08] B [-14.77 5.14] C [17.54 9.39]17print(reducer.transform([[3.3, 0.5, 0.7]]).round(2)) # [[-15.82 5.54]] -> next to BUse when
- +You want a 2-D/3-D map of large data (10⁵–10⁷ points): UMAP is typically several times faster than t-SNE.
- +You need to embed new points later (
transform) or reuse the embedding as features. - +You want 10–50 output dimensions as preprocessing for density-based clustering (HDBSCAN).
- +You have non-Euclidean data: cosine for text embeddings, Jaccard for sets, a custom metric.
Avoid when
- −You need distances, densities or cluster sizes in the picture to mean something quantitatively.
- −The data are small or roughly linear: PCA is exact, deterministic and interpretable.
- −You need a reproducible, invertible or explainable mapping (axes have no meaning; inverse_transform is approximate).
- −You might over-interpret: UMAP can show clusters in pure noise when n_neighbors is small.
Where it's used
Single-cell genomics
Standard 2-D maps of cell types in Scanpy and Seurat, built from the PCA of gene expression and the same kNN graph used for Leiden clustering.
Embedding inspection
Visualising sentence, image or LLM embeddings with the cosine metric to spot topics, duplicates and label noise.
Clustering pipelines
UMAP to about 10 dimensions followed by HDBSCAN, e.g. BERTopic's topic model.
Data and model debugging
Colouring a UMAP of a network's penultimate-layer features by label or error to find confused classes and outliers.
Check yourself
A point's nearest neighbour is 2.5 away. Before symmetrisation, what membership v does it assign to that neighbour?
Check yourself
With n_neighbors = 4, a point's three neighbours are at distances 1.0, 1.5 and 2.0. Its memberships are…
Check yourself
Point i says v₍j|i₎ = 0.5 and point j says v₍i|j₎ = 0.4. The symmetric edge weight w_ij is…
Check yourself
In the worked example, which change makes the embedding reproduce the input's ordering of group distances (A–B < B–C < A–C)?
Remember
- ✓UMAP builds a fuzzy kNN graph: with set so memberships sum to , then symmetrises by fuzzy union .
- ✓The embedding uses , where come from
min_dist. The layout minimises the cross-entropy between and : attraction along edges, repulsion between everything else. - ✓SGD with edge sampling and negative sampling makes each epoch linear in n, which is why UMAP scales further than t-SNE and supports transform and any output dimension.
- ✓
n_neighborstrades local detail for global structure (it decides which bridges exist);min_distcontrols how tightly points pack. - ✓Read neighbourhoods off a UMAP plot, not distances, densities or cluster sizes, and check any cluster you find across settings and seeds.