Definition
t-SNE (t-distributed Stochastic Neighbour Embedding) turns the distances between high-dimensional points into neighbour probabilities — "how likely is to be picked as 's neighbour" — and then places points in 2-D (or 3-D) so that the same probabilities, measured with a heavy-tailed Student-t kernel, , match them as closely as possible in Kullback–Leibler divergence. The output is a map for looking at data, not a model.
At a glance
- Task
- Nonlinear dimensionality reduction for visualization
- Input → output
- points → map points (one per training point — no function for new points)
- Learns
- the coordinates directly, by gradient descent on
- Training
- exact: time and memory per iteration; Barnes–Hut: ; FFT-interpolated (FIt-SNE):
- Prediction
- none — a new point needs a new run (or an out-of-sample extension such as openTSNE's
transform) - Hyperparameters
- perplexity (5–50), learning rate, iterations, early exaggeration, initialization (PCA vs random), metric
- Assumes
- local Euclidean distances are meaningful (scale features, often PCA to ~50 dims first); only neighbourhoods are worth preserving
The problem
You have 10,000 cells with 20,000 gene counts each, or 70,000 images of handwritten digits, and you want to see whether they form groups. PCA (see the PCA article) projects onto the directions of largest variance. It is linear, so two clusters that are separated along a curved surface, or along many small directions at once, are often drawn on top of each other. Classical MDS tries to preserve all pairwise distances, so it spends its effort on the large distances — which in high dimensions are the least informative — and squashes the local structure.
What a person reading a scatter plot actually wants is: points that are neighbours in the original space should be neighbours on the screen, and groups should look like groups. t-SNE optimizes exactly that, and gives up on everything else.
The idea
Put a Gaussian on each data point and ask: if picked a neighbour at random, in proportion to that Gaussian, how likely would it pick ? That gives a probability that is large for close points and practically zero for far ones. Do the same in the map, but with a Student-t (Cauchy) kernel. Then move the map points until the two sets of probabilities agree. Pairs that are neighbours in the data but far apart in the map attract; pairs that are close in the map but not neighbours in the data repel.
Vocabulary
- Affinity p(j|i)
- conditional probability that picks as its neighbour, from a Gaussian of width centred on
- Joint affinity pᵢⱼ
- symmetrized version ; all sum to 1
- Bandwidth σᵢ
- the width of point i's Gaussian; chosen separately for every point, small in dense regions, large in sparse ones
- Perplexity
- — the effective number of neighbours each point has; the user sets it and every is solved to match it
- Map / embedding
- the low-dimensional points that are optimized
- Student-t kernel
- — a Cauchy density; it decays like instead of
- Early exaggeration
- multiplying all by – for the first iterations so clusters form tightly before they are positioned
- Attractive / repulsive force
- the and halves of the gradient: data neighbours pull, every map point pushes
The math
Step 1 — affinities in the data. For each point, a Gaussian turns squared distances into a probability distribution over the other points:
- the i-th data point
- bandwidth of point i's Gaussian (one per point)
- probability that picks as a neighbour;
Step 2 — choose each from the perplexity. A single global would give points in dense regions hundreds of neighbours and isolated points none. Instead each is found by binary search so that the distribution has a fixed entropy:
- Shannon entropy of point i's neighbour distribution, in bits
- perplexity: a distribution spread evenly over neighbours has perplexity exactly
increases monotonically with (a wider Gaussian spreads probability over more points), so bisection on converges in a few dozen steps. Effect of perplexity: it is a smooth version of "" in a -nearest-neighbour graph. Small perplexity (2–5) means each point only cares about its closest few neighbours, so the map shatters into many small clumps; large perplexity (50+) makes nearly uniform over a big neighbourhood and the map keeps more global layout but blurs small clusters. Perplexity must be smaller than , and in practice smaller than the size of the clusters you hope to see.
Step 3 — symmetrize. in general (an outlier picks its neighbour, but nobody picks the outlier). t-SNE uses joint probabilities
- number of points; dividing by makes the whole matrix a single distribution, and guarantees every point has , so outliers still get attached to the map
Step 4 — affinities in the map use a Student-t distribution with one degree of freedom, normalized over all pairs:
- map position of point i — the only thing being learned
- unnormalized similarity in the map (Cauchy kernel; no bandwidth to tune)
- normalizer: sum over all ordered pairs
Why the heavy tail? At map distance 3 the t-kernel is while a Gaussian gives — about 810 times smaller. To reproduce a moderate affinity , the t-kernel lets the pair sit much farther apart in the map. That opens up room for the crowded neighbourhoods and pushes separate clusters away from each other.
Step 5 — the objective is the Kullback–Leibler divergence from to :
- cost to minimize; 0 iff Q = P
KL is asymmetric, and that asymmetry is the whole personality of t-SNE. A pair with that the map renders with costs ; the reverse error (, ) contributes only . Tearing neighbours apart is expensive; drawing non-neighbours close is cheap. So t-SNE preserves local structure faithfully and is nearly indifferent to large distances.
Step 6 — derive the gradient. Since and ,
Point appears in the pairs and , and because . The first sum therefore contributes . For the normalizer, , so . Adding them:
- pair is under-represented in the map → attraction (gradient descent moves y_i toward y_j)
- pair is too close in the map → repulsion
- damps the force between pairs that are already far apart
It reads like a spring system: each pair is a spring whose stiffness is . With a Gaussian map kernel instead (symmetric SNE) the factor is missing, , and vanishes so quickly that clusters cannot repel each other from a distance.
How each knob changes the math
- Perplexity
- sets every , hence how many entries of each row of are non-negligible (≈ perplexity of them)
- Early exaggeration α
- replaces by in the gradient for the first ~250 iterations; attraction dominates, so natural clusters contract into tight groups that can then move through each other freely
- Learning rate η
- step size of . Too small: the map is still compressed when you stop. Too large: points fly out into a uniform "ball". scikit-learn's "auto" uses
- Momentum & gains
- velocity accumulates past steps (0.5, then 0.8); per-coordinate gains grow by 0.2 while the gradient keeps its sign and shrink ×0.8 when it flips
- Degrees of freedom
- generalized kernel ; gives heavier tails and more separated, finer clusters; recovers the Gaussian (SNE)
Worked example: six points in 3-D
Two groups of three points, near the origin and around . We use perplexity 2 — each point should behave as if it had two neighbours — and start the map from a deliberately scrambled 2-D layout in which the groups are interleaved.
| Point | x₁ | x₂ | x₃ | Group | Y⁽⁰⁾ |
|---|---|---|---|---|---|
| 1 | 0 | 0 | 0 | A | (0, 0) |
| 2 | 1 | 0 | 0 | A | (2, 0) |
| 3 | 0 | 2 | 0 | A | (1, 1) |
| 4 | 5 | 4 | 2 | B | (1, 0) |
| 5 | 4 | 6 | 2 | B | (0, 1) |
| 6 | 6 | 6 | 4 | B | (2, 1) |
One t-SNE iteration by hand
- 1Squared distances from point 1: to 2, 3, 4, 5, 6 they are .
- 2Find . Bisection on for perplexity 2 (entropy 1 bit) stops at , i.e. . (Check: gives perplexity 2.08, gives 1.92.)
- 3Row 1 of . , , , then 0.0008 and 0.00001; their sum is 1.4835. Dividing: , , , , . Entropy bit ✓.
- 4All six bandwidths: . Point 3 gets the narrowest Gaussian because its two neighbours (distances² 4 and 5) are nearly equidistant — a narrow kernel already splits them about evenly.
- 5Symmetrize: . The full matrix is below; 99.6% of the probability mass sits on within-group pairs.
- 6Map affinities at . Squared map distances from point 1 to 2…6 are , so . Summing over all 30 ordered pairs gives , hence , . The cost is .
- 7Gradient on point 1 — one term per other point, , in the second table. Points 2 and 3 (its true neighbours, drawn too far away) pull; points 4, 5, 6 (strangers drawn too close) push. Sum: .
- 8Step with : — toward point 2 and away from points 4 and 5. Moving all six points at once lowers the cost from 1.1962 to 1.1240.
- 9Keep going. 300 iterations (momentum, gains, and ×4 exaggeration for the first 50) reach : group A ends near , group B near . Every point's nearest map neighbour equals its nearest data neighbour (1→2, 2→1, 3→1, 4→5, 5→4, 6→5).
| pᵢⱼ | 1 | 2 | 3 | 4 | 5 | 6 |
|---|---|---|---|---|---|---|
| 1 | — | 0.1019 | 0.0802 | 0.0002 | 0.0001 | 0.0000 |
| 2 | 0.1019 | — | 0.0671 | 0.0007 | 0.0001 | 0.0000 |
| 3 | 0.0802 | 0.0671 | — | 0.0005 | 0.0004 | 0.0000 |
| 4 | 0.0002 | 0.0007 | 0.0005 | — | 0.1063 | 0.0665 |
| 5 | 0.0001 | 0.0001 | 0.0004 | 0.1063 | — | 0.0760 |
| 6 | 0.0000 | 0.0000 | 0.0000 | 0.0665 | 0.0760 | — |
| j | p₁ⱼ | q₁ⱼ | w₁ⱼ | y₁ − yⱼ | 4(p−q)·w·(y₁−yⱼ) | force |
|---|---|---|---|---|---|---|
| 2 | 0.1019 | 0.0180 | 0.200 | (−2, 0) | (−0.1343, 0) | attract |
| 3 | 0.0802 | 0.0299 | 0.333 | (−1, −1) | (−0.0670, −0.0670) | attract |
| 4 | 0.0002 | 0.0449 | 0.500 | (−1, 0) | (+0.0893, 0) | repel |
| 5 | 0.0001 | 0.0449 | 0.500 | (0, −1) | (0, +0.0897) | repel |
| 6 | 0.0000 | 0.0150 | 0.167 | (−2, −1) | (+0.0200, +0.0100) | repel |
| sum | (−0.0920, +0.0327) |
There is no "prediction" step: the result is the map. What we can check is whether it is faithful — here every neighbour relation survived, and scikit-learn's trustworthiness score (below) is 1.0.
Watch it optimize
The algorithm, step by step
t-SNE(X, perplexity, η, T)
- 1Preprocess. Standardize features; for reduce to ~50 dimensions with PCA (denoises distances, speeds up neighbour search).
- 2Distances. Compute — all pairs (exact), or only the nearest neighbours (Barnes–Hut / FIt-SNE).
- 3Bandwidths. For each , bisect on until equals the perplexity; store .
- 4Symmetrize .
- 5Initialize : PCA of the data scaled to small standard deviation (modern default), or .
- 6Loop : compute and the gradient , with = exaggeration during the early phase, 1 afterwards; update gains; ; .
- 7Stop after iterations (typically 750–1000) or when the gradient norm is tiny; report .
Complexity
- Exact
- to build , then time and memory per iteration — fine up to a few thousand points
- Barnes–Hut
- sparse via a vantage-point tree in ; repulsion approximated with a quadtree, per iteration, accuracy set by angle (0.5)
- FIt-SNE / openTSNE
- repulsion by FFT-accelerated interpolation on a grid, per iteration — millions of points
- Memory
- exact: floats (70,000 points ≈ 39 GB); approximate:
Variants and practical tuning
| Setting | Typical value | Too small | Too large |
|---|---|---|---|
| perplexity | 30 (5–50); ≈ n/100 for huge n | fragmented map: many tiny clumps, spurious clusters | clusters smaller than the perplexity merge; slower |
| learning rate η | "auto" = max(n/(4α), 50) | map still compressed; clusters overlap | points scatter into a uniform ball |
| iterations | 1000 (≥ 250 after exaggeration) | clusters not separated yet | only time; KL plateaus |
| early exaggeration α | 12 for 250 iters | clusters split into pieces stuck on either side of others | very tight blobs early; rarely harmful |
| init | "pca" | — | random init loses global arrangement between runs |
| metric | Euclidean (cosine for text / embeddings) | — | — |
Variants
- SNE (2002)
- Gaussian kernel in the map and asymmetric conditional P: suffers from crowding, hard to optimize
- Barnes–Hut t-SNE
- the standard implementation (
method="barnes_hut"in scikit-learn) — 2-D or 3-D only - FIt-SNE, openTSNE
- FFT acceleration; openTSNE can also
transformnew points into an existing map - Parametric t-SNE
- trains a neural network on the t-SNE loss, so new points are embedded by a forward pass
- Multi-scale / heavy-tailed
- mix several perplexities, or use degrees of freedom to reveal finer sub-clusters
- UMAP
- a fuzzy k-NN graph with a different loss (cross-entropy, with explicit negative sampling): faster and keeps more global layout — see the UMAP article
If you need components you can interpret — a direction per source, part or factor — t-SNE is the wrong tool: its axes mean nothing. Use PCA, or the latent-factor models in the ICA, NMF and Factor Analysis articles; t-SNE is for looking at neighbourhoods.
In code
1import numpy as np23def cond_p(D, perplexity, tol=1e-5, steps=100):4 """Row-wise binary search for beta_i = 1/(2 sigma_i^2) so that 2**H(P_i) = perplexity."""5 n = len(D); P = np.zeros((n, n)); beta = np.ones(n); target = np.log(perplexity)6 for i in range(n):7 lo, hi, d = 0.0, np.inf, np.delete(D[i], i)8 for _ in range(steps):9 p = np.exp(-(d - d.min()) * beta[i]); p /= p.sum()10 H = -(p * np.log(np.maximum(p, 1e-300))).sum() # entropy in nats11 if abs(H - target) < tol: break12 if H > target: lo = beta[i]; beta[i] = beta[i] * 2 if hi == np.inf else (lo + hi) / 213 else: hi = beta[i]; beta[i] = (lo + hi) / 214 P[i, np.arange(n) != i] = p15 return P, beta1617def joint_p(X, perplexity):18 D = ((X[:, None] - X[None]) ** 2).sum(-1) # squared distances19 P, beta = cond_p(D, perplexity)20 return (P + P.T) / (2 * len(X)), beta # symmetric, sums to 12122def kl_grad(Y, P):23 d = ((Y[:, None] - Y[None]) ** 2).sum(-1)24 W = 1 / (1 + d); np.fill_diagonal(W, 0) # Student-t kernel25 Q = W / W.sum()26 G = 4 * (((P - Q) * W)[:, :, None] * (Y[:, None] - Y[None])).sum(1)27 m = P > 028 return G, (P[m] * np.log(P[m] / Q[m])).sum()2930def tsne(X, perplexity=30, n_iter=300, eta=10.0, exag=4.0, exag_iters=50, Y0=None, seed=0):31 P, _ = joint_p(X, perplexity)32 Y = np.array(Y0, float) if Y0 is not None else 1e-2 * np.random.default_rng(seed).standard_normal((len(X), 2))33 V, gains = np.zeros_like(Y), np.ones_like(Y)34 for t in range(n_iter):35 early = t < exag_iters36 G, _ = kl_grad(Y, P * (exag if early else 1.0))37 if t > 0: # delta-bar-delta gains38 gains = np.where(np.sign(G) != np.sign(V), gains + 0.2, gains * 0.8).clip(0.01)39 V = (0.5 if early else 0.8) * V - eta * gains * G40 Y = Y + V41 return Y, kl_grad(Y, P)[1]4243X = np.array([[0,0,0],[1,0,0],[0,2,0],[5,4,2],[4,6,2],[6,6,4]], float)44Y0 = np.array([[0,0],[2,0],[1,1],[1,0],[0,1],[2,1]], float)4546P, beta = joint_p(X, perplexity=2)47print("sigma:", np.round(np.sqrt(1 / (2 * beta)), 3))48G, kl0 = kl_grad(Y0, P)49print("KL at Y0:", round(kl0, 4), " gradient on point 1:", G[0].round(4))50print("point 1 after one plain step (eta=2):", (Y0[0] - 2 * G[0]).round(4))51Y, kl = tsne(X, perplexity=2, n_iter=300, eta=2.0, Y0=Y0)52print("final KL:", round(kl, 4))53print(np.round(Y, 2))54# sigma: [1.977 1.912 1.453 1.708 1.755 1.933]55# KL at Y0: 1.1962 gradient on point 1: [-0.092 0.0327]56# point 1 after one plain step (eta=2): [ 0.184 -0.0653]57# final KL: 0.0031import numpy as np2from sklearn.manifold import TSNE, trustworthiness3from sklearn.manifold._t_sne import _joint_probabilities # private helper, used only to inspect P4from scipy.spatial.distance import squareform56X = np.array([[0,0,0],[1,0,0],[0,2,0],[5,4,2],[4,6,2],[6,6,4]], float)7Y0 = np.array([[0,0],[2,0],[1,1],[1,0],[0,1],[2,1]], float)89D = ((X[:, None] - X[None]) ** 2).sum(-1)10P = squareform(_joint_probabilities(D, 2, 0))11print("p12, p13, p14:", P[0, 1].round(4), P[0, 2].round(4), P[0, 3].round(4))1213ts = TSNE(n_components=2, perplexity=2, init=Y0, learning_rate=2.0,14 method="exact", max_iter=500, random_state=0)15Y = ts.fit_transform(X)16print("KL:", round(ts.kl_divergence_, 4))17print(np.round(Y, 2))18d = np.sqrt(((Y[:, None] - Y[None]) ** 2).sum(-1))19print("nearest neighbour in the map:", d.argsort(1)[:, 1] + 1)20print("trustworthiness (k=2):", round(trustworthiness(X, Y, n_neighbors=2), 3))21# p12, p13, p14: 0.1019 0.0802 0.000222# KL: 0.002723# nearest neighbour in the map: [2 1 1 5 4 5]24# trustworthiness (k=2): 1.02526# Real data: TSNE(perplexity=30, init="pca", learning_rate="auto").fit_transform(PCA(50).fit_transform(X))Use when
- +You want to see whether high-dimensional data has cluster structure (cells, images, word or model embeddings).
- +Local neighbourhoods matter more than global geometry, and the structure is nonlinear.
- +n is up to a few hundred thousand (with Barnes–Hut / FIt-SNE) and you will look at the result, not feed it to a model.
Avoid when
- −You need distances between clusters, cluster sizes or densities to mean something.
- −You need to embed new points later, or a deterministic, reproducible projection — use PCA, parametric methods or UMAP.
- −You want features for a downstream model: t-SNE coordinates are unstable and have no out-of-sample mapping.
- −You need more than 3 output dimensions (Barnes–Hut supports at most 3), or interpretable axes.
Where it's used
Single-cell genomics
t-SNE plots of cells (after PCA to ~50 components) are the standard way to show cell types and states in scRNA-seq studies.
Inspecting neural-network embeddings
Projecting the penultimate-layer activations of a classifier shows which classes it confuses and which it separates.
Exploring image and text collections
Maps of MNIST, CIFAR or document embeddings reveal near-duplicates, sub-styles and mislabelled examples.
Cytometry and fraud review
viSNE-style maps of flow-cytometry data and maps of transaction embeddings let analysts spot rare sub-populations by eye.
Check yourself
Point i's neighbour distribution is spread evenly over 8 points (p = 1/8 each, 0 elsewhere). Its perplexity is…
Check yourself
At map distance 3, how many times larger is the Student-t similarity (1+d²)⁻¹ than the Gaussian exp(−d²)?
Check yourself
Which mistake does KL(P‖Q) punish most?
Check yourself
You fitted scikit-learn's TSNE on 10,000 images. A new batch of 500 arrives. What can you do?
Remember
- ✓t-SNE turns distances into neighbour probabilities (Gaussian, per-point ) and (Student-t in the map) and minimizes .
- ✓Perplexity is the effective number of neighbours; each is found by binary search to match it.
- ✓Gradient: — data neighbours attract, everything else repels.
- ✓The heavy-tailed kernel solves the crowding problem; KL's asymmetry makes the map faithful locally and unreliable globally.
- ✓Cluster sizes and gaps are not meaningful; there is no transform for new points; use several perplexities and PCA initialization.