Definition
HDBSCAN (Hierarchical DBSCAN, Campello, Moulavi & Sander 2013) builds the whole family of DBSCAN clusterings for every radius at once — a tree of clusters that split as the density threshold rises — then keeps the clusters that persist longest across density levels. Points that never belong to a persistent cluster are labelled noise.
At a glance
- Task
- Density-based clustering with noise detection
- Input → output
- points and a distance → a label per point ( = noise), a membership strength in , and a cluster tree
- Learns
- a minimum spanning tree on mutual-reachability distance, the condensed cluster tree, and the flat clustering that maximizes total stability
- Training
- time with dense distances; about in low dimensions with kd-tree Borůvka MST
- Prediction
- no native predict: refit, or attach new points to the tree approximately (nearest neighbour in mutual reachability)
- Hyperparameters
min_cluster_size(smallest group worth calling a cluster),min_samples(density smoothing, defaults tomin_cluster_size),cluster_selection_method,cluster_selection_epsilon- Assumes
- clusters are regions denser than their surroundings; a meaningful distance metric; no number of clusters and no single density level
The problem
DBSCAN (see the DBSCAN article) defines a cluster as a region where points have at least min_samples neighbours within radius . That one is a single density level: everything denser is cluster, everything sparser is noise. Real data rarely has one density. A city map has dense downtown blocks and sparse suburbs; a cell atlas has huge common cell types and tiny rare ones.
Take the 14 points of the worked example below: two tight squares A1 and A2 (spacing about 0.6) separated by a gap of 1.75, a loose pentagon B (spacing about 2) and one stray point. Scan DBSCAN (min_samples = 3) over every from 0.3 to 4 and something is always wrong:
| ε range | A1 (0–3) | A2 (4–7) | B (8–12) | Point 13 |
|---|---|---|---|---|
| 0.64 – 1.74 | cluster | cluster | all noise | noise |
| 1.75 – 2.00 | merged with A2 | merged with A1 | all noise | noise |
| 2.01 – 2.08 | merged | merged | 3–4 of 5 points | noise |
| 2.09 – 2.82 | merged | merged | cluster | noise |
| ≥ 2.83 | merged | merged | merged with A | noise, then joins at ≥ 3.0 |
A small separates the two squares but dissolves B into noise; a large rescues B but fuses the squares. OPTICS (see its article) orders the points so you can see all levels in a reachability plot, but you still have to cut it. HDBSCAN makes the cut automatic, and — the crucial part — the cut can be at different levels in different parts of the data.
The idea
Imagine sweeping from down to 0. At everything is one cluster. As shrinks, sparse points drop out as noise and clusters break into pieces. Write the sweep as a tree: the root is "everything", each split is a cluster breaking in two, each leaf is a cluster that finally evaporates. Every DBSCAN result is one horizontal cut through this tree.
Instead of one horizontal cut, HDBSCAN picks a set of non-overlapping branches of the tree — possibly at different heights — that together last the longest. A cluster that appears early and survives a long range of densities is real structure; a cluster that appears and immediately dissolves is noise in the hierarchy. Two tricks make this cheap and robust: a mutual-reachability distance that smooths away single noisy points, and a condensing step that ignores "splits" which only shed a handful of points.
Vocabulary
- Core distance
- : distance from to its -th nearest point, counting itself (
min_samples). Small in dense regions, large in sparse ones. - Mutual reachability distance
- . Sparse points are pushed away from everything; dense neighbours keep their true distance.
- DBSCAN*
- DBSCAN without border points: at radius , the clusters are the connected groups of core points (points with ); everything else is noise. HDBSCAN builds DBSCAN* for every .
- Minimum spanning tree (MST)
- the edges of smallest total that connect all points. Cutting it at gives DBSCAN* at .
- Single-linkage tree
- the dendrogram from adding MST edges in increasing order; its merge heights are ε values
- λ (lambda)
- density level . A cluster is born at the λ where it splits off its parent and dies at the λ where it splits again or evaporates.
- Falling out
- at a split where one side has fewer than
min_cluster_sizepoints, those points leave the cluster (they become noise at that level) and the cluster carries on — this is not a real split. - Condensed tree
- the single-linkage tree with all fall-outs collapsed: only splits into two pieces of at least
min_cluster_sizepoints remain - Stability
- — how much density-level "lifetime" the cluster's points accumulate; the area of its bar in the condensed-tree plot
- Excess of mass (EOM)
- the selection rule: keep a cluster if it is at least as stable as its selected descendants combined, otherwise keep the descendants
- Membership strength
- : 1 for points that stay to the end, smaller for points that fall out early; noise gets 0
The math
Step 1 — a density estimate per point. The core distance is the radius of the smallest ball around that holds points:
- the -th nearest point to in the data, itself counted as the first
- min_samples
- the -nearest-neighbour density estimate at
- volume of the unit ball in dimensions
So "" is the same statement as " a threshold": shrinking means raising the density level. Larger averages over more neighbours — a smoother, more conservative density estimate in which isolated points get large core distances and more of them become noise.
Step 2 — mutual reachability. Define a new distance between points:
- the original distance (Euclidean here)
- the smallest at which and are both core points and within of each other
That legend line is the whole derivation of why this works. In DBSCAN at radius , two points are directly linked iff both are core () and — i.e. iff . So DBSCAN at = connected components of the graph whose edges have ** (points with have no edges and are noise).
Step 3 — all ε at once with one MST. Two points are connected through edges of weight in the full graph iff they are connected through such edges in its minimum spanning tree — this is the cut property of MSTs: the MST path between two points minimizes the largest edge on the path. So we compute the MST once ( edges) and read off DBSCAN* for every by deleting the edges heavier than . Adding the edges from light to heavy gives the single-linkage dendrogram of .
Step 4 — condense. Re-label heights by density, , and walk down the dendrogram from the root. At each merge (now read as a split, at level ) of a cluster into pieces and :
- min_cluster_size
Each point gets , the level at which it leaves the cluster it is in (by falling out, or because that cluster split or died). The condensed tree typically has far fewer nodes than the dendrogram — in the example, 13 merges condense to 5 clusters.
Step 5 — stability. For a density and level , the excess of mass of a cluster is how much probability mass it holds above that level. Integrating the size of the cluster over the levels where it exists gives its stability:
- number of points still in at level
- level at which split off its parent
- level at which point leaves
The second form follows by swapping sum and integral: each point contributes 1 to for every between the birth and its own departure, so it contributes exactly . Geometrically, is the area of the cluster's bar in the condensed-tree plot.
Step 6 — select. We want a set of clusters, no one an ancestor of another, with maximum total stability. On a tree this is solved exactly by one bottom-up pass:
- best total stability achievable inside the subtree of
- the two clusters born when C dies (none for a leaf)
If wins, is selected and everything below it is discarded; otherwise its children's choices stand. Leaves are always provisionally selected. The root is excluded by default (otherwise a single blob would always win), so data with no structure comes back as all noise.
Each hyperparameter, mathematically
- min_samples = k
- changes , hence every . Larger raises core distances most for isolated points, so they join later (lower ) and fall out sooner — more noise, smoother tree. gives and plain single linkage.
- min_cluster_size = m
- decides which splits are real. Pieces smaller than only fall out, so raising removes small clusters from the condensed tree entirely and can merge siblings into their parent.
- cluster_selection_epsilon
- clusters born at a distance below this value () are not split further — their ancestor alive at is reported. Prevents micro-clusters in very dense regions.
- cluster_selection_method
- "eom" maximizes total stability (above); "leaf" simply returns the leaves of the condensed tree — finer, more homogeneous clusters.
- alpha
- rescales distances to inside ; rarely needed.
Worked example: 14 points, three densities
Use min_samples (so the core distance is the distance to the 2nd nearest other point) and min_cluster_size . Points are numbered from 0, as in the code.
| Point | x | y | Group | 2nd nearest other (distance) | core₃ |
|---|---|---|---|---|---|
| 0 | 1.0 | 1.0 | A1 | 2 (0.707) | 0.707 |
| 1 | 1.6 | 1.1 | A1 | 0 (0.608) | 0.608 |
| 2 | 1.1 | 1.7 | A1 | 0 (0.707) | 0.707 |
| 3 | 1.7 | 1.6 | A1 | 2 (0.608) | 0.608 |
| 4 | 3.4 | 1.2 | A2 | 6 (0.632) | 0.632 |
| 5 | 4.0 | 1.0 | A2 | 4 (0.632) | 0.632 |
| 6 | 3.6 | 1.8 | A2 | 4 (0.632) | 0.632 |
| 7 | 4.1 | 1.6 | A2 | 5 (0.608) | 0.608 |
| 8 | 6.8 | 0.6 | B | 10 (2.408) | 2.408 |
| 9 | 8.8 | 1.2 | B | 8 (2.088) | 2.088 |
| 10 | 7.0 | 3.0 | B | 12 (2.236) | 2.236 |
| 11 | 9.0 | 3.2 | B | 10 (2.010) | 2.010 |
| 12 | 8.0 | 5.0 | B | 10 (2.236) | 2.236 |
| 13 | 4.6 | 4.8 | stray | 6 (3.162) | 3.162 |
Core distances → mutual reachability → MST
- 1Core distances (table above). Example: point 8's nearest others are 9 at 2.088 and 10 at 2.408, so .
- 2Mutual reachability. Inside a square nothing changes much: but both cores are 0.608, so . In B the cores dominate: , but .
- 3MST. Prim's algorithm on gives 13 edges (next table). The heavy ones are the bridges: 3–4 at 1.746 (A1 to A2), 5–8 at 2.828 (A to B) and 6–13 at 3.162 (the stray point).
- 4Dendrogram. Adding the edges from light to heavy: A1 is complete at , A2 at 0.632, A1+A2 merge at 1.746, B is complete at 2.408, A meets B at 2.828, and point 13 joins last at 3.162.
| Edge | d | core(a), core(b) | d_mr | λ = 1/d_mr | Effect |
|---|---|---|---|---|---|
| 1–3 | 0.510 | 0.608, 0.608 | 0.608 | 1.644 | pair in A1 |
| 5–7 | 0.608 | 0.632, 0.608 | 0.632 | 1.581 | pair in A2 |
| 4–5 | 0.632 | 0.632, 0.632 | 0.632 | 1.581 | A2 grows |
| 4–6 | 0.632 | 0.632, 0.632 | 0.632 | 1.581 | A2 complete (4 pts) |
| 0–1 | 0.608 | 0.707, 0.608 | 0.707 | 1.414 | A1 grows |
| 0–2 | 0.707 | 0.707, 0.707 | 0.707 | 1.414 | A1 complete (4 pts) |
| 3–4 | 1.746 | 0.608, 0.632 | 1.746 | 0.573 | A1 + A2 → A (8 pts) |
| 9–11 | 2.010 | 2.088, 2.010 | 2.088 | 0.479 | pair in B |
| 11–10 | 2.010 | 2.010, 2.236 | 2.236 | 0.447 | B grows |
| 11–12 | 2.059 | 2.010, 2.236 | 2.236 | 0.447 | B grows (4 pts) |
| 8–9 | 2.088 | 2.408, 2.088 | 2.408 | 0.415 | B complete (5 pts) |
| 5–8 | 2.828 | 0.632, 2.408 | 2.828 | 0.354 | A + B (13 pts) |
| 6–13 | 3.162 | 0.632, 3.162 | 3.162 | 0.316 | stray point joins |
Condense, score, select
- 1Root (C0), from . The top split at separates point 13 (1 point ) from 13 others: point 13 falls out; the root carries on.
- 2 (edge 5–8): pieces of 8 and 5 points, both — a true split. C1 = A (points 0–7) and C4 = B (points 8–12) are born at .
- 3C1 at (edge 3–4): pieces 4 and 4 — a true split. C2 = A1 and C3 = A2 are born at 0.573. C1's 8 points all leave at 0.573, so .
- 4C2 (A1): the next merges inside it (0.707) cut it into pieces smaller than 3, so all four points leave at : . Likewise C3 (A2) ends at 1.581: .
- 5C4 (B): point 8 falls out at (edge 8–9); at 0.447 the rest breaks into pieces under 3 and the cluster dies. .
- 6Excess of mass, bottom-up. Leaves C2, C3, C4 are provisionally kept. At C1: , so the two squares beat their union. The root is never selected. Result: A1 = {0–3}, A2 = {4–7}, B = {8–12}, point 13 = noise.
| Cluster | Points | Born at λ | Points leave at λ | Stability S | Selected |
|---|---|---|---|---|---|
| C0 (root) | 0–13 | 0 | 13 at 0.316; rest at 0.354 | — | never |
| C1 = A | 0–7 | 0.354 | all at 0.573 (split) | 1.752 | no (1.752 < 7.40) |
| C2 = A1 | 0–3 | 0.573 | all at 1.414 | 3.366 | yes |
| C3 = A2 | 4–7 | 0.573 | all at 1.581 | 4.034 | yes |
| C4 = B | 8–12 | 0.354 | 8 at 0.415; rest at 0.447 | 0.436 | yes |
Membership strength is within the selected cluster: every point of A1 and A2 leaves at the cluster's last level, so all have strength 1; in B, point 8 left early, ; point 13 is noise with 0.
Placing a new point q = (5.5, 1.5)
- 1Distances from q: point 7 at 1.404, then points 5 and 8 at 1.581. Counting q itself, its core distance is the 2nd nearest data point: .
- 2Mutual reachability to point 7: ; to point 8 it is . The nearest in is point 7 at 1.581, so q attaches at .
- 3Point 7 is in C3 (A2), which is alive from to 1.581. Since , q lands inside A2, with strength — a member, but a weak one. (Raw distance alone would also say "near A2", but if q were halfway to B with it would land in C1, which is not selected, and be noise.)
- 1core(x) ← dist to min_samples-th nearest
- 2d_mr(a,b) ← max(core a, core b, d(a,b))
- 3T ← minimum spanning tree of d_mr
- 4add T's edges by weight → hierarchy
- 5condense: λ = 1/ε; drop pieces < mcs
- 6S(C) ← Σ_{x∈C} (λ_x − λ_birth(C))
- 7keep C if S(C) ≥ Σ S(children)
- 8label points; the rest are noise
- 9new q → MR-nearest neighbour at λ_q
14 points: two tight squares on the left, a loose pentagon on the right, one stray point. min_cluster_size = 3, min_samples = 3.
The algorithm, step by step
HDBSCAN(X, m = min_cluster_size, k = min_samples)
- 1Core distances. For each point, the distance to its -th nearest neighbour (itself included). A kd-tree or ball tree makes this in low dimensions; brute force is .
- 2Mutual-reachability MST. Build the minimum spanning tree of . Prim on the dense matrix is time and memory; dual-tree Borůvka avoids materializing the matrix and runs in roughly for low-dimensional Euclidean data.
- 3Single-linkage tree. Sort the edges and merge with union–find: .
- 4Condense. Walk the tree top-down in ; splits with a side smaller than
min_cluster_sizebecome fall-outs. Record each point's leaving : . - 5Stability. for every condensed cluster: .
- 6Select. Bottom-up: keep if of its children's best, else propagate the children's sum. Discard descendants of kept clusters. Exclude the root unless
allow_single_cluster. - 7Label. Every point in a kept cluster's subtree gets that label and strength ; all others are noise ().
The expensive part is the MST; everything after it is linear in . Memory is the practical limit of the naive version (about 8 GB of float64 distances at ); tree-based implementations handle millions of low-dimensional points. In high dimensions the trees stop helping and run time drifts back toward .
Variants and practical tuning
| min_cluster_size | min_samples | Clusters found | Noise |
|---|---|---|---|
| 2 or 3 | 2 or 3 | A1, A2, B | 13 |
| 2 or 3 | 4 | A1, A2, B without 8 and 12 | 8, 12, 13 |
| 4 | 4 | A1, A2 | all of B, 13 |
| 5 | 2 or 3 | A (= A1 ∪ A2), B | 13 |
| 5 | 4 | none | everything |
| 6 | 3 | none | everything |
Tuning guide
- Start with min_cluster_size
- set it to the smallest group you would act on (e.g. 15 customers, 50 cells). It is the most interpretable knob and usually the only one you need.
- Then min_samples
- defaults to min_cluster_size. Lower it (e.g. 1–5) if too many points are noise; raise it to be more conservative in noisy data. It changes the distances themselves, so results can shift a lot.
- cluster_selection_epsilon
- set it to a distance below which you do not care about splits. On the example, any value ≥ 1.75 (the A1–A2 merge distance) reports A as one cluster; 1.7 or less keeps the squares apart.
- "eom" vs "leaf"
- EOM favours a few large stable clusters (sometimes one giant cluster plus small ones); leaf gives many small, homogeneous clusters. Both give A1, A2, B here.
- Soft clustering
- use
probabilities_to downweight weak members; thehdbscanpackage also offers all-points membership vectors and GLOSH outlier scores (here 0.80 for the stray point, 0.07 for point 8). - Scaling
- results are invariant to rescaling all features by one factor (λ just rescales), but not to rescaling one feature: standardize features first, as for any distance-based method.
In code
1import numpy as np23def hdbscan(X, min_cluster_size=3, min_samples=3):4 n = len(X)5 D = np.sqrt(((X[:, None] - X[None]) ** 2).sum(-1))6 core = np.sort(D, axis=1)[:, min_samples - 1] # k-th nearest, the point itself included7 M = np.maximum(D, np.maximum(core[:, None], core[None])) # mutual reachability distance89 # 1. minimum spanning tree of M (Prim, O(n^2))10 seen, best, src, edges = np.zeros(n, bool), M[0].copy(), np.zeros(n, int), []11 seen[0] = True12 for _ in range(n - 1):13 j = np.where(seen, np.inf, best).argmin()14 edges.append((best[j], src[j], j)); seen[j] = True15 upd = ~seen & (M[j] < best); best[upd], src[upd] = M[j][upd], j16 edges.sort()1718 # 2. single-linkage tree: node = (left, right, height, points)19 nodes, owner = [(None, None, 0.0, [i]) for i in range(n)], list(range(n))20 for w, a, b in edges:21 A, B = owner[a], owner[b]22 nodes.append((A, B, w, nodes[A][3] + nodes[B][3]))23 for p in nodes[-1][3]: owner[p] = len(nodes) - 12425 # 3. condensed tree: walk down in lambda = 1/distance26 clusters = [] # dicts: birth, leave {point: lambda}, children27 def new(birth):28 clusters.append(dict(birth=birth, leave={}, children=[])); return len(clusters) - 129 def walk(node, c):30 left, right, h, pts = nodes[node]31 if left is None: return32 lam, kids = 1 / h, (left, right)33 big = [k for k in kids if len(nodes[k][3]) >= min_cluster_size]34 if len(big) == 2: # a true split: two clusters are born35 for p in pts: clusters[c]["leave"][p] = lam36 for k in kids:37 child = new(lam); clusters[c]["children"].append(child); walk(k, child)38 return39 for k in kids: # small pieces fall out of c40 if len(nodes[k][3]) < min_cluster_size:41 for p in nodes[k][3]: clusters[c]["leave"][p] = lam42 if big: walk(big[0], c) # c lives on as the big piece43 walk(len(nodes) - 1, new(0.0))4445 # 4. stability and excess-of-mass selection (bottom-up; the root never counts)46 S = [sum(l - c["birth"] for l in c["leave"].values()) for c in clusters]47 best_S, keep = S[:], [False] * len(clusters)48 for k in reversed(range(len(clusters))):49 kids = clusters[k]["children"]50 below = sum(best_S[ch] for ch in kids)51 if k > 0 and (not kids or S[k] >= below):52 keep[k] = True53 else:54 best_S[k] = below55 def drop(k):56 for ch in clusters[k]["children"]: keep[ch] = False; drop(ch)57 for k in range(len(clusters)):58 if keep[k]: drop(k)5960 labels = -np.ones(n, int)61 def points(k):62 return set(clusters[k]["leave"]) | set().union(*[points(ch) for ch in clusters[k]["children"]])63 for lab, k in enumerate(k for k in range(len(clusters)) if keep[k]):64 labels[list(points(k))] = lab65 return labels, core, edges, clusters, S, keep6667X = np.array([[1.0, 1.0], [1.6, 1.1], [1.1, 1.7], [1.7, 1.6], # A1: tight square68 [3.4, 1.2], [4.0, 1.0], [3.6, 1.8], [4.1, 1.6], # A2: tight square69 [6.8, 0.6], [8.8, 1.2], [7.0, 3.0], [9.0, 3.2], [8.0, 5.0], # B: loose70 [4.6, 4.8]]) # a stray point71labels, core, edges, clusters, S, keep = hdbscan(X)72print("core ", core.round(3))73print("MST ", [(int(a), int(b), round(float(w), 3)) for w, a, b in edges])74for k, c in enumerate(clusters):75 print(f"C{k}: birth lam={c['birth']:.3f} size={len(c['leave'])} S={S[k]:.3f} keep={keep[k]}")76print("labels ", labels)7778# core [0.707 0.608 0.707 0.608 0.632 0.632 0.632 0.608 2.408 2.088 2.236 2.0179# 2.236 3.162]80# MST [(1, 3, 0.608), (5, 7, 0.632), (4, 5, 0.632), (4, 6, 0.632), (0, 1, 0.707), (0, 2, 0.707),81# (3, 4, 1.746), (9, 11, 2.088), (11, 10, 2.236), (11, 12, 2.236), (8, 9, 2.408),82# (5, 8, 2.828), (6, 13, 3.162)]83# C0: birth lam=0.000 size=14 S=4.912 keep=False84# C1: birth lam=0.354 size=8 S=1.752 keep=False85# C2: birth lam=0.573 size=4 S=3.366 keep=True86# C3: birth lam=0.573 size=4 S=4.034 keep=True87# C4: birth lam=0.354 size=5 S=0.436 keep=True88# labels [ 0 0 0 0 1 1 1 1 2 2 2 2 2 -1]1import numpy as np2from sklearn.cluster import HDBSCAN, DBSCAN34X = np.array([[1.0, 1.0], [1.6, 1.1], [1.1, 1.7], [1.7, 1.6],5 [3.4, 1.2], [4.0, 1.0], [3.6, 1.8], [4.1, 1.6],6 [6.8, 0.6], [8.8, 1.2], [7.0, 3.0], [9.0, 3.2], [8.0, 5.0],7 [4.6, 4.8]])89hdb = HDBSCAN(min_cluster_size=3, min_samples=3).fit(X)10print("labels ", hdb.labels_)11print("probabilities", hdb.probabilities_.round(3))1213# No single DBSCAN eps recovers all three groups:14for eps in [1.0, 1.7, 2.05, 2.5]:15 print(f"DBSCAN eps={eps}:", DBSCAN(eps=eps, min_samples=3).fit(X).labels_)1617# Variants18print("leaf ", HDBSCAN(min_cluster_size=3, min_samples=3, cluster_selection_method="leaf").fit(X).labels_)19print("eps=2.0 ", HDBSCAN(min_cluster_size=3, min_samples=3, cluster_selection_epsilon=2.0).fit(X).labels_)20print("mcs=5 ", HDBSCAN(min_cluster_size=5, min_samples=3).fit(X).labels_)2122# labels [ 1 1 1 1 2 2 2 2 0 0 0 0 0 -1]23# probabilities [1. 1. 1. 1. 1. 1. 1. 1. 0.928 1. 1. 1.24# 1. 0. ]25# DBSCAN eps=1.0: [ 0 0 0 0 1 1 1 1 -1 -1 -1 -1 -1 -1]26# DBSCAN eps=1.7: [ 0 0 0 0 1 1 1 1 -1 -1 -1 -1 -1 -1]27# DBSCAN eps=2.05: [ 0 0 0 0 0 0 0 0 -1 1 1 1 -1 -1]28# DBSCAN eps=2.5: [ 0 0 0 0 0 0 0 0 1 1 1 1 1 -1]29# leaf [ 1 1 1 1 2 2 2 2 0 0 0 0 0 -1]30# eps=2.0 [ 0 0 0 0 0 0 0 0 1 1 1 1 1 -1]31# mcs=5 [ 0 0 0 0 0 0 0 0 1 1 1 1 1 -1]Use when
- +Clusters have different densities, and you do not know how many there are.
- +The data contains noise or background points that should not be forced into any cluster.
- +You want one main knob with a plain meaning (smallest cluster size) instead of a fragile radius ε.
- +You want a cluster hierarchy and per-point membership strengths, not just flat labels.
Avoid when
- −Data is high-dimensional (hundreds of raw features): distances concentrate, density is meaningless and the tree speed-ups vanish — reduce dimension first (e.g. UMAP or PCA).
- −Every point must get a cluster: HDBSCAN routinely leaves a sizeable share of real data as noise.
- −Clusters are not density bumps (overlapping Gaussians of equal density): a Gaussian mixture is the better model.
- −You need a cheap, stable predict() for streaming data: the model is a tree over the training set, not a function.
Where it's used
Embedding clustering (topic modelling)
BERTopic runs UMAP on sentence embeddings and then HDBSCAN to find topics, with noise documents left unassigned.
Single-cell biology
Finding cell types of very different abundance in UMAP-reduced expression data, where a rare type may have a few dozen cells.
Geospatial hotspots
Crime, taxi pick-ups or GPS stay-points: dense downtown hotspots and sparse rural ones in a single run, with haversine distance.
Astronomy and anomaly screening
Finding star clusters and stellar streams in Gaia data; GLOSH outlier scores flag points that leave their cluster unusually early.
Check yourself
With min_samples = 3, point a has core distance 0.5, point b has core distance 2.0, and d(a, b) = 1.2. What is d_mr(a, b)?
Check yourself
A cluster is born at λ = 0.5. Of its 6 points, 2 fall out at λ = 0.7 and the remaining 4 leave at λ = 1.0 when it dies. Its stability is…
Check yourself
In the worked example, why does HDBSCAN report A1 and A2 separately instead of their union A?
Check yourself
You raise min_cluster_size from 3 to 5 on the worked example (min_samples = 3). What happens?
Remember
- ✓Core distance = distance to the -th nearest point (a density estimate); mutual reachability pushes sparse points apart.
- ✓The MST of encodes DBSCAN* for every at once: cut it at to get that level's clusters.
- ✓Condensing walks the tree in λ = 1/ε and treats splits that shed fewer than min_cluster_size points as points falling out, not new clusters.
- ✓Stability ; excess of mass keeps a cluster only if it beats its descendants, so the cut can sit at different densities in different branches.
- ✓Start tuning with min_cluster_size; min_samples controls how conservative (noisy) the result is. Noise (−1) and membership strengths are part of the answer.