One-class SVM

Separate the data from the origin in kernel space with maximum margin; points that fall on the origin's side are outliers.

17 min read1882 wordsadvanced0 viewsupdated 2026-10-08

Definition

A one-class SVM is an unsupervised anomaly detector. It learns a boundary around the training data from unlabelled examples. In a kernel feature space it finds the hyperplane that pushes the data as far as possible from the origin, while letting a fraction of at most ν\nu of the points fall on the wrong side. A new point on the data's side is normal; a point on the origin's side is an outlier. Schölkopf et al. introduced it in 2001.

At a glance

Task
Novelty / outlier detection, also called one-class classification
Input → output
x∈Rdx\in\mathbb{R}^d → score f(x)=∑iαik(xi,x)−ρf(x)=\sum_i\alpha_i k(x_i,x)-\rho; normal if f(x)≥0f(x)\ge0, outlier if f(x)<0f(x)<0
Learns
one dual weight αi∈[0,1νn]\alpha_i\in[0,\tfrac{1}{\nu n}] per training point (nonzero only for support vectors) and an offset ρ\rho
Objective
maximize the margin ρ/∥w∥\rho/\lVert w\rVert between the data and the origin, plus a hinge-type penalty on points that fall short
Training
a convex quadratic program, so there is one global optimum; SMO (libsvm) takes roughly O(n2)O(n^2)–O(n3)O(n^3) time and O(n2)O(n^2) kernel memory
Prediction
O(nSV d)O(n_{\text{SV}}\,d) kernel evaluations per point
Hyperparameters
ν∈(0,1]\nu\in(0,1] (outlier budget) and the kernel, almost always RBF with width γ\gamma
Assumes
the training set is (mostly) normal data; features are on comparable scales

The problem

A fraud team has millions of normal card transactions and a handful of known frauds. A factory has months of sensor logs from a healthy machine and none from a failing one. In both cases a classifier has nothing to train on for the second class, and the anomalies of tomorrow may not look like those of yesterday anyway.

What we can do is describe what normal looks like: draw a tight boundary around the training data and flag anything that falls outside it. The boundary should be flexible enough to follow curved or multi-modal data. It should also tolerate a few odd points in the training set, so that one bad reading does not stretch the boundary to swallow everything.

The idea

A two-class Support Vector Machine puts the widest possible street between two classes. A one-class SVM has only one class, so it invents the other: the origin of feature space plays the part of the "negative class". The algorithm looks for a hyperplane that keeps the data on one side and the origin on the other, as far from the origin as possible. The region on the data's side is the learned "normal" region.

Separating a cloud of points from the origin does not sound useful in the input space; a cloud centred near the origin cannot be separated at all. With an RBF kernel the picture changes completely. Every point is mapped to a vector of length 1 (k(x,x)=1k(x,x)=1), and every pair has a positive dot product (k(x,x′)>0k(x,x')>0). All the data sits on a patch of the unit sphere, in one orthant, well away from the origin. A hyperplane slicing that patch cuts off a spherical cap. Points in dense regions have large kernel similarities to many others and land inside the cap. Isolated points land outside it. Back in input space, the cap is a closed, curved boundary around the dense parts of the data.

Vocabulary

Feature map φ
the (implicit) map into the kernel's feature space, with k(x,x′)=ϕ(x)⊤ϕ(x′)k(x,x')=\phi(x)^\top\phi(x')
Hyperplane
{z:w⊤z=ρ}\{z: w^\top z=\rho\} in feature space; the data should lie on the side w⊤ϕ(x)≥ρw^\top\phi(x)\ge\rho
Offset ρ
how far the hyperplane is from the origin, in units of ww; its distance is ρ/∥w∥\rho/\lVert w\rVert
Slack ξᵢ
how far point ii falls short of the hyperplane: ξi=max⁡(0,ρ−w⊤ϕ(xi))\xi_i=\max(0,\rho-w^\top\phi(x_i))
ν (nu)
the outlier budget: an upper bound on the fraction of training outliers and a lower bound on the fraction of support vectors
Support vector
a training point with αi>0\alpha_i>0: either exactly on the boundary or a training outlier
Decision value
f(x)=w⊤ϕ(x)−ρ=∑iαik(xi,x)−ρf(x)=w^\top\phi(x)-\rho=\sum_i\alpha_ik(x_i,x)-\rho; its sign is the verdict

The math: the primal problem

Write the hyperplane in feature space as w⊤ϕ(x)=ρw^\top\phi(x)=\rho. The distance from the origin to it is ρ/∥w∥\rho/\lVert w\rVert. We want that distance to be large: large ρ\rho and small ∥w∥\lVert w\rVert. Points that land on the origin's side are allowed, but each one pays a slack ξi\xi_i. Schölkopf's formulation balances the three goals in one objective:

min⁡w, ξ, ρ  12∥w∥2+1νn∑i=1nξi−ρs.t.w⊤ϕ(xi)≥ρ−ξi,  ξi≥0\min_{w,\,\xi,\,\rho}\ \ \tfrac12\lVert w\rVert^2+\frac{1}{\nu n}\sum_{i=1}^n\xi_i-\rho\qquad\text{s.t.}\quad w^\top\phi(x_i)\ge\rho-\xi_i,\ \ \xi_i\ge0
ww
normal vector of the hyperplane in feature space
ρ\rho
offset; the hyperplane sits at distance ρ/‖w‖ from the origin
ξi\xi_i
slack of point i: how far it falls on the origin's side
ν\nu
outlier budget in (0, 1]; small ν = expensive slack
nn
number of training points
ϕ(xi)\phi(x_i)
feature-space image of training point i

Read the objective term by term. 12∥w∥2\tfrac12\lVert w\rVert^2 is the usual SVM regularizer. −ρ-\rho rewards pushing the hyperplane away from the origin. Each unit of slack costs 1νn\tfrac{1}{\nu n}. Raising ρ\rho by δ\delta gains δ\delta in the objective, but every point currently on or beyond the hyperplane gains δ\delta slack, which costs δ/(νn)\delta/(\nu n) each. So ρ\rho keeps rising until about νn\nu n points are on the wrong side or on the boundary, and that trade is where ν\nu gets its meaning.

From primal to dual

Attach multipliers αi≥0\alpha_i\ge0 to the margin constraints and βi≥0\beta_i\ge0 to ξi≥0\xi_i\ge0:

L=12∥w∥2+1νn∑iξi−ρ−∑iαi(w⊤ϕ(xi)−ρ+ξi)−∑iβiξi\mathcal{L}=\tfrac12\lVert w\rVert^2+\frac{1}{\nu n}\sum_i\xi_i-\rho-\sum_i\alpha_i\big(w^\top\phi(x_i)-\rho+\xi_i\big)-\sum_i\beta_i\xi_i
αi\alpha_i
multiplier of point i's margin constraint (the dual weight)
βi\beta_i
multiplier of ξᵢ ≥ 0

Set the derivatives with respect to the primal variables to zero

  1. 1∂L/∂w=w−∑iαiϕ(xi)=0\partial\mathcal{L}/\partial w=w-\sum_i\alpha_i\phi(x_i)=0, so w=∑iαiϕ(xi)w=\sum_i\alpha_i\phi(x_i). The normal vector is a weighted sum of training points.
  2. 2∂L/∂ξi=1νn−αi−βi=0\partial\mathcal{L}/\partial\xi_i=\tfrac{1}{\nu n}-\alpha_i-\beta_i=0. Since βi≥0\beta_i\ge0, this gives the box 0≤αi≤1νn0\le\alpha_i\le\tfrac{1}{\nu n}.
  3. 3∂L/∂ρ=−1+∑iαi=0\partial\mathcal{L}/\partial\rho=-1+\sum_i\alpha_i=0, so ∑iαi=1\sum_i\alpha_i=1.
  4. 4Substitute back. The ξ\xi and ρ\rho terms cancel, and 12∥w∥2−∑iαiw⊤ϕ(xi)=−12∑ijαiαjϕ(xi)⊤ϕ(xj)\tfrac12\lVert w\rVert^2-\sum_i\alpha_iw^\top\phi(x_i)=-\tfrac12\sum_{ij}\alpha_i\alpha_j\phi(x_i)^\top\phi(x_j). Replace each dot product by the kernel.
min⁡α 12∑i,jαiαj k(xi,xj)=12 α⊤Kαs.t.0≤αi≤1νn,  ∑iαi=1\min_{\alpha}\ \tfrac12\sum_{i,j}\alpha_i\alpha_j\,k(x_i,x_j)=\tfrac12\,\alpha^\top K\alpha\qquad\text{s.t.}\quad 0\le\alpha_i\le\frac{1}{\nu n},\ \ \sum_i\alpha_i=1
KK
n × n Gram matrix, Kᵢⱼ = k(xᵢ, xⱼ)
1νn\tfrac{1}{\nu n}
upper bound on each αᵢ (written C below)

This is a small quadratic program: minimize a convex quadratic over a simplex cut off by a box. Note that there are no labels, and the data enters only through the kernel. The weights form a probability distribution over the training points (∑α=1\sum\alpha=1), and no single point may carry more than C=1νnC=\tfrac{1}{\nu n} of it.

Recovering ρ. For a point with 0<αi<C0<\alpha_i<C we have βi>0\beta_i>0, so ξi=0\xi_i=0; and αi>0\alpha_i>0 makes its margin constraint tight. Such a point lies exactly on the hyperplane, which gives ρ=w⊤ϕ(xi)=∑jαjk(xj,xi)=(Kα)i\rho=w^\top\phi(x_i)=\sum_j\alpha_jk(x_j,x_i)=(K\alpha)_i. In practice we average over all these "free" support vectors. Substituting ww into w⊤ϕ(x)−ρw^\top\phi(x)-\rho gives the decision function:

f(x)=∑i=1nαi k(xi,x)−ρ{f(x)≥0normalf(x)<0outlierf(x)=\sum_{i=1}^n\alpha_i\,k(x_i,x)-\rho\qquad\begin{cases}f(x)\ge0&\text{normal}\\f(x)<0&\text{outlier}\end{cases}
k(xi,x)k(x_i,x)
kernel similarity between training point i and the query x
∑iαik(xi,x)\sum_i\alpha_ik(x_i,x)
a weighted kernel density of the support vectors at x

Three kinds of training points

The KKT conditions (αi [(Kα)i−ρ+ξi]=0\alpha_i\,[(K\alpha)_i-\rho+\xi_i]=0 and βiξi=0\beta_i\xi_i=0) sort every training point into one of three groups:

GroupDual weightPositionScore
Insideαi=0\alpha_i=0strictly on the data's side(Kα)i≥ρ(K\alpha)_i\ge\rho, f≥0f\ge0
On the boundary (free SV)0<αi<C0<\alpha_i<Cexactly on the hyperplane(Kα)i=ρ(K\alpha)_i=\rho, f=0f=0
Outlier (bounded SV)αi=C\alpha_i=Con the origin's side (or on it)(Kα)i≤ρ(K\alpha)_i\le\rho, ξi=ρ−(Kα)i\xi_i=\rho-(K\alpha)_i
The KKT conditions at the optimum. Only the last two groups are support vectors.

The ν-property

The box and the sum constraint together give ν\nu a precise meaning, which needs only two lines of arithmetic:

Why ν bounds both counts

  1. 1Outliers. Every training outlier has ξi>0\xi_i>0, so βi=0\beta_i=0 and αi=C=1νn\alpha_i=C=\tfrac{1}{\nu n}. With mm outliers, ∑α≥m⋅1νn\sum\alpha\ge m\cdot\tfrac{1}{\nu n}. Since ∑α=1\sum\alpha=1, m≤νnm\le\nu n: at most a fraction ν of the training points are outliers.
  2. 2Support vectors. Every αi\alpha_i is at most 1νn\tfrac{1}{\nu n}, and together they must add to 1. That takes at least νn\nu n nonzero weights: at least a fraction ν of the points are support vectors.
#{outliers}n ≤ ν ≤ #{support vectors}n\frac{\#\{\text{outliers}\}}{n}\ \le\ \nu\ \le\ \frac{\#\{\text{support vectors}\}}{n}
ν\nu
set by you; both bounds become equalities as n → ∞ (under mild conditions on the data distribution)

This is why ν\nu is easy to set: it is the outlier budget. If you expect about 2% contamination in your training data, start with ν=0.02\nu=0.02.

The RBF kernel and γ

k(x,x′)=exp⁡ ⁣(−γ ∥x−x′∥2)k(x,x')=\exp\!\big(-\gamma\,\lVert x-x'\rVert^2\big)
γ\gamma
inverse squared width; the similarity falls to e⁻¹ at distance 1/√γ
∥x−x′∥2\lVert x-x'\rVert^2
squared Euclidean distance in input space

γ\gamma sets how far one training point's influence reaches. With small γ\gamma every point is similar to every other. The density ∑αik(xi,x)\sum\alpha_ik(x_i,x) is a single smooth bump, and the boundary is a near-ellipse around the bulk of the data. With large γ\gamma similarities die out within a short distance. Each point becomes its own island, so even an isolated point gets a tiny bubble of "normal" around itself. The model then memorizes the training set, and almost every point becomes a support vector. scikit-learn's default gamma="scale" uses γ=1/(d⋅Var⁡(X))\gamma=1/(d\cdot\operatorname{Var}(X)), which ties the width to the spread of the (scaled) data.

Worked example: ten sensor readings

Ten two-dimensional readings from a machine. Eight form a tight cluster around (3,3)(3,3); readings 9 and 10 are far off. We fit with ν=0.4\nu=0.4 and γ=0.25\gamma=0.25, so the box is C=1νn=10.4⋅10=0.25C=\tfrac{1}{\nu n}=\tfrac{1}{0.4\cdot10}=0.25. At most 4 points can be outliers, and at least 4 must be support vectors. Every number below was checked against scikit-learn.

ix₁x₂
12.52.5
23.03.0
33.52.5
42.53.5
53.53.5
63.02.0
72.03.0
83.04.0
90.55.0
105.51.0
Training data. Highlighted: the two readings that will end up as outliers.

Step 1: the kernel matrix

  1. 1Squared distances. Neighbours on the grid are 0.50.5 apart squared (∥x1−x2∥2=0.52+0.52=0.5\lVert x_1-x_2\rVert^2=0.5^2+0.5^2=0.5), axis steps of 1 give 11, and the far points are much further: ∥x9−x2∥2=2.52+22=10.25\lVert x_9-x_2\rVert^2=2.5^2+2^2=10.25, ∥x9−x10∥2=52+42=41\lVert x_9-x_{10}\rVert^2=5^2+4^2=41.
  2. 2Kernel values e−0.25 d2e^{-0.25\,d^2}: d2=0.5→0.8825d^2=0.5\to0.8825, 1→0.77881\to0.7788, 2→0.60652\to0.6065, 2.5→0.53532.5\to0.5353, 10.25→0.077110.25\to0.0771, 41→0.0000441\to0.00004. Inside the cluster all similarities are 0.37–0.88; the two far points have similarities of at most 0.21 with anything.
  3. 3The diagonal is all 1: every point sits on the unit sphere in feature space.

Step 2: solve the dual with SMO

  1. 1Feasible start. Fill weight greedily in index order up to CC: α=(0.25,0.25,0.25,0.25,0,…,0)\alpha=(0.25,0.25,0.25,0.25,0,\dots,0). Then ∑α=1\sum\alpha=1. The gradient of 12α⊤Kα\tfrac12\alpha^\top K\alpha is g=Kαg=K\alpha, for example g2=0.25(0.8825+1+0.8825+0.8825)=0.9119g_2=0.25(0.8825+1+0.8825+0.8825)=0.9119 and g10=0.25(0.0601+0.0771+0.2096+0.0221)=0.0922g_{10}=0.25(0.0601+0.0771+0.2096+0.0221)=0.0922.
  2. 2Pick a pair. To lower the objective while keeping ∑α=1\sum\alpha=1, move weight from the point with the largest gg that can shrink (αj>0\alpha_j>0) to the point with the smallest gg that can grow (αi<C\alpha_i<C). Here that is j=2j=2 (0.9119) and i=10i=10 (0.0922), a KKT gap of 0.81970.8197.
  3. 3Step size. Moving tt changes the objective by −t (gj−gi)+12t2η-t\,(g_j-g_i)+\tfrac12t^2\eta with η=Kii+Kjj−2Kij=2−2(0.0771)=1.846\eta=K_{ii}+K_{jj}-2K_{ij}=2-2(0.0771)=1.846. The unconstrained optimum is t=0.8197/1.846=0.444t=0.8197/1.846=0.444. That is clipped to min⁡(0.444, C−α10, α2)=0.25\min(0.444,\ C-\alpha_{10},\ \alpha_2)=0.25, so point 2 hands all its weight to point 10.
  4. 4Repeat. The next step moves 0.250.25 from point 1 to point 9. After that the steps shuffle weight among the cluster's edge points until the gap falls below 10−910^{-9}, which takes about 50 steps.
iαᵢ(Kα)ᵢf = (Kα)ᵢ − ρξᵢGroup
100.4002+0.01620inside
200.4316+0.04770inside
300.4065+0.02260inside
400.4232+0.03920inside
50.03550.384000boundary SV
60.20830.384000boundary SV
70.06740.384000boundary SV
80.18880.384000boundary SV
90.25 = C0.3017−0.08230.0823outlier
100.25 = C0.2921−0.09190.0919outlier
The solution: ρ=0.3840\rho=0.3840. The four free support vectors (5–8) all have (Kα)i=ρ(K\alpha)_i=\rho exactly, which is the KKT check. Points 9 and 10 are at the box α=C\alpha=C with f<0f<0: the training outliers.

Step 3: check the solution by hand

  1. 1Sum constraint: 0.0355+0.2083+0.0674+0.1888+0.25+0.25=1.00000.0355+0.2083+0.0674+0.1888+0.25+0.25=1.0000. ✓
  2. 2ρ from a free SV, point 5: (Kα)5=1(0.0355)+0.5353(0.2083)+0.5353(0.0674)+0.8825(0.1888)+0.0601(0.25)+0.0771(0.25)=0.0355+0.1115+0.0361+0.1666+0.0150+0.0193=0.3840(K\alpha)_5=1(0.0355)+0.5353(0.2083)+0.5353(0.0674)+0.8825(0.1888)+0.0601(0.25)+0.0771(0.25)=0.0355+0.1115+0.0361+0.1666+0.0150+0.0193=0.3840. ✓ Points 6, 7, 8 give the same value.
  3. 3An outlier, point 9: (Kα)9=0.0021+0.0046+0.0141+0.0308+0.25+0≈0.3017<ρ(K\alpha)_9=0.0021+0.0046+0.0141+0.0308+0.25+0\approx0.3017<\rho, so f9=−0.0823f_9=-0.0823. Most of that 0.3017 is its own weight α9k(x9,x9)=0.25\alpha_9k(x_9,x_9)=0.25; its neighbours contribute almost nothing.
  4. 4ν-property: outliers 2/10=0.2 ≤ ν=0.4 ≤2/10=0.2\ \le\ \nu=0.4\ \le support vectors 6/10=0.66/10=0.6. ✓
  5. 5Duality: the primal objective 12α⊤Kα+C∑ξi−ρ=0.1702+0.25(0.1742)−0.3840=−0.1702\tfrac12\alpha^\top K\alpha+C\sum\xi_i-\rho=0.1702+0.25(0.1742)-0.3840=-0.1702 equals the dual value −12α⊤Kα-\tfrac12\alpha^\top K\alpha, so the gap is zero. The feature-space margin is ρ/∥w∥=0.384/0.5835=0.658\rho/\lVert w\rVert=0.384/0.5835=0.658.
SV iαᵢ‖xᵢ − x*‖²k(xᵢ, x*)αᵢ · k
50.03551.060.76720.0272
60.20830.360.91390.1904
70.06741.160.74830.0505
80.18881.960.61260.1157
90.2512.010.04970.0124
100.258.810.11050.0276
Step 4: score a new reading x∗=(3.0, 2.6)x^*=(3.0,\,2.6). Only the six support vectors contribute. ∑=0.4237\sum=0.4237 (the rounded terms add to 0.4238), so f(x∗)=0.4237−0.3840=+0.0398f(x^*)=0.4237-0.3840=+0.0398: normal.

The same arithmetic for x=(4.5, 1.5)x=(4.5,\,1.5), between the cluster and reading 10, gives ∑=0.3354\sum=0.3354 and f=−0.0486f=-0.0486: outlier. For a far corner such as (1,1)(1,1) it gives f=−0.2902f=-0.2902. The score falls smoothly as you move away from the dense region.

00112233445566x₁ · shaded green: f(x) ≥ 0 (normal)12345678910x*dual weights α_iΣα = 1 · 0 ≤ α_i ≤ CC = 1/(νn) = 0.25012345.046.217.078.199.2510.25inside · α = 0on boundary · 0<α<Coutlier · α = Cring size ∝ α · drag any point
SMO step 49/49
converged: every SV has Kα = ρ (boundary) or Kα < ρ with α = C (outlier); every α = 0 point has Kα ≥ ρ
ρ = 0.3840 · f(x*) = Σα_i k(x_i, x*) − ρ = 0.4237 − 0.3840 = +0.0398 → normal (x* = 3.0, 2.6)
outliers 2/10 = 0.20 ≤ ν = 0.40 ≤ SVs 6/10 = 0.60 ✓ ν-property
fig.The worked example, live. Green shading is the region with f(x) ≥ 0, and the black curve is the boundary f = 0. Press ▶ to replay SMO: the bars on the right are the dual weights αᵢ, and each step moves weight from the point with the largest Kα to the one with the smallest. Drag the diamond x* to score any point, drag training points to reshape the region, and change ν and γ to see the ν-property line update.

Effect of ν and γ on the example

νCSVsTraining outliersρ
0.1 – 0.31.0 – 0.3336 (5–10)none0.3326
0.40.256 (5–10)9, 100.3840
0.50.26 (5–10)6, 8, 9, 100.4530
0.60.16776, 7, 8, 9, 100.5045
0.80.12597 points0.5949
Sweeping ν at γ = 0.25. For ν ≤ 0.33 the box C=1/(νn)C=1/(\nu n) is above the largest unconstrained weight (0.297), so ν stops mattering: no outliers, same ρ. Every row satisfies outliers/n ≤ ν ≤ SVs/n.
γSVsTraining outliersShare of the 6×6 box judged normal
0.0549, 109.0%
0.159, 109.5%
0.2569, 108.8%
0.67none6.9%
17none6.2%
2.510none5.2%
510none1.3%
Sweeping γ at ν = 0.4. Larger γ shrinks the normal region and turns more points into support vectors. From γ ≈ 0.6 on, readings 9 and 10 get their own tiny bubbles and stop being outliers: the model has memorized them.

The algorithm, step by step

Train and score

  1. 1Prepare. Standardize the features using statistics from the (clean) training set only.
  2. 2Kernel. Choose γ\gamma; compute (or cache on demand) Kij=e−γ∥xi−xj∥2K_{ij}=e^{-\gamma\lVert x_i-x_j\rVert^2}.
  3. 3Solve the dual. Start from any feasible α\alpha (sum 1, inside the box). Repeat SMO steps: pick the pair with the largest KKT gap, move weight between them by the clipped exact step, update g=Kαg=K\alpha. Stop when the gap is below the tolerance.
  4. 4Offset. ρ\rho = mean of (Kα)i(K\alpha)_i over the free SVs 0<αi<C0<\alpha_i<C.
  5. 5Score. For a new xx: f(x)=∑i∈SVαik(xi,x)−ρf(x)=\sum_{i\in\text{SV}}\alpha_ik(x_i,x)-\rho. Flag f(x)<0f(x)<0, or rank points by ff and set your own threshold.

In code

python
1import numpy as np
2
3def rbf(A, B, gamma):
4 d2 = ((A[:, None, :] - B[None, :, :]) ** 2).sum(-1)
5 return np.exp(-gamma * d2)
6
7def one_class_svm(X, nu=0.4, gamma=0.25, tol=1e-9, max_iter=1000):
8 """SMO on the dual: min 1/2 a^T K a s.t. sum(a) = 1, 0 <= a_i <= C = 1/(nu n)."""
9 n = len(X)
10 C = 1.0 / (nu * n)
11 K = rbf(X, X, gamma)
12 a = np.zeros(n)
13 left = 1.0
14 for i in range(n): # feasible start: fill up to C in order
15 a[i] = min(C, left)
16 left -= a[i]
17 g = K @ a # gradient of the objective = K a
18 for _ in range(max_iter):
19 up, down = a < C - 1e-12, a > 1e-12
20 i = np.flatnonzero(up)[np.argmin(g[up])] # can grow, smallest K a
21 j = np.flatnonzero(down)[np.argmax(g[down])] # can shrink, largest K a
22 if g[j] - g[i] < tol: # KKT satisfied: done
23 break
24 eta = K[i, i] + K[j, j] - 2 * K[i, j]
25 t = min((g[j] - g[i]) / eta, C - a[i], a[j]) # exact line search, clipped to the box
26 a[i] += t
27 a[j] -= t
28 g += t * (K[:, i] - K[:, j])
29 free = (a > 1e-8) & (a < C - 1e-8)
30 rho = g[free].mean() # free SVs sit exactly on the boundary
31 return a, rho
32
33def decision(X, a, rho, gamma, Z):
34 return rbf(Z, X, gamma) @ a - rho # >= 0 normal, < 0 outlier
35
36X = np.array([[2.5, 2.5], [3.0, 3.0], [3.5, 2.5], [2.5, 3.5], [3.5, 3.5],
37 [3.0, 2.0], [2.0, 3.0], [3.0, 4.0], [0.5, 5.0], [5.5, 1.0]])
38a, rho = one_class_svm(X)
39print(a.round(4), round(rho, 4)) # [0 0 0 0 0.0355 0.2083 0.0674 0.1888 0.25 0.25] 0.384
40print(decision(X, a, rho, 0.25, np.array([[3.0, 2.6]])).round(4)) # [0.0398]
Minimal one-class SVM: SMO on the dual. Reproduces the worked example.
python
1import numpy as np
2from sklearn.svm import OneClassSVM
3from sklearn.preprocessing import StandardScaler
4from sklearn.pipeline import make_pipeline
5
6X = np.array([[2.5, 2.5], [3.0, 3.0], [3.5, 2.5], [2.5, 3.5], [3.5, 3.5],
7 [3.0, 2.0], [2.0, 3.0], [3.0, 4.0], [0.5, 5.0], [5.5, 1.0]])
8
9oc = OneClassSVM(kernel="rbf", nu=0.4, gamma=0.25, tol=1e-8).fit(X)
10nu_n = oc.nu * len(X) # libsvm scales the dual by nu*n
11print(oc.support_ + 1) # [ 5 6 7 8 9 10]
12print((oc.dual_coef_ / nu_n).round(4)) # alpha: [[0.0355 0.2083 0.0674 0.1888 0.25 0.25]]
13print((-oc.intercept_ / nu_n).round(4)) # rho: [0.384]
14x_star = [[3.0, 2.6]]
15print((oc.decision_function(x_star) / nu_n).round(4)) # f(x*) = [0.0398] -> normal
16print(oc.predict([[3.0, 2.6], [4.5, 1.5]])) # [ 1 -1]
17
18# On real data: scale first, train on (mostly) clean data, pick nu from the outlier budget.
19model = make_pipeline(StandardScaler(), OneClassSVM(kernel="rbf", nu=0.05, gamma="scale"))
20model.fit(X)
scikit-learn. libsvm scales the dual by νn, so divide by it to recover the article's α, ρ and f.

Use when

  • +You have plenty of normal data and few or no labelled anomalies.
  • +The normal region is curved or has several clusters: the RBF boundary follows it.
  • +You want a direct handle on the false-alarm rate: ν\nu bounds the training outlier fraction.
  • +You want a convex problem: one global optimum, no random restarts, reproducible results.

Avoid when

  • −nn is large (beyond ~10410^4–10510^5): kernel SMO costs about O(n2)O(n^2) memory and time, and prediction cost grows with the number of SVs.
  • −Many features, or features on different scales: RBF distances become meaningless without scaling and degrade in high dimensions.
  • −There are no labels to tune γ\gamma with: results are very sensitive to it, and unsupervised selection is guesswork.
  • −You need calibrated probabilities or explanations: f(x) is a raw, uncalibrated score.
  • −The training data is heavily contaminated: Isolation Forest or LOF are often more robust and faster.

Where it's used

Industrial monitoring

Train on sensor data from a healthy machine; alarm when readings leave the learned region.

Fraud and intrusion detection

Model normal transactions or network flows; flag the rare ones that look like nothing seen before.

Novelty detection for ML systems

Detect inputs far from the training distribution before trusting a downstream model.

Quality control

Learn the feature profile of good parts; reject parts outside it, including defect types never seen.

Check yourself

You fit a one-class SVM with ν=0.05\nu=0.05 on 2,000 points. Which statement is guaranteed?

Check yourself

At the optimum a training point has 0<αi<1/(νn)0<\alpha_i<1/(\nu n). Where is it?

Check yourself

In the worked example, α=(0,0,0,0,0.0355,0.2083,0.0674,0.1888,0.25,0.25)\alpha=(0,0,0,0,0.0355,0.2083,0.0674,0.1888,0.25,0.25), ρ=0.384\rho=0.384 and γ=0.25\gamma=0.25. A new point xx has kernel values (0.29,0.54,0.12,0.12,0.73)(0.29,0.54,0.12,0.12,0.73) with SVs 5, 6, 7, 8 and 10, and ≈0\approx0 with SV 9. Its verdict is…

Check yourself

You raise γ\gamma from 0.25 to 5 on the same data. What happens?

Remember

  • ✓A one-class SVM separates the data from the origin in kernel feature space with maximum margin ρ/∥w∥\rho/\lVert w\rVert. With RBF, all points lie on the unit sphere, so the cut is a cap around the dense regions.
  • ✓Primal: min⁡12∥w∥2+1νn∑ξi−ρ\min\tfrac12\lVert w\rVert^2+\tfrac{1}{\nu n}\sum\xi_i-\rho. Dual: min⁡12α⊤Kα\min\tfrac12\alpha^\top K\alpha with ∑α=1\sum\alpha=1 and 0≤αi≤1νn0\le\alpha_i\le\tfrac{1}{\nu n}.
  • ✓Decision: f(x)=∑iαik(xi,x)−ρf(x)=\sum_i\alpha_ik(x_i,x)-\rho, a thresholded kernel density of the support vectors. ρ\rho is (Kα)i(K\alpha)_i at any free SV.
  • ✓ν-property: outlier fraction ≤ν≤\le\nu\le support-vector fraction, so ν\nu is the training outlier budget.
  • ✓Always scale features. γ\gamma is the critical knob: too small ignores structure, too large memorizes points.
← back to ML Algorithms roadmap