# One-class SVM

> Separate the data from the origin in kernel space with maximum margin; points that fall on the origin's side are outliers.

Source: https://www.mlfoundry.in/learn/one-class-svm  
Section: ML Algorithms › Unsupervised · Anomaly Detection  
Level: advanced · 17 min read · updated 2026-10-08

**A one-class SVM:** is an unsupervised anomaly detector. It learns a boundary around the training data from **unlabelled** examples. In a kernel feature space it finds the hyperplane that pushes the data as far as possible from the **origin**, while letting a fraction of at most $\nu$ of the points fall on the wrong side. A new point on the data's side is **normal**; a point on the origin's side is an **outlier**. Schölkopf et al. introduced it in 2001.

- **Task:** Novelty / outlier detection, also called one-class classification
- **Input → output:** $x\in\mathbb{R}^d$ → score $f(x)=\sum_i\alpha_i k(x_i,x)-\rho$; normal if $f(x)\ge0$, outlier if $f(x)<0$
- **Learns:** one dual weight $\alpha_i\in[0,\tfrac{1}{\nu n}]$ per training point (nonzero only for support vectors) and an offset $\rho$
- **Objective:** maximize the margin $\rho/\lVert w\rVert$ between the data and the origin, plus a hinge-type penalty on points that fall short
- **Training:** a convex quadratic program, so there is one global optimum; SMO (libsvm) takes roughly $O(n^2)$–$O(n^3)$ time and $O(n^2)$ kernel memory
- **Prediction:** $O(n_{\text{SV}}\,d)$ kernel evaluations per point
- **Hyperparameters:** $\nu\in(0,1]$ (outlier budget) and the kernel, almost always RBF with width $\gamma$
- **Assumes:** the training set is (mostly) normal data; features are on comparable scales

## The problem

A fraud team has millions of normal card transactions and a handful of known frauds. A factory has months of sensor logs from a healthy machine and none from a failing one. In both cases a classifier has nothing to train on for the second class, and the anomalies of tomorrow may not look like those of yesterday anyway.

What we can do is describe **what normal looks like**: draw a tight boundary around the training data and flag anything that falls outside it. The boundary should be flexible enough to follow curved or multi-modal data. It should also tolerate a few odd points in the training set, so that one bad reading does not stretch the boundary to swallow everything.

## The idea

A two-class *Support Vector Machine* puts the widest possible street between two classes. A one-class SVM has only one class, so it invents the other: the **origin** of feature space plays the part of the "negative class". The algorithm looks for a hyperplane that keeps the data on one side and the origin on the other, as far from the origin as possible. The region on the data's side is the learned "normal" region.

Separating a cloud of points from the origin does not sound useful in the input space; a cloud centred near the origin cannot be separated at all. With an **RBF kernel** the picture changes completely. Every point is mapped to a vector of length 1 ($k(x,x)=1$), and every pair has a positive dot product ($k(x,x')>0$). All the data sits on a patch of the unit sphere, in one orthant, well away from the origin. A hyperplane slicing that patch cuts off a spherical cap. Points in **dense** regions have large kernel similarities to many others and land inside the cap. **Isolated** points land outside it. Back in input space, the cap is a closed, curved boundary around the dense parts of the data.

**Vocabulary**

- **Feature map φ:** the (implicit) map into the kernel's feature space, with $k(x,x')=\phi(x)^\top\phi(x')$
- **Hyperplane:** $\{z: w^\top z=\rho\}$ in feature space; the data should lie on the side $w^\top\phi(x)\ge\rho$
- **Offset ρ:** how far the hyperplane is from the origin, in units of $w$; its distance is $\rho/\lVert w\rVert$
- **Slack ξᵢ:** how far point $i$ falls short of the hyperplane: $\xi_i=\max(0,\rho-w^\top\phi(x_i))$
- **ν (nu):** the outlier budget: an upper bound on the fraction of training outliers and a lower bound on the fraction of support vectors
- **Support vector:** a training point with $\alpha_i>0$: either exactly on the boundary or a training outlier
- **Decision value:** $f(x)=w^\top\phi(x)-\rho=\sum_i\alpha_ik(x_i,x)-\rho$; its sign is the verdict

## The math: the primal problem

Write the hyperplane in feature space as $w^\top\phi(x)=\rho$. The distance from the origin to it is $\rho/\lVert w\rVert$. We want that distance to be large: large $\rho$ and small $\lVert w\rVert$. Points that land on the origin's side are allowed, but each one pays a slack $\xi_i$. Schölkopf's formulation balances the three goals in one objective:

$$
\min_{w,\,\xi,\,\rho}\ \ \tfrac12\lVert w\rVert^2+\frac{1}{\nu n}\sum_{i=1}^n\xi_i-\rho\qquad\text{s.t.}\quad w^\top\phi(x_i)\ge\rho-\xi_i,\ \ \xi_i\ge0
$$

- $w$: normal vector of the hyperplane in feature space
- $\rho$: offset; the hyperplane sits at distance ρ/‖w‖ from the origin
- $\xi_i$: slack of point i: how far it falls on the origin's side
- $\nu$: outlier budget in (0, 1]; small ν = expensive slack
- $n$: number of training points
- $\phi(x_i)$: feature-space image of training point i

Read the objective term by term. $\tfrac12\lVert w\rVert^2$ is the usual SVM regularizer. $-\rho$ rewards pushing the hyperplane away from the origin. Each unit of slack costs $\tfrac{1}{\nu n}$. Raising $\rho$ by $\delta$ gains $\delta$ in the objective, but every point currently on or beyond the hyperplane gains $\delta$ slack, which costs $\delta/(\nu n)$ each. So $\rho$ keeps rising until about $\nu n$ points are on the wrong side or on the boundary, and that trade is where $\nu$ gets its meaning.

## From primal to dual

Attach multipliers $\alpha_i\ge0$ to the margin constraints and $\beta_i\ge0$ to $\xi_i\ge0$:

$$
\mathcal{L}=\tfrac12\lVert w\rVert^2+\frac{1}{\nu n}\sum_i\xi_i-\rho-\sum_i\alpha_i\big(w^\top\phi(x_i)-\rho+\xi_i\big)-\sum_i\beta_i\xi_i
$$

- $\alpha_i$: multiplier of point i's margin constraint (the dual weight)
- $\beta_i$: multiplier of ξᵢ ≥ 0

**Set the derivatives with respect to the primal variables to zero**

1. $\partial\mathcal{L}/\partial w=w-\sum_i\alpha_i\phi(x_i)=0$, so $w=\sum_i\alpha_i\phi(x_i)$. The normal vector is a weighted sum of training points.
2. $\partial\mathcal{L}/\partial\xi_i=\tfrac{1}{\nu n}-\alpha_i-\beta_i=0$. Since $\beta_i\ge0$, this gives the **box** $0\le\alpha_i\le\tfrac{1}{\nu n}$.
3. $\partial\mathcal{L}/\partial\rho=-1+\sum_i\alpha_i=0$, so $\sum_i\alpha_i=1$.
4. Substitute back. The $\xi$ and $\rho$ terms cancel, and $\tfrac12\lVert w\rVert^2-\sum_i\alpha_iw^\top\phi(x_i)=-\tfrac12\sum_{ij}\alpha_i\alpha_j\phi(x_i)^\top\phi(x_j)$. Replace each dot product by the kernel.

$$
\min_{\alpha}\ \tfrac12\sum_{i,j}\alpha_i\alpha_j\,k(x_i,x_j)=\tfrac12\,\alpha^\top K\alpha\qquad\text{s.t.}\quad 0\le\alpha_i\le\frac{1}{\nu n},\ \ \sum_i\alpha_i=1
$$

- $K$: n × n Gram matrix, Kᵢⱼ = k(xᵢ, xⱼ)
- $\tfrac{1}{\nu n}$: upper bound on each αᵢ (written C below)

This is a small quadratic program: minimize a convex quadratic over a simplex cut off by a box. Note that there are no labels, and the data enters only through the kernel. The weights form a probability distribution over the training points ($\sum\alpha=1$), and no single point may carry more than $C=\tfrac{1}{\nu n}$ of it.

**Recovering ρ.** For a point with $0<\alpha_i<C$ we have $\beta_i>0$, so $\xi_i=0$; and $\alpha_i>0$ makes its margin constraint tight. Such a point lies exactly on the hyperplane, which gives $\rho=w^\top\phi(x_i)=\sum_j\alpha_jk(x_j,x_i)=(K\alpha)_i$. In practice we average over all these "free" support vectors. Substituting $w$ into $w^\top\phi(x)-\rho$ gives the decision function:

$$
f(x)=\sum_{i=1}^n\alpha_i\,k(x_i,x)-\rho\qquad\begin{cases}f(x)\ge0&\text{normal}\\f(x)<0&\text{outlier}\end{cases}
$$

- $k(x_i,x)$: kernel similarity between training point i and the query x
- $\sum_i\alpha_ik(x_i,x)$: a weighted kernel density of the support vectors at x

> **A density threshold in disguise:** With an RBF kernel, $\sum_i\alpha_ik(x_i,x)$ is a weighted Gaussian kernel density estimate built from the support vectors, and $\rho$ is a threshold on it. The one-class SVM never models the whole density. It only learns where the density crosses one level, the level that leaves about a $\nu$ fraction of the data outside. For an RBF kernel it is also equivalent to **SVDD** (Tax & Duin), which encloses the data in the smallest feature-space ball.

## Three kinds of training points

The KKT conditions ($\alpha_i\,[(K\alpha)_i-\rho+\xi_i]=0$ and $\beta_i\xi_i=0$) sort every training point into one of three groups:

The KKT conditions at the optimum. Only the last two groups are support vectors.

| Group | Dual weight | Position | Score |
| --- | --- | --- | --- |
| Inside | $\alpha_i=0$ | strictly on the data's side | $(K\alpha)_i\ge\rho$, $f\ge0$ |
| On the boundary (free SV) | $0<\alpha_i<C$ | exactly on the hyperplane | $(K\alpha)_i=\rho$, $f=0$ |
| Outlier (bounded SV) | $\alpha_i=C$ | on the origin's side (or on it) | $(K\alpha)_i\le\rho$, $\xi_i=\rho-(K\alpha)_i$ |

## The ν-property

The box and the sum constraint together give $\nu$ a precise meaning, which needs only two lines of arithmetic:

**Why ν bounds both counts**

1. **Outliers.** Every training outlier has $\xi_i>0$, so $\beta_i=0$ and $\alpha_i=C=\tfrac{1}{\nu n}$. With $m$ outliers, $\sum\alpha\ge m\cdot\tfrac{1}{\nu n}$. Since $\sum\alpha=1$, $m\le\nu n$: **at most a fraction ν of the training points are outliers.**
2. **Support vectors.** Every $\alpha_i$ is at most $\tfrac{1}{\nu n}$, and together they must add to 1. That takes at least $\nu n$ nonzero weights: **at least a fraction ν of the points are support vectors.**

$$
\frac{\#\{\text{outliers}\}}{n}\ \le\ \nu\ \le\ \frac{\#\{\text{support vectors}\}}{n}
$$

- $\nu$: set by you; both bounds become equalities as n → ∞ (under mild conditions on the data distribution)

This is why $\nu$ is easy to set: it is the **outlier budget**. If you expect about 2% contamination in your training data, start with $\nu=0.02$.

## The RBF kernel and γ

$$
k(x,x')=\exp\!\big(-\gamma\,\lVert x-x'\rVert^2\big)
$$

- $\gamma$: inverse squared width; the similarity falls to e⁻¹ at distance 1/√γ
- $\lVert x-x'\rVert^2$: squared Euclidean distance in input space

$\gamma$ sets how far one training point's influence reaches. With **small** $\gamma$ every point is similar to every other. The density $\sum\alpha_ik(x_i,x)$ is a single smooth bump, and the boundary is a near-ellipse around the bulk of the data. With **large** $\gamma$ similarities die out within a short distance. Each point becomes its own island, so even an isolated point gets a tiny bubble of "normal" around itself. The model then memorizes the training set, and almost every point becomes a support vector. scikit-learn's default `gamma="scale"` uses $\gamma=1/(d\cdot\operatorname{Var}(X))$, which ties the width to the spread of the (scaled) data.

> **Why not a linear kernel?:** With $k(x,x')=x^\top x'$ the model separates the raw points from the origin with a straight line. The "normal" region is a half-plane, and where it falls depends on where you put the coordinate origin. That rarely matches what you mean by normal. The RBF kernel is translation-invariant and gives closed boundaries, which is why it is the default.

## Worked example: ten sensor readings

Ten two-dimensional readings from a machine. Eight form a tight cluster around $(3,3)$; readings 9 and 10 are far off. We fit with $\nu=0.4$ and $\gamma=0.25$, so the box is $C=\tfrac{1}{\nu n}=\tfrac{1}{0.4\cdot10}=0.25$. At most 4 points can be outliers, and at least 4 must be support vectors. Every number below was checked against scikit-learn.

Training data. Highlighted: the two readings that will end up as outliers.

| i | x₁ | x₂ |
| --- | --- | --- |
| 1 | 2.5 | 2.5 |
| 2 | 3.0 | 3.0 |
| 3 | 3.5 | 2.5 |
| 4 | 2.5 | 3.5 |
| 5 | 3.5 | 3.5 |
| 6 | 3.0 | 2.0 |
| 7 | 2.0 | 3.0 |
| 8 | 3.0 | 4.0 |
| 9 | 0.5 | 5.0 |
| 10 | 5.5 | 1.0 |

**Step 1: the kernel matrix**

1. Squared distances. Neighbours on the grid are $0.5$ apart squared ($\lVert x_1-x_2\rVert^2=0.5^2+0.5^2=0.5$), axis steps of 1 give $1$, and the far points are much further: $\lVert x_9-x_2\rVert^2=2.5^2+2^2=10.25$, $\lVert x_9-x_{10}\rVert^2=5^2+4^2=41$.
2. Kernel values $e^{-0.25\,d^2}$: $d^2=0.5\to0.8825$, $1\to0.7788$, $2\to0.6065$, $2.5\to0.5353$, $10.25\to0.0771$, $41\to0.00004$. Inside the cluster all similarities are 0.37–0.88; the two far points have similarities of at most 0.21 with anything.
3. The diagonal is all 1: every point sits on the unit sphere in feature space.

**Step 2: solve the dual with SMO**

1. **Feasible start.** Fill weight greedily in index order up to $C$: $\alpha=(0.25,0.25,0.25,0.25,0,\dots,0)$. Then $\sum\alpha=1$. The gradient of $\tfrac12\alpha^\top K\alpha$ is $g=K\alpha$, for example $g_2=0.25(0.8825+1+0.8825+0.8825)=0.9119$ and $g_{10}=0.25(0.0601+0.0771+0.2096+0.0221)=0.0922$.
2. **Pick a pair.** To lower the objective while keeping $\sum\alpha=1$, move weight from the point with the **largest** $g$ that can shrink ($\alpha_j>0$) to the point with the **smallest** $g$ that can grow ($\alpha_i<C$). Here that is $j=2$ (0.9119) and $i=10$ (0.0922), a KKT gap of $0.8197$.
3. **Step size.** Moving $t$ changes the objective by $-t\,(g_j-g_i)+\tfrac12t^2\eta$ with $\eta=K_{ii}+K_{jj}-2K_{ij}=2-2(0.0771)=1.846$. The unconstrained optimum is $t=0.8197/1.846=0.444$. That is clipped to $\min(0.444,\ C-\alpha_{10},\ \alpha_2)=0.25$, so point 2 hands all its weight to point 10.
4. **Repeat.** The next step moves $0.25$ from point 1 to point 9. After that the steps shuffle weight among the cluster's edge points until the gap falls below $10^{-9}$, which takes about 50 steps.

The solution: $\rho=0.3840$. The four free support vectors (5–8) all have $(K\alpha)_i=\rho$ exactly, which is the KKT check. Points 9 and 10 are at the box $\alpha=C$ with $f<0$: the training outliers.

| i | αᵢ | (Kα)ᵢ | f = (Kα)ᵢ − ρ | ξᵢ | Group |
| --- | --- | --- | --- | --- | --- |
| 1 | 0 | 0.4002 | +0.0162 | 0 | inside |
| 2 | 0 | 0.4316 | +0.0477 | 0 | inside |
| 3 | 0 | 0.4065 | +0.0226 | 0 | inside |
| 4 | 0 | 0.4232 | +0.0392 | 0 | inside |
| 5 | 0.0355 | 0.3840 | 0 | 0 | boundary SV |
| 6 | 0.2083 | 0.3840 | 0 | 0 | boundary SV |
| 7 | 0.0674 | 0.3840 | 0 | 0 | boundary SV |
| 8 | 0.1888 | 0.3840 | 0 | 0 | boundary SV |
| 9 | **0.25 = C** | 0.3017 | **−0.0823** | 0.0823 | **outlier** |
| 10 | **0.25 = C** | 0.2921 | **−0.0919** | 0.0919 | **outlier** |

**Step 3: check the solution by hand**

1. **Sum constraint:** $0.0355+0.2083+0.0674+0.1888+0.25+0.25=1.0000$. ✓
2. **ρ from a free SV**, point 5: $(K\alpha)_5=1(0.0355)+0.5353(0.2083)+0.5353(0.0674)+0.8825(0.1888)+0.0601(0.25)+0.0771(0.25)=0.0355+0.1115+0.0361+0.1666+0.0150+0.0193=0.3840$. ✓ Points 6, 7, 8 give the same value.
3. **An outlier**, point 9: $(K\alpha)_9=0.0021+0.0046+0.0141+0.0308+0.25+0\approx0.3017<\rho$, so $f_9=-0.0823$. Most of that 0.3017 is its own weight $\alpha_9k(x_9,x_9)=0.25$; its neighbours contribute almost nothing.
4. **ν-property:** outliers $2/10=0.2\ \le\ \nu=0.4\ \le$ support vectors $6/10=0.6$. ✓
5. **Duality:** the primal objective $\tfrac12\alpha^\top K\alpha+C\sum\xi_i-\rho=0.1702+0.25(0.1742)-0.3840=-0.1702$ equals the dual value $-\tfrac12\alpha^\top K\alpha$, so the gap is zero. The feature-space margin is $\rho/\lVert w\rVert=0.384/0.5835=0.658$.

Step 4: score a new reading $x^*=(3.0,\,2.6)$. Only the six support vectors contribute. $\sum=0.4237$ (the rounded terms add to 0.4238), so $f(x^*)=0.4237-0.3840=+0.0398$: normal.

| SV i | αᵢ | ‖xᵢ − x*‖² | k(xᵢ, x*) | αᵢ · k |
| --- | --- | --- | --- | --- |
| 5 | 0.0355 | 1.06 | 0.7672 | 0.0272 |
| 6 | 0.2083 | 0.36 | 0.9139 | 0.1904 |
| 7 | 0.0674 | 1.16 | 0.7483 | 0.0505 |
| 8 | 0.1888 | 1.96 | 0.6126 | 0.1157 |
| 9 | 0.25 | 12.01 | 0.0497 | 0.0124 |
| 10 | 0.25 | 8.81 | 0.1105 | 0.0276 |

The same arithmetic for $x=(4.5,\,1.5)$, between the cluster and reading 10, gives $\sum=0.3354$ and $f=-0.0486$: **outlier**. For a far corner such as $(1,1)$ it gives $f=-0.2902$. The score falls smoothly as you move away from the dense region.

> Figure: The worked example, live. Green shading is the region with f(x) ≥ 0, and the black curve is the boundary f = 0. Press ▶ to replay SMO: the bars on the right are the dual weights αᵢ, and each step moves weight from the point with the largest Kα to the one with the smallest. Drag the diamond x* to score any point, drag training points to reshape the region, and change ν and γ to see the ν-property line update.

## Effect of ν and γ on the example

Sweeping ν at γ = 0.25. For ν ≤ 0.33 the box $C=1/(\nu n)$ is above the largest unconstrained weight (0.297), so ν stops mattering: no outliers, same ρ. Every row satisfies outliers/n ≤ ν ≤ SVs/n.

| ν | C | SVs | Training outliers | ρ |
| --- | --- | --- | --- | --- |
| 0.1 – 0.3 | 1.0 – 0.333 | 6 (5–10) | none | 0.3326 |
| **0.4** | **0.25** | **6 (5–10)** | **9, 10** | **0.3840** |
| 0.5 | 0.2 | 6 (5–10) | 6, 8, 9, 10 | 0.4530 |
| 0.6 | 0.167 | 7 | 6, 7, 8, 9, 10 | 0.5045 |
| 0.8 | 0.125 | 9 | 7 points | 0.5949 |

Sweeping γ at ν = 0.4. Larger γ shrinks the normal region and turns more points into support vectors. From γ ≈ 0.6 on, readings 9 and 10 get their own tiny bubbles and stop being outliers: the model has memorized them.

| γ | SVs | Training outliers | Share of the 6×6 box judged normal |
| --- | --- | --- | --- |
| 0.05 | 4 | 9, 10 | 9.0% |
| 0.1 | 5 | 9, 10 | 9.5% |
| **0.25** | **6** | **9, 10** | **8.8%** |
| 0.6 | 7 | none | 6.9% |
| 1 | 7 | none | 6.2% |
| 2.5 | 10 | none | 5.2% |
| 5 | 10 | none | 1.3% |

## The algorithm, step by step

**Train and score**

1. **Prepare.** Standardize the features using statistics from the (clean) training set only.
2. **Kernel.** Choose $\gamma$; compute (or cache on demand) $K_{ij}=e^{-\gamma\lVert x_i-x_j\rVert^2}$.
3. **Solve the dual.** Start from any feasible $\alpha$ (sum 1, inside the box). Repeat SMO steps: pick the pair with the largest KKT gap, move weight between them by the clipped exact step, update $g=K\alpha$. Stop when the gap is below the tolerance.
4. **Offset.** $\rho$ = mean of $(K\alpha)_i$ over the free SVs $0<\alpha_i<C$.
5. **Score.** For a new $x$: $f(x)=\sum_{i\in\text{SV}}\alpha_ik(x_i,x)-\rho$. Flag $f(x)<0$, or rank points by $f$ and set your own threshold.

## In code

```python
import numpy as np

def rbf(A, B, gamma):
    d2 = ((A[:, None, :] - B[None, :, :]) ** 2).sum(-1)
    return np.exp(-gamma * d2)

def one_class_svm(X, nu=0.4, gamma=0.25, tol=1e-9, max_iter=1000):
    """SMO on the dual: min 1/2 a^T K a  s.t. sum(a) = 1, 0 <= a_i <= C = 1/(nu n)."""
    n = len(X)
    C = 1.0 / (nu * n)
    K = rbf(X, X, gamma)
    a = np.zeros(n)
    left = 1.0
    for i in range(n):                       # feasible start: fill up to C in order
        a[i] = min(C, left)
        left -= a[i]
    g = K @ a                                # gradient of the objective = K a
    for _ in range(max_iter):
        up, down = a < C - 1e-12, a > 1e-12
        i = np.flatnonzero(up)[np.argmin(g[up])]      # can grow, smallest K a
        j = np.flatnonzero(down)[np.argmax(g[down])]  # can shrink, largest K a
        if g[j] - g[i] < tol:                # KKT satisfied: done
            break
        eta = K[i, i] + K[j, j] - 2 * K[i, j]
        t = min((g[j] - g[i]) / eta, C - a[i], a[j])  # exact line search, clipped to the box
        a[i] += t
        a[j] -= t
        g += t * (K[:, i] - K[:, j])
    free = (a > 1e-8) & (a < C - 1e-8)
    rho = g[free].mean()                     # free SVs sit exactly on the boundary
    return a, rho

def decision(X, a, rho, gamma, Z):
    return rbf(Z, X, gamma) @ a - rho        # >= 0 normal, < 0 outlier

X = np.array([[2.5, 2.5], [3.0, 3.0], [3.5, 2.5], [2.5, 3.5], [3.5, 3.5],
              [3.0, 2.0], [2.0, 3.0], [3.0, 4.0], [0.5, 5.0], [5.5, 1.0]])
a, rho = one_class_svm(X)
print(a.round(4), round(rho, 4))             # [0 0 0 0 0.0355 0.2083 0.0674 0.1888 0.25 0.25] 0.384
print(decision(X, a, rho, 0.25, np.array([[3.0, 2.6]])).round(4))   # [0.0398]
```

Minimal one-class SVM: SMO on the dual. Reproduces the worked example.

```python
import numpy as np
from sklearn.svm import OneClassSVM
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline

X = np.array([[2.5, 2.5], [3.0, 3.0], [3.5, 2.5], [2.5, 3.5], [3.5, 3.5],
              [3.0, 2.0], [2.0, 3.0], [3.0, 4.0], [0.5, 5.0], [5.5, 1.0]])

oc = OneClassSVM(kernel="rbf", nu=0.4, gamma=0.25, tol=1e-8).fit(X)
nu_n = oc.nu * len(X)                        # libsvm scales the dual by nu*n
print(oc.support_ + 1)                       # [ 5  6  7  8  9 10]
print((oc.dual_coef_ / nu_n).round(4))       # alpha: [[0.0355 0.2083 0.0674 0.1888 0.25 0.25]]
print((-oc.intercept_ / nu_n).round(4))      # rho:   [0.384]
x_star = [[3.0, 2.6]]
print((oc.decision_function(x_star) / nu_n).round(4))  # f(x*) = [0.0398] -> normal
print(oc.predict([[3.0, 2.6], [4.5, 1.5]]))  # [ 1 -1]

# On real data: scale first, train on (mostly) clean data, pick nu from the outlier budget.
model = make_pipeline(StandardScaler(), OneClassSVM(kernel="rbf", nu=0.05, gamma="scale"))
model.fit(X)
```

scikit-learn. libsvm scales the dual by νn, so divide by it to recover the article's α, ρ and f.

> **Reading scikit-learn's output:** `predict` returns $+1$ (normal) or $-1$ (outlier). `decision_function` is $\nu n\cdot f(x)$: same sign, different scale. `score_samples` adds the offset back, giving the unshifted density score. Points exactly on the boundary have $f=0$ up to rounding, so `predict` may label them either way. For large datasets, `SGDOneClassSVM` with a `Nystroem` kernel approximation scales linearly in $n$.

**Strengths**

- You have plenty of normal data and few or no labelled anomalies.
- The normal region is curved or has several clusters: the RBF boundary follows it.
- You want a direct handle on the false-alarm rate: $\nu$ bounds the training outlier fraction.
- You want a convex problem: one global optimum, no random restarts, reproducible results.

**Limitations**

- $n$ is large (beyond ~$10^4$–$10^5$): kernel SMO costs about $O(n^2)$ memory and time, and prediction cost grows with the number of SVs.
- Many features, or features on different scales: RBF distances become meaningless without scaling and degrade in high dimensions.
- There are no labels to tune $\gamma$ with: results are very sensitive to it, and unsupervised selection is guesswork.
- You need calibrated probabilities or explanations: f(x) is a raw, uncalibrated score.
- The training data is heavily contaminated: Isolation Forest or LOF are often more robust and faster.

- **Industrial monitoring:** Train on sensor data from a healthy machine; alarm when readings leave the learned region.
- **Fraud and intrusion detection:** Model normal transactions or network flows; flag the rare ones that look like nothing seen before.
- **Novelty detection for ML systems:** Detect inputs far from the training distribution before trusting a downstream model.
- **Quality control:** Learn the feature profile of good parts; reject parts outside it, including defect types never seen.

> **Scale your features:** The RBF kernel uses one $\gamma$ for every direction. If one feature is in metres and another in millimetres, the second dominates every distance, and the boundary becomes a thin sliver along the first. Standardize (or robust-scale) the features using statistics from the training set, and put the scaler and the model in one `Pipeline`.

> **γ decides everything, and there are no labels:** Too small a $\gamma$ gives a blob that ignores structure. Too large a $\gamma$ wraps every training point in its own bubble: in the example, from $\gamma\approx0.6$ the two far readings stop being outliers. Warning signs are a support-vector fraction far above $\nu$ and a normal region that is a cloud of islands. If you have even a few labelled anomalies, tune $\gamma$ on them (for example by precision at a fixed alarm rate). Otherwise start from `gamma="scale"` and check the boundary on held-out normal data.

> **ν is not the test-time alarm rate:** $\nu$ bounds the outlier fraction on the **training** set. On new data the alarm rate is close to $\nu$ only if the new data comes from the same distribution. Do not set $\nu$ to 0.5 "to be safe": you are telling the model that half your training data is abnormal, and it will tighten the boundary to match.

**Question:** You fit a one-class SVM with $\nu=0.05$ on 2,000 points. Which statement is guaranteed?

- A. Exactly 100 training points are outliers
- B. At most 100 training points are outliers, and at least 100 are support vectors
- C. At least 100 training points are outliers
- D. 5% of future test points will be flagged

**Answer:** B. At most 100 training points are outliers, and at least 100 are support vectors — $\nu n=100$. Outliers have $\alpha=1/(\nu n)$ and the weights sum to 1, so there are at most 100. Each weight is at most $1/(\nu n)$, so at least 100 are needed to reach 1. Nothing is guaranteed about test data.

**Question:** At the optimum a training point has $0<\alpha_i<1/(\nu n)$. Where is it?

- A. Strictly inside the normal region
- B. Exactly on the boundary f = 0
- C. Outside the boundary
- D. It could be anywhere

**Answer:** B. Exactly on the boundary f = 0 — $\alpha_i<C$ forces $\beta_i>0$, so $\xi_i=0$; $\alpha_i>0$ makes the constraint tight. Together: $(K\alpha)_i=\rho$, which is exactly how $\rho$ is computed.

**Question:** In the worked example, $\alpha=(0,0,0,0,0.0355,0.2083,0.0674,0.1888,0.25,0.25)$, $\rho=0.384$ and $\gamma=0.25$. A new point $x$ has kernel values $(0.29,0.54,0.12,0.12,0.73)$ with SVs 5, 6, 7, 8 and 10, and $\approx0$ with SV 9. Its verdict is…

- A. Normal, f ≈ +0.05
- B. Outlier, f ≈ −0.05
- C. On the boundary
- D. Cannot tell without ‖w‖

**Answer:** B. Outlier, f ≈ −0.05 — $\sum\alpha_ik=0.0355(0.29)+0.2083(0.54)+0.0674(0.12)+0.1888(0.12)+0.25(0.73)\approx0.010+0.112+0.008+0.023+0.183=0.336$, and $0.336-0.384\approx-0.05<0$. This is the point $(4.5,1.5)$ from the text.

**Question:** You raise $\gamma$ from 0.25 to 5 on the same data. What happens?

- A. The normal region grows into one large smooth blob
- B. Nothing, since ν alone controls the boundary
- C. The region shrinks into small bubbles around individual points, and nearly every point becomes a support vector
- D. The model becomes a linear kernel

**Answer:** C. The region shrinks into small bubbles around individual points, and nearly every point becomes a support vector — Large $\gamma$ makes the kernel very local. In the example, $\gamma=5$ makes all 10 points SVs, the normal area drops from 8.8% to 1.3% of the box, and readings 9 and 10 are no longer flagged. That is overfitting.

**Key points**

- A one-class SVM separates the data from the **origin** in kernel feature space with maximum margin $\rho/\lVert w\rVert$. With RBF, all points lie on the unit sphere, so the cut is a cap around the dense regions.
- Primal: $\min\tfrac12\lVert w\rVert^2+\tfrac{1}{\nu n}\sum\xi_i-\rho$. Dual: $\min\tfrac12\alpha^\top K\alpha$ with $\sum\alpha=1$ and $0\le\alpha_i\le\tfrac{1}{\nu n}$.
- Decision: $f(x)=\sum_i\alpha_ik(x_i,x)-\rho$, a thresholded kernel density of the support vectors. $\rho$ is $(K\alpha)_i$ at any free SV.
- ν-property: outlier fraction $\le\nu\le$ support-vector fraction, so $\nu$ is the training outlier budget.
- Always scale features. $\gamma$ is the critical knob: too small ignores structure, too large memorizes points.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
