# Elliptic Envelope

> Fit a robust Gaussian to the data and flag points whose Mahalanobis distance exceeds a threshold.

Source: https://www.mlfoundry.in/learn/elliptic-envelope  
Section: ML Algorithms › Unsupervised · Anomaly Detection  
Level: core · 16 min read · updated 2026-10-08

**An elliptic envelope:** is an outlier detector that assumes the normal data come from a single multivariate Gaussian. It estimates the centre $\mu$ and covariance $\Sigma$ **robustly** (so the outliers cannot drag them), measures every point's Mahalanobis distance from the centre, and flags the points outside an ellipse whose size is set by the expected fraction of outliers, the `contamination`.

- **Task:** Unsupervised outlier / novelty detection
- **Input → output:** $x\in\mathbb{R}^p$ → a score $-d^2(x)$ and a label: $+1$ inlier, $-1$ outlier
- **Learns:** a centre $\mu$, a covariance $\Sigma$ (via the Minimum Covariance Determinant) and a threshold $\tau$
- **Decision rule:** outlier if $d^2(x)=(x-\mu)^\top\Sigma^{-1}(x-\mu)>\tau$
- **Training:** FastMCD: roughly $O(n\,p^2)$ per C-step over a few hundred random starts
- **Prediction:** $O(p^2)$ per point: one quadratic form
- **Hyperparameters:** `contamination` (sets τ), `support_fraction` (size of the MCD subset)
- **Assumes:** inliers are roughly Gaussian, so one ellipsoid describes them; n comfortably larger than p

## The idea

Picture a scatter of measurements from a healthy machine: temperature and vibration, say. They form a tilted, elongated cloud because the two rise together. A faulty reading is not necessarily far from the centre in plain distance; it is far **relative to the shape of the cloud**, for example high vibration at low temperature. The elliptic envelope captures that shape with a Gaussian, whose density contours are ellipses, and draws one ellipse that should contain the normal points. Anything outside it is an outlier.

Three questions make up the method: how to measure distance relative to the cloud (the **Mahalanobis distance**), how big to make the ellipse (the **threshold**), and how to estimate the cloud's shape when the outliers you are hunting are mixed into the data (a **robust** estimate).

## Mahalanobis distance

$$
d^2(x)=(x-\mu)^\top\,\Sigma^{-1}\,(x-\mu)
$$

- $\mu$: centre of the inlier cloud (mean vector)
- $\Sigma$: $p\times p$ covariance matrix of the inliers
- $d^2(x)$: squared Mahalanobis distance of x from the centre

With $\Sigma=I$ this is the ordinary squared Euclidean distance. In general $\Sigma^{-1}$ rescales and de-correlates: a step along a direction in which the data vary a lot costs little, and a step along a direction in which they barely vary costs a lot. Because of that, the distance is **affine invariant**: rescale a feature, change units or rotate the axes, and every $d^2$ stays the same. Feature scaling makes no difference to it.

> **Why d² follows a χ² distribution:** Factor $\Sigma=LL^\top$ (Cholesky) and set $z=L^{-1}(x-\mu)$. If $x\sim\mathcal N(\mu,\Sigma)$ then $z\sim\mathcal N(0,I)$: $p$ independent standard normals. And $d^2=(x-\mu)^\top L^{-\top}L^{-1}(x-\mu)=z^\top z=z_1^2+\dots+z_p^2$, a sum of $p$ squared standard normals, which is exactly the definition of $\chi^2_p$. So for Gaussian inliers $d^2$ has mean $p$, and a fraction $1-q$ of them land beyond the $\chi^2_p$ quantile $\chi^2_{p,q}$. In 2-D the quantile has a closed form, $\chi^2_{2,q}=-2\ln(1-q)$: the 97.5% cut-off is $7.378$.

## The geometry: ellipses from eigenvectors

Every set $\{x: d^2(x)=\tau\}$ is an ellipse (an ellipsoid in $p$ dimensions). To see its shape, diagonalize the covariance:

$$
\Sigma=Q\Lambda Q^\top \;\Rightarrow\; d^2(x)=\sum_{i=1}^{p}\frac{\big(q_i^\top(x-\mu)\big)^2}{\lambda_i}
$$

- $q_i$: i-th eigenvector of Σ: the direction of an axis of the ellipse
- $\lambda_i$: i-th eigenvalue: the variance of the data along that axis

In the rotated coordinates $u_i=q_i^\top(x-\mu)$ the contour $d^2=\tau$ is $\sum_i u_i^2/(\tau\lambda_i)=1$: an axis-aligned ellipse with **semi-axis $\sqrt{\tau\,\lambda_i}$ along $q_i$**. Large variance gives a long axis. Its area (volume in $p$ dimensions) grows with $\sqrt{\det\Sigma}$; in 2-D it is $\pi\,\tau\sqrt{\det\Sigma}$. Raising $\tau$ scales the whole ellipse up without changing its shape or tilt.

## Choosing the threshold: contamination

The fitted model gives every training point a distance. The threshold $\tau$ is set so that a chosen fraction of training points falls outside. That fraction is the `contamination` $c$, your prior guess of how much of the data is anomalous:

$$
\tau=\text{quantile}_{1-c}\big\{d^2(x_1),\dots,d^2(x_n)\big\},\qquad f(x)=\tau-d^2(x)
$$

- $c$: contamination: expected outlier fraction (scikit-learn default 0.1)
- $f(x)$: decision function: positive inside the envelope, negative outside, zero on it

Prediction is the sign: $f(x)\ge 0$ means inlier ($+1$), $f(x)<0$ means outlier ($-1$). In scikit-learn, `score_samples(x)` returns $-d^2(x)$ (higher means more normal), `offset_` stores $-\tau$, and `decision_function` is `score_samples − offset_` $=\tau-d^2$. Note that `mahalanobis()` returns the **squared** distance.

> **Contamination does not change the fit:** The centre and covariance are estimated first, independently of `contamination`; it only moves $\tau$, that is, how large the same ellipse is drawn. If you would rather not guess a fraction, a principled alternative is the Gaussian cut-off $\tau=\chi^2_{p,0.975}$, which flags about 2.5% of genuine Gaussian inliers no matter how many outliers there are. Use `score_samples` and apply that threshold yourself.

## Why the plain mean and covariance fail: masking

The obvious estimates are the sample mean and covariance of all points. But the outliers are part of "all points". A cluster of outliers pulls $\mu$ toward itself and stretches $\Sigma$ in its own direction, so the ellipse grows to accommodate them, and their distances shrink. This is **masking**: the outliers hide themselves. The side effect is **swamping**: the distorted ellipse no longer follows the inliers, so some perfectly normal points end up looking extreme.

A single far-away point is enough to wreck the classical estimates: their **breakdown point** is $0\%$. The fix is to estimate $\mu$ and $\Sigma$ from the points that look most like a tight Gaussian core. Scikit-learn's `EllipticEnvelope` uses the **Minimum Covariance Determinant (MCD)**: among all subsets of $h\approx(n+p+1)/2$ points, find the one whose covariance has the smallest determinant, which is the smallest-volume ellipse holding half the data. How MCD is computed (FastMCD and its C-steps) and why it survives up to about 50% contamination is the topic of the **Robust Covariance** article. Here we use it as a black box.

**EllipticEnvelope.fit(X), as scikit-learn does it**

1. **Robust core.** Run MCD with $h=\lceil(n+p+1)/2\rceil$ (or `support_fraction`·$n$). This gives a raw centre and covariance from the $h$ most concentrated points.
2. **Consistency correction.** The covariance of the tightest half underestimates the spread, so rescale it until the median $d^2$ of all points equals the $\chi^2_p$ median ($1.386$ for $p=2$).
3. **Reweight.** Keep every point with corrected $d^2<\chi^2_{p,0.975}$ and recompute an ordinary mean and covariance from those points. This step recovers efficiency: good points left out of the half-sample come back in.
4. **Threshold.** Compute $d^2$ for every training point under the reweighted fit, then set $\tau$ to their $(1-c)$ quantile.
5. **Predict.** For any $x$, return the sign of $f(x)=\tau-d^2(x)$.

## Worked example: twelve points, two outliers

Ten inliers lie along a rising diagonal; two outliers, P11 and P12, sit together at the bottom right, off the trend. Every number below was checked against scikit-learn's `EllipticEnvelope(contamination=2/12, random_state=0)` and NumPy.

The data, and the squared Mahalanobis distance of every point under each fit. Highlighted: the outliers, plus P8, the honest point that the classical fit gets wrong.

| Point | x₁ | x₂ | classical d² | robust d² | classical flag (τ = 4.46) | robust flag (τ = 10.76) |
| --- | --- | --- | --- | --- | --- | --- |
| P1 | 1.0 | 1.5 | 2.540 | 3.004 |  |  |
| P2 | 1.5 | 1.0 | 2.305 | 2.246 |  |  |
| P3 | 2.0 | 2.5 | 0.910 | 1.814 |  |  |
| P4 | 2.5 | 2.0 | 0.252 | 0.403 |  |  |
| P5 | 3.0 | 3.5 | 1.538 | 2.543 |  |  |
| P6 | 3.5 | 2.5 | 0.076 | 1.814 |  |  |
| P7 | 3.0 | 2.5 | 0.073 | 0.202 |  |  |
| **P8** | 4.5 | 4.5 | **4.571** | 3.887 | **outlier** (swamped) |  |
| P9 | 4.5 | 3.5 | 1.764 | 3.004 |  |  |
| P10 | 2.0 | 1.5 | 0.997 | 1.084 |  |  |
| **P11** | 5.0 | 1.0 | 4.523 | **45.777** | outlier | **outlier** |
| **P12** | 5.5 | 1.5 | 4.451 | **45.154** | inlier (masked) | **outlier** |

The two fits. The outliers drag the classical centre to the right and down, inflate the spread of x₁, and destroy most of the correlation (0.87 → 0.28).

| Quantity | Classical (all 12 points) | Robust (EllipticEnvelope) |
| --- | --- | --- |
| μ | (3.167, 2.292) | (2.750, 2.500) |
| Σ | [[1.931, 0.410], [0.410, 1.102]] | [[1.263, 1.000], [1.000, 1.050]] |
| det Σ | 1.960 | 0.326 |
| Σ⁻¹ | [[0.562, −0.209], [−0.209, 0.985]] | [[3.225, −3.071], [−3.071, 3.877]] |
| correlation | 0.28 | 0.87 |
| eigenvalues λ₁, λ₂ | 2.099, 0.934 | 2.162, 0.151 |
| major axis angle | 22.3° | 42.0° |
| 97.5% ellipse semi-axes | 3.94 × 2.63 (area 32.5) | 3.99 × 1.05 (area 13.2) |

**Where the robust numbers come from**

1. **MCD subset.** $h=\lceil(12+2+1)/2\rceil=8$. Of the $\binom{12}{8}=495$ subsets, the one with the smallest covariance determinant ($0.1426$) is {P1, P2, P3, P4, P6, P7, P9, P10}. Raw centre $(2.500, 2.125)$.
2. **Correction.** The median raw $d^2$ over all 12 points is $3.356$; dividing by the $\chi^2_2$ median $1.386$ gives a factor of $2.421$, which inflates the raw covariance.
3. **Reweight.** With the corrected fit, P5 ($d^2=3.82$) and P8 ($5.80$) are under $7.378$ and come back in; P11 and P12 ($\approx 25$ and $23$) stay out. The final $\mu$ and $\Sigma$ are the plain mean and covariance of P1 to P10: exactly the ten real inliers.
4. **Threshold.** The sorted robust $d^2$ end $\dots, 3.004, 3.887, 45.154, 45.777$. The $1-\tfrac{2}{12}$ quantile sits at position $\tfrac{10}{12}\cdot 11=9.17$, so $\tau=3.887+0.17\,(45.154-3.887)=10.765$. Exactly P11 and P12 exceed it.

**One distance by hand: P11 = (5.0, 1.0)**

1. **Robust.** $x-\mu=(2.25,\,-1.50)$. Then $d^2=3.225(2.25)^2+2(-3.071)(2.25)(-1.50)+3.877(-1.50)^2=16.32+20.73+8.72=45.78$. The cross term is large and positive because P11 moves *against* the strong positive correlation: right, but down.
2. **Classical.** $x-\mu=(1.833,\,-1.292)$. Then $d^2=0.562(1.833)^2+2(-0.209)(1.833)(-1.292)+0.985(-1.292)^2=1.89+0.99+1.64=4.52$. With the correlation mostly gone, moving against it barely costs anything.
3. **Verdict.** Robustly, P11 is $\sqrt{45.78}=6.8$ "standard deviations" out, with a $\chi^2_2$ tail probability of about $10^{-10}$. Classically it is unremarkable: below the 97.5% cut-off of $7.378$, and in fact the classical fit flags **no point at all** at that cut-off. That is masking in its purest form.

Now score two new points with the robust model. $(3, 3)$ has $d^2=0.40$ and $f=10.765-0.40=10.36$: deep inside. $(4, 2)$ is only $1.28$ away from $\mu$ in plain distance, but it lies off the diagonal: $d^2=9.85$ and $f=0.92$. It is still an inlier, but only just, and it would be flagged under the stricter $\chi^2_{2,0.975}=7.378$ cut-off. The decision function tells you more than the label does: how far inside or outside the boundary a point is.

> Figure: The worked example, live. The solid ellipse is the active detector's envelope d² = τ, with its principal axes (√(τλᵢ) along each eigenvector) and faint rings at d² = 1 and 4; the dashed ellipse is the other fit at its own threshold. Rose points are flagged. The bars show each point's distance d = √d² against √τ and the χ²₂ 97.5% line. Switch to the classical detector and P12 slips inside while P8 is pushed out. Drag P11 and P12 further away: the classical ellipse stretches to follow them, while the robust one does not move. Slide the contamination up and the ellipse shrinks onto the core. Drag the ◆ probe to read the decision function f(x) = τ − d².

## In code

```python
import numpy as np
from scipy.stats import chi2
from sklearn.covariance import EllipticEnvelope

X = np.array([[1.0, 1.5], [1.5, 1.0], [2.0, 2.5], [2.5, 2.0], [3.0, 3.5], [3.5, 2.5],
              [3.0, 2.5], [4.5, 4.5], [4.5, 3.5], [2.0, 1.5], [5.0, 1.0], [5.5, 1.5]])

def mahalanobis_sq(X, mu, cov):
    diff = X - mu
    return np.einsum("ij,jk,ik->i", diff, np.linalg.inv(cov), diff)

# Classical (non-robust) envelope: sample mean and MLE covariance of all 12 points
mu, cov = X.mean(axis=0), np.cov(X, rowvar=False, bias=True)
d2 = mahalanobis_sq(X, mu, cov)
print(d2.round(2))                         # outliers P11, P12 get only 4.52, 4.45
print((d2 > chi2.ppf(0.975, df=2)).sum())  # 0 -> masked: nothing exceeds 7.38

# Robust envelope (MCD inside), threshold from contamination
ee = EllipticEnvelope(contamination=2/12, random_state=0).fit(X)
print(ee.location_, ee.covariance_)        # [2.75 2.5]  [[1.2625 1.] [1. 1.05]]
print(ee.mahalanobis(X).round(2))          # squared distances: P11 45.78, P12 45.15
print(-ee.offset_)                         # threshold tau = 10.76
print(ee.predict(X))                       # -1 for P11 and P12 only
print(ee.decision_function([[3, 3], [4, 2]]).round(3))  # tau - d^2: [10.361  0.918]
```

scikit-learn on the worked example, against the classical fit. The comments show the actual outputs.

```python
import numpy as np
from itertools import combinations
from scipy.stats import chi2

class EllipticEnvelopeScratch:
    def __init__(self, contamination=0.1):
        self.contamination = contamination

    def fit(self, X):
        n, p = X.shape
        h = int(np.ceil((n + p + 1) / 2))
        # 1. MCD: the h points whose covariance has the smallest determinant
        #    (exhaustive here; FastMCD searches with random starts + C-steps)
        best = min(combinations(range(n), h),
                   key=lambda s: np.linalg.det(np.cov(X[list(s)], rowvar=False, bias=True)))
        S = X[list(best)]
        mu, cov = S.mean(axis=0), np.cov(S, rowvar=False, bias=True)
        # 2. consistency: rescale so the median d^2 matches the chi2_p median
        d2 = mahalanobis_sq(X, mu, cov)
        cov = cov * np.median(d2) / chi2.ppf(0.5, p)
        # 3. reweight: refit on every point inside the 97.5% chi2 ellipse
        keep = mahalanobis_sq(X, mu, cov) < chi2.ppf(0.975, p)
        self.location_ = X[keep].mean(axis=0)
        self.covariance_ = np.cov(X[keep], rowvar=False, bias=True)
        # 4. threshold: the (1 - contamination) quantile of the training d^2
        self.threshold_ = np.percentile(self.mahalanobis(X), 100 * (1 - self.contamination))
        return self

    def mahalanobis(self, X):
        return mahalanobis_sq(np.asarray(X, float), self.location_, self.covariance_)

    def decision_function(self, X):
        return self.threshold_ - self.mahalanobis(X)    # > 0 inside, < 0 outside

    def predict(self, X):
        return np.where(self.decision_function(X) >= 0, 1, -1)

m = EllipticEnvelopeScratch(contamination=2/12).fit(X)
print(m.location_, m.threshold_.round(3))   # [2.75 2.5] 10.765
print(m.predict(X))                         # [ 1  1  1  1  1  1  1  1  1  1 -1 -1]
```

From scratch: exhaustive MCD (fine for tiny n), correction, reweighting and the contamination threshold. It reproduces scikit-learn's numbers on this data.

**Strengths**

- The normal data really are one roughly Gaussian blob: sensor readings, measurement errors, process-control variables.
- You want a transparent, nearly instant detector. The model is just μ, Σ and τ, and a score is one quadratic form.
- Correlations matter: it catches points that are ordinary in every single feature but unusual in combination.
- Features have different units: Mahalanobis distance is affine invariant, so no scaling is needed.
- The score has a calibrated meaning: under the Gaussian model, d² follows χ²_p, so a cut-off maps to a false-alarm rate.

**Limitations**

- The inliers are multimodal, curved or heavy-tailed: a single ellipse covers empty space and misses structure.
- High dimensions: Σ has p(p+1)/2 parameters, MCD needs n well above p, and the fit becomes slow and unstable.
- Features are discrete, binary or highly skewed, which makes the Gaussian assumption a poor fit.
- Contamination is unknown, since τ hinges on it; a wrong guess flags too many or too few.
- Features are collinear: Σ becomes singular and the distances blow up.

- **Industrial monitoring:** Multivariate statistical process control: a robust Hotelling T² (which is d²) on correlated sensor channels flags a drifting machine early.
- **Data cleaning:** Screen a dataset for gross errors and entry mistakes before fitting a regression or a classifier, especially when the features are correlated.
- **Finance & fraud:** Flag transactions or daily returns whose combination of features (amount, frequency, time) is unusual under the typical joint behaviour.
- **Novelty detection:** Fit on clean reference data, then score incoming points with `decision_function`, for example to catch out-of-distribution inputs to a deployed model.

> **Non-Gaussian and multimodal data:** The method sees only one centre and one covariance. With two clusters of normal behaviour, the best single ellipse either covers both, including the empty gap between them where an anomaly would go unnoticed, or MCD locks onto the larger cluster and flags the whole smaller one. Curved or ring-shaped data fail the same way. Look at the data first. For several modes use a Gaussian mixture, or density-based detectors such as Local Outlier Factor or Isolation Forest; transform skewed features (for example with a log) before you fit.

> **High dimensions:** The covariance has $p(p+1)/2$ entries to estimate. MCD needs its subset of $h\approx n/2$ points to have a non-singular covariance, so $n$ must exceed roughly $2p$; a common rule of thumb is $n>5p$. With fewer points the minimum determinant collapses toward 0, scikit-learn warns that the covariance matrix is not full rank, and the distances become meaningless. FastMCD also slows down quickly as $p$ grows. Reduce dimension first (PCA, feature selection), or switch to a method that does not estimate a full $\Sigma$.

> **Contamination is a guess, not a measurement:** With `contamination=0.1` (the default), $\tau$ is set so that about 10% of the training data are flagged, even if the data are perfectly clean. In the figure, raise contamination to 25% and the honest P8 turns rose alongside P11 and P12. If you do not know the rate, rank points by `score_samples` and inspect the top, or use a $\chi^2_{p,q}$ cut-off. For novelty detection, fit on clean data and choose $\tau$ on a validation set.

**Question:** In 2-D, a point has squared Mahalanobis distance d² = 9 under a Gaussian model. Roughly what fraction of genuine inliers lie at least this far out?

- A. About 0.3%, the same as a 3σ event in 1-D
- B. About 1.1%
- C. About 5%
- D. About 11%

**Answer:** B. About 1.1% — $d^2\sim\chi^2_2$, whose tail is $P(d^2>t)=e^{-t/2}$, so $e^{-4.5}\approx0.011$. The familiar 1-D figure of 0.3% for 3σ is wrong in 2-D, because two squared normals add up to larger values more easily than one.

**Question:** You refit an EllipticEnvelope with contamination raised from 0.05 to 0.2. What changes?

- A. Both the robust covariance and the threshold
- B. Only the threshold τ: the same ellipse is drawn smaller
- C. Only the covariance: the ellipse is reshaped around fewer points
- D. Nothing: contamination only matters at predict time

**Answer:** B. Only the threshold τ: the same ellipse is drawn smaller — The MCD fit (and its support_fraction) does not depend on contamination. Contamination only sets τ, the (1 − c) quantile of the training distances, so the ellipse keeps its centre, tilt and aspect ratio and shrinks.

**Question:** In the worked example the classical fit gives the outlier P12 a d² of only 4.45, while the robust fit gives 45.15. Why?

- A. The classical fit uses Euclidean distance
- B. P11 and P12 pulled the mean toward themselves and inflated the covariance in their direction, so their own distances shrank
- C. The robust fit ignores the correlation between features
- D. The classical fit standardizes the features first

**Answer:** B. P11 and P12 pulled the mean toward themselves and inflated the covariance in their direction, so their own distances shrank — This is masking. Included in the estimate, the outliers shift μ to (3.17, 2.29) and cut the correlation from 0.87 to 0.28, so moving against the trend (right and down) looks cheap. MCD estimates the shape from the tight core of ten points, where the correlation is strong.

**Question:** Under a fitted envelope, the major axis of the ellipse $d^2=\tau$ has length (end to end)…

- A. $2\sqrt{\tau\,\lambda_1}$, along the top eigenvector of Σ
- B. $2\tau\lambda_1$, along the top eigenvector of Σ
- C. $2\sqrt{\tau/\lambda_1}$, along the top eigenvector of Σ⁻¹
- D. $2\sqrt{\tau}$, along the first feature axis

**Answer:** A. $2\sqrt{\tau\,\lambda_1}$, along the top eigenvector of Σ — In eigen-coordinates $d^2=\sum_i u_i^2/\lambda_i$, so $d^2=\tau$ has semi-axes $\sqrt{\tau\lambda_i}$ along the eigenvectors $q_i$. In the example: $2\sqrt{7.378\cdot2.162}=7.99$ for the robust 97.5% ellipse.

**Key points**

- Model the inliers as one Gaussian: score each point by $d^2=(x-\mu)^\top\Sigma^{-1}(x-\mu)$ and flag it if $d^2>\tau$.
- For Gaussian inliers $d^2\sim\chi^2_p$. Its contours are ellipses with semi-axes $\sqrt{\tau\lambda_i}$ along the eigenvectors of $\Sigma$.
- `contamination` only sets $\tau$, as the $(1-c)$ quantile of the training $d^2$. The decision function is $f(x)=\tau-d^2(x)$, and its sign gives the label.
- Classical mean and covariance are masked by the outliers they should detect, and they can swamp honest points. EllipticEnvelope fits a robust MCD estimate instead (see Robust Covariance).
- Works for unimodal, roughly Gaussian data with n well above p. For multimodal, curved or high-dimensional data, use other detectors.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
