Definition
An elliptic envelope is an outlier detector that assumes the normal data come from a single multivariate Gaussian. It estimates the centre and covariance robustly (so the outliers cannot drag them), measures every point's Mahalanobis distance from the centre, and flags the points outside an ellipse whose size is set by the expected fraction of outliers, the contamination.
At a glance
- Task
- Unsupervised outlier / novelty detection
- Input → output
- → a score and a label: inlier, outlier
- Learns
- a centre , a covariance (via the Minimum Covariance Determinant) and a threshold
- Decision rule
- outlier if
- Training
- FastMCD: roughly per C-step over a few hundred random starts
- Prediction
- per point: one quadratic form
- Hyperparameters
contamination(sets τ),support_fraction(size of the MCD subset)- Assumes
- inliers are roughly Gaussian, so one ellipsoid describes them; n comfortably larger than p
The idea
Picture a scatter of measurements from a healthy machine: temperature and vibration, say. They form a tilted, elongated cloud because the two rise together. A faulty reading is not necessarily far from the centre in plain distance; it is far relative to the shape of the cloud, for example high vibration at low temperature. The elliptic envelope captures that shape with a Gaussian, whose density contours are ellipses, and draws one ellipse that should contain the normal points. Anything outside it is an outlier.
Three questions make up the method: how to measure distance relative to the cloud (the Mahalanobis distance), how big to make the ellipse (the threshold), and how to estimate the cloud's shape when the outliers you are hunting are mixed into the data (a robust estimate).
Mahalanobis distance
- centre of the inlier cloud (mean vector)
- covariance matrix of the inliers
- squared Mahalanobis distance of x from the centre
With this is the ordinary squared Euclidean distance. In general rescales and de-correlates: a step along a direction in which the data vary a lot costs little, and a step along a direction in which they barely vary costs a lot. Because of that, the distance is affine invariant: rescale a feature, change units or rotate the axes, and every stays the same. Feature scaling makes no difference to it.
The geometry: ellipses from eigenvectors
Every set is an ellipse (an ellipsoid in dimensions). To see its shape, diagonalize the covariance:
- i-th eigenvector of Σ: the direction of an axis of the ellipse
- i-th eigenvalue: the variance of the data along that axis
In the rotated coordinates the contour is : an axis-aligned ellipse with semi-axis along . Large variance gives a long axis. Its area (volume in dimensions) grows with ; in 2-D it is . Raising scales the whole ellipse up without changing its shape or tilt.
Choosing the threshold: contamination
The fitted model gives every training point a distance. The threshold is set so that a chosen fraction of training points falls outside. That fraction is the contamination , your prior guess of how much of the data is anomalous:
- contamination: expected outlier fraction (scikit-learn default 0.1)
- decision function: positive inside the envelope, negative outside, zero on it
Prediction is the sign: means inlier (), means outlier (). In scikit-learn, score_samples(x) returns (higher means more normal), offset_ stores , and decision_function is score_samples − offset_ . Note that mahalanobis() returns the squared distance.
Why the plain mean and covariance fail: masking
The obvious estimates are the sample mean and covariance of all points. But the outliers are part of "all points". A cluster of outliers pulls toward itself and stretches in its own direction, so the ellipse grows to accommodate them, and their distances shrink. This is masking: the outliers hide themselves. The side effect is swamping: the distorted ellipse no longer follows the inliers, so some perfectly normal points end up looking extreme.
A single far-away point is enough to wreck the classical estimates: their breakdown point is . The fix is to estimate and from the points that look most like a tight Gaussian core. Scikit-learn's EllipticEnvelope uses the Minimum Covariance Determinant (MCD): among all subsets of points, find the one whose covariance has the smallest determinant, which is the smallest-volume ellipse holding half the data. How MCD is computed (FastMCD and its C-steps) and why it survives up to about 50% contamination is the topic of the Robust Covariance article. Here we use it as a black box.
EllipticEnvelope.fit(X), as scikit-learn does it
- 1Robust core. Run MCD with (or
support_fraction·). This gives a raw centre and covariance from the most concentrated points. - 2Consistency correction. The covariance of the tightest half underestimates the spread, so rescale it until the median of all points equals the median ( for ).
- 3Reweight. Keep every point with corrected and recompute an ordinary mean and covariance from those points. This step recovers efficiency: good points left out of the half-sample come back in.
- 4Threshold. Compute for every training point under the reweighted fit, then set to their quantile.
- 5Predict. For any , return the sign of .
Worked example: twelve points, two outliers
Ten inliers lie along a rising diagonal; two outliers, P11 and P12, sit together at the bottom right, off the trend. Every number below was checked against scikit-learn's EllipticEnvelope(contamination=2/12, random_state=0) and NumPy.
| Point | x₁ | x₂ | classical d² | robust d² | classical flag (τ = 4.46) | robust flag (τ = 10.76) |
|---|---|---|---|---|---|---|
| P1 | 1.0 | 1.5 | 2.540 | 3.004 | ||
| P2 | 1.5 | 1.0 | 2.305 | 2.246 | ||
| P3 | 2.0 | 2.5 | 0.910 | 1.814 | ||
| P4 | 2.5 | 2.0 | 0.252 | 0.403 | ||
| P5 | 3.0 | 3.5 | 1.538 | 2.543 | ||
| P6 | 3.5 | 2.5 | 0.076 | 1.814 | ||
| P7 | 3.0 | 2.5 | 0.073 | 0.202 | ||
| P8 | 4.5 | 4.5 | 4.571 | 3.887 | outlier (swamped) | |
| P9 | 4.5 | 3.5 | 1.764 | 3.004 | ||
| P10 | 2.0 | 1.5 | 0.997 | 1.084 | ||
| P11 | 5.0 | 1.0 | 4.523 | 45.777 | outlier | outlier |
| P12 | 5.5 | 1.5 | 4.451 | 45.154 | inlier (masked) | outlier |
| Quantity | Classical (all 12 points) | Robust (EllipticEnvelope) |
|---|---|---|
| μ | (3.167, 2.292) | (2.750, 2.500) |
| Σ | [[1.931, 0.410], [0.410, 1.102]] | [[1.263, 1.000], [1.000, 1.050]] |
| det Σ | 1.960 | 0.326 |
| Σ⁻¹ | [[0.562, −0.209], [−0.209, 0.985]] | [[3.225, −3.071], [−3.071, 3.877]] |
| correlation | 0.28 | 0.87 |
| eigenvalues λ₁, λ₂ | 2.099, 0.934 | 2.162, 0.151 |
| major axis angle | 22.3° | 42.0° |
| 97.5% ellipse semi-axes | 3.94 × 2.63 (area 32.5) | 3.99 × 1.05 (area 13.2) |
Where the robust numbers come from
- 1MCD subset. . Of the subsets, the one with the smallest covariance determinant () is {P1, P2, P3, P4, P6, P7, P9, P10}. Raw centre .
- 2Correction. The median raw over all 12 points is ; dividing by the median gives a factor of , which inflates the raw covariance.
- 3Reweight. With the corrected fit, P5 () and P8 () are under and come back in; P11 and P12 ( and ) stay out. The final and are the plain mean and covariance of P1 to P10: exactly the ten real inliers.
- 4Threshold. The sorted robust end . The quantile sits at position , so . Exactly P11 and P12 exceed it.
One distance by hand: P11 = (5.0, 1.0)
- 1Robust. . Then . The cross term is large and positive because P11 moves against the strong positive correlation: right, but down.
- 2Classical. . Then . With the correlation mostly gone, moving against it barely costs anything.
- 3Verdict. Robustly, P11 is "standard deviations" out, with a tail probability of about . Classically it is unremarkable: below the 97.5% cut-off of , and in fact the classical fit flags no point at all at that cut-off. That is masking in its purest form.
Now score two new points with the robust model. has and : deep inside. is only away from in plain distance, but it lies off the diagonal: and . It is still an inlier, but only just, and it would be flagged under the stricter cut-off. The decision function tells you more than the label does: how far inside or outside the boundary a point is.
The classical fit disagrees with the robust one: it misses P12 (masked — the outliers inflated Σ toward themselves) and it wrongly flags P8 (swamped — the shifted μ and weakened correlation make an honest point look extreme).
drag the points and the ◆ probe · ringed = raw MCD subset (h = 8) · solid = active envelope d² = τ · dashed = the other fit · faint rings d² = 1, 4
In code
1import numpy as np2from scipy.stats import chi23from sklearn.covariance import EllipticEnvelope45X = np.array([[1.0, 1.5], [1.5, 1.0], [2.0, 2.5], [2.5, 2.0], [3.0, 3.5], [3.5, 2.5],6 [3.0, 2.5], [4.5, 4.5], [4.5, 3.5], [2.0, 1.5], [5.0, 1.0], [5.5, 1.5]])78def mahalanobis_sq(X, mu, cov):9 diff = X - mu10 return np.einsum("ij,jk,ik->i", diff, np.linalg.inv(cov), diff)1112# Classical (non-robust) envelope: sample mean and MLE covariance of all 12 points13mu, cov = X.mean(axis=0), np.cov(X, rowvar=False, bias=True)14d2 = mahalanobis_sq(X, mu, cov)15print(d2.round(2)) # outliers P11, P12 get only 4.52, 4.4516print((d2 > chi2.ppf(0.975, df=2)).sum()) # 0 -> masked: nothing exceeds 7.381718# Robust envelope (MCD inside), threshold from contamination19ee = EllipticEnvelope(contamination=2/12, random_state=0).fit(X)20print(ee.location_, ee.covariance_) # [2.75 2.5] [[1.2625 1.] [1. 1.05]]21print(ee.mahalanobis(X).round(2)) # squared distances: P11 45.78, P12 45.1522print(-ee.offset_) # threshold tau = 10.7623print(ee.predict(X)) # -1 for P11 and P12 only24print(ee.decision_function([[3, 3], [4, 2]]).round(3)) # tau - d^2: [10.361 0.918]1import numpy as np2from itertools import combinations3from scipy.stats import chi245class EllipticEnvelopeScratch:6 def __init__(self, contamination=0.1):7 self.contamination = contamination89 def fit(self, X):10 n, p = X.shape11 h = int(np.ceil((n + p + 1) / 2))12 # 1. MCD: the h points whose covariance has the smallest determinant13 # (exhaustive here; FastMCD searches with random starts + C-steps)14 best = min(combinations(range(n), h),15 key=lambda s: np.linalg.det(np.cov(X[list(s)], rowvar=False, bias=True)))16 S = X[list(best)]17 mu, cov = S.mean(axis=0), np.cov(S, rowvar=False, bias=True)18 # 2. consistency: rescale so the median d^2 matches the chi2_p median19 d2 = mahalanobis_sq(X, mu, cov)20 cov = cov * np.median(d2) / chi2.ppf(0.5, p)21 # 3. reweight: refit on every point inside the 97.5% chi2 ellipse22 keep = mahalanobis_sq(X, mu, cov) < chi2.ppf(0.975, p)23 self.location_ = X[keep].mean(axis=0)24 self.covariance_ = np.cov(X[keep], rowvar=False, bias=True)25 # 4. threshold: the (1 - contamination) quantile of the training d^226 self.threshold_ = np.percentile(self.mahalanobis(X), 100 * (1 - self.contamination))27 return self2829 def mahalanobis(self, X):30 return mahalanobis_sq(np.asarray(X, float), self.location_, self.covariance_)3132 def decision_function(self, X):33 return self.threshold_ - self.mahalanobis(X) # > 0 inside, < 0 outside3435 def predict(self, X):36 return np.where(self.decision_function(X) >= 0, 1, -1)3738m = EllipticEnvelopeScratch(contamination=2/12).fit(X)39print(m.location_, m.threshold_.round(3)) # [2.75 2.5] 10.76540print(m.predict(X)) # [ 1 1 1 1 1 1 1 1 1 1 -1 -1]Use when
- +The normal data really are one roughly Gaussian blob: sensor readings, measurement errors, process-control variables.
- +You want a transparent, nearly instant detector. The model is just μ, Σ and τ, and a score is one quadratic form.
- +Correlations matter: it catches points that are ordinary in every single feature but unusual in combination.
- +Features have different units: Mahalanobis distance is affine invariant, so no scaling is needed.
- +The score has a calibrated meaning: under the Gaussian model, d² follows χ²_p, so a cut-off maps to a false-alarm rate.
Avoid when
- −The inliers are multimodal, curved or heavy-tailed: a single ellipse covers empty space and misses structure.
- −High dimensions: Σ has p(p+1)/2 parameters, MCD needs n well above p, and the fit becomes slow and unstable.
- −Features are discrete, binary or highly skewed, which makes the Gaussian assumption a poor fit.
- −Contamination is unknown, since τ hinges on it; a wrong guess flags too many or too few.
- −Features are collinear: Σ becomes singular and the distances blow up.
Where it's used
Industrial monitoring
Multivariate statistical process control: a robust Hotelling T² (which is d²) on correlated sensor channels flags a drifting machine early.
Data cleaning
Screen a dataset for gross errors and entry mistakes before fitting a regression or a classifier, especially when the features are correlated.
Finance & fraud
Flag transactions or daily returns whose combination of features (amount, frequency, time) is unusual under the typical joint behaviour.
Novelty detection
Fit on clean reference data, then score incoming points with decision_function, for example to catch out-of-distribution inputs to a deployed model.
Check yourself
In 2-D, a point has squared Mahalanobis distance d² = 9 under a Gaussian model. Roughly what fraction of genuine inliers lie at least this far out?
Check yourself
You refit an EllipticEnvelope with contamination raised from 0.05 to 0.2. What changes?
Check yourself
In the worked example the classical fit gives the outlier P12 a d² of only 4.45, while the robust fit gives 45.15. Why?
Check yourself
Under a fitted envelope, the major axis of the ellipse has length (end to end)…
Remember
- ✓Model the inliers as one Gaussian: score each point by and flag it if .
- ✓For Gaussian inliers . Its contours are ellipses with semi-axes along the eigenvectors of .
- ✓
contaminationonly sets , as the quantile of the training . The decision function is , and its sign gives the label. - ✓Classical mean and covariance are masked by the outliers they should detect, and they can swamp honest points. EllipticEnvelope fits a robust MCD estimate instead (see Robust Covariance).
- ✓Works for unimodal, roughly Gaussian data with n well above p. For multimodal, curved or high-dimensional data, use other detectors.