Definition
Autoencoder anomaly detection trains a neural network to copy its input to its output through a narrow bottleneck, using only normal data. The network can only afford to remember the structure normal data shares, so normal inputs come back almost unchanged while unusual ones come back distorted. The reconstruction error is the anomaly score, and a threshold on it gives the alarm.
At a glance
- Task
- Unsupervised (or semi-supervised) anomaly / novelty detection
- Input → output
- → score , and a flag
- Learns
- an encoder and a decoder , with : in effect a -dimensional surface (manifold) that normal data lies near
- Training
- gradient descent (backprop) on the mean squared reconstruction error of normal examples; no anomaly labels needed
- Prediction
- one forward pass per point, then compare its error to the threshold
- Hyperparameters
- bottleneck size , depth/width, activation, regularization (weight decay, dropout, noise), epochs, the threshold
- Assumes
- the training set is (almost) all normal, and normal data occupies a lower-dimensional structure that anomalies fall off
The problem
Anomalies are rare and varied. A card-fraud team might have a few hundred confirmed frauds among millions of transactions, and next month's fraud will look different from last month's. A classifier trained on labelled anomalies only learns the kinds it has already seen. What we do have in abundance is normal data. So instead of describing anomalies, we describe normality, as tightly as we can, and flag whatever does not fit.
Density models (Gaussians, elliptic envelopes) do this well in a few dimensions, but they struggle with images, spectra or hundreds of correlated sensor channels. There, normal data usually lies near a curved, low-dimensional surface inside the huge input space: neighbouring pixels move together, and sensor readings are tied together by physics. An autoencoder learns that surface directly.
The idea
Force the data through a narrow pipe. The encoder squeezes each -dimensional input into a code with only numbers; the decoder rebuilds the input from that code. Training minimizes the rebuilding error on normal data, so the code is pushed to keep exactly the directions in which normal points differ from each other, and to drop everything else.
An anomaly differs from normal data in the dropped directions: that is what makes it unusual. The encoder has no slot for that difference, so the decoder rebuilds the nearest normal-looking point instead. The gap between the input and that rebuilt point is the score. Geometrically, is a point on the learned manifold and is (roughly) how far sits from it.
Vocabulary
- Encoder
- , maps an input to its code
- Code / latent / bottleneck
- , the numbers that must carry everything the decoder gets to see
- Decoder
- , maps a code back to input space; its image is the learned manifold
- Reconstruction
- Reconstruction error
- ; averaged over the training set it is the loss, and for one point it is the anomaly score
- Undercomplete
- : the bottleneck itself prevents copying. Overcomplete autoencoders () need other regularization or they learn the identity
- Threshold τ
- the error above which a point is called an anomaly; set from the error distribution of normal data
The math
We use a single hidden (code) layer, which is all the worked example needs; deep autoencoders just stack more layers on each side and backprop through them the same way.
- input (standardized features)
- encoder weights and bias
- activation, applied elementwise (identity for a linear autoencoder; tanh or ReLU otherwise)
- the code, with
- decoder weights and bias
- reconstruction of x
Training minimizes the mean squared reconstruction error over the normal training points:
- all parameters
- squared reconstruction error of point ; the same quantity is the anomaly score
Some libraries divide by as well ("MSE per feature"). That rescales every score by the same constant, so it changes the threshold's units but never which points are flagged.
Backprop for the tiny network
For one point, write the residual and the encoder's pre-activation . The chain rule, applied from the loss backwards, gives every gradient in four lines:
- residual: where the reconstruction misses
- outer product, a matrix like
- error signal at the code layer: the residual sent back through the decoder weights
- elementwise product
- activation slope; equals 1 for a linear autoencoder
The batch gradient of is the average of these per-point gradients, and each step updates with learning rate . Notice that the decoder is told where the output missed (), and the encoder is told how its code should have moved ().
Scoring and the threshold
- anomaly score: the reconstruction error of the trained network
- decision threshold
- the -th percentile (e.g. 95th or 99th) of the scores of normal points
The percentile is a direct dial on the false-alarm rate: at the 99th percentile about 1% of normal traffic will be flagged. Training errors are optimistic, because the network has fitted those exact points, so it is better to compute the percentile on a held-out set of normal data. If a validation set with some labelled anomalies exists, pick to optimize what you care about: recall at a fixed false-positive rate, F1, or expected cost (see precision, recall and ROC). Rules such as mean standard deviations are common too, but error distributions are skewed, so percentiles are safer.
A linear autoencoder is PCA
Take = identity. Then : the reconstruction is an affine map of rank at most , so all reconstructions lie in a -dimensional flat. Minimizing squared error over rank- maps is exactly the problem PCA solves, so (Baldi and Hornik, 1989) at the global minimum:
- mean of the training data; the biases learn to pass the flat through it
- the top- eigenvectors of the covariance matrix (principal components)
- eigenvalues of the covariance; the minimum loss is the variance in the dropped directions
The span of equals the principal subspace, but the weights themselves need not be orthonormal (any invertible mixing , gives the same reconstruction). The anomaly score is then the squared distance to the principal subspace, which process engineers have used for decades as the SPE or Q-statistic. So a linear autoencoder is just a slow way to compute PCA; the reason to train a network is the nonlinear , which lets the manifold bend.
Worked example: a 2 → 1 → 2 autoencoder
Eight normal points lie near the line , a 1-D manifold in 2-D. We train a linear autoencoder with a one-number bottleneck, then score four new points. Every number below was computed with NumPy and matches the interactive figure.
| Point | x | z | x̂ | error ‖x − x̂‖² |
|---|---|---|---|---|
| A | (−3, −1.6) | −3.211 | (−3.046, −1.517) | 0.0090 |
| B | (−2, −0.8) | −2.022 | (−1.917, −0.952) | 0.0301 |
| C | (−1, −0.6) | −1.104 | (−1.046, −0.517) | 0.0090 |
| D | (0, 0.2) | 0.085 | (0.083, 0.048) | 0.0301 |
| E | (0.5, 0.1) | 0.454 | (0.433, 0.223) | 0.0196 |
| F | (1, 0.4) | 1.004 | (0.954, 0.483) | 0.0090 |
| G | (2, 1.2) | 2.193 | (2.083, 1.048) | 0.0301 |
| H | (3, 1.4) | 3.111 | (2.954, 1.483) | 0.0090 |
Initialization (fixed, deliberately poor): , , , . The decoder line slopes down while the data slopes up, and the mean error is .
| Quantity | Formula | Value |
|---|---|---|
| code | 0.88 | |
| reconstruction | (0.528, −0.176) | |
| residual | (−1.472, −1.376) | |
| error | 4.0602 | |
| decoder weight grad | (−2.5907, −2.4218) | |
| decoder bias grad | (−2.944, −2.752) | |
| code error signal | −1.216 | |
| encoder weight grad | (−2.432, −1.4592) | |
| encoder bias grad | −1.216 | |
| after one step | , , , | |
| new error on G | , | 2.2079 (was 4.0602) |
Every gradient is negative, so every weight grows: the step stretches the decoder towards G and tilts it upward ('s second entry goes from to ). Real training averages the same gradients over all eight points per step. With full-batch gradient descent, , 1000 epochs:
| Epoch | Mean error J | Decoder slope | Encoder–decoder angle |
|---|---|---|---|
| 0 | 3.2472 | −0.333 | 81.9° |
| 1 | 2.4651 | −0.167 | 64.5° |
| 3 | 0.7221 | 0.129 | 38.3° |
| 5 | 0.1172 | 0.330 | 24.5° |
| 10 | 0.0205 | 0.484 | 16.4° |
| 100 | 0.0192 | 0.501 | 12.9° |
| 1000 | 0.0182 | 0.500 | 2.0° |
Final weights: , , , . Check against PCA: the data mean is , the covariance has eigenvalues and (the first component keeps 99.6% of the variance), and the first eigenvector has slope . The autoencoder's line has slope and passes through the mean, and its loss is within 0.2% of PCA's optimum . The per-point errors also match PCA's to within .
Threshold. The 95th percentile of the eight training errors is . (With only eight points that is essentially the maximum; with real data use thousands of held-out normal points.) Now score new points:
| Point | x | z | x̂ | score | score / τ | verdict |
|---|---|---|---|---|---|---|
| N | (1.5, 0.8) | 1.598 | (1.519, 0.766) | 0.0015 | 0.05 | normal |
| P | (0.5, 0.9) | 0.816 | (0.776, 0.394) | 0.332 | 11 | anomaly (subtle) |
| Q | (−1, 1.5) | −0.155 | (−0.146, −0.067) | 3.184 | 106 | anomaly |
| R | (2, −1) | 1.199 | (1.140, 0.576) | 3.224 | 107 | anomaly |
N lies on the manifold and reconstructs better than most training points. P, Q and R have ordinary-looking coordinates one at a time: their and values are all inside the training range. What gives them away is the combination, the component perpendicular to the line, which the one-number code cannot carry. The decoder snaps each one back onto the line, and the gap is the score.
- 1init We=(.2,.4) be=0 Wd=(.6,−.2) bd=0
- 2for epoch = 1 … 1000:
- 3 z = We·x + be # encode → 1 number
- 4 x̂ = Wd·z + bd # decode → back to 2-D
- 5 L = mean ‖x̂ − x‖²
- 6 backprop ∂L/∂Wd, ∂L/∂bd, ∂L/∂We, ∂L/∂be
- 7 θ ← θ − η·∇θL (η = 0.05)
- 8τ = 95th percentile of training errors
Fixed start. The decoder line {bd + t·Wd} points along Wd = (0.60, −0.20): the wrong way, so every reconstruction lands far from its point: the mean error is 3.247.
The algorithm, step by step
Autoencoder anomaly detection
- 1Collect normal data and remove known anomalies. Split it into train and a held-out normal set (plus a labelled validation set, if you have one).
- 2Scale each feature using statistics from the training split only (standardize, or min-max for images).
- 3Choose an architecture with a real bottleneck: well below . Start small; MLP for tabular data, convolutional for images, LSTM/1-D conv/transformer for windows of a time series.
- 4Train on normal data to minimize mean reconstruction error, with early stopping on the held-out normal loss.
- 5Set the threshold: a high percentile of the held-out normal scores, or the value that best trades recall against false alarms on the labelled validation set.
- 6Score each new input with one forward pass; flag . Inspect the per-feature residuals to see what is abnormal.
- 7Monitor: recompute the score distribution on recent normal traffic and retrain when normal behaviour drifts.
In code
1import numpy as np23X = np.array([[-3,-1.6],[-2,-0.8],[-1,-0.6],[0,0.2],[0.5,0.1],[1,0.4],[2,1.2],[3,1.4]])4We, be = np.array([0.2, 0.4]), 0.0 # encoder 2 -> 1 (linear)5Wd, bd = np.array([0.6, -0.2]), np.zeros(2) # decoder 1 -> 26eta = 0.0578for epoch in range(1000):9 Z = X @ We + be # codes, shape (n,)10 Xh = np.outer(Z, Wd) + bd # reconstructions, shape (n, 2)11 R = Xh - X # residuals12 D = 2 * R @ Wd # delta at the code layer13 Wd -= eta * 2 * (R * Z[:, None]).mean(0)14 bd -= eta * 2 * R.mean(0)15 We -= eta * (D[:, None] * X).mean(0)16 be -= eta * D.mean()1718score = lambda x: (((x @ We + be)[..., None] * Wd + bd - x) ** 2).sum(-1)19tau = np.percentile(score(X), 95) # 0.030120test = np.array([[1.5, 0.8], [0.5, 0.9], [-1, 1.5], [2, -1]])21print(score(test).round(4), score(test) > tau) # [0.0015 0.332 3.1841 3.2241] [F T T T]1import numpy as np, torch, torch.nn as nn2from sklearn.preprocessing import StandardScaler34# X_normal: (n, d) normal rows only. X_new: rows to score.5X_tr, X_val = X_normal[:int(.8 * len(X_normal))], X_normal[int(.8 * len(X_normal)):]6scaler = StandardScaler().fit(X_tr) # fit on training normals only7t = lambda a: torch.tensor(scaler.transform(a), dtype=torch.float32)8Xtr, Xval = t(X_tr), t(X_val)910d, k = Xtr.shape[1], 411model = nn.Sequential(12 nn.Linear(d, 32), nn.ReLU(), nn.Linear(32, k), # encoder -> bottleneck13 nn.Linear(k, 32), nn.ReLU(), nn.Linear(32, d), # decoder14)15opt = torch.optim.Adam(model.parameters(), lr=1e-3, weight_decay=1e-5)16loader = torch.utils.data.DataLoader(Xtr, batch_size=256, shuffle=True)1718def scores(x):19 model.eval()20 with torch.no_grad():21 return ((model(x) - x) ** 2).mean(dim=1).numpy() # per-row MSE = anomaly score2223best, patience = np.inf, 024for epoch in range(200):25 model.train()26 for xb in loader:27 loss = ((model(xb) - xb) ** 2).mean()28 opt.zero_grad(); loss.backward(); opt.step()29 val = scores(Xval).mean() # early stopping on normal data30 if val < best - 1e-5: best, patience, state = val, 0, model.state_dict()31 elif (patience := patience + 1) >= 10: break32model.load_state_dict(state)3334tau = np.percentile(scores(Xval), 99) # ~1% false alarms on normals35s_new = scores(t(X_new))36is_anomaly = s_new > tauUse when
- +You have lots of normal data and few or no labelled anomalies.
- +Data is high-dimensional and structured (images, spectra, multichannel sensors, sequences), where density estimates break down.
- +You want an explanation: per-feature or per-pixel residuals show where the input disagrees with normal.
- +Scoring must be fast: one forward pass, easy to deploy on GPUs or edge devices.
Avoid when
- −Tabular data with a few dozen features: Isolation Forest or PCA are usually as good, much cheaper and need less tuning.
- −The training data may be contaminated with anomalies: the network learns to reconstruct them too.
- −You need a calibrated probability: the score is an error in feature units, not a likelihood.
- −Hyperparameters (bottleneck, depth, epochs) are hard to choose without some labelled anomalies to validate on.
Where it's used
Fraud and abuse
Reconstruct transaction or session features; unusual combinations of amount, merchant, time and device reconstruct badly.
Industrial sensors
Reconstruct windows of vibration, temperature and pressure signals; a rising error warns of a failing bearing or pump before it breaks (predictive maintenance).
Visual inspection
A convolutional autoencoder trained on good parts; the per-pixel error map localizes scratches and dents (benchmarks such as MVTec AD).
Networks and IT
Intrusion detection from flow statistics; anomalous log or metric patterns in monitoring.
Medical
Flag unusual ECG beats or scans for review, trained on healthy examples.
Check yourself
An autoencoder trained on normal data is used for anomaly detection. What is the anomaly score?
Check yourself
A linear autoencoder with k code units is trained to its global minimum of squared error. What has it learned?
Check yourself
Your autoencoder reconstructs anomalies about as well as normal points. Which change is most likely to help?
Check yourself
In the worked example, why does P = (0.5, 0.9) get a score 11 times the threshold even though both of its coordinates are within the training range?
Remember
- ✓Train through a bottleneck on normal data only; score new points by .
- ✓Backprop: , , .
- ✓Set τ as a high percentile of scores on held-out normal data (it fixes the false-alarm rate), or tune it on labelled validation anomalies.
- ✓A linear autoencoder recovers PCA's subspace and the PCA reconstruction error; nonlinear activations let the manifold curve.
- ✓Keep the training data clean, the bottleneck real and the features scaled, or the detector will quietly learn to reconstruct anomalies.