Definition
MFCC (mel-frequency cepstral coefficients) are the first few coefficients of the discrete cosine transform of a frame's log mel-filterbank energies. Davis and Mermelstein ("Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences", IEEE Trans. ASSP, 1980) showed that they beat LPC-based features for word recognition. They became the default speech feature for the next three decades.
The word cepstrum (Bogert, Healy and Tukey, 1963) means "the spectrum of a log spectrum". Speech is roughly a source (glottal pulses at F0) filtered by the vocal tract. In the log spectrum the filter and the source add: a slow envelope (the formants) plus a fast ripple (the harmonics). A cosine transform separates slow from fast. Its low-order coefficients describe the envelope, which carries the phone identity. High orders carry the pitch ripple and noise, so they are dropped.
The input is a 25 ms frame; the output is about 13 numbers, versus 201–257 FFT bins or 40 log-mel energies. Classic recognisers modelled these vectors with Gaussian mixture HMMs using diagonal covariances, which only works if the dimensions are nearly uncorrelated. That is exactly what the DCT provides.
The pipeline
- log energy of mel filter in this frame, = 20–40 filters
- -th cepstral coefficient; is proportional to the mean log energy, so it is often replaced by a separate energy term
- orthonormal scaling (
norm="ortho"), so and keeping all coefficients rebuilds exactly
Why the DCT decorrelates. Neighbouring mel filters overlap by half and see the same formant, and overall loudness moves every band together. So the 40 log energies are strongly correlated. For such smooth, highly correlated vectors the DCT is close to the optimal decorrelating transform (the Karhunen–Loève transform, i.e. PCA), and it needs no training data. In 400 synthetic vowel frames with random formants, F0 and ±20 dB level, the mean absolute correlation between log-mel bands was 0.74 (0.94 between neighbours), but only 0.27 among .
Lifter, deltas and normalisation
- lifter length, 22 in HTK and Kaldi. It boosts mid-order coefficients (×12 at ) so that all have similar variance; it does not change the information
- delta: least-squares slope of each coefficient over frames ( is usual). Applying it again gives . Static + Δ + ΔΔ = 13 × 3 = 39 dims
Cepstral mean normalisation (CMN). A fixed microphone or channel multiplies the spectrum by . After the log that becomes an additive constant, and after the DCT it is still a constant vector added to every frame. Subtracting the per-utterance mean of each coefficient removes it. Dividing by the standard deviation as well (CMVN) also equalises the scales.
Common settings
- HTK-style recipe
- pre-emphasis 0.97, 25 ms Hamming / 10 ms, 26 filters, 12 cepstra + energy, lifter 22, + Δ + ΔΔ → 39
- Kaldi defaults
- 23 mel bins, 13 cepstra (with replaced by log energy), lifter 22
- librosa
librosa.feature.mfcc: DCT-II (norm="ortho") of the dB mel spectrogram,n_mfcc=20, no lifter by default- Dimensions
- 1 s of 16 kHz audio → 98–101 frames × 39 values, versus 16,000 samples
| k | L_k | Rebuilt from c0, c1 | Error |
|---|---|---|---|
| 0 | 4 | 3.4571 | 0.5429 |
| 1 | 2 | 2.6036 | −0.6036 |
| 2 | 1 | 1.3964 | −0.3964 |
| 3 | 1 | 0.5429 | 0.4571 |
The coefficients are . is the scaled mean, and is the tilt from low to high bands. Energy is preserved: . Keeping two coefficients retains of it, and the reconstruction is the smooth falling line in the table.
A delta by hand (M = 2)
- 1One coefficient over frames : .
- 2Numerator: .
- 3Denominator: , so per frame.
Shapes for 1 s of 16 kHz speech
- 1 samples; 400-sample frames every 160 → frames.
- 2Each frame: 512-point FFT → 257 power bins → 40 mel energies → 40 log values.
- 3DCT, keep 13 → ; add Δ and ΔΔ → numbers, versus log-mels or 16,000 samples.
1import numpy as np23def mfcc(x, sr=16000, n_fft=512, win=400, hop=160, n_mels=40, n_ceps=13, lifter=22):4 x = np.append(x[0], x[1:] - 0.97 * x[:-1]) # pre-emphasis5 frames = np.lib.stride_tricks.sliding_window_view(x, win)[::hop] * np.hamming(win)6 power = np.abs(np.fft.rfft(frames, n_fft)) ** 2 # (T, 257)7 mel = lambda f: 2595 * np.log10(1 + f / 700)8 pts = 700 * (10 ** (np.linspace(0, mel(sr / 2), n_mels + 2) / 2595) - 1)9 f = np.arange(n_fft // 2 + 1) * sr / n_fft10 H = np.maximum(0, np.minimum((f - pts[:-2, None]) / (pts[1:-1, None] - pts[:-2, None]),11 (pts[2:, None] - f) / (pts[2:, None] - pts[1:-1, None])))12 logmel = np.log(power @ H.T + 1e-10) # (T, 40)13 n, k = np.arange(n_ceps)[:, None], np.arange(n_mels)[None]14 dct = np.sqrt(np.where(n == 0, 1, 2) / n_mels) * np.cos(np.pi * n * (k + 0.5) / n_mels)15 c = logmel @ dct.T # DCT-II, keep 1316 c *= 1 + (lifter / 2) * np.sin(np.pi * np.arange(n_ceps) / lifter)17 c -= c.mean(axis=0) # CMN18 def delta(c, N=2):19 p = np.pad(c, ((N, N), (0, 0)), mode="edge")20 T = len(c)21 return sum(m * (p[N + m:N + m + T] - p[N - m:N - m + T])22 for m in range(1, N + 1)) / (2 * sum(m * m for m in range(1, N + 1)))23 d = delta(c)24 return np.hstack([c, d, delta(d)]).T # (39, T)2526t = np.arange(16000) / 1600027x = np.sin(2 * np.pi * 220 * t) * (1 + 0.5 * np.sin(2 * np.pi * 3 * t))28print(mfcc(x).shape) # (39, 98)29# librosa.feature.mfcc(y=x, sr=16000, n_mfcc=13) is similar but not identical:30# Slaney filters, dB scaling, no pre-emphasis, lifter or CMN by default.Use when
- +Compact: 13 numbers per 10 ms, nearly decorrelated, ideal for diagonal-covariance GMMs
- +Separates vocal-tract envelope from pitch; CMN removes fixed channel effects
- +Cheap and training-free
Avoid when
- −Truncation discards detail that deep networks can use
- −The DCT mixes all bands into each coefficient, so a local noise burst corrupts every one
- −Not invertible to audio without the dropped coefficients and phase
Where it's used
Classic ASR
GMM-HMM and hybrid DNN-HMM recognisers (HTK, Kaldi recipes), where 39-dim MFCCs with CMVN were the default input
Speaker recognition
GMM-UBM and i-vector systems; MFCCs or log-mels remain common inputs to x-vector style embeddings
Small devices
keyword spotting and simple sound classifiers where 13–40 numbers per frame fit a tiny model
Check yourself
A recording passes through a microphone with a fixed frequency response. What happens to its MFCCs, and how is it removed?
Check yourself
Why do classic recognisers keep only about 13 of the 40 DCT coefficients?
Remember
- ✓MFCC = DCT-II of log mel energies, keep ; Davis and Mermelstein (1980).
- ✓Pipeline: pre-emphasis 0.97 → 25 ms / 10 ms Hamming frames → |FFT|² → mel → log → DCT → lifter 22 → CMN → Δ, ΔΔ → 39 dims.
- ✓The log makes source and filter additive; low cepstral orders = envelope (formants), high orders = pitch ripple.
- ✓The DCT approximates PCA for correlated log-mel bands, which diagonal-covariance GMM-HMMs needed.
- ✓Deep models learn from 80/128 log-mels instead; MFCCs remain for small, classical or Gaussian models.