# MFCC

> Mel-frequency cepstral coefficients: the DCT-II of a log-mel frame, truncated to about 13 values, plus deltas and delta-deltas (39 dims). A compact, decorrelated spectral-envelope feature that dominated ASR before deep models switched to log-mels.

Source: https://www.mlfoundry.in/learn/mfcc  
Section: Deep Learning › Speech and Audio Deep Learning  
Level: core · 8 min read · updated 2026-10-15

**MFCC (mel-frequency cepstral coefficients):** are the first few coefficients of the discrete cosine transform of a frame's log mel-filterbank energies. Davis and Mermelstein ("Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences", IEEE Trans. ASSP, 1980) showed that they beat LPC-based features for word recognition. They became the default speech feature for the next three decades.

The word *cepstrum* (Bogert, Healy and Tukey, 1963) means "the spectrum of a log spectrum". Speech is roughly a source (glottal pulses at F0) filtered by the vocal tract. In the log spectrum the filter and the source **add**: a slow envelope (the formants) plus a fast ripple (the harmonics). A cosine transform separates slow from fast. Its low-order coefficients describe the envelope, which carries the phone identity. High orders carry the pitch ripple and noise, so they are dropped.

The input is a 25 ms frame; the output is about 13 numbers, versus 201–257 FFT bins or 40 log-mel energies. Classic recognisers modelled these vectors with Gaussian mixture HMMs using **diagonal** covariances, which only works if the dimensions are nearly uncorrelated. That is exactly what the DCT provides.

## The pipeline

> Figure: The MFCC recipe at 16 kHz. Row 1: pre-emphasis (y[n] = x[n] − 0.97·x[n−1]) flattens the spectral tilt, 400-sample Hamming frames are taken every 160 samples, and a 512-point FFT gives 257 power bins. Row 2 (right to left): 26–40 mel triangles, a log, and a DCT-II; 13 coefficients are kept and liftered. Row 3: cepstral mean normalisation, Δ and ΔΔ regression, and the 39-dim vector per 10 ms frame. Side note: in a real cepstrum the envelope sits at low quefrency and the pitch at 1/F0 (8.3 ms for 120 Hz).

$$
c_n=\sqrt{\tfrac{2-\delta_{n0}}{K}}\sum_{k=0}^{K-1}L_k\cos\!\Big(\frac{\pi n\,(k+\tfrac12)}{K}\Big),\quad n=0,\dots,12,\qquad L_k=\ln\big(M_k+\varepsilon\big)
$$

- $L_k$: log energy of mel filter $k$ in this frame, $K$ = 20–40 filters
- $c_n$: $n$-th cepstral coefficient; $c_0$ is proportional to the mean log energy, so it is often replaced by a separate energy term
- $\sqrt{\cdot}$: orthonormal scaling (`norm="ortho"`), so $\sum c_n^2=\sum L_k^2$ and keeping all $K$ coefficients rebuilds $L$ exactly

**Why the DCT decorrelates.** Neighbouring mel filters overlap by half and see the same formant, and overall loudness moves every band together. So the 40 log energies are strongly correlated. For such smooth, highly correlated vectors the DCT is close to the optimal decorrelating transform (the Karhunen–Loève transform, i.e. PCA), and it needs no training data. In 400 synthetic vowel frames with random formants, F0 and ±20 dB level, the mean absolute correlation between log-mel bands was 0.74 (0.94 between neighbours), but only 0.27 among $c_1,\dots,c_{13}$.

> **Real cepstrum vs MFCC:** The original real cepstrum is the inverse DFT of $\log|X(f)|$ over all FFT bins; a voiced frame shows a clear peak at quefrency $1/F_0$. MFCCs change two things: the log is taken of 20–40 mel energies, not of every bin, and a DCT replaces the inverse DFT. The mel pooling already smooths most harmonics away above about 1 kHz, so MFCCs keep less pitch information than the real cepstrum.

## Lifter, deltas and normalisation

$$
\tilde c_n=\Big(1+\frac{L}{2}\sin\frac{\pi n}{L}\Big)c_n,\qquad \Delta c_t=\frac{\sum_{m=1}^{M}m\,(c_{t+m}-c_{t-m})}{2\sum_{m=1}^{M}m^2}
$$

- $L$: lifter length, 22 in HTK and Kaldi. It boosts mid-order coefficients (×12 at $n=11$) so that all have similar variance; it does not change the information
- $\Delta c_t$: delta: least-squares slope of each coefficient over $\pm M$ frames ($M=2$ is usual). Applying it again gives $\Delta\Delta$. Static + Δ + ΔΔ = 13 × 3 = 39 dims

**Cepstral mean normalisation (CMN).** A fixed microphone or channel multiplies the spectrum by $G(f)$. After the log that becomes an additive constant, and after the DCT it is still a constant vector added to every frame. Subtracting the per-utterance mean of each coefficient removes it. Dividing by the standard deviation as well (CMVN) also equalises the scales.

**Common settings**

- **HTK-style recipe:** pre-emphasis 0.97, 25 ms Hamming / 10 ms, 26 filters, 12 cepstra + energy, lifter 22, + Δ + ΔΔ → 39
- **Kaldi defaults:** 23 mel bins, 13 cepstra (with $c_0$ replaced by log energy), lifter 22
- **librosa:** `librosa.feature.mfcc`: DCT-II (`norm="ortho"`) of the dB mel spectrogram, `n_mfcc`=20, no lifter by default
- **Dimensions:** 1 s of 16 kHz audio → 98–101 frames × 39 values, versus 16,000 samples

> Figure: Cepstral truncation on one frame of a synthetic vowel: 40 HTK log-mel energies (dots), their orthonormal DCT-II (bars), and the envelope rebuilt from the first K coefficients (warm line). Defaults (/ɑ/, F0 120 Hz, pre-emphasis, K = 13): the rebuilt envelope is within 2.13 dB RMS of the log-mel and keeps 97.5% of the energy in c1…c39. K = 4 gives 6.49 dB (76.4%); K = 40 is exact. Raise F0 to 300 Hz and K = 13 gives 5.64 dB (81.9%): sparse harmonics leave dips between mel bands that need high-order terms. The lifter multiplies c1 by 2.57 and c11 by 12.

Worked example: K = 4 log-mel energies L = (4, 2, 1, 1), orthonormal DCT-II, keep c0 and c1

| k | L_k | Rebuilt from c0, c1 | Error |
| --- | --- | --- | --- |
| 0 | 4 | 3.4571 | 0.5429 |
| 1 | 2 | 2.6036 | −0.6036 |
| 2 | 1 | 1.3964 | −0.3964 |
| 3 | 1 | 0.5429 | 0.4571 |

The coefficients are $c=(4,\;2.2304,\;1,\;0.1585)$. $c_0=\tfrac12\sum L_k=4$ is the scaled mean, and $c_1$ is the tilt from low to high bands. Energy is preserved: $\sum c_n^2=16+4.9749+1+0.0251=22=\sum L_k^2$. Keeping two coefficients retains $20.97/22=95.3\%$ of it, and the reconstruction is the smooth falling line in the table.

**A delta by hand (M = 2)**

1. One coefficient over frames $t-2,\dots,t+2$: $(1,\,2,\,4,\,5,\,5)$.
2. Numerator: $1\cdot(5-2)+2\cdot(5-1)=3+8=11$.
3. Denominator: $2\,(1^2+2^2)=10$, so $\Delta c_t=1.1$ per frame.

**Shapes for 1 s of 16 kHz speech**

1. $16{,}000$ samples; 400-sample frames every 160 → $\lfloor(16{,}000-400)/160\rfloor+1=98$ frames.
2. Each frame: 512-point FFT → 257 power bins → 40 mel energies → 40 log values.
3. DCT, keep 13 → $98\times13$; add Δ and ΔΔ → $98\times39=3{,}822$ numbers, versus $98\times40=3{,}920$ log-mels or 16,000 samples.

```python
import numpy as np

def mfcc(x, sr=16000, n_fft=512, win=400, hop=160, n_mels=40, n_ceps=13, lifter=22):
    x = np.append(x[0], x[1:] - 0.97 * x[:-1])                    # pre-emphasis
    frames = np.lib.stride_tricks.sliding_window_view(x, win)[::hop] * np.hamming(win)
    power = np.abs(np.fft.rfft(frames, n_fft)) ** 2               # (T, 257)
    mel = lambda f: 2595 * np.log10(1 + f / 700)
    pts = 700 * (10 ** (np.linspace(0, mel(sr / 2), n_mels + 2) / 2595) - 1)
    f = np.arange(n_fft // 2 + 1) * sr / n_fft
    H = np.maximum(0, np.minimum((f - pts[:-2, None]) / (pts[1:-1, None] - pts[:-2, None]),
                                 (pts[2:, None] - f) / (pts[2:, None] - pts[1:-1, None])))
    logmel = np.log(power @ H.T + 1e-10)                          # (T, 40)
    n, k = np.arange(n_ceps)[:, None], np.arange(n_mels)[None]
    dct = np.sqrt(np.where(n == 0, 1, 2) / n_mels) * np.cos(np.pi * n * (k + 0.5) / n_mels)
    c = logmel @ dct.T                                            # DCT-II, keep 13
    c *= 1 + (lifter / 2) * np.sin(np.pi * np.arange(n_ceps) / lifter)
    c -= c.mean(axis=0)                                           # CMN
    def delta(c, N=2):
        p = np.pad(c, ((N, N), (0, 0)), mode="edge")
        T = len(c)
        return sum(m * (p[N + m:N + m + T] - p[N - m:N - m + T])
                   for m in range(1, N + 1)) / (2 * sum(m * m for m in range(1, N + 1)))
    d = delta(c)
    return np.hstack([c, d, delta(d)]).T                          # (39, T)

t = np.arange(16000) / 16000
x = np.sin(2 * np.pi * 220 * t) * (1 + 0.5 * np.sin(2 * np.pi * 3 * t))
print(mfcc(x).shape)                                              # (39, 98)
# librosa.feature.mfcc(y=x, sr=16000, n_mfcc=13) is similar but not identical:
# Slaney filters, dB scaling, no pre-emphasis, lifter or CMN by default.
```

39-dim MFCCs in numpy: pre-emphasis, Hamming frames, HTK mel, DCT-II, lifter, CMN, deltas. Run with numpy 2.5.

> **Why deep models moved to log-mels:** The DCT is a fixed linear map, so a network's first layer can learn it, or something better, from data. Truncation throws information away, and convolutions and attention exploit the local frequency structure that the DCT scrambles. Mohamed, Hinton and Penn (2012) found that filterbank features beat MFCCs for deep-network acoustic models. Today ASR, TTS and audio tagging almost all use 80- or 128-band log-mels (see **Mel spectrograms**). MFCCs survive where models are small or Gaussian: speaker verification baselines, keyword spotting on microcontrollers, and quick features for classical classifiers.

> **MFCCs are not one standard:** Toolkits disagree on the mel scale, number of filters, log base (ln vs dB), DCT normalisation, $c_0$ vs energy, the lifter and pre-emphasis. Two "13-dim MFCC" files can differ by large factors per coefficient. Match the exact extractor a model or a paper used, and normalise (CMVN) per speaker or utterance.

**Strengths**

- Compact: 13 numbers per 10 ms, nearly decorrelated, ideal for diagonal-covariance GMMs
- Separates vocal-tract envelope from pitch; CMN removes fixed channel effects
- Cheap and training-free

**Limitations**

- Truncation discards detail that deep networks can use
- The DCT mixes all bands into each coefficient, so a local noise burst corrupts every one
- Not invertible to audio without the dropped coefficients and phase

- **Classic ASR:** GMM-HMM and hybrid DNN-HMM recognisers (HTK, Kaldi recipes), where 39-dim MFCCs with CMVN were the default input
- **Speaker recognition:** GMM-UBM and i-vector systems; MFCCs or log-mels remain common inputs to x-vector style embeddings
- **Small devices:** keyword spotting and simple sound classifiers where 13–40 numbers per frame fit a tiny model

**Question:** A recording passes through a microphone with a fixed frequency response. What happens to its MFCCs, and how is it removed?

- A. Every coefficient is multiplied by a constant; divide by the maximum
- B. A constant vector is added to every frame's coefficients; subtract the per-utterance mean (CMN)
- C. Only c0 changes; drop c0
- D. Nothing; MFCCs are channel-invariant

**Answer:** B. A constant vector is added to every frame's coefficients; subtract the per-utterance mean (CMN) — The channel multiplies the power spectrum by $G(f)$. The log turns that into $+\log G$, and the linear DCT turns it into a constant vector added to all frames. Subtracting the mean over the utterance cancels it (along with the average speech spectrum).

**Question:** Why do classic recognisers keep only about 13 of the 40 DCT coefficients?

- A. The DCT returns only 13 values
- B. Low orders describe the smooth envelope that identifies the phone; high orders mostly carry pitch harmonics and noise
- C. High-order coefficients are always zero
- D. To make the lifter work

**Answer:** B. Low orders describe the smooth envelope that identifies the phone; high orders mostly carry pitch harmonics and noise — The envelope (formants) varies slowly across bands and is captured by low cosine frequencies. Pitch makes faster ripple across bands, which lands in higher orders and varies with the speaker, not the phone. In the figure, 13 coefficients rebuild the /ɑ/ envelope to 2.13 dB RMS.

**Key points**

- MFCC = DCT-II of log mel energies, keep $c_0\dots c_{12}$; Davis and Mermelstein (1980).
- Pipeline: pre-emphasis 0.97 → 25 ms / 10 ms Hamming frames → |FFT|² → mel → log → DCT → lifter 22 → CMN → Δ, ΔΔ → 39 dims.
- The log makes source and filter additive; low cepstral orders = envelope (formants), high orders = pitch ripple.
- The DCT approximates PCA for correlated log-mel bands, which diagonal-covariance GMM-HMMs needed.
- Deep models learn from 80/128 log-mels instead; MFCCs remain for small, classical or Gaussian models.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
