# Short-time Fourier transform (STFT)

> The STFT applies a DFT to overlapping windowed frames (n_fft 400, hop 160 at 16 kHz); window length trades time for frequency resolution, overlap-add inverts it exactly, and Griffin–Lim recovers a phase from magnitude.

Source: https://www.mlfoundry.in/learn/short-time-fourier-transform-stft  
Section: Deep Learning › Speech and Audio Deep Learning  
Level: core · 8 min read · updated 2026-10-15

**Short-time Fourier transform:** is the sequence of DFTs of short, overlapping, windowed frames of a signal, indexed by frame (time) and bin (frequency). Its magnitude is the **spectrogram**, the standard input of speech and audio models. The time–frequency view goes back to Gabor, "Theory of Communication" (1946). The analysis–synthesis framework was set out by Allen and Rabiner (Proc. IEEE, 1977). Griffin and Lim, "Signal Estimation from Modified Short-Time Fourier Transform" (IEEE Trans. ASSP, 1984), showed how to rebuild a signal from magnitudes alone.

Speech and music change over time. One **Fourier transform** of a whole utterance gives its overall frequency content but says nothing about *when* each frequency occurs. A rising pitch and a falling pitch can have the same spectrum. The STFT applies the transform locally: over 20–40 ms speech is close to stationary, so a frame's spectrum describes one phone-sized slice.

## Definition and parameters

$$
S[k,m]=\sum_{n=0}^{N-1}x[n+mH]\,w[n]\,e^{-j2\pi kn/N},\qquad k=0,\dots,\tfrac{N}{2},\quad m=0,\dots,T-1
$$

- $N$: `n_fft`: FFT size; gives $N/2+1$ bins spaced $f_s/N$
- $w$: window of `win_length` $\le N$ samples (zero-padded to $N$ in the centre), usually periodic Hann
- $H$: `hop_length`: samples between frame starts; the frame rate is $f_s/H$
- $m$: frame index; with centring, frame $m$ is centred on sample $mH$

The speech default is a 25 ms window every 10 ms. At 16 kHz this is `n_fft = win_length = 400` and `hop = 160`, giving 201 bins 40 Hz apart and 100 frames per second; consecutive frames overlap by 240 samples (60%). With centring (`center=True` in torch and librosa) the signal is padded by $N/2$ on both sides, so the frame count depends only on the hop:

$$
T_{\text{centre}}=1+\Big\lfloor\frac{L}{H}\Big\rfloor,\qquad T_{\text{no centre}}=1+\Big\lfloor\frac{L-N}{H}\Big\rfloor
$$

- $L$: signal length in samples; 1 s at 16 kHz with $N=400$, $H=160$ gives 101 or 98 frames

> Figure: The STFT at Whisper-style settings. Top: the 16 kHz signal is padded by n_fft/2 = 200 samples and cut into 400-sample Hann windows every 160 samples; frame 3 covers samples 280–679 and is centred on sample 480. Middle: each frame becomes one column of 201 complex bins, so 1 s gives a 201 × 101 matrix (98 frames without centring). Bottom: overlap-add with window normalisation inverts a complex STFT exactly; Griffin–Lim estimates a phase when only |S| is known.

STFT settings used by well-known models and libraries

| Setting | fs | n_fft / win | hop | Notes |
| --- | --- | --- | --- | --- |
| ASR front end (Kaldi-style, Whisper) | 16 kHz | 400 / 400 (25 ms) | 160 (10 ms) | Whisper: Hann, 80 mel bands (128 in large-v3), 30 s → 3000 frames |
| Tacotron 2 | 24 kHz | 50 ms window | 12.5 ms | 80 mel bands, 125 Hz – 7.6 kHz |
| HiFi-GAN (LJSpeech) | 22.05 kHz | 1024 / 1024 (46.4 ms) | 256 (11.6 ms) | 80 mel bands, fmax 8 kHz |
| librosa defaults | any | 2048 / 2048 | 512 | hop = win_length / 4 |
| torch.stft defaults | any | n_fft required | n_fft // 4 | window = None means rectangular |

## The time–frequency trade-off

Bin spacing is $f_s/N$ and the window lasts $N/f_s$, so their product is always 1. A long window separates close partials but smears events in time. A short window localises clicks and onsets but blurs harmonics. Gabor's uncertainty relation makes this a law rather than an engineering limit. For the RMS widths of any window in time and in frequency, $\sigma_t\,\sigma_f\ge 1/(4\pi)$, with equality only for a Gaussian.

Speech settings sit in between. A 25 ms window holds 2–3 pitch periods of a 100 Hz voice and resolves formants, and a 10 ms hop follows changes between phones. Music analysis often uses longer windows (2048 samples at 44.1 kHz = 46 ms) to resolve notes.

> Figure: Spectrogram of 0.5 s at 8 kHz: a 2000 → 3800 Hz chirp, tones at 600 and 700 Hz, and two clicks 6 ms apart (hop 25, centred, 161 frames). Default (Hann, 256 samples = 32 ms): Δf = 31.25 Hz, the tones are 3.2 bins apart and resolved with an 11.6 dB dip, but the clicks merge into one stripe. At 64 samples (8 ms): Δf = 125 Hz, the tones merge, and the clicks separate with a 4.9 dB dip. At 128 neither pair is resolved. At 512 the tone dip grows to 27.5 dB, while the chirp sweeps 230.4 Hz within one window.

## Inversion: overlap-add

Each column can be inverted with an IFFT, which returns the windowed frame. Multiplying again by a synthesis window, adding the frames at their offsets and dividing by the summed squared window recovers $x$ exactly:

$$
\hat x[n]=\frac{\sum_m w[n-mH]\;y_m[n-mH]}{\sum_m w^2[n-mH]},\qquad y_m=\mathrm{IFFT}(S[\cdot,m])
$$

- $\sum_m w^2[n-mH]$: window envelope; must never be 0 (the NOLA condition)
- $\text{COLA}$: stronger: $\sum_m w[n-mH]$ constant. Periodic Hann sums to 1 at $H=N/2$ and to 2 at $H=N/4$

The speech setting 400 / 160 is *not* COLA for Hann: the window sum ripples between 1.19 and 1.31. It is NOLA, so `torch.istft` still inverts it exactly, because it divides by the envelope.

## Magnitude, power and the missing phase

Models usually take the magnitude $|S|$ or the power $|S|^2$, then a mel filterbank and a log; see **Mel spectrograms**. In dB the two are identical, $10\log_{10}|S|^2 = 20\log_{10}|S|$; with a natural log, log power is twice log magnitude. Dropping phase halves the data, and the magnitude is stable under small shifts of the frame.

To produce audio from a magnitude, a phase must be invented. **Griffin–Lim** starts from random phase and alternates: ISTFT to a signal, STFT again, keep the new phase, and restore the target magnitude. Each iteration does not increase $\sum\big(|\mathrm{STFT}(x)|-|S|\big)^2$. Tacotron (2017) used it as its vocoder. Its metallic, phasey artefacts are why **WaveNet**, **WaveRNN** and **HiFi-GAN** vocoders replaced it.

## Worked example: shapes at 16 kHz, n_fft 400, hop 160

Frame counts and sizes (centre padding unless stated)

| Input | Frames T | Spectrogram | Note |
| --- | --- | --- | --- |
| 1 s (16,000 samples) | 1 + ⌊16000/160⌋ = 101 | 201 × 101 complex = 20,301 | frame rate 100 per second |
| 1 s, center=False | 1 + ⌊15600/160⌋ = 98 | 201 × 98 | frames end at sample 15,920; the last 80 samples are in no frame |
| 2 s | 1 + 32000/160 = 201 | 201 × 201 | T grows by exactly 100 per second |
| 30 s Whisper window | 1 + 480000/160 = 3001 | Whisper drops the last: 80 mel × 3000 | encoder conv stride 2 → 1500 |
| Redundancy | 201 complex per 160 new samples | 402 reals / 160 = 2.51× | so not every array is a valid STFT |

```python
import numpy as np

def stft(x, n_fft=400, hop=160):
    """Like torch.stft(center=True, pad_mode='reflect', window=hann_window(n_fft))."""
    w = 0.5 - 0.5 * np.cos(2 * np.pi * np.arange(n_fft) / n_fft)        # periodic Hann
    xp = np.pad(x, n_fft // 2, mode="reflect")
    frames = np.lib.stride_tricks.sliding_window_view(xp, n_fft)[::hop]   # (T, n_fft)
    return np.fft.rfft(frames * w, axis=1).T, w                          # (n_fft/2 + 1, T)

def istft(S, w, hop, length):
    """Overlap-add with window-envelope normalisation (needs NOLA, not COLA)."""
    n_fft = len(w)
    frames = np.fft.irfft(S.T, n=n_fft, axis=1) * w
    out = np.zeros(n_fft + hop * (len(frames) - 1))
    env = np.zeros_like(out)
    for m, f in enumerate(frames):
        out[m * hop:m * hop + n_fft] += f
        env[m * hop:m * hop + n_fft] += w ** 2
    s = slice(n_fft // 2, n_fft // 2 + length)                          # drop the centre padding
    return out[s] / env[s]

fs = 16_000
t = np.arange(fs) / fs
x = np.sin(2 * np.pi * (300 * t + 1_500 * t ** 2))                     # chirp 300 -> 3300 Hz in 1 s
S, w = stft(x)
print(S.shape)                                                          # (201, 101)
power_db = 10 * np.log10(np.abs(S) ** 2 + 1e-10)                        # power spectrogram in dB
print(power_db[:, 50].argmax() * fs / 400, "Hz")                        # 1800.0 at t = 0.5 s
print(np.abs(istft(S, w, 160, len(x)) - x).max())                       # ~9e-16: exact inversion
```

STFT and ISTFT from scratch in numpy, matching torch.stft(center=True, window=hann_window(400)) (runs as is)

> **torch.stft's default window is rectangular:** If `window` is omitted, `torch.stft` uses all ones and warns, giving −13 dB sidelobes instead of Hann's −31 dB. Always pass `window=torch.hann_window(win_length)`, and the same window to `torch.istft`. Also check `center` and `pad_mode`: they change the frame count and the first frames, and a model trained with one setting will see shifted features with another.

> **Not every spectrogram is an STFT:** Overlapping frames make the STFT about 2.5× redundant at 400 / 160, so neighbouring columns must agree. An arbitrary magnitude array, such as one predicted by an acoustic model, usually has no signal whose STFT matches it exactly. Griffin–Lim finds a signal whose STFT magnitude is close to it. Neural vocoders learn to produce a waveform whose spectrogram is close to it.

**Question:** A 2 s clip at 16 kHz goes through torch.stft(n_fft=400, hop_length=160, center=True). What is the output shape?

- A. (201, 200)
- B. (201, 201)
- C. (400, 201)
- D. (201, 198)

**Answer:** B. (201, 201) — Bins: $400/2+1=201$. Frames with centring: $1+\lfloor 32000/160\rfloor = 201$. Without centring it would be $1+\lfloor 31600/160\rfloor = 198$.

**Question:** You must tell apart two clicks 6 ms apart at 8 kHz. Which window works?

- A. 512 samples (64 ms) Hann
- B. 256 samples (32 ms) Hann
- C. 64 samples (8 ms) Hann
- D. Any window with 75% overlap

**Answer:** C. 64 samples (8 ms) Hann — The two clicks must fall mostly in different windows. A 64-sample Hann window has an effective width of only a few milliseconds, and the figure shows a 4.9 dB dip between the clicks. The 256- and 512-sample windows cover both clicks at once. A smaller hop only samples the same blurred picture more densely.

**Key points**

- STFT: DFT of windowed frames $x[n+mH]w[n]$; output $(N/2+1)\times T$ complex.
- Speech default at 16 kHz: n_fft = win = 400 (25 ms), hop 160 (10 ms), Hann → 201 bins at 40 Hz, 100 frames/s.
- Frames: 1 + ⌊L/hop⌋ with centring (N/2 padding), 1 + ⌊(L − n_fft)/hop⌋ without; Whisper's 30 s gives 3001, trimmed to 3000.
- Window length × bin spacing = 1: long windows resolve frequency, short windows resolve time (Gabor: σ_t·σ_f ≥ 1/4π).
- Overlap-add with division by Σw² inverts exactly whenever the envelope is non-zero (NOLA); Hann at hop N/2 or N/4 is also COLA.
- Magnitude or power spectrograms drop phase; Griffin–Lim (1984) or a neural vocoder must supply it to make audio.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
