# Encoder–decoder architecture

> An encoder reads the input into a representation and a decoder generates the output from it, one token at a time, as a conditional language model. Squeezing the input into one fixed vector is the weak point; attention and skip connections remove that bottleneck.

Source: https://www.mlfoundry.in/learn/encoder-decoder-architecture  
Section: Deep Learning › Recurrent Neural Networks and Sequence Models  
Level: core · 12 min read · updated 2026-10-12

**Encoder–decoder architecture:** is a network made of two parts. The **encoder** maps an input $x$ (a sentence, an image, an audio clip) to an intermediate representation $c=\mathrm{enc}(x)$. The **decoder** produces the output $y$ from that representation, usually autoregressively: $p(y\mid x)=\prod_t p(y_t\mid y_{<t},c)$. Input and output can have different lengths, different vocabularies and even different modalities.

Many tasks map one structured object to another of a different shape: translation (a French sentence of length 7 to an English sentence of length 9), speech recognition (audio frames to characters), captioning (an image to words), summarisation, segmentation (an image to a per-pixel mask). A single recurrent network that emits one output per input step cannot handle unequal lengths or reordering. Splitting the job in two is the fix: first *understand* the whole input, then *generate* the output conditioned on that understanding.

The pattern is old in the form of the **autoencoder**, where the decoder reconstructs the encoder's own input (Hinton and Salakhutdinov, 2006). For sequence transduction it arrived in 2013–2014. Kalchbrenner and Blunsom (2013) encoded a source sentence with a convolutional model and decoded it with a recurrent language model. Cho, van Merriënboer, Gulcehre, Bahdanau, Bougares, Schwenk and Bengio (EMNLP 2014) named the **RNN Encoder–Decoder**: one RNN reads the source into a fixed-length vector, a second RNN generates the target from it (the same paper introduced the GRU). Sutskever, Vinyals and Le (2014) scaled the idea up with deep LSTMs (see **Sequence-to-sequence (Seq2Seq)**). The same two-part design then spread to images (Show and Tell, U-Net), and the Transformer (Vaswani et al., 2017) is an encoder–decoder too.

## The conditional language model

$$
c=\mathrm{enc}(x_1,\dots,x_T),\qquad p(y_1,\dots,y_{T'}\mid x)=\prod_{t=1}^{T'} p\big(y_t\mid y_1,\dots,y_{t-1},\,c\big)
$$

- $x_1..x_T$: the input sequence (or image, or audio), length $T$
- $c$: the encoder's representation: one fixed vector (classic RNN encoder–decoder) or one vector per input position (attention, Transformer)
- $y_1..y_{T'}$: the output sequence, length $T'$ chosen by the decoder itself: it stops when it emits an end-of-sequence token
- $p(y_t\mid y_{<t},c)$: the decoder: a language model over the output vocabulary whose every step is conditioned on $c$

The factorisation is exact (it is the chain rule of probability), so the decoder is just a language model with an extra input. Training maximises $\log p(y\mid x)=\sum_t\log p(y_t\mid y_{<t},c)$ over (input, output) pairs: a cross-entropy loss per output token, with the ground-truth prefix $y_{<t}$ fed in (see **Teacher forcing**). The gradient flows from every output token back through the decoder **into the encoder**, so both halves are trained jointly, end to end, for the one task. At inference the decoder generates token by token, by greedy choice, sampling, or **Beam search**.

In the RNN version the encoder's final hidden state is the representation and becomes the decoder's initial state: $h_t=f_{\text{enc}}(h_{t-1},x_t)$, $c=h_T$, $s_0=c$ (or $s_0=\tanh(Wc)$), $s_t=f_{\text{dec}}(s_{t-1},y_{t-1},c)$, $p(y_t\mid\cdot)=\mathrm{softmax}(W_o s_t)$. Cho et al. fed $c$ into every decoder step as well, so it would not fade from the decoder state.

**At a glance**

- **Encoder:** any network that reads the whole input: RNN/LSTM/GRU, CNN, Transformer encoder (bidirectional, sees everything)
- **Decoder:** a conditional generator, usually autoregressive and causal: sees only $y_{<t}$ and the encoder output
- **Interface:** one fixed vector (RNN encoder–decoder, Show and Tell), or all encoder states read by attention (Bahdanau, Transformer cross-attention), or skip connections (U-Net)
- **Loss:** $-\sum_t\log p(y_t\mid y_{<t},x)$, summed over output tokens; trained end to end
- **Lengths:** $T$ and $T'$ are independent; the decoder ends with an `<eos>` token
- **Weak point:** a fixed-size $c$ must hold the whole input: quality drops on long inputs (Cho et al., 2014b)

## Across modalities

The same two-part pattern in different systems

| system | encoder | representation | decoder |
| --- | --- | --- | --- |
| RNN Encoder–Decoder (Cho et al., 2014) | GRU over source words | final hidden state $c$, fed to every decoder step | GRU language model over target words |
| Seq2Seq (Sutskever et al., 2014) | 4-layer LSTM over the reversed source | final $(h,c)$ of all layers | 4-layer LSTM, initialised from the encoder |
| RNNsearch (Bahdanau et al., 2015) | bidirectional GRU | one annotation per source word, read by attention | GRU with a new context vector each step |
| Show and Tell (Vinyals et al., 2015) | ImageNet CNN | image embedding, given to the LSTM once, before the first word | LSTM caption generator |
| U-Net (Ronneberger et al., 2015) | contracting conv path | bottleneck feature map plus skip connections at every scale | expanding path with up-convolutions to a per-pixel map |
| Autoencoder | MLP or CNN | low-dimensional code $z$ | reconstructs the input $x$ |
| Transformer, T5, BART, Whisper | stack of self-attention layers | one vector per input token | causal self-attention plus cross-attention to every encoder vector |

Two lessons sit in the table. First, the encoder and the decoder need not be the same kind of network: Show and Tell (Vinyals, Toshev, Bengio and Erhan, CVPR 2015) pairs a convolutional image encoder with an LSTM text decoder, and Whisper (Radford et al., 2022) pairs an audio Transformer encoder with a text decoder. Second, the systems that win on long or detailed inputs all pass **more than one vector** across: attention reads every encoder state, U-Net copies feature maps across at every resolution. T5 (Raffel et al., JMLR 2020) cast every NLP task as text-to-text with an encoder–decoder Transformer, and in its comparison of architectures the encoder–decoder with a denoising objective did best.

## The bottleneck of a fixed vector

If the whole input must pass through a vector of $d$ numbers, there is a hard limit on what the decoder can know, and it gets worse as the input grows. Cho, van Merriënboer, Bahdanau and Bengio ("On the Properties of Neural Machine Translation: Encoder–Decoder Approaches", SSST 2014) measured it: the BLEU score of their RNN encoder–decoder fell steadily as sentences got longer, and they named the fixed-length vector as the likely cause. Bahdanau, Cho and Bengio's attention model, published the same year, was the answer: keep all $T$ encoder states and let the decoder choose which to read at each step.

The figure makes the limit exact with a linear model. A linear RNN with a $d$-dimensional state, $h_t=\rho Q h_{t-1}+b\,x_t$ ($Q$ a random rotation, $b$ a random unit vector), reads $T$ random numbers $x_t\sim\mathcal N(0,1)$. Its final state is $c=Mx$ for a $d\times T$ matrix $M$. The decoder is the best possible linear decoder when every number it receives has noise of standard deviation $\sigma=0.05$. Its recovered share of token $t$'s variance is $r_t\in[0,1]$, and because $M$ has rank at most $d$,

$$
\sum_{t=1}^{T} r_t\;\le\;d\qquad\Longrightarrow\qquad \overline{R^2}=\frac1T\sum_t r_t\;\le\;\min\!\Big(1,\frac dT\Big)
$$

- $r_t$: $[(M^\top M+\sigma^2I)^{-1}M^\top M]_{tt}$, the fraction of $\operatorname{Var}(x_t)$ the best linear decoder recovers from $c$
- $d$: the size of the fixed vector
- $\overline{R^2}$: mean recovered variance over the sequence

> Figure: A linear RNN encoder squeezes T random numbers into a vector of d = 8 numbers; the best linear decoder reconstructs them (orange dots over the blue source bars). The bottom-left bars show how much of each token survives and the right curve shows mean R² against T, with the bound min(1, d/T) dashed. With ρ = 1, mean R² is 0.998 at T = 1, 0.952 at T = 8, 0.798 at T = 10, 0.399 at T = 20 and 0.200 at T = 40: once T > d it tracks d/T, and the 8 numbers never carry more than 8 tokens' worth (Σ r_t = 7.99 at T = 20). Lower ρ makes the encoder forget: at ρ = 0.8, T = 20 the first token keeps 0.3% of its variance and the last 96.7%. Switch the decoder to read all encoder states, as attention does, and mean R² is 0.999 at every length.

The curve has the shape Cho et al. saw in BLEU: fine while the input fits, then a steady decline. A nonlinear encoder can pack information more cleverly than a linear one, but no encoder can beat the counting argument in general: $d$ real numbers at finite precision hold a bounded number of bits, while the input's information grows with $T$. The forgetting factor $\rho$ adds a second effect. A recurrent encoder overwrites old information with new, so the **first** tokens are the worst preserved, exactly the ones a left-to-right decoder needs first. That is why Sutskever et al. reversed the source sentence, and why attention, which skips the bottleneck entirely, made the trick unnecessary.

## Worked example: one number for two tokens

Take $d=1$, $T=2$, $\rho=1$. In one dimension a rotation is $\pm1$, and the figure's encoder gives $M=(-1,\,-1)$, so $c=-(x_1+x_2)$. With noise variance $\sigma^2=0.0025$:

1. $M^\top M=\begin{pmatrix}1&1\\1&1\end{pmatrix}$. Its eigenvalues are 2 (direction $(1,1)$) and 0 (direction $(1,-1)$).
2. The decoder recovers the sum and nothing of the difference: $(M^\top M+\sigma^2I)^{-1}M^\top M=\frac{1}{2+\sigma^2}\begin{pmatrix}1&1\\1&1\end{pmatrix}$.
3. So $r_1=r_2=1/(2+0.0025)=0.4994$ and $\hat x_1=\hat x_2=\tfrac{x_1+x_2}{2.0025}$: each token is half recovered.
4. $\sum_t r_t=0.9988\le d=1$, mean $R^2=0.4994\le d/T=0.5$. The figure's library returns the same 0.4994.
5. Give the decoder both states $h_1=-x_1$ and $h_2=-(x_1+x_2)$ instead, and $x_1=-h_1$, $x_2=h_1-h_2$: the bottleneck is gone.

The output side of the same model is the chain-rule factorisation. If a trained decoder assigns the three reference tokens of a translation the probabilities 0.6, 0.5 and 0.9 given their prefixes, then $p(y\mid x)=0.6\cdot0.5\cdot0.9=0.27$ and the training loss for this pair is $-\ln 0.6-\ln 0.5-\ln 0.9=0.511+0.693+0.105=1.309$ nats, or 0.436 per token.

```python
import torch, torch.nn as nn, torch.nn.functional as F

class EncoderDecoder(nn.Module):
    """Cho et al. (2014): c = last encoder state, fed to every decoder step."""
    def __init__(self, src_vocab, tgt_vocab, emb=256, hid=512):
        super().__init__()
        self.src_emb = nn.Embedding(src_vocab, emb)
        self.tgt_emb = nn.Embedding(tgt_vocab, emb)
        self.encoder = nn.GRU(emb, hid, batch_first=True)
        self.init = nn.Linear(hid, hid)
        self.decoder = nn.GRU(emb + hid, hid, batch_first=True)   # input = [y_{t-1}; c]
        self.out = nn.Linear(hid + hid, tgt_vocab)                 # logits from [s_t; c]

    def encode(self, src):                       # src: (N, T) token ids
        _, h_n = self.encoder(self.src_emb(src)) # h_n: (1, N, hid)
        return h_n[-1]                           # c: (N, hid), the whole input in one vector

    def forward(self, src, tgt_in):              # tgt_in: (N, T') = <sos> y_1 .. y_{T'-1}
        c = self.encode(src)
        s0 = torch.tanh(self.init(c)).unsqueeze(0)
        ctx = c.unsqueeze(1).expand(-1, tgt_in.size(1), -1)
        s, _ = self.decoder(torch.cat([self.tgt_emb(tgt_in), ctx], dim=-1), s0)
        return self.out(torch.cat([s, ctx], dim=-1))  # (N, T', tgt_vocab)

model = EncoderDecoder(src_vocab=8000, tgt_vocab=8000)
src = torch.randint(0, 8000, (32, 17))
tgt = torch.randint(0, 8000, (32, 12))        # <sos> ... <eos>
logits = model(src, tgt[:, :-1])                # teacher forcing: shifted ground truth
loss = F.cross_entropy(logits.reshape(-1, 8000), tgt[:, 1:].reshape(-1))

# The Transformer is the same split, but the decoder cross-attends to ALL encoder vectors.
tf = nn.Transformer(d_model=512, nhead=8, num_encoder_layers=6,
                    num_decoder_layers=6, batch_first=True)
x, y = torch.randn(32, 17, 512), torch.randn(32, 11, 512)  # embedded source / shifted target
memory = tf.encoder(x)                                      # (32, 17, 512): one vector per token
mask = nn.Transformer.generate_square_subsequent_mask(11)   # causal mask for the decoder
h = tf.decoder(y, memory, tgt_mask=mask)                    # (32, 11, 512)
```

A GRU encoder–decoder with a fixed context vector, and the same pattern as nn.Transformer

> **The decoder can ignore the encoder:** A strong autoregressive decoder can lower its loss by modelling the output language alone and treating $c$ as noise, especially early in training or when $c$ is weak. Symptoms: fluent outputs unrelated to the input, the same caption for every image. Feeding $c$ into every decoder step (Cho et al.), attention to the encoder states, and checking that the loss actually rises when you shuffle the inputs within a batch are the usual guards.

> **Attention is a wider interface, not a new architecture:** Bahdanau attention, Luong attention and Transformer cross-attention keep the encoder–decoder split and change only what crosses it: all $T$ encoder vectors instead of one, with the decoder computing at each step a weighted average of them (see **Attention mechanism** and **Self-Attention**). U-Net's skip connections play the same role for images. In the figure, switching the decoder to read every encoder state lifts mean R² from 0.399 to 0.999 at T = 20, d = 8.

> **Encoder–decoder vs decoder-only:** GPT-style models drop the separate encoder: input and output are concatenated into one sequence and a single causal Transformer models it. Encoder–decoders remain common where the input is long, fully known in advance and of a different kind from the output: translation, speech recognition (Whisper), summarisation and many T5 and BART fine-tunes. The encoder can then attend in both directions and its cost is paid once, not at every generated token.

**Strengths**

- Input and output lengths, vocabularies and even modalities are decoupled
- One principled loss: the conditional log-likelihood $\sum_t\log p(y_t\mid y_{<t},x)$, trained end to end
- The encoder sees the whole input (bidirectionally) before any output is produced
- Encoder and decoder can be swapped or pre-trained separately (CNN encoder + LSTM decoder, pretrained T5 encoder)

**Limitations**

- A single fixed vector caps what is transmitted: quality falls with input length
- Autoregressive decoding is sequential: output latency grows with output length
- Training with ground-truth prefixes but generating from its own outputs causes exposure bias
- Two networks to size and tune; the decoder may learn to ignore a weak encoder

**Question:** An RNN encoder–decoder passes a single 256-dimensional vector from encoder to decoder. Translation quality is good for 10-word sentences but poor for 50-word ones. Which change addresses the cause most directly?

- A. Let the decoder attend over all encoder hidden states
- B. Double the decoder's depth
- C. Use a larger output vocabulary
- D. Train with a smaller learning rate

**Answer:** A. Let the decoder attend over all encoder hidden states — The symptom is the fixed-vector bottleneck: everything about a 50-word sentence must fit in the same 256 numbers as a 10-word one. A deeper decoder still reads only that vector. Attention passes one vector per source word, so the interface grows with the input, which is what Bahdanau et al. (2015) did after Cho et al. (2014) reported the length effect.

**Question:** In the figure's linear model with d = 4, what is the largest possible mean R² for a sequence of T = 16 tokens?

- A. 1
- B. 0.5
- C. 0.25
- D. 0.0625

**Answer:** C. 0.25 — $\sum_t r_t\le d=4$, so the mean over 16 tokens is at most $4/16=0.25$. The library gives 0.249 for $\rho=1$. A vector of 4 numbers can carry at most 4 tokens' worth of variance however it is arranged.

**Key points**

- Encoder: input $\to$ representation $c$. Decoder: a conditional language model $p(y\mid x)=\prod_t p(y_t\mid y_{<t},c)$.
- Trained end to end with per-token cross-entropy; generation is token by token until an end-of-sequence token.
- The pattern spans modalities: RNN Seq2Seq, Show and Tell captioning, U-Net, autoencoders, Transformer, T5, Whisper.
- A fixed vector of $d$ numbers can carry at most $d$ tokens' worth of a linear code, so quality falls with input length (Cho et al., 2014).
- Attention and skip connections widen the interface to one vector per input position and remove the bottleneck.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
