Definition
Encoder–decoder transformer is a sequence-to-sequence model built only from attention and feed-forward layers. An encoder of layers turns the source into a memory using unmasked self-attention. A decoder of layers generates one token at a time using causal self-attention over its own prefix, cross-attention to in every layer, and a softmax over the vocabulary. It models and is trained end to end with cross-entropy.
The model is the Transformer of Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser and Polosukhin, "Attention Is All You Need" (NeurIPS 2017). In 2017 the best translation systems were recurrent or convolutional encoder–decoders with attention (see Sequence-to-sequence (Seq2Seq) and Bahdanau attention). Recurrence forces sequential computation over the positions of a sentence, so the work cannot be parallelised within a training example. The paper kept the encoder–decoder structure and the attention between the two halves, and replaced every recurrent and convolutional layer with self-attention. A single big model trained for 3.5 days on eight P100 GPUs set new records on WMT'14 English→German and English→French at a fraction of the training cost of the previous best models.
The same architecture was later pre-trained as a general text-to-text model, notably in T5 (Raffel et al., JMLR 2020) and BART (Lewis et al., ACL 2020). It remains the standard choice when the input and output are distinct sequences: translation, summarisation, speech recognition (Whisper), and many structured prediction tasks.
The architecture
- one embedding matrix shared by the source embedding, the target embedding and the output projection (the paper used a shared source–target vocabulary for this)
- sinusoidal positional encodings, , (see Positional encoding)
- self-attention + FFN with residuals and post-LN (see Transformer encoder)
- masked self-attention, cross-attention to , FFN (see Transformer decoder)
- the encoder output of the last layer, read by the cross-attention of every decoder layer
Attention appears in three forms, with three mask patterns. In encoder self-attention an matrix is fully visible except for source padding. In decoder self-attention a matrix is lower-triangular. In cross-attention a matrix has queries from the decoder and keys and values from , and it is fully visible except for source padding. Each uses heads of width (see Multi-head attention) and the scaled dot product (see Scaled dot-product attention).
The paper compared costs per layer. A self-attention layer costs with sequential operations and maximum path length . A recurrent layer costs with sequential operations and path length . For sentences, where , self-attention is also cheaper in FLOPs. Every source and target position is connected to every other through a constant number of layers.
| model | N | d_model | d_ff | h | d_k = d_v | P_drop | ε_ls | train steps | params |
|---|---|---|---|---|---|---|---|---|---|
| base | 6 | 512 | 2048 | 8 | 64 | 0.1 | 0.1 | 100K | 65M |
| big | 6 | 1024 | 4096 | 16 | 64 | 0.3 | 0.1 | 300K | 213M |
Training setup in the paper
- Data
- WMT 2014 En–De: about 4.5M sentence pairs, byte-pair encoding with a shared source–target vocabulary of about 37,000 tokens. En–Fr: 36M sentences, 32,000 word-piece vocabulary
- Batches
- sentence pairs batched by approximate length; about 25,000 source and 25,000 target tokens per batch
- Hardware
- one machine with 8 NVIDIA P100 GPUs; base 0.4 s per step (100K steps, 12 hours), big 1.0 s per step (300K steps, 3.5 days)
- Optimiser
- Adam, , , ; with warmup = 4000
- Regularisation
- dropout on every sublayer output and on the embedding + PE sums; label smoothing ; the big En–Fr model used
- Inference
- averaged the last 5 checkpoints (base) or 20 (big); beam size 4, length penalty α = 0.6, max length input + 50
Results
| model | EN-DE | EN-FR | training FLOPs (EN-DE) |
|---|---|---|---|
| GNMT + RL | 24.6 | 39.92 | 2.3 · 10¹⁹ |
| ConvS2S | 25.16 | 40.46 | 9.6 · 10¹⁸ |
| GNMT + RL ensemble | 26.30 | 41.16 | |
| ConvS2S ensemble | 26.36 | 41.29 | |
| Transformer (base) | 27.3 | 38.1 | 3.3 · 10¹⁸ |
| Transformer (big) | 28.4 | 41.8 | 2.3 · 10¹⁹ |
The big model beat the best previous results, ensembles included, by more than 2 BLEU on English→German. On English→German the base model already beat all of them, with training FLOPs against for ConvS2S and for GNMT + RL. The paper computed the FLOP estimates as training time × number of GPUs × an assumed sustained 9.5 TFLOPS per P100. The English→French score of the big model is 41.8 in the revised arXiv version. The NeurIPS proceedings version reports 41.0. The same architecture, with 4 layers and , also reached 91.3 F1 on WSJ constituency parsing with little task-specific tuning.
Worked example: counting Transformer base and its schedule
| component | formula | parameters | share |
|---|---|---|---|
| shared embedding / output | V·d = 37,000 · 512 | 18,944,000 | 30.0% |
| encoder, 6 layers | 6 · (12d² + 13d) | 18,914,304 | 30.0% |
| decoder self-attention + LN | 6 · (4d² + 4d + 2d) | 6,309,888 | 10.0% |
| decoder cross-attention + LN | 6 · (4d² + 4d + 2d) | 6,309,888 | 10.0% |
| decoder FFN + LN | 6 · (8d² + 5d + 2d) | 12,604,416 | 20.0% |
| total | 63,082,496 | 100% |
Parameters, compute and learning rate
- 1Per layer with , : encoder , decoder . Six of each give , and the embedding adds , for M. The paper reports 65M. The difference most likely comes from the exact BPE vocabulary, which is given only as "about 37000".
- 2Training cost of base: FLOPs, the of Table 2. Big: .
- 3Learning rate: rises linearly to its peak at , . Then it decays as : at and at .
- 4Allowed query–key pairs per head and layer for , : (encoder) (decoder) (cross) .
T5 and BART: pre-trained encoder–decoders
T5 (Raffel, Shazeer, Roberts, Lee, Narang, Matena, Zhou, Li and Liu, "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer", JMLR 2020) casts every task as text in, text out: "translate English to German: …", "summarize: …", or a GLUE label written as a word. It pre-trains on C4, a cleaned Common Crawl corpus of about 750 GB, with span corruption. 15% of tokens are dropped in spans of mean length 3. Each span is replaced by a sentinel token in the input, and the target lists the dropped spans after their sentinels. In the paper's controlled comparison, the encoder–decoder with this denoising objective beat decoder-only and prefix-LM variants in comparable settings. Architecturally T5 changes the 2017 model in small ways: pre-LN with a scale-only LayerNorm, no bias terms, and relative position biases (32 buckets, a learned scalar per head added to the attention logits) in place of sinusoids. The released sizes were Small (60M), Base (220M), Large (770M), 3B and 11B.
BART (Lewis, Liu, Goyal, Ghazvininejad, Mohamed, Levy, Stoyanov and Zettlemoyer, "BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension", ACL 2020) is a BERT-like bidirectional encoder joined to a GPT-like autoregressive decoder. It is pre-trained to reconstruct a document from a corrupted version. The paper compared token masking, token deletion, text infilling (spans with Poisson() lengths, each replaced by a single [MASK] token, so the model must also predict how many tokens are missing), sentence permutation and document rotation. Text infilling plus sentence permutation worked best. BART-large has 12 encoder and 12 decoder layers of width 1024, uses GELU and learned position embeddings, and was trained like RoBERTa on 160 GB of text. It matched RoBERTa on GLUE and SQuAD, gained up to 6 ROUGE on abstractive summarisation, and added 1.1 BLEU over a back-translation system on WMT Romanian–English.
A useful accounting from the T5 paper: an encoder–decoder with layers in each stack has about as many parameters as a -layer decoder-only model. Its compute is roughly that of an -layer one, because each token passes through only one of the two stacks. The cost is an architectural split: the input must be known in advance and is encoded once, which suits conditional tasks and not open-ended continuation (see Decoder-only transformer).
1import math, torch, torch.nn as nn, torch.nn.functional as F23PAD, BOS, EOS = 0, 1, 245def sinusoid(L, d):6 pos = torch.arange(L)[:, None]7 ang = pos / 10000 ** (torch.arange(0, d, 2) / d)8 pe = torch.zeros(L, d)9 pe[:, 0::2], pe[:, 1::2] = torch.sin(ang), torch.cos(ang)10 return pe1112def causal(T, device): # True = may NOT attend13 return torch.ones(T, T, dtype=torch.bool, device=device).triu(1)1415class Seq2SeqTransformer(nn.Module):16 def __init__(self, vocab, d=512, h=8, N=6, d_ff=2048, p=0.1, max_len=1024):17 super().__init__()18 self.d, self.emb = d, nn.Embedding(vocab, d, padding_idx=PAD) # shared src/tgt/output19 self.register_buffer("pe", sinusoid(max_len, d), persistent=False)20 self.tf = nn.Transformer(d_model=d, nhead=h, num_encoder_layers=N, num_decoder_layers=N,21 dim_feedforward=d_ff, dropout=p, batch_first=True)22 self.drop = nn.Dropout(p)2324 def embed(self, ids):25 return self.drop(self.emb(ids) * math.sqrt(self.d) + self.pe[: ids.size(1)])2627 def forward(self, src, tgt_in): # src (N, S), tgt_in (N, T)28 sp, tp = src == PAD, tgt_in == PAD29 z = self.tf(self.embed(src), self.embed(tgt_in), tgt_mask=causal(tgt_in.size(1), src.device),30 src_key_padding_mask=sp, tgt_key_padding_mask=tp, memory_key_padding_mask=sp)31 return z @ self.emb.weight.T # tied output projection -> (N, T, V)3233 @torch.no_grad()34 def greedy(self, src, max_new=50):35 sp = src == PAD36 mem = self.tf.encoder(self.embed(src), src_key_padding_mask=sp) # encoder runs once37 ys = torch.full((src.size(0), 1), BOS, device=src.device)38 for _ in range(max_new): # no KV cache here: re-runs the prefix39 z = self.tf.decoder(self.embed(ys), mem, tgt_mask=causal(ys.size(1), src.device),40 memory_key_padding_mask=sp)41 nxt = (z[:, -1] @ self.emb.weight.T).argmax(-1, keepdim=True)42 ys = torch.cat([ys, nxt], 1)43 if (nxt == EOS).all():44 break45 return ys4647V = 3700048model = Seq2SeqTransformer(V)49print(sum(p.numel() for p in model.parameters())) # 63084544 (nn.Transformer adds 2 final LNs: +2048)5051opt = torch.optim.Adam(model.parameters(), lr=1.0, betas=(0.9, 0.98), eps=1e-9)52sched = torch.optim.lr_scheduler.LambdaLR(53 opt, lambda s: 512 ** -0.5 * min((s + 1) ** -0.5, (s + 1) * 4000 ** -1.5))5455src = torch.randint(3, V, (2, 10)); src[1, 8:] = PAD56tgt = torch.randint(3, V, (2, 9)); tgt[:, 0] = BOS; tgt[1, 7:] = PAD57tgt_in, tgt_out = tgt[:, :-1], tgt[:, 1:] # shift right58logits = model(src, tgt_in)59print(logits.shape) # torch.Size([2, 8, 37000])60loss = F.cross_entropy(logits.flatten(0, 1), tgt_out.flatten(), ignore_index=PAD, label_smoothing=0.1)61loss.backward(); opt.step(); sched.step()6263model.eval()64print(model.greedy(src, max_new=5).shape) # torch.Size([2, 6]) unless both rows emit EOS earlyUse when
- +Bidirectional encoding of the input plus exact autoregressive generation of the output
- +Fully parallel training: no recurrence in either stack
- +Cross-attention to all source states in every decoder layer; no fixed-length bottleneck
- +Source encoded once; about the parameters of a single stack at roughly single-stack compute per token
Avoid when
- −Quadratic attention in and
- −Generation is sequential, one decoder pass per output token
- −Two stacks and three mask types: more plumbing than decoder-only
- −Needs a clear input/output split; less natural for open-ended continuation or chat
Check yourself
Which pair of hyperparameters distinguishes Transformer big from Transformer base, apart from dropout and training length?
Check yourself
In an encoder–decoder Transformer, where do the keys and values of decoder layer 3's cross-attention come from?
Remember
- ✓Encoder–decoder: with full self-attention; decoder with causal self-attention and cross-attention to the last-layer in every layer.
- ✓Transformer (Vaswani et al., NeurIPS 2017): base N = 6, d = 512, d_ff = 2048, h = 8 (65M), 27.3 BLEU En–De; big d = 1024, d_ff = 4096, h = 16 (213M), 28.4 En–De, 41.8 En–Fr.
- ✓Three attention patterns: full, causal, cross; one shared embedding matrix, , sinusoidal positions.
- ✓Training: Adam (0.9, 0.98, ), warm-up 4000 then , dropout 0.1, label smoothing 0.1; decoding with beam 4, α = 0.6.
- ✓T5: text-to-text, span corruption on C4, pre-LN, relative position biases; BART: denoising (text infilling + sentence permutation) with a bidirectional encoder and an autoregressive decoder.