Model families and model selection

Pick the family by input/output contract (decoder-only, encoder, encoder–decoder, diffusion, multimodal), then the model by your own eval set: the cheapest that clears the quality bar within latency, context and licence limits.

8 min read793 wordscore0 viewsupdated 2026-10-21

Definition

Model selection is choosing, for one task, the model family and then the specific model that meets fixed requirements on quality, latency, cost, context length, licence and data handling at the lowest total cost. A model family is a group of architectures that share an input–output contract and training objective: decoder-only language models, encoder-only models, encoder–decoder models, diffusion models and multimodal models.

Teams get this wrong in two directions. They default to the largest general model for everything and pay 10–100× more per request than needed, or they pick a small model from a leaderboard and find it fails on their own data. Selection is an engineering decision, made with measurements on your task. It belongs in a system design next to prompt design and retrieval, and it is revisited whenever a new model ships.

The families

Decoder-only models (see Decoder-only models and GPT architecture) predict the next token with causal attention and generate by repeating that step. They are the default for open-ended text, code, chat, tool use and RAG answers. Llama, Qwen, Mistral and GPT models belong here. Mixture-of-experts variants keep the same contract but route each token to a few expert feed-forward blocks. Mixtral 8×7B (Jiang et al., 2024) has 46.7B parameters but uses about 12.9B per token.

Encoder-only models (BERT, Sentence embeddings models, cross-encoder rerankers) read the whole input with bidirectional attention and output a vector or a label. They do not generate text, but for classification, retrieval and reranking they are often as accurate as an LLM at a tiny fraction of the cost. Encoder–decoder models (T5, BART, Whisper) encode an input sequence and decode a different one, a good fit for translation, transcription and rewriting.

Diffusion models (see Diffusion models and Latent diffusion models) generate images, video or audio by iteratively denoising a latent conditioned on a prompt. Multimodal models connect a vision or audio encoder to a language model (see Vision-language models and Multimodal LLMs), or align two encoders in a shared space as CLIP does.

families: what goes in, what comes outselectiondecoder-onlyLLMprompt → next token, repeatGPT, Llama, Qwen, Mistralchat, code, agents, RAG answersencoder-onlybidirectionaltext → vector or labelBERT, embedding modelssearch, classify, rerankencoder–decoderseq2seqinput seq → output seqT5, BART, Whispertranslate, transcribe, rewritediffusioniterative denoisenoise + prompt → imageStable Diffusion, DiTimages, video, audiomultimodalencoders + LLMimage + text → textCLIP, vision-language modelscaptions, VQA, documents1. task + modality→ which family2. hard limits: context,licence, data residency3. eval set from realtraffic (100s of items)4. measure quality, p95latency, $ per request5. cheapest that passes;re-test on every updatea mixture-of-experts LLM is still decoder-only: it routes each token to a few expert FFNs
fig.Left: five families by their input–output contract, with example models and typical jobs. The contract decides the family: if the output is a label or a vector, an encoder is usually enough; if it is free text, you need a decoder. Right: the selection flow. Hard limits (context, licence, data residency) filter before any quality test, and the final choice is the cheapest model that passes your own eval set.
FamilyObjectiveOutputTypical jobArticle
Decoder-onlynext-token likelihoodtext, code, tool callschat, generation, agentsGPT architecture
Encoder-onlymasked-token prediction, contrastivevector or labelsearch, classification, rerankingBERT
Encoder–decodersequence-to-sequence likelihoodnew sequencetranslation, ASR, summarisationT5
Diffusiondenoising (noise prediction)image, video, audiotext-to-image, editingDiffusion models
Multimodalcontrastive or next-token over mixed inputstext or embeddingscaptioning, VQA, document readingVision-language models
Families at a glance

The criteria, as numbers

Quality comes from your own eval set. The other criteria can be computed before you run anything. Cost per request for a token-priced model:

c=nin pin+nout pout106,Cmonth=V cc=\frac{n_\text{in}\,p_\text{in}+n_\text{out}\,p_\text{out}}{10^6},\qquad C_\text{month}=V\,c
nin, noutn_\text{in},\,n_\text{out}
input and output tokens per request (prompt, retrieved context, answer)
pin, poutp_\text{in},\,p_\text{out}
price per million tokens; output tokens usually cost several times more than input
VV
requests per month

Latency for a streamed answer is time to first token plus one decode step per output token. At batch size 1, each decode step must read all the weights from GPU memory, which sets a floor on the time per output token (TPOT):

T≈TTFT+nout⋅TPOT,TPOT  ≳  Nactive⋅bmemory bandwidthT\approx\text{TTFT}+n_\text{out}\cdot\text{TPOT},\qquad \text{TPOT}\;\gtrsim\;\frac{N_\text{active}\cdot b}{\text{memory bandwidth}}
TTFT\text{TTFT}
time to first token: queueing plus prefill of the ninn_\text{in} prompt tokens
NactiveN_\text{active}
parameters used per token (all of them for a dense model, the routed experts for MoE)
bb
bytes per parameter: 2 for bf16, 1 for int8, about 0.5 for 4-bit (see Quantization)

Example: an 8B dense model in bf16 is 16 GB of weights. On a GPU with 3.35 TB/s of memory bandwidth (H100 SXM), one decode step takes at least 16/3350 s = 4.8 ms, so at most about 209 tokens/s for one stream. A 70B model in bf16 needs 140 GB for weights alone, more than one 80 GB GPU; at 4-bit it is about 35 GB. KV cache memory comes on top and grows with context length and batch size.

Stable reference points

BERT-base
110M parameters, 512-token input limit (Devlin et al., 2019)
Whisper
Encoder–decoder speech model trained on 680,000 hours of labelled audio (Radford et al., 2022)
Stable Diffusion
Latent diffusion (Rombach et al., CVPR 2022): a 512 × 512 image is denoised as a 64 × 64 × 4 latent, 8× downsampling per side
CLIP
Dual encoder trained contrastively on 400M image–text pairs (Radford et al., 2021)
Mixtral 8×7B
8 experts, top-2 routing per token, 46.7B total and about 12.9B active parameters (Jiang et al., 2024)
Weight memory
parameters × bytes per parameter: 7B ≈ 14 GB in bf16, ≈ 3.5 GB at 4-bit, before KV cache and activations

Selecting on your constraints

RAG question answering: 2000 in + 200 out tokens, 1M requests/monthhypothetical candidates; quality = your eval set, cost = V·(n_in·p_in + n_out·p_out)/10⁶0.40.60.81.0$10.00$100$1,000$10k$100kmonthly cost (log scale)qualityquality bar 0.80234567# modelhostedqualitylatency$/monthverdict1 enc-110Mselfn/a0.01 s$20.00cannot generate2 dec-3Bself0.681.65 s$120quality < bar3 dec-8Bself0.762.48 s$240quality < bar4 MoE-47B/13Bself0.823.15 s$720passes5 dec-70Bself0.866.30 s$1,580too slow6 api-smallapi0.801.95 s$420PICK: cheapest pass7 api-largeapi0.914.60 s$9,000passespick api-small: quality 0.80, 1.95 s, $420/month
task
data residency
fig.Seven hypothetical candidates (archetypes, not real products) with illustrative prices and eval scores. Defaults: RAG QA (2,000 in, 200 out tokens), 1M requests/month, quality bar 0.80, latency budget 5 s. dec-70B fails on latency (6.30 s); api-small is picked at $420/month (0.80, 1.95 s). Require in-house data: MoE-47B/13B at $720. Raise the bar to 0.85: only api-large passes, at $9,000; relax latency to 7 s and dec-70B wins at $1,580. Switch to ticket classification at a 0.90 bar and 1 s: the 110M encoder wins at $3/month (0.93, 0.01 s).

Two patterns show up. The cost spread between candidates that pass is often 10× or more, so the quality bar should be set from what the product needs, not from the best score available. And for label or vector outputs the encoder family dominates: in the classification setting it costs $3\text{\textdollar}3 a month where the largest API model costs $975\text{\textdollar}975.

Worked example: the default RAG case

Modelp_in / p_out ($/1M)QualityLatency$/monthVerdict
dec-3B0.05 / 0.100.680.05 + 1.60 = 1.65 s100 + 20 = 120quality < bar
dec-8B0.10 / 0.200.760.08 + 2.40 = 2.48 s200 + 40 = 240quality < bar
MoE-47B/13B0.30 / 0.600.820.15 + 3.00 = 3.15 s600 + 120 = 720passes
dec-70B0.70 / 0.900.860.30 + 6.00 = 6.30 s1,400 + 180 = 1,580too slow
api-small0.15 / 0.600.800.35 + 1.60 = 1.95 s300 + 120 = 420pick
api-large3.00 / 15.000.910.60 + 4.00 = 4.60 s6,000 + 3,000 = 9,000passes
2,000 input + 200 output tokens, 1,000,000 requests/month, quality ≥ 0.80, latency ≤ 5 s. Cost = 10⁶ × (2000·p_in + 200·p_out)/10⁶; latency = TTFT + 200·TPOT. All prices and scores illustrative

The pick costs 21× less than api-large, and its quality just meets the bar. On a bar that tight, check the confidence interval of the eval score: with 300 items, a score of 0.80 has a 95% interval of roughly ±0.045.

python
1cands = {
2 # name: (hosted, ttft_s, tpot_s, $/1M in, $/1M out, eval quality)
3 "dec-3B": ("self", 0.05, 0.008, 0.05, 0.10, 0.68),
4 "dec-8B": ("self", 0.08, 0.012, 0.10, 0.20, 0.76),
5 "MoE-47B/13B": ("self", 0.15, 0.015, 0.30, 0.60, 0.82),
6 "dec-70B": ("self", 0.30, 0.030, 0.70, 0.90, 0.86),
7 "api-small": ("api", 0.35, 0.008, 0.15, 0.60, 0.80),
8 "api-large": ("api", 0.60, 0.020, 3.00, 15.0, 0.91),
9}
10
11def choose(cands, n_in, n_out, volume, q_bar, lat_budget, in_house=False):
12 passing = []
13 for name, (hosted, ttft, tpot, p_in, p_out, q) in cands.items():
14 latency = ttft + n_out * tpot
15 monthly = volume * (n_in * p_in + n_out * p_out) / 1e6
16 if q >= q_bar and latency <= lat_budget and not (in_house and hosted == "api"):
17 passing.append((monthly, name))
18 return min(passing)[1] if passing else None
19
20print(choose(cands, 2000, 200, 1_000_000, q_bar=0.80, lat_budget=5.0)) # api-small
21print(choose(cands, 2000, 200, 1_000_000, q_bar=0.80, lat_budget=5.0, in_house=True)) # MoE-47B/13B
Filter candidates on hard limits and the quality bar, then take the cheapest; eval scores come from running each model on your own labelled set (runs as is)

Use when

  • +Smaller or specialised models: lower cost and latency, can run on your own hardware, easier to fine-tune.
  • +Larger general models: higher quality on open-ended and multi-step tasks, fewer prompt workarounds.

Avoid when

  • −Smaller models fail more often on reasoning, long context and rare knowledge; check on your data.
  • −Larger models cost more per token, are slower per stream, and may only be available through an API (see Open-weight vs closed models).

Check yourself

You need to route 5M support tickets a month into 40 categories, and you have 20,000 labelled tickets. Which family should you try first?

Check yourself

A dense 13B model in bf16 runs at batch size 1 on a GPU with 2 TB/s of memory bandwidth. What is the best possible decode speed for one stream?

Remember

  • ✓Choose the family by input–output contract: decoder for free text, encoder for labels and vectors, encoder–decoder for sequence-to-sequence, diffusion for media, multimodal for mixed inputs.
  • ✓Cost per request =(ninpin+noutpout)/106=(n_\text{in}p_\text{in}+n_\text{out}p_\text{out})/10^6; latency ≈\approx TTFT + nout⋅+\,n_\text{out}\cdotTPOT; TPOT at batch 1 is bounded by active weight bytes ÷ memory bandwidth.
  • ✓Apply hard limits first (context, licence, data residency), then pick the cheapest model that clears the quality bar on your own eval set.
  • ✓Costs among passing models often differ by 10× or more; routing easy traffic to small models cuts the blended cost.
  • ✓Rankings and prices change fast: re-run the eval when candidates change, rather than trusting leaderboards.
← back to Generative AI roadmap