# LLM-as-a-judge

> A strong LLM grades or compares outputs against a rubric. Run pairwise verdicts in both orders, control for length, and validate against human labels with agreement and Cohen's κ before trusting the scores.

Source: https://www.mlfoundry.in/learn/llm-as-a-judge  
Section: Generative AI › Evaluation of Generative Models  
Level: core · 8 min read · updated 2026-10-21

**LLM-as-a-judge:** is the use of a language model as the evaluator of other models' outputs. Given a prompt, one or two candidate answers and a rubric, the judge returns a score or a preference, usually with a written rationale. Zheng et al. named and studied the practice in "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena" (NeurIPS 2023 Datasets and Benchmarks), where GPT-4's pairwise verdicts agreed with expert humans over 80% of the time, about as often as two experts agreed with each other.

Open-ended outputs (chat answers, summaries, code reviews, agent traces) have no answer key, so exact-match metrics do not apply. Overlap metrics such as **BLEU, ROUGE and METEOR** and **BERTScore** need a reference and reward wording, not usefulness. Human raters are the gold standard but cost minutes per item (see **Human evaluation and arena-style ranking**). A judge model costs one API call per item, so it can run on every pull request, every prompt change and every fine-tuning checkpoint.

In a real system the judge sits in the offline eval harness and sometimes online. Offline it compares a candidate against a baseline on a fixed prompt set. Online it scores a sample of production traffic for groundedness, policy violations or task success, feeding dashboards and alerts. **LLM evaluation and benchmarking** covers the benchmark side; this article is about building and trusting the judge itself.

## Three formats

**Pairwise comparison**: the judge sees two answers and picks one or a tie. It is the most sensitive format and matches how humans compare. **Single-answer grading**: the judge scores one answer on a scale, say 1–10. It is cheaper to scale (one call per answer, not per pair) but scores drift between judge versions. **Reference-guided grading**: the judge also gets a reference solution, which Zheng et al. found important for maths and reasoning, where the judge may fail to solve the problem itself.

> Figure: Top: a pairwise judgment run twice with the answer order swapped. Answer A is short and correct, B is long and wrong. Both calls pick slot 1, so the verdicts disagree and the pair is recorded as a tie, not as a win for either. Middle: single-answer grading that reads the score-token probabilities (0.05, 0.10, 0.40, 0.45 for 2–5) and reports Σ p(s)·s = 4.25 instead of the argmax 5. Bottom: the four numbers to measure against human labels before trusting the judge.

## Known biases

**Position bias**: verdicts depend on which answer comes first. Wang et al. ("Large Language Models are not Fair Evaluators", 2023) showed that swapping the order could flip rankings between models, and proposed scoring both orders and averaging. **Verbosity bias**: longer answers win even when the extra text adds nothing or adds errors. **Self-preference**: a judge tends to favour outputs from its own model family. **Limited competence**: a judge cannot reliably grade a proof or a bug fix that it cannot produce itself.

$$
s_{AB}=\tfrac12\big[v(A,B)+1-v(B,A)\big],\qquad \operatorname{logit}P(A\succ B)=\theta+\varphi\,\Delta\ell,\qquad \text{LC win rate}=\sigma(\theta)
$$

- $v(X,Y)$: 1 if the judge prefers the answer in slot 1 when X is shown first, 0 otherwise
- $s_{AB}$: order-balanced score: 1 or 0 when both orders agree, 0.5 when they split
- $\Delta\ell$: length difference of A over B (standardised)
- $\varphi,\ \theta$: fitted length effect and the remaining preference; $\sigma(\theta)$ is the win rate predicted at equal lengths

The length model is the idea behind length-controlled AlpacaEval (Dubois et al., 2024). Its regression has terms for the model, the length difference and the instruction, and the reported win rate is the prediction with the length term set to zero. Arena-style ratings use the same trick under the name style control (see **Human evaluation and arena-style ranking**).

## Measuring the judge

$$
\kappa=\frac{p_o-p_e}{1-p_e},\qquad p_e=\sum_k p^{\text{judge}}_k\,p^{\text{human}}_k,\qquad \text{score}_{\text{G-Eval}}=\sum_{s}p(s)\,s
$$

- $p_o$: observed agreement between judge and human labels
- $p_e$: agreement expected by chance from the two label distributions; $\kappa=0$ is chance, 1 is perfect
- $p(s)$: the judge's probability for score token s (G-Eval, Liu et al., EMNLP 2023)

Raw agreement flatters a judge when one label dominates; $\kappa$ (Cohen, 1960) removes the chance part. Compare it with human–human $\kappa$ on the same items: a judge cannot be expected to beat the humans' own consistency. Probability-weighted scores give a continuous value and fewer ties than reading the argmax token, which helps rank close systems.

**Reference points**

- **MT-Bench:** 80 multi-turn questions in 8 categories (writing, roleplay, extraction, reasoning, math, coding, STEM, humanities), graded by GPT-4 on a 1–10 scale (Zheng et al., 2023).
- **AlpacaEval:** 805 instructions; pairwise win rate against a reference model, judged by an LLM. The length-controlled version (2024) was introduced to remove the length bias.
- **G-Eval:** Liu et al., EMNLP 2023: the judge first writes evaluation steps (chain of thought), then fills a form; scores are weighted by token probabilities.
- **Open judges:** Fine-tuned evaluator models such as Prometheus (Kim et al., ICLR 2024) take a user-written rubric and can run locally.
- **Juries:** Verga et al. (2024) found that a panel of smaller judges from different families can match a single large judge at lower cost and with less self-preference.

> Figure: 300 synthetic prompts where model A is slightly worse (humans prefer A 44.0% of the time) but longer. Each judge call adds position and length terms to the true quality gap. Defaults (position bias 0.8, length bias 0.6, A always first): the judge reports a win rate of 0.747, length control alone 0.652, agreement with humans 0.660 and κ = 0.358. Random order: 0.583 raw and 0.445 length-controlled. Both orders, tie if split: 117 ties (consistency 0.610), 0.555 raw, 0.459 length-controlled, and 158 of 183 decided verdicts (0.863) match the humans. A bias-free judge agrees on 242/300 (κ = 0.611).

Two lessons from the figure. Length control fixes length bias but not position bias: with A always first, it still reports 0.652. Swapping fixes position bias at the cost of ties; the decided verdicts get more accurate, but the tie rate itself is a warning that the judge is unreliable on those items.

## Worked example: is the judge good enough?

100 human-labelled pairs; judge vs majority human label

| Judge ↓ / human → | Human: A | Human: B | Judge total |
| --- | --- | --- | --- |
| Judge: A | 40 | 15 | 55 |
| Judge: B | 10 | 35 | 45 |
| Human total | 50 | 50 | 100 |

1. Observed agreement: $p_o=(40+35)/100=0.75$.
2. Chance agreement: $p_e=0.55\cdot0.50+0.45\cdot0.50=0.50$.
3. $\kappa=(0.75-0.50)/(1-0.50)=0.50$: moderate. If two humans reach $\kappa=0.6$ on these items, the judge is close to, but below, human consistency.
4. The 15 + 10 disagreements are the useful output: read them, look for a pattern (longer answer wins, first answer wins, maths), then fix the rubric or add a reference answer and re-measure.

```python
import json

JUDGE_PROMPT = """You are an impartial judge. Compare the two answers to the question.
Judge accuracy first, then helpfulness. Ignore answer order and length.
Question: {q}
[Answer 1]
{a1}
[Answer 2]
{a2}
Reply with JSON: {{"reasoning": "...", "winner": "1" | "2" | "tie"}}"""

def pairwise(ask, q, a, b):
    """Score A vs B in both orders; returns 1 (A), 0 (B) or 0.5 (tie / inconsistent)."""
    v1 = json.loads(ask(JUDGE_PROMPT.format(q=q, a1=a, a2=b)))["winner"]
    v2 = json.loads(ask(JUDGE_PROMPT.format(q=q, a1=b, a2=a)))["winner"]
    s1 = {"1": 1.0, "2": 0.0, "tie": 0.5}[v1]           # A was in slot 1
    s2 = {"1": 0.0, "2": 1.0, "tie": 0.5}[v2]           # A was in slot 2
    return s1 if s1 == s2 else 0.5

# stand-in for an LLM API call: a judge that always prefers slot 1
always_first = lambda prompt: '{"reasoning": "...", "winner": "1"}'
print(pairwise(always_first, "Explain a p-value.", "P(data this extreme | H0).", "P(H0 is true)."))
# 0.5: the verdict flips with the order, so it is recorded as a tie
```

Order-balanced pairwise judging. `ask` is any function that sends a prompt to an LLM API and returns its text; here a stand-in judge that always picks slot 1 shows the swap catching position bias (runs as is)

> **Do not let the judge grade its own family:** If the candidate and the judge come from the same model family, self-preference inflates the candidate's score. Use a judge from a different family, or a jury of several, and never tune prompts against the same judge you report with: the system learns the judge's taste, not the users'. Pin the judge model version too; an upgraded judge changes every historical score.

> **Specific rubrics beat general ones:** "Which answer is better?" invites the judge's own biases. "Does the answer state the refund window from the policy? Does it invent conditions?" turns judgment into checking, raises agreement and makes disagreements explainable. Binary or 3-point criteria are usually more reliable than a 1–10 scale.

**Question:** A judge picks A when A is shown first and B when B is shown first. What should an order-balanced protocol record?

- A. A win, since A was the first verdict
- B. B win, since the last verdict counts
- C. A tie (0.5), and count it as an inconsistency
- D. Discard the prompt from the eval set

**Answer:** C. A tie (0.5), and count it as an inconsistency — The two verdicts disagree, so the judge has no stable preference: $s_{AB}=\tfrac12[1+1-1]=0.5$. Keeping the pair as a tie avoids biasing the win rate, and the consistency rate (share of pairs where both orders agree) is itself a quality metric for the judge.

**Question:** Model B's answers are on average twice as long as A's, and an LLM judge prefers B in 70% of pairs. What is the first check?

- A. Nothing: B is better
- B. Fit the win rate against the length difference and report it at equal length
- C. Switch to single-answer grading
- D. Raise the judge temperature

**Answer:** B. Fit the win rate against the length difference and report it at equal length — Verbosity bias is a known failure. Regressing verdicts on $\Delta\ell$ (as length-controlled AlpacaEval does) separates a real quality gain from a length effect. A sample of human labels on the same pairs settles it.

**Key points**

- A judge LLM scores or compares outputs against a rubric; pairwise is the most sensitive format, single grading the cheapest.
- Biases: position, verbosity, self-preference and limited competence on hard maths or code.
- Run both orders, record a tie when they split; control for length with $\operatorname{logit}P=\theta+\varphi\Delta\ell$.
- Validate on human labels: agreement, Cohen's $\kappa$, position consistency, length slope; compare with human–human $\kappa$.
- Use specific rubrics, reference answers for reasoning tasks, a judge from another family, and a pinned judge version.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
