Factuality evaluation

Measure how much of a model's output is true. Split it into atomic facts and check each against a knowledge source (FActScore), or score consistency with a source by QA or NLI. Validate every metric against human labels.

8 min read758 wordsadvanced0 viewsupdated 2026-10-15

Definition

Factuality evaluation scores the factual precision of a model or of a set of its outputs: the share of claims a trusted knowledge source supports. One recipe is decompose-then-verify (FActScore, Min et al., EMNLP 2023). Another is consistency with a given source, measured by question answering (QAGS, QuestEval) or entailment (SummaC). Fixed benchmarks include FEVER and TruthfulQA.

Hallucination detection flags one output at inference time, often with no reference. Factuality evaluation is the offline measurement. It takes many outputs and labelled evidence, and returns a number to compare models, decoding settings or training runs. It can be slow and use retrieval, because nobody waits on it.

The task: input a prompt xx, the model's response yy, and a knowledge source C\mathcal{C} (Wikipedia, a document, search results). Output a score in [0,1][0,1]. It is hard because long text mixes true and false claims, often within one sentence. Overlap metrics such as ROUGE and BLEU also barely see a one-word error like "Kraków" for "Warsaw". Maynez et al. (ACL 2020) found that entailment-based measures agreed with human faithfulness judgments better than ROUGE.

Decompose, then verify

FActScore splits each response into atomic facts: short statements that carry one piece of information. In the original work an LLM proposed the facts and human annotators corrected them. Each fact is then labelled supported or not against C\mathcal{C}, and the score is the supported share, averaged over prompts:

f(M)=Ex∈X[1∣Ay∣∑a∈Ay1[a is supported by C]],y=M(x)f(\mathcal{M})=\mathbb{E}_{x\in\mathcal{X}}\Bigg[\frac{1}{|\mathcal{A}_y|}\sum_{a\in\mathcal{A}_y}\mathbb{1}\big[a\text{ is supported by }\mathcal{C}\big]\Bigg],\qquad y=\mathcal{M}(x)
Ay\mathcal{A}_y
atomic facts of response y
C\mathcal{C}
knowledge source; FActScore uses the Wikipedia page of the person in the prompt
X\mathcal{X}
prompts the model actually answers; abstentions are reported separately

Human annotation of people biographies (“Tell me a bio of …”) put ChatGPT at only 58%. The paper then built an automatic estimator: retrieve passages from the page and ask an LM whether each fact is true. Its error against the human FActScore was under 2%, and it was used to score 6,500 generations from 13 recent LMs.

decompose-then-verify: FActScore (Min et al., EMNLP 2023)generationprompt: “Tell me abio of Marie Curie.”“Marie Curie was aPolish-born physicistand chemist. … Shewas born in Kraków in1867. … She died in1938.”atomic facts (13)Polish-borna physicista chemistwon Nobel Physics… in 1903won Nobel Chemistry… in 1911born in Krakówborn in 1867+ 4 moreverifierretrieve top-kpassages for thetopic; ask an LM“True or False?”(or an NLI model)supported if thefact is entailedby the passagesverdictsPolish-born✓a physicist✓a chemist✓won Nobel Physics✓… in 1903✓won Nobel Chemistry✓… in 1911✓born in Kraków✗born in 1867✓+ 4: 3 ✓, died 1938 ✗knowledge sourceWikipedia: M. Curie“b. 1867, Warsaw”“d. 4 July 1934”why atomic facts, not sentences?S3 “born in Kraków in 1867” is half right:atomic credit 1 of 2; sentence credit 0.Sentences fully supported: 3 of 5.FActScore = 11 / 13 = 0.846 · sentence-level 3 / 5 = 0.600averaged over prompts; responses that abstain are not scoredQA-based: QAGS, QuestEvalgenerate questions from theoutput, answer them on thesource and on the output,score the answer agreementNLI-based: SummaCNLI matrix of source sentences× output sentences; max overthe source, mean over theoutput (FactCC: a classifier)benchmarksFEVER: 185,445 claims vsWikipedia, Supported / Refuted/ NotEnoughInfo; TruthfulQA:817 misconception questionsother ways to score factual consistency
fig.FActScore on one biography. The response is split into 13 atomic facts. A verifier checks each against retrieved Wikipedia passages. Two are false (born in Kraków, died 1938), so FActScore = 11/13 = 0.846. Scoring whole sentences would give 3/5, because one false fact sinks a sentence. Below: QA- and NLI-based consistency metrics and two standard benchmarks.

Consistency with a source: QA and NLI

When the reference is a given document, as in summarisation or RAG answers, the question becomes faithfulness to that document. QAGS (Wang, Cho and Lewis, ACL 2020) generates questions from the summary and answers each one twice, from the source and from the summary. The score is the mean similarity of the answer pairs. QuestEval (Scialom et al., EMNLP 2021) also asks questions in the other direction, from the source, which adds a recall component.

SummaC (Laban, Schnabel, Bennett and Hearst, TACL 2022) fixes a granularity mismatch. NLI models are trained on sentence pairs, so it scores every (source sentence, summary sentence) pair and aggregates:

SummaCZS(D,S)=1∣S∣∑j=1∣S∣max⁡i  eij,eij=NLI(di⇒sj)\mathrm{SummaC}_{\mathrm{ZS}}(D,S)=\frac{1}{|S|}\sum_{j=1}^{|S|}\max_{i}\;e_{ij},\qquad e_{ij}=\mathrm{NLI}\big(d_i\Rightarrow s_j\big)
di, sjd_i,\,s_j
i-th source sentence, j-th summary sentence
eije_{ij}
entailment score of the pair from an MNLI-style model
max⁡i\max_i
each summary sentence needs support from only one source sentence; SummaC-Conv instead learns from the histogram of the column

FactCC (Kryściński et al., EMNLP 2020) takes another route: a classifier trained on synthetic corruptions such as swapped entities, numbers and negations. FRANK (Pagnoni et al., NAACL 2021) annotates a typology of error types, so you can see which errors each metric misses.

Benchmarks and reference results

FEVER
Thorne et al., NAACL 2018. 185,445 claims made by altering Wikipedia sentences, labelled Supported, Refuted or NotEnoughInfo, with evidence sentences
TruthfulQA
Lin, Hilton and Evans, ACL 2022. 817 questions in 38 categories that target common misconceptions. The best model then was truthful on 58%, humans on 94%. See LLM evaluation and benchmarking
FActScore
Min et al., EMNLP 2023. People biographies scored against Wikipedia. Human FActScore of ChatGPT: 58%. Automatic estimator error: under 2%
SAFE / LongFact
Wei et al., 2024. An LLM agent checks each fact with Google Search. It agreed with crowd annotators on 72% of facts and won 76% of 100 disagreements, at over 20× lower cost

Is the metric itself right?

An automatic factuality metric is a model, so it must be validated against people. There are three levels. At the fact level, measure the agreement or Cohen's κ\kappa of the verdicts with human labels. At the output level, use correlation or balanced accuracy on consistent-vs-inconsistent summaries; SummaC's benchmark uses balanced accuracy. At the system level, ask whether the estimated scores rank models in the human order and how far each score is off.

system A, one biography → 13 atomic facts · verifier: retrieved passages + LM · τ = 0.50S1Polish-born0.86 ✓physicist0.97 ✓chemist0.95 ✓✓ / ✓S2Nobel Physics0.96 ✓… in 19030.91 ✓Nobel Chem.0.95 ✓… in 19110.90 ✓✓ / ✓S3born Kraków0.05 ✗born 18670.93 ✓✗ / ✗S4polonium0.94 ✓radium0.96 ✓with Pierre0.88 ✓✓ / ✓S5died 19380.10 ✗✗ / ✗fill = verifier verdict (p ≥ τ) · border = human label · dashed = disagree · right: auto / humanthis bio: FActScore human 11/13 = 0.846, auto 11/13 = 0.846 · sentences human 3/5, auto 3/5systemhumanautoerroragreeFActScore, 50 facts each (○ human, ● auto)A0.800.78−0.02098%B0.620.68+0.06094%C0.440.54+0.10078%00.250.50.751ranking: human A > B > C · automatic A > B > C (same order)all 150 facts: 135 verdicts match humans · 11 wrongly accepted · 4 wrongly rejectedretrieval grounds each verdict in a passage, so the estimate tracks the human scoreFActScore = supported / all atomic facts; a sentence counts only if all its facts hold
verifier
fig.Three toy systems, 50 human-labelled atomic facts each. With the retrieval verifier at τ = 0.50, the automatic FActScores are 0.78, 0.68 and 0.54, against human 0.80, 0.62 and 0.44. The ranking is preserved, and 135 of 150 verdicts match the humans. Switch to LM only (no retrieval) and the verifier accepts Kraków and 1938. The scores become 0.82, 0.84 and 0.70, so B now outranks A. A's error is only +0.02, yet 15 of its 50 verdicts are wrong (8 accepted, 7 rejected); the errors cancel.

Worked example

SentenceAtomic factsSupportedSentence fully supported?
S1 Polish-born physicist and chemistPolish-born · physicist · chemist3 / 3yes
S2 Nobel Physics 1903, Chemistry 1911Physics prize · in 1903 · Chemistry prize · in 19114 / 4yes
S3 born in Kraków in 1867born in Kraków · born in 18671 / 2no
S4 discovered polonium and radium with Pierrepolonium · radium · with Pierre Curie3 / 3yes
S5 died in 1938died in 19380 / 1no
Total13 facts11 / 13 = 0.8463 / 5 = 0.600
The diagram's biography, scored by hand against Wikipedia (born in Warsaw, 7 November 1867; died 4 July 1934)

SummaC-ZS on a 2 × 2 matrix

  1. 1Source sentences d1,d2d_1, d_2; summary sentences s1,s2s_1, s_2. NLI entailment scores e=[0.90.10.20.3]e=\begin{bmatrix}0.9&0.1\\0.2&0.3\end{bmatrix} (rows: source, columns: summary).
  2. 2Column maxima: s1→max⁡(0.9,0.2)=0.9s_1\to\max(0.9,0.2)=0.9, s2→max⁡(0.1,0.3)=0.3s_2\to\max(0.1,0.3)=0.3.
  3. 3Mean over summary sentences: (0.9+0.3)/2=0.60(0.9+0.3)/2=0.60. Sentence s2s_2 is weakly supported, so this summary is suspect.
python
1from transformers import pipeline
2from sklearn.metrics import cohen_kappa_score
3
4nli = pipeline("text-classification", model="microsoft/deberta-large-mnli", top_k=None)
5
6passage = ("Maria Salomea Skłodowska-Curie (7 November 1867 – 4 July 1934) was a Polish "
7 "and naturalised-French physicist and chemist ... She was born in Warsaw ...")
8facts = ["Marie Curie was Polish-born.", "Marie Curie was a physicist.",
9 "Marie Curie was born in Kraków.", "Marie Curie was born in 1867.",
10 "Marie Curie died in 1938."]
11human = [1, 1, 0, 1, 0]
12
13def p_entail(premise, hypothesis):
14 out = nli([{"text": premise, "text_pair": hypothesis}])[0]
15 return next(d["score"] for d in out if d["label"] == "ENTAILMENT")
16
17auto = [int(p_entail(passage, f) >= 0.5) for f in facts]
18print("FActScore (auto) :", sum(auto) / len(auto))
19print("FActScore (human):", sum(human) / len(human)) # 0.6
20print("Cohen's kappa :", cohen_kappa_score(human, auto))
Verify given atomic facts against a passage with an NLI model, then check agreement with human labels (sketch; not run here)

Where it's used

Model comparison

Rank LLMs, decoding settings or RAG pipelines on long-form factual precision (biographies, LongFact prompts).

Summarisation

Gate abstractive summaries by SummaC or QA-based consistency with the source document.

Training signal

Use automatic factuality scores as rewards or preference labels for fine-tuning toward fewer unsupported claims.

Check yourself

A response has 4 sentences with 3 atomic facts each. Exactly one fact in each sentence is false. What are FActScore and the sentence-level score?

Check yourself

On 50 facts, an automatic verifier's FActScore is within 0.02 of the human FActScore. What can you conclude?

Remember

  • ✓Factuality evaluation measures precision offline over many outputs. Hallucination detection flags single outputs at inference.
  • ✓FActScore == mean share of atomic facts supported by the knowledge source. Human-annotated ChatGPT bios: 58%. The automatic estimator's error was under 2%.
  • ✓Source consistency: QAGS and QuestEval compare answers to generated questions. SummaC takes, for each summary sentence, the max NLI entailment over source sentences, then averages.
  • ✓Benchmarks: FEVER (185,445 claims, Supported / Refuted / NotEnoughInfo) and TruthfulQA (817 misconception questions).
  • ✓Validate metrics against humans at the fact level (agreement, κ), the output level (correlation, balanced accuracy) and the system level (ranking).
  • ✓Precision alone rewards short or abstaining answers. Report coverage, or use F1@K.
← back to Deep Learning roadmap