# Document summarization

> LLMs summarize documents longer than their context by stuffing, map-reduce or refine. Chunking decides what each call can see, the reduce or refine steps decide what survives, and citations plus claim checks keep the summary faithful.

Source: https://www.mlfoundry.in/learn/document-summarization  
Section: Generative AI › Applications  
Level: core · 8 min read · updated 2026-10-21

**Document summarization:** with LLMs condenses one or more long documents into a shorter text for a stated reader and purpose, usually by prompting an instruction-tuned model rather than training a summarizer. Documents longer than the context window are handled with multi-call strategies: **map-reduce** (summarize chunks, then summarize the summaries) and **refine** (update a running summary chunk by chunk). Wu et al. (2021) applied the recursive form to whole books.

The model internals and the classic extractive and abstractive models are covered in **Text summarization**. This article is about the applied problem: reports, contracts, transcripts and papers that are longer than a prompt, read by someone who needs specific things from them and will act on the result.

Three things make it hard. The document may not fit. Even when it fits, models use the middle of a long input less reliably than the start and end (Liu et al., 2023). And an abstractive summary can state things the source never said, which is worse in a summary than elsewhere because readers use it in place of the source.

## Three strategies

> Figure: Map-reduce over a fictional 96k-token annual report. It is split on section headings, each section is summarized by the same map prompt with page citations, a reduce call merges and ranks the partials for the reader, and a verification pass checks every claim against its cited span before the final summary. Below: the call shapes of stuff (one call), map-reduce (n parallel calls, then one) and refine (n calls in sequence).

Strategies for documents of length L with chunks of C tokens

| Strategy | Calls | Sequential steps | Strength | Weakness |
| --- | --- | --- | --- | --- |
| Stuff | 1 | 1 | the model sees everything at once; cross-references stay intact | only works if L fits; the middle is under-used |
| Map-reduce | n + 1 (more if reduce overflows) | $1+\lceil\log_g n\rceil$ | map calls run in parallel; each chunk is read closely | a chunk loses context defined elsewhere; reduce sees only partials |
| Refine | n | n | each step sees the running summary, so later chunks are read in context | slow; each rewrite can drop earlier details, which biases the summary toward the end |

## Cost and depth

$$
n=\Big\lceil\frac{L}{C}\Big\rceil,\qquad T_{\text{map-reduce}}\approx L+n\,p+(n\,s+p),\qquad T_{\text{refine}}\approx n\,(C+p)+(n-1)\,s
$$

- $L,\;C$: document length and chunk size in tokens
- $p,\;s$: instruction tokens per call and tokens per partial (or running) summary
- $T$: total input tokens read across all calls

$$
g=\Big\lfloor\frac{W-p}{s}\Big\rfloor,\qquad \text{depth}=1+\big\lceil\log_g n\big\rceil
$$

- $W$: the model's context window
- $g$: fan-in: how many partial summaries one reduce call can read
- $\text{depth}$: sequential rounds of map-reduce; refine always needs n

Both strategies read the document once, so their token totals are close. The difference is latency and what each call sees. Map-reduce runs its map calls in parallel and finishes in 2 rounds when the partials fit one reduce call. Refine has to wait for each call before starting the next.

**Reference points**

- **Wu et al. (2021):** OpenAI summarized whole books by recursively summarizing sections and then the summaries, with GPT-3 fine-tuned on human feedback
- **Goyal, Li and Durrett (2022):** human raters preferred zero-shot GPT-3 news summaries to those of fine-tuned summarizers, while ROUGE did not show this
- **Chain of Density (2023):** Adams et al.: repeatedly add missing salient entities at a fixed length; readers preferred summaries denser than a plain GPT-4 summary but not the densest
- **Faithfulness metrics:** QAGS (Wang et al., 2020) asks questions of summary and source; SummaC (Laban et al., 2022) applies NLI sentence by sentence (see **Hallucination and groundedness evaluation**)
- **Datasets:** CNN/DailyMail: about 300k news articles with highlight bullets; XSum (Narayan et al., 2018): 226,711 BBC articles with one-sentence summaries

## Strategies in a simulation

> Figure: A 96k-token report with 30 facts; the reference is the 10 most important. The simulated model keeps the facts it judges most important, under-uses the middle of long inputs, and under-rates facts whose context lies before the chunk it sees. Default (map-reduce, 16k chunks, 10-fact summary): 9 of 10 key facts in 7 calls and 2 steps, 98,240 tokens; the miss is the fact just after 64k whose context sits in the previous chunk. Stuffing a 32k window keeps 5 (the only key facts it can see). Refine keeps 3 with 4k chunks over 24 sequential calls, all from the second half, and 8 with 32k chunks.

The grid shows the two failure directions. Small chunks cut facts off from their context, so map-reduce at 4k keeps 7. Very large chunks bring back the lost-in-the-middle effect inside each map call, so 32k also keeps 7. Refine improves as chunks grow because fewer rewrites means less erosion of early facts.

## Worked example: a 200k-token contract bundle

L = 200,000, C = 8,000, p = 300, s = 400, W = 32,000 (from the equations above)

| Quantity | Map-reduce | Refine |
| --- | --- | --- |
| Calls | 25 map + 1 reduce = 26 | 25 |
| Input tokens | 200,000 + 25 × 300 + (25 × 400 + 300) = 217,800 | 25 × 8,300 + 24 × 400 = 217,100 |
| Fan-in g | ⌊31,700 / 400⌋ = 79 | n/a |
| Sequential steps | 1 + ⌈log₇₉ 25⌉ = 2 | 25 |
| At L = 2,000,000 (n = 250) | 4 reduce calls, then 1: 3 steps | 250 steps |

The token bills differ by 0.3%. The wait does not: if a call takes about the same time either way, refine takes 25 call-times and map-reduce about 2. At 250 chunks the 250 partials (100,000 tokens) overflow one reduce call, so they are reduced in 4 groups of at most 79 and the 4 results in a final call.

```python
import re
from concurrent.futures import ThreadPoolExecutor

def n_tokens(text):                          # swap in the model's tokenizer
    return max(1, round(len(text) / 4))

def split_sections(doc, max_tok=8000):
    """Split on headings, then pack paragraphs so no chunk exceeds max_tok."""
    chunks = []
    for sec in re.split(r"(?m)^(?=#+ )", doc):
        cur = ""
        for para in sec.split("\n\n"):
            if cur and n_tokens(cur + para) > max_tok:
                chunks.append(cur)
                cur = ""
            cur += para + "\n\n"
        if cur.strip():
            chunks.append(cur)
    return chunks

MAP = ("Summarize this part of a report for a CFO in at most 5 bullets. Keep figures, "
       "dates and owners exactly. End each bullet with its page in [brackets].\n\n{text}")
REDUCE = ("Merge these partial summaries into at most 8 bullets, most important first. "
          "Remove duplicates. Keep every [page] citation.\n\n{text}")

def summarize(doc, llm, window=32000, chunk=8000):
    with ThreadPoolExecutor(max_workers=8) as pool:            # map calls run in parallel
        parts = list(pool.map(lambda c: llm(MAP.format(text=c)), split_sections(doc, chunk)))
    while True:                                                # reduce, recursively if needed
        joined = "\n".join(parts)
        if n_tokens(REDUCE.format(text=joined)) <= window:
            return llm(REDUCE.format(text=joined))
        groups, cur = [], []
        for p in parts:                                        # pack partials into windows
            if cur and n_tokens(REDUCE.format(text="\n".join(cur + [p]))) > window:
                groups.append(cur)
                cur = []
            cur.append(p)
        if len(groups) + 1 >= len(parts):                      # no progress: partials too long
            raise ValueError("partial summaries do not fit; shorten the map output")
        parts = [llm(REDUCE.format(text="\n".join(g))) for g in groups + [cur]]

# offline stand-in for the LLM: keeps the cited lines of whatever it is given
fake = lambda prompt: "\n".join(l for l in prompt.splitlines() if "[p." in l)
doc = "# Risks\n\nOne supplier provides 60% of parts [p.18]\n\n# MD&A\n\nRevenue rose 12% to $4.1B [p.41]"
print(summarize(doc, fake))
# One supplier provides 60% of parts [p.18]
# Revenue rose 12% to $4.1B [p.41]
```

Map-reduce with heading-aware chunks, parallel map calls and recursive reduce (plain Python; run with an offline stand-in for the LLM)

> **Fluent is not faithful:** Summaries invite unchecked trust: wrong numbers, merged entities ("the CEO" from one section, "the CFO" from another), and claims of certainty the source hedged. Make the map prompt cite a page or span for each claim, check each claim against its citation with an NLI model or an LLM judge, and drop what fails. Numbers should be copied, not recomputed.

> **Say who the summary is for:** "Summarize this" leaves the model to guess what matters. Name the reader and the decision ("a CFO deciding whether to refinance"), the length, the format and what to always include (dates, amounts, obligations, open risks). The reduce step should rank by that goal; otherwise it ranks by what was repeated most across chunks.

**Question:** A 2,000,000-token archive is summarized with 8k chunks, 400-token partials, a 300-token instruction and a 32k window. How many sequential rounds does map-reduce need?

- A. 1
- B. 2
- C. 3
- D. 250

**Answer:** C. 3 — n = 250 chunks. The fan-in is ⌊31,700/400⌋ = 79, so the 250 partials need ⌈250/79⌉ = 4 reduce calls, and their 4 outputs need one more: map, reduce, reduce = 1 + ⌈log₇₉ 250⌉ = 3 rounds. Refine would need 250.

**Question:** A refine-style summary of a long transcript mentions mostly what was said in the last hour. What is the most likely cause?

- A. The tokenizer drops early tokens
- B. Each rewrite of the running summary re-compresses earlier content, so early details are dropped step by step while new chunks arrive at full detail
- C. The model's temperature rises over time
- D. Map-reduce was used by mistake

**Answer:** B. Each rewrite of the running summary re-compresses earlier content, so early details are dropped step by step while new chunks arrive at full detail — In refine, an early fact must survive every later rewrite to reach the output, while a late fact needs to survive only a few. The simulation shows this: with 4k chunks, all 10 kept facts come from the second half of the document.

**Key points**

- Stuff if the document fits; map-reduce or refine if it does not. Split on structure (sections, speakers), not fixed sizes, where you can.
- Map-reduce: $n=\lceil L/C\rceil$ parallel calls and $1+\lceil\log_g n\rceil$ rounds; refine: $n$ sequential calls, similar token cost.
- Small chunks lose context, large ones under-use their middle, and refine erodes early content.
- Ask for citations in every partial and check claims against their spans before the reader sees the summary.
- Specify reader, purpose, length and must-keep items; ROUGE is a weak guide to summary quality for LLM output.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
