# Named entity recognition (NER)

> NER finds and types entity spans by tagging every token in BIO; BiLSTM-CRF (Lample et al., 2016) reached 90.94 F1 on CoNLL-2003 and fine-tuned BERT-large 92.8, both scored by exact-match entity F1.

Source: https://www.mlfoundry.in/learn/named-entity-recognition-ner  
Section: Deep Learning › Deep Learning for NLP  
Level: core · 8 min read · updated 2026-10-15

**Named entity recognition:** (NER) is the task of finding the spans of a text that name real-world entities and assigning each span a type, such as person (PER), organisation (ORG), location (LOC) or miscellaneous (MISC) in the CoNLL-2003 shared task (Tjong Kim Sang and De Meulder, 2003). The term comes from the MUC-6 evaluation (1995). The standard neural recipe is a **sequence labeller**: tag every token with a BIO label, then read the spans off the tags.

Input: a tokenised sentence. Output: a set of (start, end, type) spans, e.g. {(Angela Merkel, PER), (Deutsche Bank, ORG), (New York, LOC)}. Spans have variable length, they cannot overlap in the flat setting, and the same string changes type with context: "Washington" is a person, a city or a government.

Before deep learning, NER systems were HMMs, maximum-entropy taggers and CRFs over hand-built features: capitalisation, prefixes and suffixes, word shape, part-of-speech tags and gazetteers (lists of known names). The best CoNLL-2003 English system (Florian et al., 2003) combined several such classifiers and reached 88.76 F1.

## Spans as tags: BIO and BIOES

The span set is encoded as one label per token. In **BIO** (IOB2), `B-X` opens an entity of type $X$, `I-X` continues it, and `O` is outside any entity. With 4 types that is $2\cdot4+1=9$ labels. **BIOES** (also called BILOU) adds `E-X` for the last token and `S-X` for single-token entities, $4\cdot4+1=17$ labels. It makes boundaries explicit and often helps slightly.

Not every tag sequence is valid. `O I-PER` is illegal in BIO, because `I-PER` must follow `B-PER` or `I-PER`. A tagger that predicts each token independently can emit such sequences; a CRF output layer learns to avoid them.

> Figure: NER as token classification. The sentence is split into WordPieces; the encoder (BERT, or a BiLSTM over character and word embeddings in Lample et al.) gives one vector per piece; a 9-way softmax or a CRF predicts a BIO tag. Only each word's first piece carries a label: "##kel" and the special tokens get −100 and are ignored by the loss. B opens a span and I extends it.

## BiLSTM-CRF

Lample, Ballesteros, Subramanian, Kawakami and Dyer ("Neural Architectures for Named Entity Recognition", NAACL 2016) built each word vector from a pre-trained embedding plus a character-level BiLSTM, which handles capitalisation and unseen names. A word-level **Bidirectional LSTM** then produces emission scores $P_{i,y}$ for every token $i$ and tag $y$. Huang, Xu and Yu (2015) proposed the same BiLSTM-CRF combination for tagging.

The CRF (Lafferty, McCallum and Pereira, 2001) scores the whole tag sequence with a learned transition matrix $A$ and normalises over all $K^n$ sequences:

$$
s(X,y)=\sum_{i=0}^{n}A_{y_i,\,y_{i+1}}+\sum_{i=1}^{n}P_{i,\,y_i},\qquad \log p(y\mid X)=s(X,y)-\log\sum_{\tilde y}e^{\,s(X,\tilde y)}
$$

- $P_{i,y}$: emission score of tag $y$ at token $i$, from the BiLSTM (or BERT) output through a linear layer
- $A_{y,y'}$: transition score from tag $y$ to $y'$; $y_0$ and $y_{n+1}$ are added start and end tags
- $\log\sum_{\tilde y}$: log-partition over all tag sequences, computed by the forward algorithm in $O(nK^2)$

Training maximises $\log p(y\mid X)$ of the gold tags. Prediction is $y^*=\arg\max_y s(X,y)$, found exactly by Viterbi decoding in $O(nK^2)$ (see **Part-of-speech tagging**). Because $A_{\text{O},\,\text{I-PER}}$ is learned to be very negative, illegal sequences lose. The model reached 90.94 F1 on CoNLL-2003 English without gazetteers; Ma and Hovy (2016) reported 91.21 with a character CNN instead of the character LSTM.

## Fine-tuned transformers and subword alignment

**BERT** turns NER into token classification: a linear layer and softmax on each final hidden state, trained with cross-entropy, no CRF. Words are split into WordPieces, so labels must be aligned. BERT used the representation of each word's **first** sub-token. In the Hugging Face token-classification examples, the other pieces get label $-100$, which `CrossEntropyLoss` ignores.

$$
\mathcal L=-\frac{1}{|\mathcal F|}\sum_{i\in\mathcal F}\log\operatorname{softmax}\big(Wh_i+b\big)_{y_i},\qquad W\in\mathbb R^{K\times H}
$$

- $\mathcal F$: positions holding the first piece of a word (all others are labelled $-100$)
- $h_i$: final-layer hidden state of piece $i$; $H=768$ for BERT-base
- $K$: number of tags, 9 for CoNLL-2003 BIO

BERT reported CoNLL-2003 F1 of 92.4 (base) and 92.8 (large) on the test set, 96.4 and 96.6 on dev. A CRF layer on top of BERT is still a common option when tag-sequence validity matters, for example with BIOES or on small training sets.

**Benchmarks and scoring**

- **CoNLL-2003 English:** Reuters news; 4 types (PER, ORG, LOC, MISC); 14,041 / 3,250 / 3,453 sentences in train / dev / test with 23,499 / 5,942 / 5,648 entities
- **OntoNotes 5.0:** 18 entity types, including dates, money, quantities and cardinal numbers, across several genres
- **Metric:** micro-averaged **entity-level** F1: a predicted span counts only if its start, end and type all match a gold span (the `conlleval` script; `seqeval` in Python)
- **Reference scores:** pre-neural 88.76 (Florian et al., 2003); BiLSTM-CRF 90.94; BiLSTM-CNN-CRF 91.21; BERT-large 92.8 (test F1)

> Figure: Toggle typical tagger errors and watch both metrics. At the default ("Deutsche" dropped from Deutsche Bank) token accuracy is 13/15 = 86.7% but entity F1 is 0.750, because the truncated span counts as one false positive and one miss. Tagging everything O still scores 53.3% accuracy with entity F1 0. Turn on only "I- without B": lenient conlleval decoding still recovers Angela Merkel (entity F1 1.000) while token F1 falls to 0.857; strict IOB2 decoding drops the span and F1 falls to 0.857.

## Worked example: why the CRF fixes "I-PER I-PER"

Take "Angela Merkel", tags {O, B-PER, I-PER}. Emissions: Angela $(0.5,\,1.5,\,1.8)$, Merkel $(0.2,\,0.8,\,2.0)$. Per-token softmax gives Angela $(0.135,\,0.368,\,0.497)$, so an independent tagger outputs the illegal `I-PER I-PER`. Add transitions: start→I-PER $=-4$, O→I-PER $=-4$, B→I $=1$, I→I $=0.5$, B→B and I→B $=-1$, everything else 0.

All 9 sequences scored by s(X, y); log Z = 4.6695 (forward algorithm and brute force agree)

| Tags | Score s(X, y) | p(y \| X) |
| --- | --- | --- |
| B-PER I-PER | 0 + 1.5 + 1.0 + 2.0 = 4.50 | 0.8441 |
| B-PER O | 0 + 1.5 + 0 + 0.2 = 1.70 | 0.0513 |
| B-PER B-PER | 0 + 1.5 − 1 + 0.8 = 1.30 | 0.0344 |
| O B-PER | 0 + 0.5 + 0 + 0.8 = 1.30 | 0.0344 |
| O O | 0 + 0.5 + 0 + 0.2 = 0.70 | 0.0189 |
| I-PER I-PER | −4 + 1.8 + 0.5 + 2.0 = 0.30 | 0.0127 |
| O I-PER, I-PER O, I-PER B-PER | −1.50, −2.00, −2.40 | 0.0021, 0.0013, 0.0009 |

Viterbi returns `B-PER I-PER` with probability 0.844. The emissions alone preferred I-PER at "Angela"; the start→I-PER penalty of $-4$ overturns that, which is exactly what a CRF adds on top of per-token classification.

```python
def align_labels(word_labels, word_ids, label2id):
    """Label the first subword of each word; mask the rest and the special tokens with -100."""
    out, prev = [], None
    for w in word_ids:                       # from a fast tokenizer: enc.word_ids()
        out.append(-100 if w is None or w == prev else label2id[word_labels[w]])
        prev = w
    return out

def spans(tags):
    """BIO tags -> set of (start, end, type), conlleval-style (a stray I- opens a span)."""
    out, cur = set(), None
    for i, t in enumerate(tags + ["O"]):
        if cur and (t == "O" or t.startswith("B-") or t[2:] != cur[2]):
            out.add(tuple(cur)); cur = None
        if t != "O" and cur is None:
            cur = [i, i, t[2:]]
        elif cur:
            cur[1] = i
    return out

def entity_f1(gold, pred):
    g, p = spans(gold), spans(pred)
    tp = len(g & p)
    prec, rec = tp / max(len(p), 1), tp / max(len(g), 1)
    return 2 * prec * rec / max(prec + rec, 1e-12)

label2id = {"O": 0, "B-PER": 1, "I-PER": 2, "B-ORG": 3, "I-ORG": 4, "B-LOC": 5, "I-LOC": 6}
labels = ["B-PER", "I-PER", "O", "B-ORG", "I-ORG"]          # Angela Merkel met Deutsche Bank
word_ids = [None, 0, 1, 1, 2, 3, 4, None]                   # [CLS] Angela Mer ##kel met Deutsche Bank [SEP]
print(align_labels(labels, word_ids, label2id))   # [-100, 1, 2, -100, 0, 3, 4, -100]

gold = "B-PER I-PER O B-ORG I-ORG O O B-LOC I-LOC O O O B-ORG O O".split()
pred = "B-PER I-PER O O B-ORG O O B-LOC I-LOC O O O B-ORG O O".split()   # "Deutsche" missed
acc = sum(a == b for a, b in zip(gold, pred)) / len(gold)
print(f"token acc {acc:.3f}  entity F1 {entity_f1(gold, pred):.3f}")    # token acc 0.867  entity F1 0.750
```

Label alignment for subword models, and entity-level F1 versus token accuracy (pure Python, runs as is)

> **Report entity F1, not token accuracy:** Most tokens are O, so token accuracy is high even for a useless tagger: in the figure's sentence, all-O scores 53.3%, and on news text the share of O tokens is far higher. Entity F1 gives no credit for a partly right span. A boundary error costs a false positive **and** a false negative.

> **Flat tags cannot nest:** BIO assigns one label per token, so "Bank of China" (ORG) cannot also contain "China" (LOC). Nested NER (ACE, GENIA) uses span-based models that score every candidate span $(i,j)$ directly, or generative models that write the entities out as text.

**Question:** Gold has "New York" as one LOC entity. The tagger outputs New = B-LOC, York = O. How is this scored at the entity level?

- A. One true positive (most of the span is right)
- B. One false positive and one false negative
- C. One false negative only
- D. Half a true positive

**Answer:** B. One false positive and one false negative — The predicted span (New, LOC) does not exactly match the gold span (New York, LOC). It is a false positive, and the gold entity is still missed, a false negative. That is why boundary errors hurt entity F1 twice.

**Question:** A softmax tagger outputs O, I-PER, I-PER for "met Angela Merkel". What would a CRF output layer change?

- A. Nothing, it uses the same emissions
- B. It rescales the emissions to sum to 1
- C. Its transition scores make O → I-PER unlikely, so joint decoding prefers O, B-PER, I-PER
- D. It splits the span into two entities

**Answer:** C. Its transition scores make O → I-PER unlikely, so joint decoding prefers O, B-PER, I-PER — The CRF scores whole sequences: emissions plus learned transitions. $A_{\text{O},\,\text{I-PER}}$ is strongly negative after training, so Viterbi picks the best **valid** sequence, as in the worked example.

**Key points**

- NER = tag each token with BIO (9 labels for CoNLL-2003) or BIOES (17), then decode spans.
- BiLSTM-CRF: $s(X,y)=\sum A_{y_i,y_{i+1}}+\sum P_{i,y_i}$, trained with the forward algorithm, decoded with Viterbi; 90.94 F1 (Lample et al., 2016).
- BERT: linear + softmax per token on each word's first subword, other pieces labelled −100; 92.8 test F1 for BERT-large.
- Score with exact-match entity-level F1; token accuracy is inflated by the O class.
- Flat BIO cannot represent nested entities; span-based or generative models can.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
