Definition
Hybrid search retrieves with a lexical ranker and a dense ranker in parallel and merges their result lists into one. The lexical side is almost always BM25, from the Okapi system (Robertson et al., TREC-3, 1994; reviewed in Robertson and Zaragoza, "The Probabilistic Relevance Framework: BM25 and Beyond", 2009). The dense side is a bi-encoder such as DPR (Karpukhin et al., EMNLP 2020). The two standard merges are reciprocal rank fusion (Cormack, Clarke and Büttcher, SIGIR 2009) and a convex combination of normalised scores, analysed by Bruch, Gai and Ingber ("An Analysis of Fusion Functions for Hybrid Retrieval", ACM TOIS 2023).
The two rankers fail in opposite ways. BM25 matches surface terms only. It finds "E42", part numbers, names and rare jargon, but scores "internet is really slow" against "pages load slowly" as zero, because "slow" and "slowly" are different terms without stemming. A bi-encoder (see Embedding models) matches meaning and handles that paraphrase, but compresses a chunk into one vector, where error codes E42 and E50 land almost on top of each other.
In a RAG system hybrid search is the first stage: it must get the right chunks into the top 50–100 candidates that the Reranking stage then orders (see RAG architecture). Both indexes are built from the same chunks (see Document chunking), and many Vector databases and search engines now host both.
The two scores
- chunks containing term , total chunks; rare terms get large idf (Lucene's always-positive form)
- occurrences of in chunk ; the fraction saturates towards
- term-frequency saturation; Lucene's default is 1.2, with 0 meaning presence only
- length normalisation, 0 (none) to 1 (full); default 0.75; is the mean chunk length
The dense score is a cosine between unit-normalised embeddings, found by approximate search (see Approximate nearest neighbor search (HNSW, IVF, PQ)). BM25 is unbounded and depends on query length; cosine lies in and real models crowd most scores into a narrow band. Adding the raw numbers would let whichever scale is larger dominate.
Fusion
- 1-based position in ranker r's list; a chunk the ranker did not return adds 0
- 60 in Cormack et al.; larger k flattens the gap between rank 1 and rank 10
- a score min-max normalised over the candidate list:
- weight of the dense side, tuned on labelled queries; is pure BM25, pure dense
RRF needs no score calibration and no training data, which is why it is the default in many engines. It throws away how far apart scores are: a chunk that wins one list by a wide margin counts the same as one that wins narrowly. The convex combination keeps that information but needs normalisation and one tuned weight. Bruch et al. found that a tuned convex combination beat RRF both in and out of domain, needed only a small labelled set to tune , and that RRF was more sensitive to its than usually assumed.
Reference points
- RRF (2009)
- Cormack et al. fused TREC and LETOR runs with k = 60; the title states the result: RRF outperformed Condorcet fusion and individual learning-to-rank methods
- BEIR (2021)
- Thakur et al., 18 zero-shot retrieval datasets: BM25 was a strong baseline, and several dense retrievers trained on MS MARCO fell below it out of domain; the motivation for keeping a lexical lane
- DPR vs BM25
- Natural Questions top-20 retrieval accuracy 78.4% for DPR vs 59.1% for BM25 (Karpukhin et al., 2020): dense wins in domain, on the data it was trained for
- Learned sparse
- SPLADE (Formal, Piwowarski and Clinchant, SIGIR 2021) predicts weights for expansion terms with BERT and still serves from an inverted index; it can replace BM25 in the lexical lane
- Engine knobs
- as of 2024: Lucene/Elasticsearch BM25 defaults k1 = 1.2, b = 0.75 and an RRF retriever with rank constant 60; Weaviate's hybrid query takes alpha (1 = pure vector, 0 = pure keyword)
Worked example: where RRF and α disagree
Query "router stuck during firmware update"; gold chunks d9 (the lights blink during an update) and d1 (E42, update failed). , mean length terms after stop-word removal.
BM25 for d9 (8 terms, k1 = 1.2, b = 0.75)
- 1Length factor , so each term seen once gets .
- 2idf: "during" is in 1 chunk, ; "firmware" and "update" are in 3, ; "router" is in 6, ; "stuck" is in none and adds nothing.
- 3, rank 1. The rare word "during" is worth more than "router" and "update" put together.
| Chunk | BM25 (rank) | cosine (rank) | RRF, k = 60 | convex α = 0.5 | convex α = 0.3 |
|---|---|---|---|---|---|
| d5 | 3.666 (2) | 0.929 (1) | 1/62 + 1/61 = 0.032522 | 0.8283 | 0.7597 |
| d9 (gold) | 5.583 (1) | 0.593 (3) | 1/61 + 1/63 = 0.032266 | 0.8191 | 0.8915 |
| d1 (gold) | 3.651 (3) | 0.879 (2) | 1/63 + 1/62 = 0.032002 | 0.8002 | 0.7417 |
| d12 | 0.871 (5) | 0.285 (6) | 1/65 + 1/66 = 0.030536 | 0.2314 | 0.2012 |
Under RRF, d5 wins on ranks 2 and 1, although BM25 scored d9 52% above it. The convex mix sees that margin: after min-max scaling d9 has against d5's , while d5 leads on the dense side to . The two tie at ; below that d9 is first. Recall@3 is 1.0 under every fusion here; only the order, and so MRR, changes.
In practice
Start with RRF at and a candidate window of 50–100 per lane. Build a labelled set of a few hundred real queries with their gold chunks, then measure recall@50 for each lane alone and fused (see RAG evaluation). If the labels exist, try the convex mix with swept from 0 to 1 and keep the best. Keep the tokeniser of the BM25 lane in step with the content: stemming fixes "slow"/"slowly" but can merge codes, so index identifiers as exact tokens.
1import numpy as np2from rank_bm25 import BM25Okapi3from sentence_transformers import SentenceTransformer45chunks = [...] # list[str]6bm25 = BM25Okapi([c.lower().split() for c in chunks], k1=1.2, b=0.75)7enc = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")8E = enc.encode(chunks, normalize_embeddings=True) # offline; ANN index in production910def hybrid(q, n=100, fusion="rrf", k=60, alpha=0.5):11 s_bm = bm25.get_scores(q.lower().split())12 s_de = E @ enc.encode(q, normalize_embeddings=True)13 top_bm = [i for i in np.argsort(-s_bm)[:n] if s_bm[i] > 0] # what an inverted index returns14 top_de = list(np.argsort(-s_de)[:n])15 cand = set(top_bm) | set(top_de)16 if fusion == "rrf":17 score = {d: 0.0 for d in cand}18 for lst in (top_bm, top_de):19 for pos, d in enumerate(lst):20 score[d] += 1 / (k + pos + 1)21 else: # convex over min-max normalised scores22 def mm(s):23 v = s[list(cand)]24 return {d: (s[d] - v.min()) / (v.max() - v.min() + 1e-9) for d in cand}25 nb, nd = mm(s_bm), mm(s_de)26 score = {d: alpha * nd[d] + (1 - alpha) * nb[d] for d in cand}27 return sorted(cand, key=score.get, reverse=True)[:n] # hand to the rerankerUse when
- +Catches both exact-term and paraphrase matches; more robust out of domain than either ranker alone
- +RRF needs no labels, no score calibration and one constant
- +The two lanes run in parallel, so latency is about that of the slower one
Avoid when
- −Two indexes to build, update and keep consistent with deletions and access rules
- −Convex fusion needs labelled queries and re-tuning after model or corpus changes
- −RRF ignores score margins and can promote a chunk both rankers rate as mediocre
Check yourself
Chunk A is 1st in BM25 and 40th in dense; chunk B is 5th in both. Which ranks higher under RRF with k = 60?
Check yourself
Why is min-max normalisation (or rank fusion) needed before combining BM25 and cosine scores?
Remember
- ✓BM25 catches exact rare terms (codes, names); dense retrieval catches paraphrase. Each fails where the other succeeds.
- ✓BM25: idf × saturating tf with length normalisation; defaults , .
- ✓RRF: ; label-free and scale-free, but blind to score margins.
- ✓Convex: after min-max scaling; needs a tuned α and often beats RRF when tuned.
- ✓Hybrid is a first stage: optimise recall of the candidate set and leave the final order to a reranker.