Hybrid Search - Combining Semantic and Keyword Search

Hybrid search refers to combining two search methods that work differently and complement each other: semantic search based on embeddings and classic keyword search. Instead of choosing one path, hybrid search merges the results of both. This is especially decisive in the retrieval step of a RAG knowledge base, because answer quality depends on whether the right evidence is found.

The Two Building Blocks

• Semantic search (dense): Question and documents are represented as vectors. What is found is what is similar in meaning - even if not a single word matches. Strong with paraphrases, synonyms, and natural-language questions. Weak when an exact word matters.
• Keyword search (sparse, BM25): The classic full-text match, usually via the proven ranking method BM25. It scores how often and how distinctively a search term appears in a document. Strong with exact terms - product numbers, proper names, error codes, technical vocabulary. Weak when the user uses different words than the document.

Why the Combination Delivers Better Results

The strengths of the two methods lie exactly in the other's gaps:

1. Exact terms: Vector search can "smear over" a rare article number or a proper name, because such tokens carry little semantic content. BM25 hits them precisely.
2. Meaning over wording: If someone phrases a question differently than the document, keyword search finds nothing - semantic search understands the intent anyway.
3. Robustness: If one method misses a query, the other catches the hit. This reduces the number of cases where no matching evidence is found at all.

How the Results Are Merged

Both methods first produce their own result lists. These are fused into a common ranking - a common approach is to combine the positions from both lists so that documents ranked high in both move to the front. The method therefore weighs ranks rather than raw scores, which balances the differing value ranges of the two searches.

Reranking as a Second Stage

Hybrid search is often combined with reranking. The fast hybrid search first delivers a manageable set of candidates. A specialized reranker model then checks each candidate directly against the question and reorders the list so the truly most relevant passages end up at the top. These two stages - search broadly and fast, then reorder precisely - deliver the most stable retrieval quality in practice. How good the candidates are in the first place also depends on clean chunking of the documents.

Hybrid Search at Elasticbrains

At Elasticbrains we combine vector and keyword search, including reranking, so your knowledge base reliably answers both exact technical terms and freely phrased questions. See how this becomes a verifiable internal assistant on our service page internal AI assistants / RAG knowledge base.