Vector vs Keyword Search, Playable

EP 01

The literal librarian — search by shared words

Classic search (BM25, the engine behind decades of search boxes) scores a document by which of your query words it contains, weighting rare words more than common ones. Below are the 9 articles in our help-desk KB. Type a query: matched words light up, and the score is the fraction of your query's weight the document covers.

The pattern: when you happen to use the author's words (“reset password”), keyword search is fast, cheap and exactly right. Say the same thing in different words (“car won't start” vs “engine fails to turn over”) and it returns junk or nothing — it literally cannot see synonyms. This is the vocabulary-mismatch problem that killed a thousand intranet search boxes.

EP 02

The meaning map — words as points in space

Vector search starts by giving every word (and document) coordinates in a “meaning space”, so that similar meanings sit close together. This map shows our vocabulary's hand-built vectors, flattened to 2D. Click any word — its nearest neighbours by cosine similarity light up. Then click “charge” and look closely at who its neighbours are.

Vectors here are hand-built over 7 topic dimensions so the demo is transparent — real embeddings learn 1,000+ dimensions from data, but distance means the same thing.

The catch you just saw: “charge” gets one point on the map, but it means two things (battery charge / billing charge). Its vector ends up a compromise, dragged toward whichever sense is more common. One-vector-per-word is why classic embeddings botch polysemy — and why modern systems embed whole sentences in context instead of single words.

EP 03

Head-to-head — “car won't start” meets its true answer

Same query, same 9 documents, both librarians side by side. Keyword scores by shared words; vector search embeds your query and ranks by cosine similarity to each document's vector. The ★ marks the document a human would want. Try “car won't start” — it shares zero content words with “Engine fails to turn over on cold mornings”.

Keyword ranking

Vector ranking

Why it matters: users describe symptoms; authors describe solutions. The overlap in actual words is often zero. Vector search bridges that gap by matching meaning — this is the retrieval half of every RAG pipeline, and the reason “semantic search” became the default for docs, support and code search.

EP 04

The revenge of the exact string — error 0x80070057

Now flip the tables. You hit error 0x80070057 and paste it into search. That hex code has no “meaning” — it's an arbitrary identifier the embedding has essentially never seen, so it contributes almost nothing to the query vector. Watch the vector librarian confidently hand you the thematically similar but wrong article, while boring keyword search nails it.

Vector ranking

Keyword ranking

The rule of thumb: anything whose value lies in its exact characters — error codes, part numbers, function names, people's names, SKUs — is where pure vector search quietly fails. It rounds your query to the nearest theme. Real incident postmortems are full of “semantic search returned the general guide, not the KB article with the actual code”.

EP 05

Hire both librarians — the hybrid dial

Production search blends both scores: final = (1−α)·keyword + α·vector. Drag α and watch two hard queries at once: one needs meaning, one needs an exact string. Green means the blended ranking puts the correct document first; red means it doesn't. Find the zone where both are green.

α (keyword ← → vector) 0.50

“car won't start” — blended top 3

“error 0x80070057” — blended top 3

This is real: Elasticsearch, Vespa, Weaviate and most RAG stacks ship exactly this blend (often as “reciprocal rank fusion”). Neither engine is the answer; the dial is. At α = 0 you can't find synonyms, at α = 1 you can't find error codes — production systems live in the middle, plus a re-ranker on top.