Embeddings, Playable

EP 01

45 words, one map — click around and feel the geometry

Every dot below is a word placed so that distance = difference in meaning. Animals huddle top-left, feelings top-right, money bottom-right. Click any word (or pick from the menu) and the map draws lines to its five nearest neighbors, with a live similarity score for each. Then click bank and see something strange.

Select

Click any word on the map.

Real embeddings live in 300–3,000 dimensions and are learned from billions of sentences. This map is a 2-D, hand-built miniature — but nearness-as-meaning works the same way.

Why it matters: “happy” finding “joyful” and “glad” with no dictionary — just distance — is exactly how semantic search, recommendation feeds and every modern LLM decide what is related to what. And “bank” sitting awkwardly between the river and the money districts is a real bug you'll meet again in EP 05.

EP 02

Type a word, watch it land — or crash

An embedding model is a function: word in, coordinates out. Type a word this little model knows and it drops onto the map right next to its relatives. Type a word it has never seen and it still returns coordinates — it has no choice — but they mean nothing. Try “monarch”, then try “zorblax”.

Type a word and press Embed it.

This toy knows the 45 map words plus ~30 relatives via a hand-built lookup. Real models learn the lookup from data — but they too output something for junk input.

The trap: an embedding model never says “unknown word” — out-of-vocabulary input just gets arbitrary coordinates, and everything downstream (search, clustering, RAG) silently trusts them. This is why typos, product codes and new slang quietly wreck semantic search in production.

EP 03

Word algebra: king − man + woman ≈ queen

Because meanings are vectors, you can do arithmetic on them. Subtracting man from king isolates a direction that means roughly “royalty”; add woman and you land on queen. Watch the arrows do it — then build a nonsense equation and watch the same arithmetic produce confident garbage.

− +

The gold arrow is the difference A − B; the same arrow is then re-drawn starting at C. The star is the computed point; the circled word is whatever happens to be nearest.

Why it matters: this parallelogram trick, discovered in word2vec (2013), was the moment people realized geometry had captured relations — gender, tense, capital-of — not just similarity. But the math never refuses: a meaningless equation still returns a “nearest word”, just with a giveaway huge distance. Always check the distance, not just the answer.

EP 04

The similarity dial — where do you draw the line?

Every real system that uses embeddings has a threshold: “count two things as related if similarity ≥ X”. Drag the dial. Strict → clean synonym pairs. Loose → the map floods with edges and cat becomes “related” to money. Both edge counts below are computed live from all 990 word pairs.

Min similarity

The engineering truth: there is no universally correct threshold. Too strict and your search engine misses “car” when you typed “automobile” (false negatives); too loose and it returns everything (false positives). Tuning this one number is half the job of building semantic search — and the red cross-cluster edges are exactly the junk results users complain about.

EP 05

The polysemy trap — one word, two meanings, one point

“Bank” means a river's edge and a place for money — but a classic word embedding must assign it exactly one point. So it gets parked in no-man's-land between both neighborhoods. Put “bank” into each sentence below and watch the point refuse to move; then check an unambiguous word for contrast.

Pick a sentence.

How the real world fixed it: this failure is why contextual embeddings (BERT, GPT) were such a leap — they embed the word in its sentence, so “bank” gets a different vector each time. But sentence-level and document-level embeddings used in search and RAG still average meanings together, so the trap you just triggered is alive in production systems today.