trendingAI & ML — INTRODUCTION
What Are Embeddings? How Text Becomes Numbers, Explained
Search “cheap flights” and a good search engine also shows you results for “affordable plane tickets” — even though you never typed those exact words. Embeddings are the reason that works.
The Quick Answer
An embedding turns a word, sentence, or document into a long list of numbers — a position in a kind of numeric map of meaning. Text with a similar meaning ends up positioned close together on that map, and unrelated text ends up far apart. That’s the entire trick: comparing numbers turns out to be a surprisingly good stand-in for comparing meaning.
What the Experts Say
OpenAI describes its own embedding models in similarly plain terms: an embedding is just a list of numbers, and the mathematical distance between two of these lists is what signals how related the underlying text actually is. Their production embedding models output lists of 1,536 numbers by default for the smaller model, and 3,072 for the larger one — every single piece of text you send in gets reduced to a position defined by that many coordinates.
How It Actually Works
- 1
Every word or sentence becomes a list of numbers. An embedding model reads text and outputs a fixed-length list of numbers — often over a thousand of them — that represents that text’s meaning, not its spelling.
- 2
Similar meanings end up close together. Words like "dog" and "puppy" get placed near each other in this numeric space, while unrelated words like "dog" and "bicycle" end up far apart — even though none of that closeness was hand-programmed.
- 3
Closeness is measured, not guessed. There’s a standard way to measure how "close" two of these number-lists are — it’s just geometry, checking the angle between two positions in that numeric space rather than comparing the words themselves.
- 4
This is the foundation of semantic search. Search that understands "cheap flights" and "affordable plane tickets" mean roughly the same thing is only possible because both phrases land in nearly the same spot in embedding space.
Where Do These Words Live?
Real embeddings use hundreds or thousands of numbers per word — impossible to draw. But the same core idea holds even if you shrink it down to just 2 numbers per word, which you can actually plot on a simple map. Click any word below and see which other words it lands closest to.
The Famous “Word Math” Example
One result made embeddings famous back in 2013: take the embedding for “king,” subtract “man,” add “woman,” and the closest resulting position in that numeric space turns out to be “queen” — meaning encoded well enough that it survives actual arithmetic.
Here’s the honest part often left out: that clean result only shows up once you deliberately exclude the three input words themselves from the search. Leave them in, and the nearest match to that same equation is usually just “king” again — the arithmetic barely nudges it away from where it started. Try both versions below.
Fun Fact
One of OpenAI’s current embedding models turns every piece of text you give it — a single word or several paragraphs — into exactly 3,072 numbers by default. A haiku and an entire chapter both come out the same fixed length; only the position they land at differs.
Test Yourself
What does an embedding model actually output for a piece of text?
Why do "dog" and "puppy" end up close together in embedding space, while "dog" and "bicycle" don’t?
Ready for the actual formula behind “closeness,” how embedding models are trained, and how to pick one for a real system? Continue to the Intermediate level →