Computers need numbers, not words
Start with a basic fact. Computers work with numbers. They do not really deal in letters or meaning. So before an AI can do anything clever with language, it has to turn words into numbers.
The naive way would be to just number the words: cat is 1, dog is 2, and so on. But that is useless, because the numbers would not mean anything. There is no real sense in which cat being 1 and dog being 2 tells you they are both pets. An embedding is a smarter way to turn words into numbers, one where the numbers actually carry meaning.
A map of meaning
The best way to picture an embedding is as a location on a giant map. Imagine a map where every word has a spot, and words with similar meanings sit close together.
On this map, “cat,” “dog,” and “hamster” would all be clustered in one neighborhood, because they are all pets. “Car,” “truck,” and “bus” would sit in a different neighborhood. “Happy” and “joyful” would be almost on top of each other, while “happy” and “sad” would be far apart.
An embedding is just that spot on the map, written as a list of numbers. Close numbers mean close meaning. So the AI can measure how related two words are simply by checking how close their spots are. It has turned “are these two things similar?” into “how far apart are these numbers?”
Meaning, not spelling
The clever thing is that embeddings capture meaning, not just how a word is spelled. “Big” and “large” look nothing alike as words, but they mean nearly the same thing, so they sit close together on the map. “Bank” the riverbank and “bank” the place for money are spelled the same but mean different things, and a good system can place them apart based on how they are used.
This all comes from the same learning process behind other AI. By seeing which words show up in similar situations across huge amounts of text, the model works out which words belong near each other. Nobody places the words on the map by hand.
What embeddings are used for
This one idea, meaning as a location, powers a surprising amount of the technology you use.
- → Smart search that finds results by meaning, so a search for “cheap flights” also matches “affordable airfare.”
- → Recommendations that suggest items similar to what you liked.
- → Grouping similar documents, reviews, or support tickets automatically.
- → Helping a chatbot pull up the right piece of information to answer a question.
How embeddings fit with tokens
If you have read about tokens, here is how the two connect. First, text is broken into tokens, the small chunks a model reads. Then each token is turned into an embedding, the list of numbers that stands for its meaning. Tokens are the pieces; embeddings are the meaning of those pieces, written in a form a computer can work with. Together they are the first steps that let an AI do anything at all with your words.