What Is an Embedding? How AI Understands Meaning

Type "cheap places to stay in Baku" into a good search box and you'll get back a hostel near Fountain Square, a guesthouse in Icherisheher, maybe a budget hotel by the boulevard. Look closely at those results and there's a good chance none of them contains the word "cheap". Nobody writes "cheap" on their own listing. They write "affordable", "budget", "great value", or they just put the price.

So how did the search engine know? It didn't match your words. It matched what you meant. The thing that makes that possible is called an embedding, and once you get it, half of what people call "AI" stops looking like magic.

A word becomes a list of numbers

Computers are bad with words and very good with numbers. An embedding model takes a piece of text — a word, a sentence, a whole product description — and turns it into a long list of numbers. Something like this:

"cat" → [0.021, -0.344, 0.187, 0.903, -0.052, ... ]

That list usually has a few hundred to a few thousand numbers in it. 768 is common for open models, OpenAI's text-embedding-3-small gives you 1,536. You can't read them one by one, and that's fine: no single number means "animal" or "furry". What matters is the whole list taken together.

The easiest way to think about it: the list is a set of coordinates. A point on a map has two (latitude and longitude). An embedding is a point in a space with 768 directions instead of two. We can't picture that, so in the video above and in most explanations you'll see it squashed down to a flat 2D plane. Keep in mind it's a simplification, but a useful one.

Similar meaning, nearby points

Here's the part that matters. The model is trained on huge amounts of text, and it learns to put words that show up in similar contexts close to each other. "Cat" and "dog" both get fed, sleep on the sofa and go to the vet, so they land near each other. "Car" gets parked, refuelled and stuck in traffic on Heydar Aliyev Avenue, so it lands somewhere else entirely.

Nobody told the model that cats and dogs are pets. It figured out the neighbourhood from how people write. That's both the strength and the weakness of the whole approach, and we'll come back to it.

How "close" is measured

You'd think you'd just measure the distance between two points with a ruler. In practice most systems look at the angle between two vectors instead — draw an arrow from the origin to each point and see how far apart the arrows point. This is called cosine similarity.

• Arrows pointing almost the same way (a small angle) → score close to 1 → similar meaning.

• Arrows at a right angle → score near 0 → unrelated.

Why the angle and not the distance? Because the length of a vector often carries stuff you don't care about, like how long or wordy the text was. The direction is where the meaning lives. A short review that says "terrible service" and a long angry paragraph saying the same thing should point the same way, even if one arrow is longer.

The famous trick: king − man + woman

In 2013 a team at Google led by Tomas Mikolov released word2vec, and with it an example that has been in every AI talk since. Take the vector for "king", subtract "man", add "woman", and look for the nearest word to the result. You get "queen".

It works because the direction from "man" to "woman" is roughly the same as the direction from "king" to "queen". The model has picked up something like a "gender" direction in its space, without anyone defining it. The same happened with countries and capitals, verb tenses and so on.

A small honest footnote: in those demos, the original words are excluded from the answer. Without that rule, the closest point to "king − man + woman" is often just "king". The effect is real, but it's less clean than the slides make it look.

Still, the point stands. Meaning has become geometry, and geometry you can calculate with.

Where this is actually used

This isn't just a lecture topic. If you've used any of these in the last year, you've used embeddings:

Search that understands you

Instead of matching keywords, the system turns your query into a vector, then finds the documents whose vectors point in the closest direction. That's how "cheap places to stay" finds "budget guesthouse", and how "laptop for a student" finds a light notebook under 1,000 AZN, even though none of those words overlap.

Recommendations

"People who liked this also liked…" can be done by putting products, articles or films into the same space and suggesting neighbours. An online shop doesn't need a year of purchase history to start recommending — the descriptions alone are enough to get going.

Chatbots that know your business (RAG)

When a company wants a chatbot that answers from its own documents — price lists, policies, contracts — the usual approach is RAG, retrieval-augmented generation. All the documents are cut into chunks and embedded once. When a customer asks a question, the question is embedded too, the closest chunks are pulled out, and only those are passed to the language model to write an answer. The model doesn't have to "remember" your company; it just reads the right page at the right moment.

What it looks like in code

Less scary than it sounds. If your project already runs on Postgres (Supabase included), the pgvector extension lets you store embeddings right next to your normal data. Finding the five most similar products comes down to one query:

select title, price
from products
order by embedding <=> $1
limit 5;

Here $1 is the embedding of the user's query, and <=> is cosine distance. No separate vector database needed for most small and medium projects.

It's cheap, too. Embedding a catalogue of ten thousand products with a hosted model usually costs less than a cup of coffee, and you only do it once — then again only for items that change.

Where embeddings let you down

We like this technology a lot, which is exactly why it's worth being straight about its limits:

• Exact things. Product codes, phone numbers, invoice IDs, model names like "iPhone 15" vs "iPhone 16". Semantically they're almost identical, so a pure vector search happily mixes them up. Here a plain old keyword search is better.

• Negation. "Hotel with a pool" and "hotel without a pool" sit very close in vector space. The model sees the same topic and misses the one word that flips the meaning.

• Language. Many models were trained mostly on English. Azerbaijani and Russian work fine with good multilingual models, but quality varies a lot. Test on your own real data before you choose.

• Chunking. In RAG, how you cut documents matters as much as the model. Cut a price table in half and the chatbot will confidently quote the wrong number.

That's why in real projects we rarely use vectors alone. The usual setup is hybrid search: keyword search and vector search run side by side, and the results are merged. Exact matches stay exact, and fuzzy queries still find what people mean.

In one sentence

An embedding is meaning turned into numbers, so that "similar" becomes something a computer can measure. Almost everything that feels smart about modern search, recommendations and business chatbots sits on top of that one idea.

If you're thinking about smart search for your shop, a chatbot that answers from your own documents, or just want to know whether AI makes sense for your business at all, take a look at what we do with AI integration or drop us a message. We'll tell you honestly if a simple filter would do the job better.