What is a vector, in the context of AI and machine learning?

A vector is simply an ordered list of numbers, which can be pictured as an arrow pointing from an origin to a specific position. In AI, an embedding — the numeric representation of a word, sentence, or image — is a vector, usually with hundreds or thousands of numbers instead of just 2 or 3.

What does the dot product actually measure?

The dot product multiplies two vectors position-by-position and adds up the results into a single number. That number measures how much two vectors point in the same direction: a large positive number means they align closely, near zero means they are roughly unrelated in direction, and a negative number means they point opposite ways.

How is the dot product used in real AI systems?

Two major uses: comparing embeddings for semantic search (finding which stored piece of text is closest in meaning to a query, via cosine similarity, which is built on the dot product), and scoring relevance inside a transformer's attention mechanism, where a query vector is dotted against key vectors to decide which other tokens to pay the most attention to.

AI & ML — INTRODUCTION

Vectors & Dot Products: The Math Behind Embeddings

Every time an AI system decides that “puppy” and “kitten” mean similar things, or that one search result is more relevant than another, there’s a vector and a dot product doing the actual work underneath. This page covers what both of those really are — not abstract math-class theory, but the exact mechanism embeddings and semantic search are built on.

The Quick Answer

A vector is just an ordered list of numbers — you can picture it as an arrow pointing from a starting point to some position. A dot product is a way of combining two vectors into a single number: multiply each pair of matching positions together, then add up all those results. That single number turns out to be extremely useful — it measures how much two vectors point in the same direction, which is exactly the building block behind comparing embeddings for meaning, and behind how attention works inside a transformer.

What Is a Vector?

A vector is an ordered list of numbers. (3, 4) is a vector. So is (0.9, 0.1, 0.05, 0.1). The simplest way to picture one: draw an arrow starting at the origin (0, 0) and ending at the point those numbers describe. The vector (3, 4) is an arrow that goes 3 steps right and 4 steps up.

Try it below — click a vector to draw it as an arrow, or click several to compare them:

Adding Two Vectors

Vectors can be added together, and the picture makes it intuitive: walk vector A, then from wherever you land, walk vector B. Where you end up is A + B. Arithmetically it’s even simpler — just add the matching positions: (3, 1) + (1, 3) = (4, 4).

What Is a Dot Product?

The dot product takes two vectors of the same length and combines them into a single number. The recipe is simple: multiply each pair of matching positions, then add everything up. For (2, 3) and (4, 1): (2×4) + (3×1) = 8 + 3 = 11.

Try the step-by-step version below on a few different vector pairs:

Why This Number Measures “Similarity”

A large positive dot product means the two vectors have large numbers in the same positions — they generally point the same way. A dot product near zero means the vectors don’t line up in any particular direction relative to each other. A negative dot product means they point in roughly opposite directions. This is the entire reason vectors are useful for AI at all: two embeddings representing similar meanings end up as vectors that point in similar directions — and the dot product is how a computer actually checks that.

See it in action: guess which of two candidate sentences is closer in meaning to an anchor sentence, using toy embedding vectors, then check your guess against the real computed similarity:

Fun Fact

One of OpenAI’s current production embedding models turns any piece of text you send it — a single word or several paragraphs — into a vector of exactly 3,072 numbers by default. A haiku and an entire book chapter both come out the same fixed length; only the direction that vector points in differs.

Source: OpenAI embeddings guide

Test Yourself

What is a vector, most simply?

If two vectors have a dot product close to zero, what does that suggest?

Why does an AI embedding count as a vector?

Brainstorm

No right answers here — just questions worth sitting with for a minute before you expand the model answer.

Before reading this page, if someone told you a computer "understands" that a cat and a dog are more similar than a cat and a car, what did you imagine was happening under the hood?

A 2D arrow on paper and a 1,536-number embedding vector are both called "vectors." What do they actually have in common, and what is genuinely different between them?

Why do you think multiplying matching positions and adding them up (the dot product) turns out to be a useful measure of "how similar" two things are, rather than some other arithmetic operation?

If a search engine could only compare exact words (no embeddings, no vectors at all), what kinds of searches would break first?

Picture a vector with 3 dimensions instead of 2 — you can still sort of imagine it in 3D space. What do you think actually changes, mathematically, once you go from 3 dimensions to 300? Is it a difference in kind, or just in scale?

Ready for the real formula behind cosine similarity, the geometric meaning of the angle between two vectors, and why dot-product attention needs a scaling factor? Continue to the Intermediate level →