What is the geometric formula for the dot product?

A · B = |A| |B| cos(θ), where |A| and |B| are the magnitudes (lengths) of the two vectors and θ is the angle between them. This is mathematically equivalent to the multiply-and-sum formula, but makes explicit why the dot product depends on both how long the vectors are and the angle between them.

Why does cosine similarity divide the dot product by the magnitudes of both vectors?

Dividing by both magnitudes cancels out length entirely, leaving only the cosine of the angle between the vectors — a pure measure of direction. Without this division, a longer vector (for example, from a longer piece of text) could produce a larger raw dot product purely from its length, which would wrongly look like a stronger match.

What is a unit vector and why does it matter for similarity search?

A unit vector is a vector that has been rescaled to have a magnitude (length) of exactly 1, without changing its direction. If every vector in a collection is normalized to a unit vector ahead of time, a plain dot product between any two of them already equals their cosine similarity directly, which is why some embedding systems normalize vectors up front.

← Back to the Introduction level

AI & ML — INTERMEDIATE

The Dot Product Geometrically: Angle, Magnitude & Cosine Similarity

The Introduction level covered the dot product as pure arithmetic: multiply, then add. This level covers what that number actually means geometrically, why cosine similarity divides it by magnitude, and how a real similarity search ranks several candidates against one query.

The Quick Answer

The dot product has a geometric formula: A · B = |A| |B| cos(θ), where |A| and |B| are the lengths (magnitudes) of the two vectors and θ is the angle between them. This means the dot product mixes together two separate things — how long the vectors are, and how aligned their directions are. Cosine similarity isolates just the second part by dividing out both magnitudes, which is why it’s the standard measure used to compare embeddings regardless of how long the underlying text was.

The Geometric Meaning: A · B = |A| |B| cos(θ)

Fix a vector A, and rotate a second vector B around it. The dot product changes as the angle between them changes — even if their lengths never do. At 0° (pointing exactly the same way), the dot product is at its largest positive value. At 90° (perpendicular), it’s zero. Past 90°, it goes negative, reaching its most negative value at 180° (pointing exactly opposite).

Drag the angle below and watch the dot product respond:

Why Magnitude Can Mislead

Because the formula includes |A| and |B|, two vectors can have a large dot product simply because one of them is long — even if their directions haven’t changed at all. That’s a real problem for comparing embeddings: a longer document doesn’t necessarily produce a longer embedding vector by design, but if it did, you wouldn’t want “longer text” to be mistaken for “more relevant text.”

Unit Vectors and Normalization

A unit vector is any vector rescaled to have a magnitude of exactly 1, with its direction left untouched. Once a vector is normalized this way, a plain dot product against another normalized vector already equals cosine similarity directly — dividing by magnitude has been done once, ahead of time, instead of on every single comparison.

Ranking Real Candidates by Similarity

This is the exact operation a semantic search system performs at query time: take a query’s embedding, compute its cosine similarity against every candidate document’s embedding, and sort by the result. Try it below with a small toy collection of five documents:

How Many Dimensions Do Real Embeddings Use?

Everything above was drawn or computed in 2, 3, or 5 dimensions to stay legible. Real production embedding models use vectors with hundreds or thousands of dimensions — the math is identical, just with far more terms in the sum. See the real scale below:

Fun Fact

Some production embedding models are trained so their output vectors come pre-normalized to unit length. When that’s true, a plain dot product between two embeddings already is their cosine similarity — no separate normalization step needed at query time, which is a real, meaningful speed optimization at large scale.

Test Yourself

In the formula A · B = |A| |B| cos(θ), what happens to the dot product as θ approaches 90°?

Why does cosine similarity divide the dot product by |A| times |B|?

What does normalizing a vector into a unit vector actually do to it?

In the multi-document similarity ranking demo, what determines which document ranks first for a given query?

Brainstorm

Open-ended questions worth pausing on — expand each one to see one reasonable way to think it through.

The formula A · B = |A||B|cos(θ) and the "multiply matching positions and add them up" formula from the Introduction level compute the exact same number. Why do you think it is useful to have both descriptions of the same thing?

If you were designing a search engine and had to choose between comparing raw embedding vectors with a plain dot product, or first normalizing every vector and then comparing them, which would you pick, and why?

The dimension-count slider in this section shows real embeddings can have 1,536 or more dimensions. Why do you think embedding models use so many numbers instead of, say, just 10?

In the similarity ranking demo, the top-ranked document for a query never shared an exact word with it. What does that tell you about what "search" even means once embeddings are involved?

Try to picture, even loosely, what "perpendicular" could mean for two 300-dimensional vectors, given you cannot draw them. What do you think perpendicular vectors would represent in terms of meaning, for two embeddings?

Ready for dot-product attention itself — scoring a query against several keys, scaling by √d_k, and seeing exactly which key wins — plus the real computational cost of brute-force similarity search at scale? Continue to the Advanced level →