Cosine similarity vs Euclidean distance: what matters for image search.

Search by image is a magic trick

Photograph a stranger's sneakers and, a second later, a search engine hands you that exact pair from a catalog of millions. No text was typed. Nobody tagged the photo. The machine simply looked — and understood.

It's a genuinely good trick, and like all good tricks it collapses into a few small moves once you've seen them. Here is the collapse: a computer never sees a picture — it sees a grid of color numbers, one set per pixel. The first piece of real engineering is a way to squeeze that grid into a short list of numbers describing the picture's content, so that every photo of that sneaker becomes nearly the same list. A picture becomes a point in space — "space" in the mathematical sense: a coordinate system with a few thousand axes. From that moment, "find photos like this" stops being a vision problem and becomes a geometry problem: which points sit close to mine?

This post walks the chain one link at a time — five dots, each raising exactly the question the next dot answers — and ends with the two-line proof that ties the field's two famous distance measures together. The one piece of genuine machine learning you need to take on faith is that a neural network can arrange images so similar ones land close together. Everything after that is arithmetic you can check with a pencil.

First dot: every picture is a point

To a computer, a photo is millions of numbers describing color at positions. That representation is useless for search because it is loyal to the wrong things: shift the camera one centimeter and every number changes, while the meaning — "a sneaker" — stays exactly where it was. Search needs the opposite kind of summary: blind to the meaningless differences, faithful to the meaningful one.

So the image is passed through a neural network trained on millions of labeled photographs — classically a convolutional neural network (CNN), the model family that powered the image-recognition boom. Instead of taking the network's final classification ("28% sneaker, 9% boot, …"), you take the numbers of its second-to-last layer — its internal description of the image, written just before it commits to a label. That list of about 2,000 numbers is called an embedding: coordinates that place the image inside a space with about 2,000 axes. (Real systems use anywhere from a few hundred to 2,048 dimensions; we'll say 2,000 and mean it loosely.)

The property that makes the whole story work is this: images with similar content get placed at nearby points. Two photos of the same sneaker, shot from different angles in different light, land almost on top of each other; a sneaker and a sailboat land far apart. Nobody writes that rule down — the network learns it, because "similar images should get similar answers" is precisely what it is graded on during training.

From here on, the photo is gone; only the point remains. Searching a library of images is now literally searching a library of points — which leaves exactly one decision: what does "close" mean?

Second dot: two rulers for "close"

Mathematics offers two natural ways to measure distance between points, and you already know both.

Euclidean distance is the straight line between two points — how far you would have to walk from one to reach the other. Cosine similarity works differently: draw an arrow — a vector — from the origin to each point, and measure the angle between the two arrows. Do they point the same way (cos θ = 1), at right angles (cos θ = 0), or in opposite directions (cos θ = −1)?

Two rulers for the same job — and, left alone, they disagree. That disagreement is easiest to feel with real numbers. Describe each book with two numbers: its romance pages and its rocket-science pages. That pair is the book's embedding, in a miniature two-dimensional space. Book A = (5, 1): light on rockets. Book B = (20, 5): also mostly romance — just a longer book. Book C = (2, 5): mostly rockets.

• Straight-line distance from A to B: √(15² + 4²) ≈ 15.5. From A to C: √(3² + 4²) = 5. By the straight line, C is more than three times closer to A than B is.
• Angle from A: B sits in almost the same direction — cos θ ≈ 0.999, practically the same book. C points noticeably elsewhere — cos θ ≈ 0.55, a different subject.

So Euclidean says A's nearest book is C; cosine says it's B, with C a distant stranger. Who is right? Both — the answer depends on whether the length of the vector carries meaning. Here the vector's length is essentially the book's thickness (its total page count); if you care about subject, thickness is noise and the angle is the honest ruler. If raw amounts matter — someone wants the big romance novel — the straight line has a case.

Image embeddings face exactly this fork. A beach photo and a badly overexposed photo of the same beach are the same content at different magnitudes — as vectors, they point the same way but sit at different distances from the origin. Search-by-content wants the exposure ignored. So real systems reach for the angle — and then something curious happens. The argument between the rulers is about to end, not by crowning a winner, but by removing the thing they were fighting over.

Third dot: one division ends the argument

Every serious image-search system does one thing before anything else: it divides each embedding by its own length. The operation is called L2 normalization — L2 because it uses the straight-line (Euclidean) length from two paragraphs ago — and it moves every point onto the sphere of radius 1 around the origin. Each vector keeps its direction and loses its size. One division per image, done once, at index time.

After that, the fight is over, and the proof takes two lines. Writing ‖a‖ for the length of vector a, and a·b for the dot product — multiply matching coordinates, then add up the products — the squared Euclidean distance between two vectors expands as:

‖a − b‖² = ‖a‖² − 2(a·b) + ‖b‖²

On the unit sphere ‖a‖ = ‖b‖ = 1. And for unit vectors — only for unit vectors — the dot product equals the cosine of the angle between them: a·b = cos θ. What remains:

‖a − b‖² = 2 − 2 cos θ

Read it slowly: squared distance is a function of the angle alone. Two pairs of images with the same cosine are exactly the same distance apart — no approximation. Ranking by one ruler and ranking by the other produce the identical list, every time.

Three anchor points make the curve intuitive: identical directions (θ = 0°) sit at distance 0; perpendicular directions (θ = 90°) at √2, the distance between any two perpendicular unit vectors; opposite directions (180°) at 2, the farthest apart two unit points can ever be.

This is why tutorials and production code seem to talk past each other. Tutorials say "cosine similarity"; libraries like FAISS — the open-source similarity-search standard — happily search by straight-line distance, because after normalization the two are the same geography, and computing dot products is the one thing CPUs and GPUs do fastest. The equivalence isn't a party trick; it's why nobody ever has to choose.

Panel (b) holds the deeper point. Similarity was never in how long a vector was — only in where it pointed. The network's real job was to arrange directions on a sphere so similar content subtends small angles; magnitude was noise all along, and normalization is the honesty of throwing it away. (Keep the books in mind for the rare cases where length was the meaning — there, you simply skip the division.)

Fourth dot: why "check everything" dies

Now let the library grow. Say 100 million images, each embedded into 2,048 numbers — a concrete stand-in for our "about 2,000" — each stored as a 4-byte floating-point value.

• Memory: 10⁸ × 2,048 × 4 B ≈ 800 GB of vectors. Large, but one serious machine can hold it.
• Search: exact search compares the query against every stored point. Comparing two 2,048-dimensional vectors takes roughly 2 × 2,048 ≈ 4,000 multiply-and-add operations; across the whole library that is 4,000 × 10⁸ ≈ 4 × 10¹¹ operations — several hundred billion — for a single search.

The memory is a footnote; the search is the story. That 4 × 10¹¹ grows in a straight line with library size: double the catalog, double the work, forever. Hardware gets faster, but a straight line outlives every server purchase. Exact search is not slow because the code is bad — it is slow by definition.

The standard escape is to stop checking everything. The inverted-file (IVF) index borrows an idea older than computers: don't search the building, search the right shelf. Offline, the points are clustered into about √N cells — for 10 million points, √10,000,000 ≈ 3,160 cells — and each cell ends up holding about √N points. At query time, the search compares the query only against the 8 cells nearest to it (the common default), so it examines about 8 × √N points instead of N. At 10 million images: about 25,000 points instead of 10,000,000 — 0.25% of the work, a ≈400× speedup, the gap drawn in the figure.

Nothing is free: a true nearest neighbor sitting in a cell you didn't probe gets missed. But this is a dial, not a defect. Turn "cells probed" up and the index slides toward exactness, buying recall back with time. This is why serious retrieval systems report recall measured against exact search — so the price of the dial is always on the record.

Fifth dot: two numbers that name the disease

Built and deployed — but is it any good? Two numbers answer that, and the pair matters more than either alone. First, set the scene: each query returns a short list of neighbors — the 10 nearest points, say — and for evaluation we know which library points are truly similar to the query (say, photos of the same object class).

Precision: of the 10 returned points, how many were truly similar? It punishes wrong answers. Recall: of all the truly-similar points in the library, how many made it into the 10? It punishes misses.

Two systems can post the same F1 — the standard harmonic blend of precision and recall — while carrying opposite diagnoses: one needs an index tweak, the other a retrain. Read the shape of the pair before trusting any single number. The same discipline governs the IVF dial: every turn of the knob moves you along a precision/recall curve that you, not a benchmark, have to judge.

Connecting the dots

Look back up the chain. A photograph became a point — that was the learned part. "Similar" became a distance — geometry. One division revealed that the two famous distance measures were always the same ranking — algebra, two lines. "Find the nearest" became an organized search with a tunable dial — data structures. And "is it any good" became two numbers that name the disease — evaluation.

Five dots, each small enough to explain at dinner. The neural network gets the headlines, but it is only the first dot; the system lives or dies on the geometry around it — and the geometry is yours to reason about with a pencil. A ruler for the straight line, a protractor for the angle, and the knowledge that after one honest division, they never disagree.

One last connection, free of charge: nothing above is special to images. Replace the picture with a sentence, an audio clip, or a product description — modern systems embed them all into the same kind of space, normalize, and measure with the same rulers. Learn the geometry once, and every "search" you ever meet is the same search.

Next: 99.6% accuracy, completely useless → Get in touch