Definition
The dot product of two vectors is the number — multiply matching coordinates and add. Geometrically it equals , where is the angle between them.
At a glance
- Notation
- , ,
- Input → output
- two vectors of the same length → one number
- Cost
- multiplications, additions
- Key property
- ⇔ orthogonal (for nonzero vectors);
- Prerequisites
- coordinates, Pythagoras, cosine
- Used in
- neurons, attention, cosine similarity, projections, every matrix product
The intuition
A vector is an arrow from the origin — or, equally, a list of numbers such as one row of a dataset. The dot product asks how much two arrows point the same way, weighted by their lengths. Same direction gives a large positive number, perpendicular gives exactly zero, opposite gives a large negative number.
Read as arithmetic, is a weighted sum: each feature times its importance . That single operation is a linear model, one neuron, and one entry of every matrix product.
The math
- -th coordinate of
- Euclidean length of a
Directly from the sum: it is symmetric (), linear in each argument (), and with equality only for .
Why the cosine. The triangle with sides , and obeys the law of cosines, . Expanding the left side by linearity gives . Comparing the two:
Solve for the cosine and you get cosine similarity, which ignores length. Divide by instead and you get the scalar projection — the signed length of 's shadow on :
- the vector shadow of on the line through
Because , the dot product can never exceed the product of the lengths: (Cauchy–Schwarz), with equality only when the vectors are parallel.
See it
Worked example
, . Then , , , so and . The shadow of on has length , and the projection vector is — exactly .
In code
1import numpy as np23a, b = np.array([3.0, 1.0]), np.array([1.0, 2.0])4dot = a @ b # 5.0 (same as np.dot(a, b))5cos = dot / (np.linalg.norm(a) * np.linalg.norm(b)) # ≈ 0.70716theta = np.degrees(np.arccos(cos)) # ≈ 45.07proj = (dot / (b @ b)) * b # array([1., 2.])89# score 3 samples (rows) against one weight vector10X = np.array([[1, 0], [0, 1], [2, 2]]); w = np.array([0.5, -1.0])11X @ w # array([ 0.5, -1. , -1. ])Where it's used
Neurons & linear models
scores how aligned an input is with the learned weights.
Attention
Scores decide how much each token reads from every other.
Similarity search
Cosine similarity between embeddings ranks documents, products and images.
Least squares
Regression projects onto the columns of — built entirely from dot products.
Check yourself
, . What is the angle between them?
Check yourself
, and . What is ?
Remember
- ✓.
- ✓Sign gives the angle: positive below 90°, zero exactly at 90°, negative beyond.
- ✓Cosine similarity removes length; is the projection length.
- ✓Neurons, attention and similarity search are all dot products.