Norms & Distances

A norm measures a vector's size; L1, L2 and L∞ measure it differently, and each choice changes what 'close' means.

5 min read397 wordscore4 viewsupdated 2026-10-03

Definition

A norm is a function ∥⋅∥\lVert\cdot\rVert that gives every vector a non-negative length, is zero only for the zero vector, scales with the vector (∥αx∥=∣α∣ ∥x∥\lVert\alpha x\rVert=|\alpha|\,\lVert x\rVert) and obeys the triangle inequality ∥x+y∥≤∥x∥+∥y∥\lVert x+y\rVert\le\lVert x\rVert+\lVert y\rVert. The distance between two points is the norm of their difference, d(x,y)=∥x−y∥d(x,y)=\lVert x-y\rVert.

At a glance

Notation
∥x∥p\lVert x\rVert_p for the LpL_p norm; plain ∥x∥\lVert x\rVert usually means L2L_2
Input → output
a vector → a non-negative number
The common three
L1L_1 = sum of absolute values · L2L_2 = Euclidean length · L∞L_\infty = largest absolute coordinate
Key property
∥x∥∞≤∥x∥2≤∥x∥1\lVert x\rVert_\infty\le\lVert x\rVert_2\le\lVert x\rVert_1 for every xx
Prerequisites
vectors & the dot product
Used in
k-NN, k-means, lasso and ridge, adversarial robustness, gradient clipping

The intuition

Length sounds unambiguous until you ask how to combine coordinates. A taxi on a street grid travels ∣Δx∣+∣Δy∣|\Delta x|+|\Delta y| (L1L_1). A bird flies Δx2+Δy2\sqrt{\Delta x^2+\Delta y^2} (L2L_2). A chess king needs max⁡(∣Δx∣,∣Δy∣)\max(|\Delta x|,|\Delta y|) moves (L∞L_\infty). Same two points, three valid distances — and every algorithm built on distance inherits whichever one you pick.

The clearest picture of a norm is its unit ball {x:∥x∥≤1}\{x:\lVert x\rVert\le1\}: a diamond for L1L_1, a disc for L2L_2, a square for L∞L_\infty. Every property of the norm is visible in that shape.

The math

∥x∥p=(∑i=1n∣xi∣p)1/p  (p≥1),∥x∥∞=max⁡i∣xi∣=lim⁡p→∞∥x∥p\lVert x\rVert_p=\Big(\sum_{i=1}^n|x_i|^p\Big)^{1/p}\ \ (p\ge1),\qquad \lVert x\rVert_\infty=\max_i|x_i|=\lim_{p\to\infty}\lVert x\rVert_p
pp
order of the norm, 1≤p≤∞1\le p\le\infty
nn
number of coordinates
∥x∥1=∑i∣xi∣,∥x∥2=∑ixi2=x⋅x,∥x∥∞=max⁡i∣xi∣\lVert x\rVert_1=\sum_i|x_i|,\qquad \lVert x\rVert_2=\sqrt{\textstyle\sum_i x_i^2}=\sqrt{x\cdot x},\qquad \lVert x\rVert_\infty=\max_i|x_i|

Only L2L_2 comes from a dot product, which is why it alone has angles and orthogonal projections. Below p=1p=1 the formula breaks the triangle inequality — the "ball" stops being convex — so it is not a norm.

Why the shape matters for regularization. Minimizing a loss subject to ∥w∥1≤t\lVert w\rVert_1\le t means finding where the loss contours first touch a diamond. That usually happens at a corner, where some coordinates are exactly zero. This is why the lasso (L1L_1) produces sparse weights, while ridge (L2L_2, a round ball) only shrinks them.

See it

‖x‖2 = 1ab
L1 = 5.00L2 = 3.61L∞ = 3.00L2 = 3.61

Drag a and b. staircase = L1 path, straight = L2, bar = L∞ (largest gap). Dotted: all points at the same Lp distance from a as b.

fig.Slide p from 1 to ∞: the unit ball morphs from diamond to disc to square. Drag the points — for a = (−1.5, −0.5), b = (1.5, 1.5): L1 = 5, L2 ≈ 3.61, L∞ = 3. The dotted curve is every point as far from a as b is.

Worked example

x=(3,−4)x=(3,-4): ∥x∥1=7\lVert x\rVert_1=7, ∥x∥2=5\lVert x\rVert_2=5, ∥x∥∞=4\lVert x\rVert_\infty=4, and in between ∥x∥3=911/3≈4.498\lVert x\rVert_3=91^{1/3}\approx4.498 — the ordering ∞≤3≤2≤1\infty\le3\le2\le1 holds. Normalizing for L2L_2 gives the unit vector (0.6,−0.8)(0.6,-0.8).

The choice changes answers. From the origin, p1=(2,2)p_1=(2,2) and p2=(3,0)p_2=(3,0) are at L1L_1 distances 4 and 3, L2L_2 distances 2.832.83 and 3, L∞L_\infty distances 2 and 3. The nearest neighbour is p2p_2 under L1L_1 and p1p_1 under the other two.

In code

python
1import numpy as np
2from scipy.spatial.distance import cdist
3
4x = np.array([3.0, -4.0])
5np.linalg.norm(x, 1), np.linalg.norm(x), np.linalg.norm(x, np.inf) # (7.0, 5.0, 4.0)
6
7origin = np.array([[0, 0]]); pts = np.array([[2, 2], [3, 0]])
8cdist(origin, pts, "cityblock") # [[4., 3.]] L1
9cdist(origin, pts, "euclidean") # [[2.83, 3.]] L2
10cdist(origin, pts, "chebyshev") # [[2., 3.]] L∞
Norms of one vector, and pairwise distances under three metrics.

Where it's used

Nearest neighbours & clustering

The metric decides who is close; L1 is less dominated by a single large coordinate gap than L2.

Regularization

Lasso (L1) zeroes weights, ridge (L2) shrinks them, elastic net mixes both.

Adversarial robustness

Attacks are bounded in L∞L_\infty (each pixel changes by at most ε\varepsilon) or in L2L_2.

Training stability

Gradient clipping rescales any gradient whose L2 norm exceeds a threshold; embeddings are L2-normalized before cosine search.

Check yourself

For x=(1,−2,2)x=(1,-2,2), what are ∥x∥1\lVert x\rVert_1, ∥x∥2\lVert x\rVert_2 and ∥x∥∞\lVert x\rVert_\infty?

Check yourself

Why does L1 regularization produce weights that are exactly zero, while L2 does not?

Remember

  • ✓∥x∥p=(∑∣xi∣p)1/p\lVert x\rVert_p=(\sum|x_i|^p)^{1/p}: L1L_1 sum, L2L_2 Euclidean, L∞L_\infty max.
  • ✓Unit balls: diamond, disc, square — the shape is the norm.
  • ✓Only L2 has angles and projections; L1's corners create sparsity.
  • ✓Pick the norm deliberately, and standardize features first.
← back to Mathematics roadmap