Definition
Matrix multiplication combines an matrix and an matrix into the matrix , whose entry is the dot product of row of with column of . As maps, is the single linear map that applies first and then .
At a glance
- Notation
- ,
- Input → output
- — inner dimensions must match
- Cost
- multiply-adds; for square matrices
- Key properties
- associative and distributive; not commutative; ;
- Prerequisites
- vectors & the dot product
- Used in
- every neural-network layer, batch prediction, PCA, Markov chains, graphics
The intuition
A matrix is a machine that moves vectors: . It is completely described by where it sends the basis vectors — column of is the image of . Everything else follows by linearity, so lines stay straight, the origin stays put, and grid lines stay parallel and evenly spaced, though the grid may be stretched, sheared, rotated or flipped.
Two moves in a row are again a linear move, and its matrix is the product. Order matters for the same reason that "rotate, then shear" and "shear, then rotate" end in different places.
The math
- entry in row , column of
- the shared inner dimension
Three views of the same product
- 1Dot products: = (row of ) · (column of ).
- 2Columns: is a weighted sum of 's columns — so column of is applied to column of .
- 3Composition: for every . The definition is chosen precisely to make this true.
The determinant is the factor by which a square matrix scales area (volume in higher dimensions); its sign says whether orientation flips. For a matrix:
Composition multiplies scale factors. A determinant of zero means the map flattens the plane onto a line or a point: information is lost, and no matrix can undo it.
See it
Worked example
Shear , rotation by 90° . Then but . Rotate-then-shear sends to ; shear-then-rotate sends it to . Both products still have determinant : neither map changes area.
In code
1import numpy as np23A = np.array([[1, 1], [0, 1]]) # shear4B = np.array([[0, -1], [1, 0]]) # rotate 90°5A @ B # [[1, -1], [1, 0]] rotate, then shear6B @ A # [[0, -1], [1, 1]] shear, then rotate7np.linalg.det(A @ B) # ≈ 1.0 — area preserved89# dense layer on a batch: (batch, in) @ (in, out) -> (batch, out)10X = np.random.randn(32, 784)11W = np.random.randn(784, 128) * 0.0112H = X @ W # (32, 128): 32·784·128 ≈ 3.2M multiply-addsWhere it's used
Neural networks
A dense layer computes ; a forward pass is a chain of matrix products with nonlinearities between them.
Batch prediction
One product scores every example against every weight vector at once — the workload GPUs are built for.
Dimensionality reduction
PCA maps data to dimensions with one product, .
Graphs & Markov chains
gives -step transition probabilities; entries of count length- paths in a graph.
Check yourself
is and is . What is the shape of , and is defined?
Check yourself
and . What is , and what does do to orientation?
Remember
- ✓ = row of · column of ; inner dimensions must match.
- ✓Columns of are the images of the basis vectors; mixes the columns.
- ✓ means apply , then — not commutative.
- ✓ = area scale factor; ; zero means information lost.