Matrix Multiplication

A matrix is a linear map; multiplying two matrices composes the two maps.

6 min read414 wordscore3 viewsupdated 2026-10-03

Definition

Matrix multiplication combines an m×nm\times n matrix AA and an n×pn\times p matrix BB into the m×pm\times p matrix C=ABC=AB, whose entry cijc_{ij} is the dot product of row ii of AA with column jj of BB. As maps, ABAB is the single linear map that applies BB first and then AA.

At a glance

Notation
C=ABC=AB,  cij=∑kaikbkj\ c_{ij}=\sum_k a_{ik}b_{kj}
Input → output
(m×n)⋅(n×p)→m×p(m\times n)\cdot(n\times p)\to m\times p — inner dimensions must match
Cost
mnpmnp multiply-adds; O(n3)O(n^3) for square matrices
Key properties
associative and distributive; not commutative; (AB)⊤=B⊤A⊤(AB)^\top=B^\top A^\top; det⁡(AB)=det⁡Adet⁡B\det(AB)=\det A\det B
Prerequisites
vectors & the dot product
Used in
every neural-network layer, batch prediction, PCA, Markov chains, graphics

The intuition

A matrix is a machine that moves vectors: x↦Axx\mapsto Ax. It is completely described by where it sends the basis vectors — column jj of AA is the image of eje_j. Everything else follows by linearity, so lines stay straight, the origin stays put, and grid lines stay parallel and evenly spaced, though the grid may be stretched, sheared, rotated or flipped.

Two moves in a row are again a linear move, and its matrix is the product. Order matters for the same reason that "rotate, then shear" and "shear, then rotate" end in different places.

The math

cij=∑k=1naik bkj,A∈Rm×n, B∈Rn×p, C∈Rm×pc_{ij}=\sum_{k=1}^{n}a_{ik}\,b_{kj},\qquad A\in\mathbb{R}^{m\times n},\ B\in\mathbb{R}^{n\times p},\ C\in\mathbb{R}^{m\times p}
aika_{ik}
entry in row ii, column kk of AA
nn
the shared inner dimension

Three views of the same product

  1. 1Dot products: cijc_{ij} = (row ii of AA) · (column jj of BB).
  2. 2Columns: Ax=x1a1+⋯+xnanAx=x_1a_1+\dots+x_na_n is a weighted sum of AA's columns — so column jj of ABAB is AA applied to column jj of BB.
  3. 3Composition: (AB)x=A(Bx)(AB)x=A(Bx) for every xx. The definition is chosen precisely to make this true.

The determinant is the factor by which a square matrix scales area (volume in higher dimensions); its sign says whether orientation flips. For a 2×22\times2 matrix:

det⁡(abcd)=ad−bc,det⁡(AB)=det⁡A⋅det⁡B\det\begin{pmatrix}a&b\\c&d\end{pmatrix}=ad-bc,\qquad \det(AB)=\det A\cdot\det B

Composition multiplies scale factors. A determinant of zero means the map flattens the plane onto a line or a point: information is lost, and no matrix can undo it.

See it

AvîĵA = [1 0.5][-0.25 1]
î → (1, -0.25)ĵ → (0.5, 1)det A = 1.13A(1, 1.5) = (1.75, 1.25)
presets
compose · A = shear, B = rotate 90°
fig.Drag the tips of î and ĵ — they are the columns of A. The shaded square's area is det A; it turns rose when orientation flips and grey at det = 0 (try 'project'). The compose buttons play shear-then-rotate and rotate-then-shear: different results.

Worked example

Shear A=(1101)A=\begin{pmatrix}1&1\\0&1\end{pmatrix}, rotation by 90° B=(0−110)B=\begin{pmatrix}0&-1\\1&0\end{pmatrix}. Then AB=(1−110)AB=\begin{pmatrix}1&-1\\1&0\end{pmatrix} but BA=(0−111)BA=\begin{pmatrix}0&-1\\1&1\end{pmatrix}. Rotate-then-shear sends (1,0)(1,0) to (1,1)(1,1); shear-then-rotate sends it to (0,1)(0,1). Both products still have determinant 1⋅1=11\cdot1=1: neither map changes area.

In code

python
1import numpy as np
2
3A = np.array([[1, 1], [0, 1]]) # shear
4B = np.array([[0, -1], [1, 0]]) # rotate 90°
5A @ B # [[1, -1], [1, 0]] rotate, then shear
6B @ A # [[0, -1], [1, 1]] shear, then rotate
7np.linalg.det(A @ B) # ≈ 1.0 — area preserved
8
9# dense layer on a batch: (batch, in) @ (in, out) -> (batch, out)
10X = np.random.randn(32, 784)
11W = np.random.randn(784, 128) * 0.01
12H = X @ W # (32, 128): 32·784·128 ≈ 3.2M multiply-adds
Order matters; a dense layer is one matrix product over a batch.

Where it's used

Neural networks

A dense layer computes XWXW; a forward pass is a chain of matrix products with nonlinearities between them.

Batch prediction

One product scores every example against every weight vector at once — the workload GPUs are built for.

Dimensionality reduction

PCA maps data to kk dimensions with one product, XVkXV_k.

Graphs & Markov chains

PtP^t gives tt-step transition probabilities; entries of AkA^k count length-kk paths in a graph.

Check yourself

AA is 3×43\times4 and BB is 4×24\times2. What is the shape of ABAB, and is BABA defined?

Check yourself

det⁡A=2\det A=2 and det⁡B=−3\det B=-3. What is det⁡(AB)\det(AB), and what does BB do to orientation?

Remember

  • ✓cijc_{ij} = row ii of AA · column jj of BB; inner dimensions must match.
  • ✓Columns of AA are the images of the basis vectors; AxAx mixes the columns.
  • ✓ABAB means apply BB, then AA — not commutative.
  • ✓det⁡\det = area scale factor; det⁡(AB)=det⁡Adet⁡B\det(AB)=\det A\det B; zero means information lost.
← back to Mathematics roadmap