The perceptron

A single neuron sign(w·x + b) that adds y·x to its weights on every mistake; it can only draw a straight line, so it cannot learn XOR.

6 min read692 wordscore0 viewsupdated 2026-10-09

Definition

The perceptron is one artificial neuron with a hard threshold, y^=sign⁡(w⋅x+b)\hat y=\operatorname{sign}(w\cdot x+b), plus a learning rule: whenever it gets an example wrong, add y xy\,x to the weights (and yy to the bias). Rosenblatt introduced it in 1957–58. It was the first neuron that learned its weights from data instead of having them set by hand.

The first trainable neuron

The McCulloch–Pitts neuron of 1943 already computed a weighted sum and fired above a threshold, but someone had to choose the weights. Rosenblatt's contribution was the update rule: show the neuron labelled examples and let its own mistakes move the weights.

Everything a modern network does with one unit is already here: a weighted sum, a bias, a nonlinearity, and weights changed by an error signal. What is missing is a smooth activation and a way to train more than one layer.

The neuron and its update rule

y^=sign⁡(s),s=w⋅x+b=w1x1+⋯+wdxd+b\hat y=\operatorname{sign}(s),\qquad s=w\cdot x+b=w_1x_1+\dots+w_dx_d+b
x∈Rdx\in\mathbb{R}^d
input vector
w∈Rd, bw\in\mathbb{R}^d,\ b
weights and bias, the only things that are learned
ss
pre-activation (score): positive means class +1, negative means −1
y^∈{−1,+1}\hat y\in\{-1,+1\}
the predicted label; the step function sign(·) is the activation
if y s≤0:w←w+η y x,b←b+η y\text{if } y\,s\le 0:\qquad w\leftarrow w+\eta\,y\,x,\qquad b\leftarrow b+\eta\,y
y∈{−1,+1}y\in\{-1,+1\}
the true label
y s≤0y\,s\le0
a mistake: the score has the wrong sign (or is exactly 0)
η>0\eta>0
learning rate; with w and b starting at 0 it only rescales them, so η = 1 is standard

Why this helps: write x~=(1,x)\tilde x=(1,x) so the bias is just weight w0w_0. After an update on (x~,y)(\tilde x,y), the new value of y sy\,s on that same point is y (w+yx~)⋅x~=y s+∥x~∥2y\,(w+y\tilde x)\cdot\tilde x=y\,s+\lVert\tilde x\rVert^2. One update always pushes the offending point's score toward the correct sign. Correctly classified points are ignored.

The decision boundary is the set w⋅x+b=0w\cdot x+b=0. In two dimensions that is a straight line with normal vector ww, and the bias shifts it away from the origin. Every update rotates or shifts that line, but it is always a single line.

Logic gates as a test bench

Boolean gates are the classic test: four inputs, two classes. Below, a 1 output is the label +1+1 and a 0 output is −1-1.

x₁x₂ANDORNANDXOR
000010
010111
100111
111100
Truth tables. AND, OR and NAND each separate one corner of the square from the other three, which one line can do. XOR pairs opposite corners.

Run the rule from w=0w=0, b=0b=0, visiting the rows in the order shown, with η=1\eta=1. Written as (b,w1,w2)(b,w_1,w_2), OR stops at (−1,2,2)(-1,2,2) after 9 updates (6 epochs, the last one clean), i.e. the line 2x1+2x2=12x_1+2x_2=1. AND stops at (−4,3,2)(-4,3,2) after 18 updates (9 epochs). NAND is AND with every sign flipped and stops at (4,−3,−2)(4,-3,-2). XOR never stops.

input space · shaded = +1 side0011x₁ →x₂ ↑w = 0: no line yet−+++OR truth tablex₁x₂ysŷ00−100 ✗01+100 ✗10+100 ✗11+100 ✗w = (b, w₁, w₂) = (0, 0, 0)press step to visit (0, 0) mistakes so far: 0points wrong now: 4 / 4w = 0: every score is 0, and a scoreof 0 counts as a mistakemistakes per epoch (max 4)1234
epoch 0 · updates 0
fig.Pick a gate and press step (one input) or epoch (all four). The line is w·x + b = 0, the shaded side is predicted +1, and a red ring marks a misclassified point. OR and AND settle with zero mistakes in an epoch. Under XOR every epoch makes 4 mistakes and w returns to (0, 0, 0), so the mistake counter grows forever.

Worked example: one XOR epoch

visitxys = w·x̃y·sw after
1(0, 0)−100 → mistake(−1, 0, 0)
2(0, 1)+1−1−1 → mistake(0, 0, 1)
3(1, 0)+100 → mistake(1, 1, 1)
4(1, 1)−13−3 → mistake(0, 0, 0)
XOR from w = (b, w₁, w₂) = (0, 0, 0) with η = 1. Every visit is a mistake, and the four updates cancel.

The sum of the four updates is ∑iyix~i=−(1,0,0)+(1,0,1)+(1,1,0)−(1,1,1)=(0,0,0)\sum_i y_i\tilde x_i=-(1,0,0)+(1,0,1)+(1,1,0)-(1,1,1)=(0,0,0). With this visiting order the weights come back to exactly where they started, so epoch 2 is a copy of epoch 1, and so on forever. Other orders give different loops, but none ever ends, because no line separates XOR.

Why no line can do XOR

A four-line proof

  1. 1A separating line needs b<0b<0 for (0,0)(0,0), w2+b>0w_2+b>0 for (0,1)(0,1), w1+b>0w_1+b>0 for (1,0)(1,0) and w1+w2+b<0w_1+w_2+b<0 for (1,1)(1,1).
  2. 2Add the middle two: w1+w2+2b>0w_1+w_2+2b>0, so w1+w2+b>−bw_1+w_2+b>-b.
  3. 3The first condition makes −b>0-b>0, so w1+w2+b>0w_1+w_2+b>0.
  4. 4That contradicts the last condition. No (w1,w2,b)(w_1,w_2,b) exists.

Geometrically: the two positive points sit on one diagonal of the square and the two negative points on the other. A half-plane is convex, so if it contains both ends of a diagonal it contains the midpoint (0.5,0.5)(0.5,0.5). Both diagonals share that midpoint, so it would have to lie strictly on the positive side and strictly on the negative side at once.

Minsky and Papert's 1969 book Perceptrons made this kind of limit rigorous for single-layer perceptrons (XOR and parity are the simplest cases). Networks with a hidden layer could compute XOR, but there was no practical way to train their hidden weights. Interest and funding in neural networks fell sharply until backpropagation was popularised in 1986 (see the multilayer perceptron and backpropagation articles).

python
1import torch
2
3X = torch.tensor([[0., 0.], [0., 1.], [1., 0.], [1., 1.]])
4Xa = torch.cat([torch.ones(4, 1), X], dim=1) # x̃ = (1, x1, x2)
5
6def train(y, epochs=20):
7 w = torch.zeros(3) # (b, w1, w2)
8 for epoch in range(1, epochs + 1):
9 mistakes = 0
10 for xi, yi in zip(Xa, y):
11 if yi * (w @ xi) <= 0: # wrong side, or on the line
12 w += yi * xi
13 mistakes += 1
14 if mistakes == 0:
15 return w, epoch
16 return w, None # never converged
17
18print(train(torch.tensor([-1., 1., 1., 1.]))) # OR : (tensor([-1., 2., 2.]), 6)
19print(train(torch.tensor([-1., 1., 1., -1.]))) # XOR: (tensor([0., 0., 0.]), None)
The perceptron rule on the four gate inputs (bias folded in as x₀ = 1).

Check yourself

A perceptron has (b,w1,w2)=(−1,2,2)(b,w_1,w_2)=(-1,2,2). Which gate does it compute on inputs in {0,1}2\{0,1\}^2?

Check yourself

Why can no single perceptron compute XOR, however long it trains?

Remember

  • ✓Perceptron: y^=sign⁡(w⋅x+b)\hat y=\operatorname{sign}(w\cdot x+b); on a mistake, w+=y xw\mathrel{+}=y\,x and b+=yb\mathrel{+}=y.
  • ✓Each update moves the misclassified point's score toward the right sign by ‖x̃‖².
  • ✓Its decision boundary is always a single straight line (a hyperplane in d dimensions).
  • ✓AND, OR and NAND are linearly separable and are learned; XOR is not, and the rule cycles forever.
  • ✓Fixing this needs a hidden layer, and training hidden layers needs differentiable activations.
← back to Deep Learning roadmap