Definition
An artificial neuron is a function : multiply each input by a weight, add them up with a bias, and pass the total through a fixed nonlinearity . It is a deliberately crude model of a brain cell. It keeps the idea of integrating many inputs and responding when the total is high, and it throws away almost everything else.
What a real neuron does
A cortical neuron receives signals on its dendrites, branching fibres that collect input from thousands of other cells. Each contact point is a synapse. When the upstream cell fires, the synapse releases neurotransmitter, which nudges the receiving cell's voltage up (excitatory) or down (inhibitory). How big the nudge is depends on the synapse's strength.
These nudges spread to the soma (cell body) and add up. At rest, the inside of the cell sits near mV. If the summed input pushes the voltage at the axon hillock past a threshold of about mV, the cell fires an action potential: an all-or-none spike, about 1 ms long, that travels down the axon to the next cells. Below threshold, nothing is sent.
The artificial neuron is that sentence turned into arithmetic: weighted sum, then threshold.
| Biology | Artificial neuron | Role |
|---|---|---|
| dendrites | inputs | receive signals from upstream units |
| synaptic strength (excitatory / inhibitory) | weight (positive / negative) | how much, and in which direction, each input counts |
| summing of potentials in the soma | combine all inputs into one number | |
| resting level vs firing threshold | bias (a threshold is ) | how much input is needed to respond |
| axon hillock: fire or not | activation | turn the total into an output |
| axon and its terminals | output , fanned out to the next layer | send the result on |
McCulloch–Pitts: the first formal neuron (1943)
- binary input: upstream neuron firing (1) or silent (0)
- fixed connection weight, set by hand (no learning yet)
- firing threshold
- binary output: fire or stay silent
Warren McCulloch and Walter Pitts showed that networks of such units can compute any Boolean function, so in principle a brain made of them could carry out any logical computation. In their model an active inhibitory input vetoes firing outright. The weights were fixed by the designer. Rosenblatt's perceptron (1958) added a rule for learning them from examples.
| AND () | OR () | |||
|---|---|---|---|---|
| 0 | 0 | 0 | 0 | 0 |
| 0 | 1 | 1 | 0 | 1 |
| 1 | 0 | 1 | 0 | 1 |
| 1 | 1 | 2 | 1 | 1 |
The modern artificial neuron
- real-valued input (a pixel, or the output of another neuron)
- learned weight; any real number, either sign
- learned bias; shifts the threshold
- pre-activation, the analogue of the summed membrane potential
- activation: step, sigmoid, ReLU, …
The hard step of McCulloch–Pitts has a derivative of zero everywhere it is defined, so gradient descent cannot train it. Modern networks swap in a smooth or piecewise-linear . The sigmoid can be read as a firing rate between 0 and 1, which is closer to how neuroscientists summarise a spike train than a single 0/1 output.
Worked example: three inputs
- 1Inputs , weights , bias . Input 2 is inhibitory.
- 2Weighted terms: , , .
- 3Sum: .
- 4Step: , so (fires). ReLU: .
- 5Sigmoid: . Read as a rate with a 100 Hz ceiling, that is about 62 Hz.
- 6To silence the step unit, strengthen the inhibitory input: , which is negative once . At , and . (Lowering alone would need .)
1import torch23x = torch.tensor([0.9, 0.2, 0.6])4w = torch.tensor([0.8, -1.2, 0.5])5b = torch.tensor(-0.3)67z = w @ x + b # tensor(0.4800)8print((z >= 0).float()) # step: 1.09print(torch.sigmoid(z)) # sigmoid: 0.617710print(torch.relu(z)) # ReLU: 0.48001112# A layer is many such neurons sharing the same inputs:13layer = torch.nn.Linear(3, 128) # 128 neurons, each with 3 weights + 1 biasWhere the analogy breaks
| Aspect | Biological neuron | Artificial neuron |
|---|---|---|
| Signal | discrete spikes; information in rate and precise timing | one real number per forward pass |
| Time | continuous dynamics; potentials leak and decay over milliseconds | no time at all (outside recurrent models); one synchronous step |
| Connections | roughly synapses per cortical neuron; billion neurons in a human brain | as many as the layer has inputs; a large model has billions of weights in total |
| Dendrites | compute nonlinearly on their own; single human cortical neurons can compute XOR in their dendrites (Gidon et al., 2020) | a plain linear sum |
| Sign of output | a given neuron is excitatory or inhibitory, not both (Dale's principle) | any weight can take either sign and flip during training |
| Learning | local rules: synapses change from the activity of the two cells they join (Hebbian learning, spike-timing plasticity) | backpropagation: a global error signal sent backwards through exact copies of the forward weights |
| Energy | whole brain runs on about 20 W | training a large model takes megawatt-hours |
Check yourself
In the artificial neuron, which quantity plays the role of synaptic strength?
Check yourself
A McCulloch–Pitts unit has two binary inputs with weights 1 and 1. Which threshold makes it compute OR?
Remember
- ✓Dendrites → inputs, synapses → weights, soma → weighted sum, axon hillock → activation, axon → output.
- ✓McCulloch–Pitts (1943): binary inputs, fixed weights, hard threshold; enough for any Boolean function.
- ✓Modern neurons use real values and smooth activations so they can be trained by gradient descent.
- ✓Real neurons spike, integrate over time, have ~10⁴ synapses with nonlinear dendrites, and learn with local rules.
- ✓Treat the brain as the source of the metaphor, not as a specification.