Definition
Receptive field of a unit in a CNN is the region of the input image whose pixels can change that unit's value. For a stack of convolutions and poolings it is a square of side (the theoretical receptive field), and its centres on neighbouring units are pixels apart (the jump, or cumulative stride).
The term comes from neuroscience. Hubel and Wiesel (1959, 1962) mapped the patch of the visual field that drives a single neuron in the cat's visual cortex, and found that cells further along the pathway have larger fields and respond to more complex patterns. Fukushima's neocognitron (1980) built that layout into a network, and every CNN since has inherited it. A unit in layer 1 sees a patch. A unit in the last stage of a ResNet sees, in principle, more than the whole image.
The receptive field decides what a unit can know. A detector whose field is smaller than the object cannot recognise the object as a whole. A segmentation network whose output units see only 100 pixels cannot use the context of a 500-pixel scene. Architecture choices such as strides, pooling, dilation and depth are, among other things, ways of growing .
The recursion
- receptive-field side of a unit in layer , in input pixels; (a pixel sees itself)
- jump: input-pixel distance between neighbouring units of layer ; . Also called the output stride
- kernel size, stride and dilation of layer l (pooling layers count, with their window and stride)
- effective kernel size: a kernel with dilation 2 spans 5 positions
Why it works: a layer- unit reads units of layer . Neighbouring units of layer sit input pixels apart, so the of them span pixels between their centres. Each one also brings its own field of pixels, which adds on either side. Summing gives the recursion. Unrolled, : every layer adds times the stride accumulated below it. A conv after four stride-2 layers adds 32 pixels; the same conv at the start adds 2.
Seen from the gradient, the field is the support of . The value of a layer- unit depends on input pixel only if some chain of kernel taps connects them, and is a sum over those chains. Pixels outside have no chain and an exactly zero gradient.
How each ingredient grows r
- Stride-1 conv, k×k
- adds : linear growth with depth. Ten layers:
- Stride or pooling
- multiplies by , so every later layer adds times more. Exponential growth if repeated
- Dilation d
- with no extra weights and no loss of resolution. Doubling each layer gives
- 1×1 conv
- adds nothing (), but a strided still multiplies
- Global pooling / attention
- the whole input in one step
- Padding
- does not change ; it changes where the field sits and how much of it falls outside the image
Worked example
| Layer | k | s | d | k̂ | j before | r after | j after |
|---|---|---|---|---|---|---|---|
| input | 1 | 1 | |||||
| conv 3×3 | 3 | 1 | 1 | 3 | 1 | 1 | |
| conv 3×3, stride 2 | 3 | 2 | 1 | 3 | 1 | 2 | |
| max-pool 2×2, stride 2 | 2 | 2 | 1 | 2 | 2 | 4 | |
| conv 3×3, dilation 2 | 3 | 1 | 2 | 5 | 4 | 4 | |
| conv 3×3 | 3 | 1 | 1 | 3 | 4 | 4 |
The dilated layer contributes 16 of the 31 pixels: it has a large effective kernel and it comes after a jump of 4. The last ordinary conv adds 8, four times what the first one added. The same recursion gives the well-known VGG-16 values: at conv5_3 and after the last pooling layer, with jump 32.
Theoretical receptive field of ResNet-50 (torchvision)
- 1Stem: conv with stride 2 gives , . The max-pool with stride 2 adds : , .
- 2Only the convs grow (the convs add nothing). Stage 1 has three of them at : .
- 3torchvision puts each stage's stride on the first conv ("ResNet v1.5"). Stage 2: four at : , .
- 4Stage 3: six at : , . Stage 4: three at : , .
- 5So a final unit has a theoretical field of 427 px on a 224 px image. The original placement (stride on the first conv, as in He et al. 2015) multiplies one layer earlier and gives 483 px. Either way, much of the field lies in the zero padding.
The effective receptive field is much smaller
Not all pixels in the field matter equally. A central pixel reaches the top unit through many chains of taps, a pixel near the border through only a few. With uniform weights and no nonlinearity, is proportional to the number of chains, and that count is the convolution of every layer's tap pattern. By the central limit theorem it approaches a Gaussian.
- layer 's taps on the input grid: equal weights spaced pixels apart
- variance of one layer's taps; for a stride-1 , so six layers give
- the ERF widens like the square root of depth while grows linearly, so the ratio shrinks like
Luo, Li, Urtasun and Zemel (NeurIPS 2016, "Understanding the Effective Receptive Field in Deep Convolutional Neural Networks") made this precise. They showed that the ERF is approximately Gaussian, that it occupies only a fraction of the theoretical field, and that this fraction shrinks as with depth . Subsampling and dilated convolutions enlarge it quickly, while dropout leaves its Gaussian shape unchanged. With ReLUs it becomes less smooth but stays concentrated near the centre, and training on real data makes it grow. The figure reproduces the uniform-weight case: in 2-D the 95% window of ResNet-50's path count covers only 22% of its theoretical area.
1import torch, torch.nn as nn23def receptive_field(layers):4 """layers: list of (k, s, d). Returns [(r, j)] after each layer."""5 r, j, out = 1, 1, []6 for k, s, d in layers:7 r += d * (k - 1) * j8 j *= s9 out.append((r, j))10 return out1112print(receptive_field([(3,1,1), (3,2,1), (2,2,1), (3,1,2), (3,1,1)]))13# [(3, 1), (5, 2), (7, 4), (23, 4), (31, 4)]1415vgg16 = [(3,1,1)]*2 + [(2,2,1)] + [(3,1,1)]*2 + [(2,2,1)] + ([(3,1,1)]*3 + [(2,2,1)])*316print(receptive_field(vgg16)[-1]) # (212, 32)1718# Effective RF: gradient of one central output unit w.r.t. the input (Luo et al. 2016)19from torchvision.models import resnet5020body = nn.Sequential(*list(resnet50(weights=None).children())[:-2]).eval()21x = torch.randn(1, 3, 448, 448, requires_grad=True) # not zeros: ReLU'(0) = 022y = body(x) # (1, 2048, 14, 14)23y[0, :, 7, 7].sum().backward() # one central location, all channels24erf = x.grad.abs().sum(1)[0] # (448, 448) map of |dy/dx|25ys, xs = (erf > 0).nonzero(as_tuple=True)26print(ys.min().item(), ys.max().item()) # nonzero support: at most 427 rows wide27# The mass is concentrated near the centre: compare the 95%-energy box with 427.Use when
- +One closed-form recursion gives the field of any conv/pool stack
- +Tells you whether a unit can see a whole object or the needed context
- +Explains design choices: strided stems, stacked 3×3 convs (VGG), dilated segmentation backbones
- +The ERF view predicts that depth alone grows useful context only like √depth
Avoid when
- −The theoretical field overstates what a unit uses: the effective field is Gaussian and much smaller
- −Ignores padding: border units have part of their field outside the image
- −Skip connections create paths of several lengths; the formula gives the longest path's field
- −Says nothing about which pixels a trained unit actually relies on: measure gradients for that
Check yourself
A stack has a 7×7 conv with stride 2, a 3×3 max-pool with stride 2, then two 3×3 convs with stride 1. What is the receptive field of the last layer?
Check yourself
You stack more 3×3, stride-1 conv layers. How do the theoretical receptive field r and the width of the effective receptive field grow with depth L?
Remember
- ✓ and , with : every layer adds its kernel span times the stride accumulated below it.
- ✓Stride-1 convs grow r linearly; strides, pooling and doubling dilations grow it exponentially.
- ✓ResNet-50 (torchvision): r = 427 px with jump 32; VGG-16 pool5: r = 212.
- ✓The effective receptive field is roughly Gaussian with width , a shrinking fraction of the theoretical field (Luo et al. 2016).
- ✓Large fields are bought with stride (losing resolution), dilation (risking gridding) or global operations (pooling, attention).