Definition
Feature map is the output of one filter of a convolutional layer: a 2-D grid that holds the response of filter at every spatial position. A layer with filters produces feature maps, stored together as an activation tensor of shape : batch, channels, height, width.
Read the tensor two ways. A slice is a map: where in image feature fires. A fibre is a -dimensional vector: what is present at position , described in the vocabulary of that layer's detectors. The next convolution reads all channels in a small window, so it builds new detectors out of combinations of the old ones.
The name comes from the earliest CNNs. Fukushima's neocognitron (1980) arranged S-cells in "cell-planes", one per feature type, and LeCun et al.'s LeNet papers (1989, 1998) called the planes feature maps: every unit in a map shares the same weights, so the map is one feature detector scanned over the image. The word "channel" came later, by analogy with the RGB channels of the input. To the next layer, the 64 maps of a conv layer are simply a 64-channel image.
How a feature map is computed
- feature map of layer ; is the image, with for RGB
- the slice of filter that reads input channel ; filter has weights in total
- every output map reads every input map: this is where features get combined
- stride; a stride-2 layer halves H and W
- the nonlinearity (usually ReLU, after BatchNorm); without it stacked layers collapse into one linear filter
The gradient tells the same story backwards. With the pre-activation, : filter is updated by correlating its error map with every input map. Because the same weights are used at every , every position contributes to one shared gradient. That is weight sharing, and it is why a single filter learns to detect its feature anywhere in the image.
At a glance
- Shape
- in PyTorch (channels-first); TensorFlow defaults to
- One map
- the response of one filter at every position: translation equivariant, a shifted input gives a shifted map
- One fibre
- , the -dim descriptor of one location, seen through the layer's receptive field there
- Typical sizes
- ResNet-50 at : after the stem conv, then , , ,
- Memory
- bytes per value, for every layer whose output backprop needs to keep
- Sparsity
- after ReLU many entries are exactly zero; a map is often blank for an image without its feature
Channels are detectors, depth builds a hierarchy
In the figure, a layer-1 map responds to one local pattern: a dark-to-bright step in one direction. Layer 2 cannot see pixels. It sees four edge maps, and its corner filter puts positive weight on "bright to the right" just below its centre and on "bright below" just to its right. Its bias of acts as a threshold. A straight edge alone gives at most and leaves the unit off. Only a place where both edges meet switches it on. That is the general mechanism: each layer is a set of soft AND/OR detectors over the previous layer's detectors.
Zeiler and Fergus (2014, "Visualizing and Understanding Convolutional Networks") made this visible in a trained ImageNet network. They projected strong activations back to pixel space with a deconvnet. Layer 1 holds oriented edges and colour blobs. Layer 2 responds to corners and edge-colour conjunctions. Layer 3 picks up textures and repeated patterns such as meshes and text. Layer 4 shows class-specific parts such as dog faces and bird legs, and layer 5 responds to whole objects across large pose changes. They also observed that the lower layers converge within a few epochs, while the upper layers only develop after 40 to 50.
Resolution down, channels up
Almost every CNN trades spatial resolution for channels as it goes deeper (see CNN architecture). Each stride-2 stage halves and , so the number of positions drops by 4. The channel count is usually doubled at the same time, so the total number of activation values per stage halves, while the weights per layer, , grow by 4. Early maps are large, cheap in weights and expensive in memory. Late maps are tiny, rich in channels, and hold most of the parameters.
| Tensor | Shape (C × H × W) | Values | fp32 |
|---|---|---|---|
| conv1 (stem conv) | 802,816 | 3.06 MiB | |
| layer1 | 802,816 | 3.06 MiB | |
| layer2 | 401,408 | 1.53 MiB | |
| layer3 | 200,704 | 0.77 MiB | |
| layer4 | 100,352 | 0.38 MiB |
The final tensor is the network's description of the image: 49 positions, each a 2048-dim fibre. Global average pooling turns it into one 2048-dim vector for the classifier. Detection and segmentation heads instead use the maps of several stages at once (a feature pyramid), because the early maps keep the spatial detail that the late ones have lost.
Worked example: one corner unit
Probe the square image at its top-left corner pixel (5, 5)
- 1Layer 1. The window at covers rows and columns 4 to 6. The square starts at row and column 5, so the Sobel- filter sees and in its right column and nothing on the left: . Sobel- gives too. The fibre is before ReLU and after it: each opposite-polarity filter is the exact negative of its partner, so ReLU keeps one of each pair.
- 2Pool. The max-pool maps to pooled cell . Around it, the "edge →" map has at and just below it at , where the edge runs straight. The "edge ↓" map has at and to its right at .
- 3Layer 2. The ⌜ filter has weight 1 on exactly those four taps: , so . Its ⌟ partner reads the opposite polarities, which are 0 here: , . The bar filter gets . One position out of 100 lights up: sparsity 1%.
- 4Shapes and memory. : values. At this scale it is nothing. A real layer, e.g.
nn.Conv2d(64, 128, 3, padding=1)on a batch, outputs values (51.4 MB in fp32) from just 73,856 weights.
Adding pixel noise in the figure shows why the bias threshold matters. At noise 0.3 every layer-1 map is nonzero at about half of all positions, because ReLU passes any positive flicker. The layer-2 corner maps stay sparse (3% and 7% nonzero), and their peaks stay at the true corners, and .
Memory: activations dominate training
Backprop needs the input of every layer to compute that layer's weight gradient, so training keeps most feature maps alive until the backward pass reaches them. Their size scales with batch size and image area, and weights do not. Counting only the conv outputs and residual sums of ResNet-50 gives about 16.8 million values per image, 64 MiB in fp32. A batch of 256 then needs over 16 GiB for activations, against about 100 MiB for the 25.6M weights. The BatchNorm and ReLU outputs that autograd also saves come on top of that.
Looking inside: forward hooks
1import torch2from torchvision.models import resnet50, ResNet50_Weights3from torchvision.models.feature_extraction import create_feature_extractor45model = resnet50(weights=ResNet50_Weights.DEFAULT).eval()67# 1) Forward hooks: called as hook(module, inputs, output) after module.forward8acts = {}9def save(name):10 def hook(module, inputs, output):11 acts[name] = output.detach()12 return hook1314names = ["conv1", "layer1", "layer2", "layer3", "layer4"]15handles = [getattr(model, n).register_forward_hook(save(n)) for n in names]1617x = torch.randn(1, 3, 224, 224) # use a real, normalised image in practice18with torch.inference_mode():19 model(x)20for n in names:21 a = acts[n]22 print(f"{n:7s} {tuple(a.shape)} {a.numel() * 4 / 2**20:.2f} MiB")23# conv1 (1, 64, 112, 112) 3.06 MiB24# layer1 (1, 256, 56, 56) 3.06 MiB25# layer2 (1, 512, 28, 28) 1.53 MiB26# layer3 (1, 1024, 14, 14) 0.77 MiB27# layer4 (1, 2048, 7, 7) 0.38 MiB28for h in handles:29 h.remove() # hooks stay attached until removed3031# 2) Or build a model that returns the maps directly32fx = create_feature_extractor(model, return_nodes={"layer2": "mid", "layer4": "top"})33feats = fx(x) # {"mid": (1,512,28,28), "top": (1,2048,7,7)}3435# Which channels fire most at the centre of the last map? (a fibre)36fibre = feats["top"][0, :, 3, 3]37print(fibre.topk(5).indices)3839# Show the first 16 channels of layer1 as a grid of images40import torchvision.utils as vutils41maps = acts["layer1"][0, :16].unsqueeze(1) # (16, 1, 56, 56)42grid = vutils.make_grid(maps, nrow=4, normalize=True, scale_each=True)Use when
- +Spatial layout is preserved, so maps can be visualised, localised (Grad-CAM, detection) and reused by dense prediction heads
- +Weight sharing: one detector per channel scanned over every position, with translation equivariance for free
- +Multi-scale features come out of one forward pass: early maps are detailed, late maps are semantic
- +ReLU maps are sparse and compress well
Avoid when
- −Activation memory scales with batch × resolution and dominates training memory
- −Early high-resolution maps cost the most memory and bandwidth for the fewest parameters
- −Individual channels are often polysemantic and hard to interpret
- −Late maps are coarse (7×7 at stride 32), which loses small objects and fine boundaries
Check yourself
A conv layer maps a (16, 64, 56, 56) input to 128 channels with a 3×3 kernel, stride 2 and padding 1. What is the output shape?
Check yourself
Doubling the input resolution of a CNN from 224 to 448 (same model, same batch) changes training memory how?
Remember
- ✓A conv layer outputs : feature maps, one per filter. A map says where its feature is, a fibre says what is at one place.
- ✓Every output channel reads every input channel, so deeper filters are combinations of earlier ones: edges → corners/textures → parts → objects (Zeiler & Fergus 2014).
- ✓Going deeper, H and W halve per stage while channels double: fewer, richer positions, and more weights per layer.
- ✓Activation memory scales with batch × resolution and usually dominates training memory; checkpointing, mixed precision and smaller inputs reduce it.
- ✓Inspect maps with
register_forward_hookorcreate_feature_extractor; remember to remove hooks and to run underinference_mode.