# Feature maps

> A conv layer outputs a stack of 2-D maps, one per filter: channel c says where in the image feature c was found. Deeper layers combine earlier channels into edges, textures, parts and objects while the maps shrink and the channels multiply.

Source: https://www.mlfoundry.in/learn/feature-maps  
Section: Deep Learning › Convolutional Neural Networks  
Level: core · 11 min read · updated 2026-10-11

**Feature map:** is the output of one filter of a convolutional layer: a 2-D grid $a_c[i,j]$ that holds the response of filter $c$ at every spatial position. A layer with $C$ filters produces $C$ feature maps, stored together as an **activation tensor** of shape $(N, C, H, W)$: batch, channels, height, width.

Read the tensor two ways. A **slice** $a[n, c, :, :]$ is a map: *where* in image $n$ feature $c$ fires. A **fibre** $a[n, :, i, j]$ is a $C$-dimensional vector: *what* is present at position $(i,j)$, described in the vocabulary of that layer's $C$ detectors. The next convolution reads all $C$ channels in a small window, so it builds new detectors out of combinations of the old ones.

The name comes from the earliest CNNs. Fukushima's neocognitron (1980) arranged S-cells in "cell-planes", one per feature type, and LeCun et al.'s LeNet papers (1989, 1998) called the planes **feature maps**: every unit in a map shares the same weights, so the map is one feature detector scanned over the image. The word "channel" came later, by analogy with the RGB channels of the input. To the next layer, the 64 maps of a conv layer are simply a 64-channel image.

## How a feature map is computed

$$
a^{(l)}_{c}[i,j]=\varphi\Big(b^{(l)}_c+\sum_{c'=1}^{C_{l-1}}\sum_{u,v}W^{(l)}_{c,c'}[u,v]\;a^{(l-1)}_{c'}[\,s i+u,\;s j+v\,]\Big)
$$

- $a^{(l)}_c$: feature map $c$ of layer $l$; $a^{(0)}$ is the image, with $C_0=3$ for RGB
- $W^{(l)}_{c,c'}$: the $k\times k$ slice of filter $c$ that reads input channel $c'$; filter $c$ has $C_{l-1}k^2$ weights in total
- $\sum_{c'}$: every output map reads every input map: this is where features get combined
- $s$: stride; a stride-2 layer halves H and W
- $\varphi$: the nonlinearity (usually ReLU, after BatchNorm); without it stacked layers collapse into one linear filter

The gradient tells the same story backwards. With $z$ the pre-activation, $\partial L/\partial W_{c,c'}[u,v]=\sum_{n,i,j}\delta_c[n,i,j]\,a_{c'}[n,si+u,sj+v]$: filter $c$ is updated by correlating its error map $\delta_c=\partial L/\partial z_c$ with every input map. Because the same weights are used at every $(i,j)$, every position contributes to one shared gradient. That is weight sharing, and it is why a single filter learns to detect its feature anywhere in the image.

**At a glance**

- **Shape:** $(N,C,H,W)$ in PyTorch (channels-first); TensorFlow defaults to $(N,H,W,C)$
- **One map:** the response of one filter at every position: translation **equivariant**, a shifted input gives a shifted map
- **One fibre:** $a[n,:,i,j]$, the $C$-dim descriptor of one location, seen through the layer's receptive field there
- **Typical sizes:** ResNet-50 at $224^2$: $64\times112^2$ after the stem conv, then $256\times56^2$, $512\times28^2$, $1024\times14^2$, $2048\times7^2$
- **Memory:** $N\cdot C\cdot H\cdot W\cdot$ bytes per value, for every layer whose output backprop needs to keep
- **Sparsity:** after ReLU many entries are exactly zero; a map is often blank for an image without its feature

> Figure: A two-layer toy CNN with hand-set weights on a 20×20 image. Layer 1 has four Sobel filters, one per edge polarity, giving maps of shape (1, 4, 20, 20); after a 2×2 max-pool, layer 2 has three filters that each read all four pooled maps (4·3·3 + 1 = 37 weights). On the square, every edge map peaks at 4 (a Sobel step response, 1+2+1) and fires on 6% of positions. The corner ⌜ unit fires at exactly one pooled position, (2, 2), the top-left corner, with value 4, and the vertical-bar unit stays silent. Switch to bars and the bar unit reaches 12 on 18% of the map. Click any map to read the channel vector at that location, and toggle pre-ReLU to see the negative half that the ReLU discards.

## Channels are detectors, depth builds a hierarchy

In the figure, a layer-1 map responds to one local pattern: a dark-to-bright step in one direction. Layer 2 cannot see pixels. It sees four edge maps, and its corner filter puts positive weight on "bright to the right" just below its centre and on "bright below" just to its right. Its bias of $-10$ acts as a threshold. A straight edge alone gives at most $4+4=8<10$ and leaves the unit off. Only a place where **both** edges meet switches it on. That is the general mechanism: each layer is a set of soft AND/OR detectors over the previous layer's detectors.

Zeiler and Fergus (2014, "Visualizing and Understanding Convolutional Networks") made this visible in a trained ImageNet network. They projected strong activations back to pixel space with a deconvnet. Layer 1 holds oriented edges and colour blobs. Layer 2 responds to corners and edge-colour conjunctions. Layer 3 picks up textures and repeated patterns such as meshes and text. Layer 4 shows class-specific parts such as dog faces and bird legs, and layer 5 responds to whole objects across large pose changes. They also observed that the lower layers converge within a few epochs, while the upper layers only develop after 40 to 50.

> **Why the hierarchy appears:** Two architectural facts force it. Each unit's **receptive field** grows with depth (8×8 input pixels for a layer-2 unit in the figure), so deeper units can only be about larger structures. And each layer's filters are built from the previous layer's channels, so their features are compositions. Nobody tells the network to learn edges first. Edges are simply the most useful thing a $3\times3$ window of pixels can detect.

## Resolution down, channels up

Almost every CNN trades spatial resolution for channels as it goes deeper (see **CNN architecture**). Each stride-2 stage halves $H$ and $W$, so the number of positions drops by 4. The channel count is usually doubled at the same time, so the total number of activation values per stage halves, while the weights per layer, $C_{in}C_{out}k^2$, grow by 4. Early maps are large, cheap in weights and expensive in memory. Late maps are tiny, rich in channels, and hold most of the parameters.

Stage outputs of torchvision ResNet-50 for one 224×224 image (values counted by the module's lib; fp32 = 4 bytes)

| Tensor | Shape (C × H × W) | Values | fp32 |
| --- | --- | --- | --- |
| conv1 (stem conv) | $64\times112\times112$ | 802,816 | 3.06 MiB |
| layer1 | $256\times56\times56$ | 802,816 | 3.06 MiB |
| layer2 | $512\times28\times28$ | 401,408 | 1.53 MiB |
| layer3 | $1024\times14\times14$ | 200,704 | 0.77 MiB |
| layer4 | $2048\times7\times7$ | 100,352 | 0.38 MiB |

The final $2048\times7\times7$ tensor is the network's description of the image: 49 positions, each a 2048-dim fibre. Global average pooling turns it into one 2048-dim vector for the classifier. Detection and segmentation heads instead use the maps of several stages at once (a feature pyramid), because the early maps keep the spatial detail that the late ones have lost.

## Worked example: one corner unit

**Probe the square image at its top-left corner pixel (5, 5)**

1. **Layer 1.** The $3\times3$ window at $(5,5)$ covers rows and columns 4 to 6. The square starts at row and column 5, so the Sobel-$x$ filter sees $+1$ and $+2$ in its right column and nothing on the left: $3$. Sobel-$y$ gives $3$ too. The fibre is $z=(3,-3,3,-3)$ before ReLU and $a=(3,0,3,0)$ after it: each opposite-polarity filter is the exact negative of its partner, so ReLU keeps one of each pair.
2. **Pool.** The $2\times2$ max-pool maps $(5,5)$ to pooled cell $(2,2)$. Around it, the "edge →" map has $3$ at $(2,2)$ and $4$ just below it at $(3,2)$, where the edge runs straight. The "edge ↓" map has $3$ at $(2,2)$ and $4$ to its right at $(2,3)$.
3. **Layer 2.** The ⌜ filter has weight 1 on exactly those four taps: $z=(3+4)+(3+4)-10=4$, so $a=4$. Its ⌟ partner reads the opposite polarities, which are 0 here: $z=-10$, $a=0$. The bar filter gets $z=-17$. One position out of 100 lights up: sparsity 1%.
4. **Shapes and memory.** $(1,1,20,20)\to(1,4,20,20)\to(1,4,10,10)\to(1,3,10,10)$: $400\to1600\to400\to300$ values. At this scale it is nothing. A real layer, e.g. `nn.Conv2d(64, 128, 3, padding=1)` on a $(32,64,56,56)$ batch, outputs $32\cdot128\cdot56\cdot56=12{,}845{,}056$ values (51.4 MB in fp32) from just 73,856 weights.

Adding pixel noise in the figure shows why the bias threshold matters. At noise 0.3 every layer-1 map is nonzero at about half of all positions, because ReLU passes any positive flicker. The layer-2 corner maps stay sparse (3% and 7% nonzero), and their peaks stay at the true corners, $(2,2)$ and $(7,7)$.

## Memory: activations dominate training

Backprop needs the input of every layer to compute that layer's weight gradient, so training keeps most feature maps alive until the backward pass reaches them. Their size scales with batch size and image area, and weights do not. Counting only the conv outputs and residual sums of ResNet-50 gives about 16.8 million values per $224^2$ image, 64 MiB in fp32. A batch of 256 then needs over 16 GiB for activations, against about 100 MiB for the 25.6M weights. The BatchNorm and ReLU outputs that autograd also saves come on top of that.

> **Out of memory? Look at the activations:** Halving the batch halves activation memory, and halving the input side divides it by 4. Weight count barely matters. The standard tools are mixed precision (fp16/bf16 activations), `torch.utils.checkpoint` (store only some layers' outputs and recompute the rest during backward), and `torch.no_grad()` or `torch.inference_mode()` at evaluation time so nothing is stored. For a model that has to fit a fixed batch, a lower input resolution is the biggest lever.

## Looking inside: forward hooks

```python
import torch
from torchvision.models import resnet50, ResNet50_Weights
from torchvision.models.feature_extraction import create_feature_extractor

model = resnet50(weights=ResNet50_Weights.DEFAULT).eval()

# 1) Forward hooks: called as hook(module, inputs, output) after module.forward
acts = {}
def save(name):
    def hook(module, inputs, output):
        acts[name] = output.detach()
    return hook

names = ["conv1", "layer1", "layer2", "layer3", "layer4"]
handles = [getattr(model, n).register_forward_hook(save(n)) for n in names]

x = torch.randn(1, 3, 224, 224)          # use a real, normalised image in practice
with torch.inference_mode():
    model(x)
for n in names:
    a = acts[n]
    print(f"{n:7s} {tuple(a.shape)}  {a.numel() * 4 / 2**20:.2f} MiB")
# conv1   (1, 64, 112, 112)  3.06 MiB
# layer1  (1, 256, 56, 56)  3.06 MiB
# layer2  (1, 512, 28, 28)  1.53 MiB
# layer3  (1, 1024, 14, 14)  0.77 MiB
# layer4  (1, 2048, 7, 7)  0.38 MiB
for h in handles:
    h.remove()                            # hooks stay attached until removed

# 2) Or build a model that returns the maps directly
fx = create_feature_extractor(model, return_nodes={"layer2": "mid", "layer4": "top"})
feats = fx(x)                             # {"mid": (1,512,28,28), "top": (1,2048,7,7)}

# Which channels fire most at the centre of the last map? (a fibre)
fibre = feats["top"][0, :, 3, 3]
print(fibre.topk(5).indices)

# Show the first 16 channels of layer1 as a grid of images
import torchvision.utils as vutils
maps = acts["layer1"][0, :16].unsqueeze(1)          # (16, 1, 56, 56)
grid = vutils.make_grid(maps, nrow=4, normalize=True, scale_each=True)
```

Capture and inspect intermediate feature maps of a pretrained ResNet-50

> **Activations versus filters:** Plotting a feature map shows **where** one image excites a channel. To see **what** a channel responds to you need other tools: plot first-layer filters directly (they live in pixel space), collect the dataset patches that activate a unit most, project activations back with a deconvnet (Zeiler & Fergus) or gradients (saliency, Grad-CAM), or synthesise an input that maximises the channel (feature visualisation). Only the first layer's weights can be read as images.

> **One unit is not one concept:** Many channels respond to several unrelated patterns, and a concept is often spread across many channels. Reading meaning into a single map from a few images is easy to overdo. Check a channel against many images, and use the fibre (the full vector) rather than individual channels for retrieval or probing tasks.

**Strengths**

- Spatial layout is preserved, so maps can be visualised, localised (Grad-CAM, detection) and reused by dense prediction heads
- Weight sharing: one detector per channel scanned over every position, with translation equivariance for free
- Multi-scale features come out of one forward pass: early maps are detailed, late maps are semantic
- ReLU maps are sparse and compress well

**Limitations**

- Activation memory scales with batch × resolution and dominates training memory
- Early high-resolution maps cost the most memory and bandwidth for the fewest parameters
- Individual channels are often polysemantic and hard to interpret
- Late maps are coarse (7×7 at stride 32), which loses small objects and fine boundaries

**Question:** A conv layer maps a (16, 64, 56, 56) input to 128 channels with a 3×3 kernel, stride 2 and padding 1. What is the output shape?

- A. (16, 128, 28, 28)
- B. (16, 128, 56, 56)
- C. (16, 64, 28, 28)
- D. (128, 16, 27, 27)

**Answer:** A. (16, 128, 28, 28) — Batch stays 16, channels become the number of filters (128), and each spatial side is $\lfloor(56+2-3)/2\rfloor+1=28$. The layer has $128\cdot64\cdot9+128=73{,}856$ parameters, whatever the batch or image size.

**Question:** Doubling the input resolution of a CNN from 224 to 448 (same model, same batch) changes training memory how?

- A. Weight memory doubles
- B. Activation memory roughly quadruples, weight memory is unchanged
- C. Nothing changes: convolutions are size-independent
- D. Activation memory doubles

**Answer:** B. Activation memory roughly quadruples, weight memory is unchanged — Every feature map has $H\times W$ entries, so all of them grow by $2^2=4$. The filters are the same size, so the parameter count, and with it weight memory, stays the same. Compute (multiply-adds) also grows by about 4.

**Key points**

- A conv layer outputs $(N,C,H,W)$: $C$ feature maps, one per filter. A map says where its feature is, a fibre $a[n,:,i,j]$ says what is at one place.
- Every output channel reads every input channel, so deeper filters are combinations of earlier ones: edges → corners/textures → parts → objects (Zeiler & Fergus 2014).
- Going deeper, H and W halve per stage while channels double: fewer, richer positions, and more weights per layer.
- Activation memory scales with batch × resolution and usually dominates training memory; checkpointing, mixed precision and smaller inputs reduce it.
- Inspect maps with `register_forward_hook` or `create_feature_extractor`; remember to remove hooks and to run under `inference_mode`.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
