# 1D, 2D and 3D CNNs

> The same sliding dot product over 1, 2 or 3 axes: Conv1d for audio, sensors and text (N, C, L), Conv2d for images (N, C, H, W), Conv3d for video and medical volumes (N, C, T, H, W). Each extra axis multiplies the weights by k and the cost by k times that axis's length.

Source: https://www.mlfoundry.in/learn/1d-2d-and-3d-cnns  
Section: Deep Learning › Convolutional Neural Networks  
Level: core · 11 min read · updated 2026-10-11

**1D, 2D and 3D convolutions:** are the same operation applied over a different number of **spatial axes**: a kernel of size $k^D$ slides along $D$ axes of the input and takes a dot product with every input channel at each position. The dimensionality names the axes the kernel moves along, not the rank of the tensor: a colour image is 3-D data, but it is processed by a **2-D** convolution because the kernel only slides over height and width.

Choose the dimensionality by asking along which axes the data is **ordered** and roughly **translation-invariant**. An audio waveform or a sensor trace has one such axis, time. An image has two, and a video or a CT scan has three. Every other axis is a channel: the 3 colours of a pixel, the 12 leads of an ECG, the 300 dimensions of a word embedding. The kernel always covers all channels at once. It slides only along the ordered axes.

All three forms are old. Waibel et al.'s time-delay neural network (1989) was a 1-D convolutional network for phoneme recognition, and LeCun et al.'s networks for digits (1989, 1998) made 2-D convolution standard for images. Ji, Xu, Yang and Yu introduced 3-D CNNs for human action recognition (ICML 2010, TPAMI 2013). Tran et al.'s C3D (2015) then showed that a plain VGG-style network of $3\times3\times3$ kernels learns good generic video features.

## One formula, D axes

$$
y_{o}[\mathbf p]=b_o+\sum_{c=1}^{C_{in}}\ \sum_{\mathbf u\in\{0,\dots,k-1\}^D} W_{o,c}[\mathbf u]\;x_c[\,s\mathbf p+d\,\mathbf u-\mathbf q\,],\qquad \#\text{params}=C_{out}\,C_{in}\,k^D+C_{out}
$$

- $\mathbf p$: output position: $t$ for 1-D, $(i,j)$ for 2-D, $(t,i,j)$ for 3-D
- $W_{o,c}$: a $k^D$ kernel slice; the weight tensor has shape $(C_{out},C_{in},k,\dots,k)$ with $D$ trailing $k$'s
- $s,\ d,\ \mathbf q$: stride, dilation and padding offset, set per axis (a 3-D kernel can be $3\times7\times7$)
- $k^D$: each extra axis multiplies the weights by $k$, and the multiply-adds by $k$ times that axis's output length

The backward pass is the same in every dimension. The gradient for $W_{o,c}$ correlates the output error $\delta_o$ with input channel $c$ over all positions and the batch, and the gradient for $x$ is a transposed convolution of $\delta$ with the same weights. Frameworks implement all three with the same kernels, and `nn.Conv1d`, `nn.Conv2d` and `nn.Conv3d` take the same arguments.

Layouts and costs of a layer with 64 → 128 channels, k = 3, stride 1, same padding (counted by the figure's lib)

| Property | Conv1d | Conv2d | Conv3d |
| --- | --- | --- | --- |
| Input layout | (N, C, L) | (N, C, H, W) | (N, C, T, H, W) or (N, C, D, H, W) |
| Weight shape | (128, 64, 3) | (128, 64, 3, 3) | (128, 64, 3, 3, 3) |
| Parameters | 24,704 | 73,856 | 221,312 |
| Typical input | 1 s of 16 kHz audio, L = 16,000 | ImageNet crop, 224 × 224 | C3D clip, 16 × 112 × 112 |
| Input values (C = 64) | 1.02M | 3.21M | 12.85M |
| Multiply-adds | 393.2M | 3.70G | 44.39G |
| Typical data | audio, ECG / sensors, text tokens, genomes | images, spectrograms | video, CT / MRI volumes, point clouds as voxels |

> Figure: The same 3-tap-per-axis cross-correlation on a 40-sample signal, a 16×16 image and a 6-frame clip of 10×10 frames. Click to see which inputs feed one output. In 1-D the edge kernel gives y[13] = x[14] − x[12] = −1.145 at the falling edge of the pulse; switch on causal and the same value appears one step later, at t = 14, because the window now ends at t; dilation 4 spreads the three taps over 9 samples. In 3-D the motion kernel (next frame minus previous frame, each averaged over 3×3) is exactly 0 on the static square in the inner frames and ±0.67 at the leading and trailing edges of the moving one. The readout counts a real 64 → 128 layer: 24,704, 73,856 and 221,312 parameters, and 0.39, 3.70 and 44.4 billion multiply-adds on typical inputs.

## Conv1d: sequences

A 1-D CNN slides along time (or position in a sequence). For raw audio the input is $(N,1,L)$, for a 12-lead ECG $(N,12,L)$. For text, the embedding dimension becomes the channel axis: Kim (2014) ran filters of widths 3, 4 and 5 over word2vec embeddings, took the max over time of each feature map, and got strong sentence classifiers with a single conv layer. In each case the network is translation-equivariant in time: a pattern is detected wherever it occurs.

Two variants matter for sequences. A **causal** convolution only reads the present and the past, $y[t]=\sum_u w_u\,x[t-u\,d]$, so it can be used for prediction and generation. A **dilated** convolution spaces its taps $d$ apart. WaveNet (van den Oord et al., 2016) stacks causal convolutions with kernel 2 and dilations $1,2,4,\dots,512$, so ten layers cover 1024 samples, and repeats that stack several times. Temporal convolutional networks (Bai, Kolter and Koltun, 2018) use the same recipe with residual blocks and match or beat recurrent networks on many sequence benchmarks.

**Worked example on the figure's signal (edge kernel w = (−1, 0, 1))**

1. Centred, dilation 1: $y[13]=-x[12]+0\cdot x[13]+x[14]=-1.104+(-0.041)=-1.145$. The pulse ends between samples 13 and 14, so the output is strongly negative there.
2. Causal, dilation 1: the window for output $t$ is $x[t-2],x[t-1],x[t]$. The same three samples now produce $y[14]=-1.145$: a causal conv is a centred one delayed by $(k-1)/2\cdot d=1$ step.
3. Causal, dilation 4: output 30 reads $x[22],x[26],x[30]$, a span of $2\cdot4+1=9$ samples with only three weights.
4. Length bookkeeping, $L_{out}=\lfloor(L+2p-d(k-1)-1)/s\rfloor+1$: on 16,000 samples, $k=3$ with no padding gives 15,998, $p=1$ keeps 16,000, and stride 2 gives 8,000. A kernel-2 layer with dilation 512 and no padding gives 15,488.

## Conv2d: images and image-like data

The 2-D case is the subject of most of this module. The kernel slides over $(H,W)$, and colour or feature maps are channels. Data that is not a photograph also uses Conv2d when it has two ordered axes. Speech models convolve log-mel **spectrograms** (frequency × time), and board-game networks treat the board as an image with one channel per piece type.

## Conv3d: video and volumes

A 3-D kernel slides over time (or depth) as well as height and width, so a single filter can respond to **motion**: a change between frames at a given place. In the figure, the motion kernel subtracts the previous frame from the next one. It gives exactly 0 on the static square in the inner frames and responds only at the leading and trailing edges of the moving square. A 2-D network applied frame by frame (the spatial kernel) sees both squares the same way and has to infer motion later from features alone.

Three designs shaped video CNNs. **C3D** (Tran et al., 2015) used $3\times3\times3$ kernels everywhere on 16-frame $112\times112$ clips and found that kernel to be the best of the depths it tried. **I3D** (Carreira and Zisserman, 2017) **inflated** a 2-D ImageNet Inception network: every $N\times N$ filter becomes $N\times N\times N$ by repeating the 2-D weights along time and dividing by the number of repeats. A video made of one repeated frame then produces the same response as the image did. **R(2+1)D** (Tran et al., 2018) factorises each $t\times d\times d$ conv into a $1\times d\times d$ spatial conv and a $t\times1\times1$ temporal conv, with an extra ReLU in between.

$$
M=\left\lfloor\frac{t\,d^2\,C_{in}C_{out}}{d^2C_{in}+t\,C_{out}}\right\rfloor,\qquad \underbrace{d^2C_{in}M+t\,M\,C_{out}}_{(2+1)\mathrm D}\approx \underbrace{t\,d^2C_{in}C_{out}}_{\text{full 3D}}
$$

- $M$: width of the intermediate (spatial) output, chosen so both versions have about the same number of weights
- $t,\ d$: temporal and spatial kernel size
- $(2+1)\mathrm D$: for $64\to128$, $t=d=3$: $M=230$ and $220{,}800$ weights versus $221{,}184$; for $64\to64$, $M=144$ and both have $110{,}592$

At equal parameter count, the factorised network has twice as many nonlinearities. Tran et al. also reported that it was easier to optimise, with lower training error than the full 3-D network of the same size. In medical imaging, the third axis is depth rather than time: 3D U-Net (Çiçek et al., 2016) and V-Net (Milletari et al., 2016) segment CT and MRI volumes of shape $(N,1,D,H,W)$ with $3\times3\times3$ kernels.

> **3-D costs grow fast:** One $3\times3\times3$ layer on a 16-frame $112^2$ clip costs 44.4 billion multiply-adds, twelve times the same layer on a $224^2$ image, and the activations grow just as fast. Video models therefore use short clips, small spatial sizes, early temporal striding, factorised (2+1)D or depthwise 3-D kernels, and often sample a few clips per video rather than processing all of it.

> **Channels-first for Conv1d:** PyTorch convolutions expect channels before the sequence axis: $(N,C,L)$. Time series and token embeddings usually arrive as $(N,L,C)$, e.g. `nn.Embedding` outputs $(N,L,E)$. Call `x.transpose(1, 2)` first. If you forget, `Conv1d` silently convolves over the embedding dimension whenever the sizes happen to match.

```python
import torch, torch.nn as nn, torch.nn.functional as F

for x, conv in [
    (torch.randn(2, 64, 16000),         nn.Conv1d(64, 128, 3, padding=1)),   # (N, C, L)
    (torch.randn(2, 64, 224, 224),      nn.Conv2d(64, 128, 3, padding=1)),   # (N, C, H, W)
    (torch.randn(2, 64, 16, 112, 112),  nn.Conv3d(64, 128, 3, padding=1)),   # (N, C, T, H, W)
]:
    n = sum(p.numel() for p in conv.parameters())
    print(tuple(conv(x).shape), tuple(conv.weight.shape), n)
# (2, 128, 16000)          (128, 64, 3)        24704
# (2, 128, 224, 224)       (128, 64, 3, 3)     73856
# (2, 128, 16, 112, 112)   (128, 64, 3, 3, 3)  221312

class CausalConv1d(nn.Conv1d):
    """Output t sees only x[t], x[t-d], ..., x[t-(k-1)d]: pad on the left only."""
    def __init__(self, cin, cout, k, dilation=1):
        super().__init__(cin, cout, k, dilation=dilation)
        self.left = (k - 1) * dilation
    def forward(self, x):
        return super().forward(F.pad(x, (self.left, 0)))

wavenet_like = nn.Sequential(*[CausalConv1d(32, 32, 2, dilation=2**i) for i in range(10)])
print(wavenet_like(torch.randn(1, 32, 4000)).shape)    # (1, 32, 4000); receptive field 1024

def conv2plus1d(cin, cout, t=3, d=3):
    m = (t * d * d * cin * cout) // (d * d * cin + t * cout)          # 230 for 64 -> 128
    return nn.Sequential(
        nn.Conv3d(cin, m, (1, d, d), padding=(0, d // 2, d // 2), bias=False),  # space
        nn.BatchNorm3d(m), nn.ReLU(inplace=True),
        nn.Conv3d(m, cout, (t, 1, 1), padding=(t // 2, 0, 0), bias=False),      # time
    )

# I3D inflation: a 2-D kernel repeated over time and divided by t
w2d = torch.randn(64, 3, 7, 7)
w3d = w2d.unsqueeze(2).repeat(1, 1, 7, 1, 1) / 7                     # (64, 3, 7, 7, 7)

# Text: embeddings are (N, L, E); Conv1d wants (N, E, L)
emb = nn.Embedding(10000, 300)(torch.randint(0, 10000, (8, 50)))      # (8, 50, 300)
feat = nn.Conv1d(300, 100, 3)(emb.transpose(1, 2)).amax(dim=2)        # max over time: (8, 100)

# Pretrained video backbones in torchvision
from torchvision.models.video import r2plus1d_18, R2Plus1D_18_Weights
video_net = r2plus1d_18(weights=R2Plus1D_18_Weights.KINETICS400_V1)  # input (N, 3, T, H, W)
```

Conv1d / 2d / 3d side by side, a causal dilated Conv1d, a (2+1)D block and I3D-style inflation

> **Pick the axes, not the data rank:** A spectrogram can be a 2-D image (frequency × time, Conv2d) or a 1-D sequence with 80 mel channels (Conv1d over time); the second treats each frequency band as its own feature and is translation-equivariant only in time. A stack of satellite images can be a 3-D volume, or a 2-D image with the time steps as channels. Convolve over an axis only if a pattern should mean the same thing wherever it appears along it.

**Strengths**

- One operation and one API for sequences, images and volumes
- Conv1d is fast and parallel over time (unlike RNNs), and causal + dilated stacks reach long contexts cheaply
- Conv3d models motion and volumetric structure directly
- 2-D weights can be inflated to initialise 3-D networks (I3D)

**Limitations**

- Parameters grow as $k^D$ and compute as $k^D$ times the input volume: Conv3d is expensive
- 3-D activations make memory the bottleneck for video and medical volumes
- Fixed receptive field: long-range dependencies need depth, dilation, pooling or attention
- Assumes translation invariance along every convolved axis, which is wrong for e.g. the frequency axis of some audio tasks

**Question:** A video batch has shape (4, 3, 16, 112, 112). Which layer applies a 3×3 spatial and 3-frame temporal kernel to it, and how many weights does a 3 → 64 version have (no bias)?

- A. nn.Conv3d(3, 64, 3): 5,184
- B. nn.Conv2d(3, 64, 3): 1,728
- C. nn.Conv3d(16, 64, 3): 27,648
- D. nn.Conv1d(3, 64, 27): 5,184

**Answer:** A. nn.Conv3d(3, 64, 3): 5,184 — The tensor is $(N,C,T,H,W)$ with 3 channels, so it needs `Conv3d` with `in_channels=3`. Weights: $64\cdot3\cdot3^3=5184$. The 16 frames are an axis the kernel slides along, not channels.

**Question:** A causal Conv1d stack uses kernel size 2 and dilations 1, 2, 4, …, 512 (ten layers). How many past samples, including the current one, can the last output see?

- A. 1024
- B. 20
- C. 512
- D. 1023

**Answer:** A. 1024 — Each layer adds $(k-1)d=d$ samples: $1+(1+2+\dots+512)=1+1023=1024$. This is WaveNet's building block: exponential context with linear depth.

**Key points**

- The dimensionality is the number of axes the kernel slides along; everything else (colours, embeddings, sensors) is channels.
- Layouts: Conv1d $(N,C,L)$, Conv2d $(N,C,H,W)$, Conv3d $(N,C,T,H,W)$; weights $(C_{out},C_{in},k^D)$, params $C_{out}C_{in}k^D+C_{out}$.
- Conv1d for audio, time series and text; causal and dilated variants (WaveNet, TCN) give long contexts for generation and forecasting.
- Conv3d for video and medical volumes: C3D (3×3×3), I3D (inflated 2-D weights), R(2+1)D (spatial then temporal conv).
- Each extra axis multiplies weights by $k$ and compute by $k$ times its length: a $64\to128$ layer costs 0.39, 3.70 and 44.4 GMACs on typical 1-D, 2-D and 3-D inputs.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
