Xception

"Extreme Inception": if an Inception module splits channels into a few groups, take it to the limit of one group per channel. Xception is 36 depthwise separable conv layers with residual connections; with fewer parameters than Inception-v3 (22.9M) it is slightly more accurate on ImageNet.

11 min read1127 wordsadvanced0 viewsupdated 2026-10-11

Definition

Xception is the CNN of François Chollet, "Xception: Deep Learning with Depthwise Separable Convolutions" (CVPR 2017). It replaces every Inception module by depthwise separable convolutions: a per-channel 3×33\times3 spatial conv followed by a 1×11\times1 conv that mixes channels. The 36 conv layers are arranged in an entry flow, a middle flow repeated 8 times and an exit flow, with residual connections around almost every block. The name is short for "Extreme Inception".

Chollet's starting point is a hypothesis about what makes Inception work. A regular conv layer learns filters over two spatial dimensions and the channel dimension at once: one kernel handles cross-channel and spatial correlations together. An Inception module first looks at cross-channel correlations with 1×11\times1 convs, mapping the input into a few smaller spaces, and then looks at spatial correlations in each of those spaces with 3×33\times3 or 5×55\times5 convs. The implicit assumption is that the two kinds of correlation are decoupled enough that it is better not to map them jointly.

A simplified Inception module (one 1×11\times1 conv, then 3×33\times3 convs on non-overlapping groups of its output channels) makes this a continuum. With 1 group it is a 1×11\times1 conv followed by a full 3×33\times3 conv. With 3 or 4 groups it resembles Inception's parallel towers. With one group per channel it becomes a depthwise separable convolution: the 1×11\times1 conv handles all cross-channel mixing and the 3×33\times3 conv handles only spatial mixing, one channel at a time. Xception asks whether the extreme end of the continuum works best, and finds that it does, at least at the scale tested.

Depthwise separable convs were not new. Sifre and Mallat studied them (2013–2014), Google Brain used them in 2014, and they were available as SeparableConv2D in TensorFlow and Keras, which Chollet created. MobileNets (Howard et al. 2017) appeared at about the same time and used them for efficiency on phones. Xception's contribution is the interpretation as extreme Inception, a clean architecture built from them, and the evidence that they match Inception at the same parameter count.

At a glance

Paper
François Chollet, CVPR 2017 (arXiv Oct. 2016)
Building block
depthwise 3×33\times3 conv (no nonlinearity) then pointwise 1×11\times1 conv, BN after each separable conv
Layout
36 conv layers in 14 modules: entry flow (4), middle flow (8 identical blocks at 19×19×728), exit flow (2)
Parameters
22,855,952 (paper, Table 3); Inception-v3 in the same table: 23,626,728
Compute
≈8.4G multiply-adds at 299×299299\times299 (computed from the layout)
ImageNet
79.0% top-1, 94.5% top-5 single crop vs Inception-v3's 78.2% / 94.1% (paper, Table 1)
In PyTorch
not in torchvision; timm.create_model("legacy_xception", pretrained=True)

Depthwise separable convolution

A regular k×kk\times k conv computes each output channel oo from all input channels at once. The separable version splits it in two. The depthwise step filters each input channel cc with its own k×kk\times k kernel DcD_c. The pointwise step mixes the filtered channels with a 1×11\times1 conv PP:

x~c=Dc∗xc(c=1..Cin),yo=∑c=1CinPoc x~c,weightssepweightsregular=k2Cin+CinCoutk2CinCout=1Cout+1k2\tilde x_c=D_c*x_c\quad(c=1..C_{in}),\qquad y_o=\sum_{c=1}^{C_{in}}P_{oc}\,\tilde x_c,\qquad \frac{\text{weights}_{\text{sep}}}{\text{weights}_{\text{regular}}}=\frac{k^2C_{in}+C_{in}C_{out}}{k^2C_{in}C_{out}}=\frac1{C_{out}}+\frac1{k^2}
DcD_c
depthwise k×kk\times k kernel of channel cc (a conv with groups=Cin\mathtt{groups}=C_{in})
PP
pointwise Cout×CinC_{out}\times C_{in} matrix, applied at every pixel
1/Cout+1/k21/C_{out}+1/k^2
cost ratio; for k=3k=3 it approaches 1/91/9 as CoutC_{out} grows

The separable conv is a regular conv whose kernel is constrained to Woc=PocDcW_{oc}=P_{oc}D_c: each output channel's filter on channel cc is a scaled copy of the same spatial kernel. Backprop follows the factorization: the pointwise gradient is ∂L/∂Poc=∑u,vgo(u,v) x~c(u,v)\partial\mathcal L/\partial P_{oc}=\sum_{u,v}g_o(u,v)\,\tilde x_c(u,v), and the depthwise kernel of channel cc receives the back-mixed error g~c=∑oPoc go\tilde g_c=\sum_oP_{oc}\,g_o, correlated with xcx_c alone. The full treatment of the operation is in Depthwise separable convolution.

Xception differs from a literal "extreme Inception" module in two ways. Its order is depthwise first, then pointwise, the reverse of Inception's 1×11\times1 first; in a deep stack this hardly matters, because the layers alternate either way. And there is no nonlinearity between the depthwise and pointwise steps. The paper tested this directly: no activation in between converged fastest and reached the best accuracy, ELU was worse and ReLU worse still. Chollet's explanation is that the depthwise step works on 1-channel spaces, which are too shallow for a nonlinearity to help and lose information to it.

Entry, middle and exit flows

Xception at 299×299×3

  1. 1Entry flow. Two regular convs (3×33\times3/2 to 32, 3×33\times3 to 64), then three blocks of [two separable convs, 3×33\times3/2 max-pool] going to 128, 256 and 728 channels. Each block's shortcut is a strided 1×11\times1 conv: 64×1472→728×19264\times147^2\to728\times19^2.
  2. 2Middle flow. 8 identical blocks of [ReLU, separable conv 728] × 3 with an identity shortcut, all at 19×19×72819\times19\times728. This is where most of the depth and 56% of the parameters sit.
  3. 3Exit flow. A block of separable convs to 728 and 1024 with max-pool and a 1×11\times1 projection shortcut (10×1010\times10), then separable convs to 1536 and 2048 without shortcut, global average pooling, and a logistic-regression layer to 1000 classes.
  4. 4Everywhere. BatchNorm after every conv and separable conv; no biases; 36 conv layers, 34 of them separable.

The residual connections matter. The paper trained a version without them: it converged much more slowly and to a clearly worse ImageNet accuracy. Chollet adds that this does not make residual connections a requirement for stacks of separable convs: he also got excellent results with non-residual VGG-style models whose convs were all replaced by separable ones.

Xception, 299×299 · 36 conv layers, 34 of them depthwise separable · click a blockentry flowmiddle flow (×8, 19×19×728)exit flowconve1+1×1e2+1×1e3+1×1m1+xm2+xm3+xm4+xm5+xm6+xm7+xm8+xx1+1×1x2fcmiddle block 1: 3 sep convs 728→728: 728×19² → 728×19² · 1,613,976 params · 581.1Mlayeroutparamsmult-addssep 3×3 728→728 (dw 6552 + pw 529984)728×19²537,992193.7Msep 3×3 728→728 (dw 6552 + pw 529984)728×19²537,992193.7Msep 3×3 728→728 (dw 6552 + pw 529984)728×19²537,992193.7Mresidual: identity shortcut, y = x + F(x) · every conv is followed by BNone layer, 728 → 728 at 19×19: 1×1 conv, then a 3×3 conv split into g = 728 segments of 1 channels(728 segments; drawn as 91 stripes)one segment per channel: depthwise separable (“extreme Inception”)1×1 pointwise529,9843×3, 728 groups6,552total 536,536 weights, 193.7M mult-adds · regular 3×3: 4,769,856 (8.89× of this)22,855,952 trainable params · 8.36 G mult-adds · paper Table 3: 22,855,952 ✓Keras also counts BN moving stats: 22,910,480 · depthwise layer above saves 8.9× vs regular
top-1 0.79 vs Inception-v3 0.782 (paper)
fig.Xception block by block, computed from the paper's layout at 299×299 (conv weights + BN γ, β + fc). Bars are parameters per block; +x marks identity shortcuts and +1×1 projection shortcuts. The default view selects middle block 1: three separable convs 728 → 728 at 19×19, 1,613,976 parameters and 581.1M mult-adds. The total, 22,855,952 trainable parameters and 8.36G mult-adds, matches the paper's Table 3 exactly. "Regular 3×3 convs" replaces every separable conv by a full 3×3 conv with the same channels: 179.1M parameters (7.8×) and 68.8G mult-adds (8.2×). The slider walks Chollet's continuum for one 728 → 728 layer: g = 1 segment (a 1×1 conv then a full 3×3) needs 5,299,840 weights, slightly more than a plain 3×3 conv's 4,769,856; g = 4 needs 1,722,448; g = 728 (depthwise) needs 536,536, 8.89× fewer than the plain conv.

Worked example: one middle-flow layer

A middle-flow separable conv maps 728×19×19728\times19\times19 to 728×19×19728\times19\times19. Weights and multiply-adds (weights × 192=36119^2=361), without BN:

VariantDepthwise 3×3Pointwise / fullTotal weightsMult-adds
regular 3×3 conv–9 · 728 · 728 = 4,769,8564,769,8561,721.9M
1×1 conv, 3×3 in 4 groups9 · 728 · 182 = 1,192,464728 · 728 = 529,9841,722,448621.8M
depthwise separable9 · 728 = 6,552728 · 728 = 529,984536,536193.7M
One 728 → 728 layer at 19×19: separable versus regular 3×3 (values from the figure's lib)

Reading the table

  1. 1The cost ratio formula gives 1/(1/728+1/9)=8.891/(1/728+1/9)=8.89, the same as 4,769,856/536,5364{,}769{,}856/536{,}536.
  2. 298.8% of the separable layer's weights are in the pointwise 1×11\times1 conv. The spatial filtering is nearly free; the channel mixing is what costs.
  3. 3With BN (2 × 728) the layer has 537,992 parameters, and a middle block of three such layers 1,613,976. Eight blocks: 12.9M of the network's 22.9M parameters.
  4. 4Because separable layers are so cheap, Xception can afford 728 channels at 19×1919\times19 and still have fewer parameters than Inception-v3. It spends the saving on width, not on lower cost: at 8.4G it needs more multiply-adds than Inception-v3's 5.7G.
python
1import torch, torch.nn as nn
2
3class SeparableConv2d(nn.Module):
4 def __init__(self, cin, cout, k=3):
5 super().__init__()
6 self.depthwise = nn.Conv2d(cin, cin, k, padding=k // 2, groups=cin, bias=False) # one k x k filter per channel
7 self.pointwise = nn.Conv2d(cin, cout, 1, bias=False) # 1 x 1 channel mixing
8 def forward(self, x):
9 return self.pointwise(self.depthwise(x)) # no nonlinearity in between (as in the paper)
10
11class MiddleBlock(nn.Module): # [ReLU, SepConv 728, BN] x 3, identity shortcut
12 def __init__(self, c=728):
13 super().__init__()
14 self.f = nn.Sequential(*[m for _ in range(3) for m in
15 (nn.ReLU(), SeparableConv2d(c, c), nn.BatchNorm2d(c))])
16 def forward(self, x):
17 return x + self.f(x)
18
19blk = MiddleBlock()
20print(sum(p.numel() for p in blk.parameters())) # 1613976
21print(blk(torch.randn(1, 728, 19, 19)).shape) # torch.Size([1, 728, 19, 19])
22print(sum(p.numel() for p in nn.Conv2d(728, 728, 3, bias=False).parameters())) # 4769856
23
24# Pretrained Xception lives in timm (named "xception" in older timm releases)
25# import timm; model = timm.create_model("legacy_xception", pretrained=True)
Separable conv and the middle-flow block in PyTorch

Use when

  • +Spatial and channel mixing are separated: about 1/91/9 of the weights of a regular 3×33\times3 conv at the same width.
  • +A simple, regular architecture (one block type per flow) instead of Inception's hand-tuned branches.
  • +Slightly better ImageNet accuracy than Inception-v3 at fewer parameters, and a larger gain on JFT.

Avoid when

  • −Depthwise convs are memory-bound and slower on GPUs than their FLOPs suggest.
  • −The saving was spent on width: 8.4G multiply-adds, more than Inception-v3.
  • −Not in torchvision; the separable constraint Woc=PocDcW_{oc}=P_{oc}D_c can hurt in very small networks.

Check yourself

A depthwise separable 3×3 conv maps 728 channels to 728. Roughly how many times fewer weights does it have than a regular 3×3 conv with the same channels?

Check yourself

What did the Xception paper find about putting a ReLU between the depthwise and the pointwise step?

Remember

  • ✓Xception (Chollet 2017) = "extreme Inception": one spatial filter per channel, with all channel mixing done by 1×1 convs.
  • ✓Depthwise separable conv: weights k2Cin+CinCoutk^2C_{in}+C_{in}C_{out} instead of k2CinCoutk^2C_{in}C_{out}, a ratio of 1/Cout+1/k21/C_{out}+1/k^2.
  • ✓36 conv layers (34 separable) in entry, middle (8 × 3 layers at 19×19×728) and exit flows, residual connections on all but the first and last modules, BN everywhere, no nonlinearity inside a separable conv.
  • ✓22,855,952 parameters vs Inception-v3's 23,626,728; 79.0% vs 78.2% ImageNet top-1, a larger gain on JFT.
  • ✓The saving is spent on width, so compute is not lower (≈8.4G mult-adds); depthwise convs are memory-bound on GPUs.
← back to Deep Learning roadmap