# Xception

> "Extreme Inception": if an Inception module splits channels into a few groups, take it to the limit of one group per channel. Xception is 36 depthwise separable conv layers with residual connections; with fewer parameters than Inception-v3 (22.9M) it is slightly more accurate on ImageNet.

Source: https://www.mlfoundry.in/learn/xception  
Section: Deep Learning › Convolutional Neural Networks  
Level: advanced · 11 min read · updated 2026-10-11

**Xception:** is the CNN of François Chollet, "Xception: Deep Learning with Depthwise Separable Convolutions" (CVPR 2017). It replaces every Inception module by **depthwise separable convolutions**: a per-channel $3\times3$ spatial conv followed by a $1\times1$ conv that mixes channels. The 36 conv layers are arranged in an **entry flow**, a **middle flow** repeated 8 times and an **exit flow**, with residual connections around almost every block. The name is short for "Extreme Inception".

Chollet's starting point is a hypothesis about what makes Inception work. A regular conv layer learns filters over two spatial dimensions and the channel dimension at once: one kernel handles **cross-channel** and **spatial** correlations together. An Inception module first looks at cross-channel correlations with $1\times1$ convs, mapping the input into a few smaller spaces, and then looks at spatial correlations in each of those spaces with $3\times3$ or $5\times5$ convs. The implicit assumption is that the two kinds of correlation are decoupled enough that it is better not to map them jointly.

A simplified Inception module (one $1\times1$ conv, then $3\times3$ convs on non-overlapping groups of its output channels) makes this a continuum. With 1 group it is a $1\times1$ conv followed by a full $3\times3$ conv. With 3 or 4 groups it resembles Inception's parallel towers. With one group per channel it becomes a **depthwise separable convolution**: the $1\times1$ conv handles all cross-channel mixing and the $3\times3$ conv handles only spatial mixing, one channel at a time. Xception asks whether the extreme end of the continuum works best, and finds that it does, at least at the scale tested.

Depthwise separable convs were not new. Sifre and Mallat studied them (2013–2014), Google Brain used them in 2014, and they were available as `SeparableConv2D` in TensorFlow and Keras, which Chollet created. MobileNets (Howard et al. 2017) appeared at about the same time and used them for efficiency on phones. Xception's contribution is the interpretation as extreme Inception, a clean architecture built from them, and the evidence that they match Inception at the same parameter count.

**At a glance**

- **Paper:** François Chollet, CVPR 2017 (arXiv Oct. 2016)
- **Building block:** depthwise $3\times3$ conv (no nonlinearity) then pointwise $1\times1$ conv, BN after each separable conv
- **Layout:** 36 conv layers in 14 modules: entry flow (4), middle flow (8 identical blocks at 19×19×728), exit flow (2)
- **Parameters:** 22,855,952 (paper, Table 3); Inception-v3 in the same table: 23,626,728
- **Compute:** ≈8.4G multiply-adds at $299\times299$ (computed from the layout)
- **ImageNet:** 79.0% top-1, 94.5% top-5 single crop vs Inception-v3's 78.2% / 94.1% (paper, Table 1)
- **In PyTorch:** not in torchvision; `timm.create_model("legacy_xception", pretrained=True)`

## Depthwise separable convolution

A regular $k\times k$ conv computes each output channel $o$ from all input channels at once. The separable version splits it in two. The **depthwise** step filters each input channel $c$ with its own $k\times k$ kernel $D_c$. The **pointwise** step mixes the filtered channels with a $1\times1$ conv $P$:

$$
\tilde x_c=D_c*x_c\quad(c=1..C_{in}),\qquad y_o=\sum_{c=1}^{C_{in}}P_{oc}\,\tilde x_c,\qquad \frac{\text{weights}_{\text{sep}}}{\text{weights}_{\text{regular}}}=\frac{k^2C_{in}+C_{in}C_{out}}{k^2C_{in}C_{out}}=\frac1{C_{out}}+\frac1{k^2}
$$

- $D_c$: depthwise $k\times k$ kernel of channel $c$ (a conv with $\mathtt{groups}=C_{in}$)
- $P$: pointwise $C_{out}\times C_{in}$ matrix, applied at every pixel
- $1/C_{out}+1/k^2$: cost ratio; for $k=3$ it approaches $1/9$ as $C_{out}$ grows

The separable conv is a regular conv whose kernel is constrained to $W_{oc}=P_{oc}D_c$: each output channel's filter on channel $c$ is a scaled copy of the same spatial kernel. Backprop follows the factorization: the pointwise gradient is $\partial\mathcal L/\partial P_{oc}=\sum_{u,v}g_o(u,v)\,\tilde x_c(u,v)$, and the depthwise kernel of channel $c$ receives the back-mixed error $\tilde g_c=\sum_oP_{oc}\,g_o$, correlated with $x_c$ alone. The full treatment of the operation is in **Depthwise separable convolution**.

Xception differs from a literal "extreme Inception" module in two ways. Its order is depthwise first, then pointwise, the reverse of Inception's $1\times1$ first; in a deep stack this hardly matters, because the layers alternate either way. And there is **no nonlinearity** between the depthwise and pointwise steps. The paper tested this directly: no activation in between converged fastest and reached the best accuracy, ELU was worse and ReLU worse still. Chollet's explanation is that the depthwise step works on 1-channel spaces, which are too shallow for a nonlinearity to help and lose information to it.

## Entry, middle and exit flows

**Xception at 299×299×3**

1. **Entry flow.** Two regular convs ($3\times3$/2 to 32, $3\times3$ to 64), then three blocks of [two separable convs, $3\times3$/2 max-pool] going to 128, 256 and 728 channels. Each block's shortcut is a strided $1\times1$ conv: $64\times147^2\to728\times19^2$.
2. **Middle flow.** 8 identical blocks of [ReLU, separable conv 728] × 3 with an **identity** shortcut, all at $19\times19\times728$. This is where most of the depth and 56% of the parameters sit.
3. **Exit flow.** A block of separable convs to 728 and 1024 with max-pool and a $1\times1$ projection shortcut ($10\times10$), then separable convs to 1536 and 2048 without shortcut, global average pooling, and a logistic-regression layer to 1000 classes.
4. **Everywhere.** BatchNorm after every conv and separable conv; no biases; 36 conv layers, 34 of them separable.

The residual connections matter. The paper trained a version without them: it converged much more slowly and to a clearly worse ImageNet accuracy. Chollet adds that this does not make residual connections a requirement for stacks of separable convs: he also got excellent results with non-residual VGG-style models whose convs were all replaced by separable ones.

> Figure: Xception block by block, computed from the paper's layout at 299×299 (conv weights + BN γ, β + fc). Bars are parameters per block; +x marks identity shortcuts and +1×1 projection shortcuts. The default view selects middle block 1: three separable convs 728 → 728 at 19×19, 1,613,976 parameters and 581.1M mult-adds. The total, 22,855,952 trainable parameters and 8.36G mult-adds, matches the paper's Table 3 exactly. "Regular 3×3 convs" replaces every separable conv by a full 3×3 conv with the same channels: 179.1M parameters (7.8×) and 68.8G mult-adds (8.2×). The slider walks Chollet's continuum for one 728 → 728 layer: g = 1 segment (a 1×1 conv then a full 3×3) needs 5,299,840 weights, slightly more than a plain 3×3 conv's 4,769,856; g = 4 needs 1,722,448; g = 728 (depthwise) needs 536,536, 8.89× fewer than the plain conv.

## Worked example: one middle-flow layer

A middle-flow separable conv maps $728\times19\times19$ to $728\times19\times19$. Weights and multiply-adds (weights × $19^2=361$), without BN:

One 728 → 728 layer at 19×19: separable versus regular 3×3 (values from the figure's lib)

| Variant | Depthwise 3×3 | Pointwise / full | Total weights | Mult-adds |
| --- | --- | --- | --- | --- |
| regular 3×3 conv | – | 9 · 728 · 728 = 4,769,856 | 4,769,856 | 1,721.9M |
| 1×1 conv, 3×3 in 4 groups | 9 · 728 · 182 = 1,192,464 | 728 · 728 = 529,984 | 1,722,448 | 621.8M |
| depthwise separable | 9 · 728 = 6,552 | 728 · 728 = 529,984 | 536,536 | 193.7M |

**Reading the table**

1. The cost ratio formula gives $1/(1/728+1/9)=8.89$, the same as $4{,}769{,}856/536{,}536$.
2. 98.8% of the separable layer's weights are in the pointwise $1\times1$ conv. The spatial filtering is nearly free; the channel mixing is what costs.
3. With BN (2 × 728) the layer has 537,992 parameters, and a middle block of three such layers 1,613,976. Eight blocks: 12.9M of the network's 22.9M parameters.
4. Because separable layers are so cheap, Xception can afford 728 channels at $19\times19$ and still have fewer parameters than Inception-v3. It spends the saving on width, not on lower cost: at 8.4G it needs more multiply-adds than Inception-v3's 5.7G.

```python
import torch, torch.nn as nn

class SeparableConv2d(nn.Module):
    def __init__(self, cin, cout, k=3):
        super().__init__()
        self.depthwise = nn.Conv2d(cin, cin, k, padding=k // 2, groups=cin, bias=False)  # one k x k filter per channel
        self.pointwise = nn.Conv2d(cin, cout, 1, bias=False)                             # 1 x 1 channel mixing
    def forward(self, x):
        return self.pointwise(self.depthwise(x))       # no nonlinearity in between (as in the paper)

class MiddleBlock(nn.Module):                          # [ReLU, SepConv 728, BN] x 3, identity shortcut
    def __init__(self, c=728):
        super().__init__()
        self.f = nn.Sequential(*[m for _ in range(3) for m in
                                 (nn.ReLU(), SeparableConv2d(c, c), nn.BatchNorm2d(c))])
    def forward(self, x):
        return x + self.f(x)

blk = MiddleBlock()
print(sum(p.numel() for p in blk.parameters()))       # 1613976
print(blk(torch.randn(1, 728, 19, 19)).shape)          # torch.Size([1, 728, 19, 19])
print(sum(p.numel() for p in nn.Conv2d(728, 728, 3, bias=False).parameters()))  # 4769856

# Pretrained Xception lives in timm (named "xception" in older timm releases)
# import timm; model = timm.create_model("legacy_xception", pretrained=True)
```

Separable conv and the middle-flow block in PyTorch

> **FLOPs are not latency:** A depthwise conv does only $k^2$ multiply-adds per output value, so it is limited by memory bandwidth rather than arithmetic, and GPUs run it far below their peak. In the paper Xception trained at 28 steps/s versus Inception-v3's 31 despite having fewer parameters. Count memory traffic, not just multiply-adds, when comparing separable and dense designs.

> **Where the gains showed up:** On ImageNet the improvement over Inception-v3 was small (79.0% vs 78.2% top-1). On Google's internal JFT dataset (350M images, 17,000 classes) it was clearly larger. Chollet reads this as a sign that Inception-v3's hyperparameters had been tuned for ImageNet, while the cleaner Xception design generalizes across datasets. Both networks have nearly the same parameter count, so the gain comes from using the parameters more efficiently.

> **Descendants:** MobileNet and MobileNetV2 use depthwise separable convs for mobile efficiency. DeepLabv3+ (Chen et al. 2018) uses a modified "aligned" Xception with more middle-flow blocks and strided separable convs in place of max-pooling. ConvNeXt's $7\times7$ depthwise conv followed by $1\times1$ MLP layers is the same split between spatial and channel mixing, and so is the token-mixing / channel-mixing split of Transformer blocks.

**Strengths**

- Spatial and channel mixing are separated: about $1/9$ of the weights of a regular $3\times3$ conv at the same width.
- A simple, regular architecture (one block type per flow) instead of Inception's hand-tuned branches.
- Slightly better ImageNet accuracy than Inception-v3 at fewer parameters, and a larger gain on JFT.

**Limitations**

- Depthwise convs are memory-bound and slower on GPUs than their FLOPs suggest.
- The saving was spent on width: 8.4G multiply-adds, more than Inception-v3.
- Not in torchvision; the separable constraint $W_{oc}=P_{oc}D_c$ can hurt in very small networks.

**Question:** A depthwise separable 3×3 conv maps 728 channels to 728. Roughly how many times fewer weights does it have than a regular 3×3 conv with the same channels?

- A. About 3×
- B. About 8.9×
- C. About 728×
- D. The same number

**Answer:** B. About 8.9× — The ratio is $1/(1/C_{out}+1/k^2)=1/(1/728+1/9)\approx8.89$: 536,536 versus 4,769,856. Almost all of the separable layer's weights are in the pointwise $1\times1$ conv, so the saving can never exceed $k^2=9$.

**Question:** What did the Xception paper find about putting a ReLU between the depthwise and the pointwise step?

- A. It was required for convergence
- B. It made no difference
- C. It hurt: no intermediate nonlinearity trained fastest and reached the best accuracy
- D. It helped only with residual connections

**Answer:** C. It hurt: no intermediate nonlinearity trained fastest and reached the best accuracy — The ablation compared no activation, ELU and ReLU between the two steps. No activation was best, then ELU, then ReLU. Chollet argues that a nonlinearity on 1-channel depthwise outputs destroys information without adding useful expressive power.

**Key points**

- Xception (Chollet 2017) = "extreme Inception": one spatial filter per channel, with all channel mixing done by 1×1 convs.
- Depthwise separable conv: weights $k^2C_{in}+C_{in}C_{out}$ instead of $k^2C_{in}C_{out}$, a ratio of $1/C_{out}+1/k^2$.
- 36 conv layers (34 separable) in entry, middle (8 × 3 layers at 19×19×728) and exit flows, residual connections on all but the first and last modules, BN everywhere, no nonlinearity inside a separable conv.
- 22,855,952 parameters vs Inception-v3's 23,626,728; 79.0% vs 78.2% ImageNet top-1, a larger gain on JFT.
- The saving is spent on width, so compute is not lower (≈8.4G mult-adds); depthwise convs are memory-bound on GPUs.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
