MobileNet

A family of mobile CNNs built from depthwise separable convolutions. V1 adds width and resolution knobs (4.2M params, 569M mult-adds), V2 inverted residuals with linear bottlenecks, and V3 a searched architecture with h-swish and squeeze-and-excitation.

13 min read936 wordscore0 viewsupdated 2026-10-11

Definition

MobileNet is a family of convolutional networks from Google designed for phones and embedded devices. Nearly all of their spatial filtering is done by cheap depthwise 3×33\times3 (or 5×55\times5) convolutions, and nearly all of their computation is in 1×11\times1 convolutions. Three versions matter: V1 (Howard et al., 2017), V2 (Sandler et al., 2018) and V3 (Howard et al., 2019).

By 2016 the accurate ImageNet models (VGG, Inception, ResNet) cost billions of multiply-adds per image, far too much for a phone CPU. Howard et al.'s "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications" (2017) took a deliberately plain approach. Stack Depthwise separable convolution blocks in a straight line, like VGG. Then give the user two global knobs to trade accuracy for cost. The result reached 70.6% ImageNet top-1 with 569M multiply-adds, close to GoogLeNet's accuracy at about a third of its compute. It became the default backbone for on-device detection (SSD-MobileNet) and segmentation.

Three versions

V1 (2017)
3×33\times3 stem + 13 depthwise separable blocks, BN + ReLU everywhere; width multiplier α\alpha, resolution multiplier ρ\rho. 4.2M params, 569M mult-adds, 70.6% top-1
V2 (2018)
inverted residual blocks: 1×11\times1 expand (t=6t=6) → depthwise 3×33\times3 → linear 1×11\times1 project, ReLU6, residual between bottlenecks. 3.4M params, 300M mult-adds, 72.0% top-1
V3 (2019)
block structure found by platform-aware NAS + NetAdapt; h-swish, squeeze-and-excitation, 5×55\times5 depthwise, cheaper head. Large: 5.4M, 219M, 75.2%; Small: 2.5M, 56M, 67.4%
Accuracy source
ImageNet top-1, 224×224 single crop, as reported in each paper
In PyTorch
torchvision.models.mobilenet_v2, mobilenet_v3_large, mobilenet_v3_small (V1 is in timm, not torchvision)

V1: separable blocks and two multipliers

Each V1 block is a depthwise 3×33\times3 conv followed by a pointwise 1×11\times1 conv, each with BatchNorm and ReLU. Downsampling happens with stride 2 in the depthwise conv. Channels go 32→64→128→⋯→102432\to64\to128\to\dots\to1024 while the map shrinks 112→7112\to7, then global average pooling and one fully connected layer. Counting depthwise and pointwise separately, the network is 28 layers deep.

The width multiplier α∈(0,1]\alpha\in(0,1] thins every layer to αC\alpha C channels. The resolution multiplier ρ\rho shrinks the input image (224 → 192, 160, 128), which shrinks every feature map. A block's cost becomes

k2 αCin (ρD)2⏟depthwise  +  αCin αCout (ρD)2⏟pointwise  ≈  α2ρ2⋅cost(α=1,ρ=1)\underbrace{k^2\,\alpha C_{in}\,(\rho D)^2}_{\text{depthwise}}\;+\;\underbrace{\alpha C_{in}\,\alpha C_{out}\,(\rho D)^2}_{\text{pointwise}}\;\approx\;\alpha^2\rho^2\cdot\text{cost}(\alpha{=}1,\rho{=}1)
α\alpha
width multiplier: cost and parameters scale roughly as α2\alpha^2 (the pointwise term dominates)
ρ\rho
resolution multiplier: cost scales as ρ2\rho^2; parameters do not change
DD
the side of the block's output feature map at full resolution

The paper also reports where the work goes: 95% of the multiply-adds and 75% of the parameters are in 1×11\times1 convolutions. That is good news for speed, because a 1×11\times1 conv is a plain matrix multiply (GEMM) with no im2col reshuffle. The depthwise convs that do all the spatial filtering account for only about 3% of the compute.

MobileNetV1 α = 1, 224×224multiply-adds per row (max 54.99M)conv 3×3 /232dw-sep 164dw-sep 2 /2128dw-sep 3128dw-sep 4 /2256dw-sep 5256dw-sep 6 /2512dw-sep 7512dw-sep 8512dw-sep 9512dw-sep 10512dw-sep 11512dw-sep 12 /21024dw-sep 131024avgpool + FC1000dw-sep 7in 512×14×14 → out 512×14×14dw 3×34.6k p · 903.2k MACspw 1×1 512→512262.1k p · 51.38M MACsdw share of this block: 1.7% of MACsa full 3×3 conv here: 462.42M MACstotal 4,210,088 params · 568.7M MACs1×1 94.9% · dw 3×3 3.1% · other 2.1% of MACspaper: 4.2M params, 569M mult-adds, 70.6% top-1Howard et al. 2017, Table 6/7
width α
resolution
fig.MobileNetV1 and V2 built layer by layer from the papers' configurations. Bars are multiply-adds per block, split into depthwise 3×3 (blue), 1×1 (orange) and other (stem, FC). Click a row for its shapes, parameters and cost. At the default V1, α = 1, 224×224, the library counts 4,210,088 parameters and 568.7M multiply-adds, matching the paper's 4.2M and 569M. The 1×1 convs take 94.9% of the multiply-adds, the depthwise convs 3.1%. The selected block (512 → 512 on 14×14) costs 52.3M multiply-adds, against 462.4M for a full 3×3 conv. Try α = 0.5 (149.5M, paper 149M) or 128×128 (186.4M, paper 186M). Switch to V2: torchvision's counting gives 3,504,872 parameters and 300.8M multiply-adds (paper 300M). The expansion slider shows the cost of t: 63.7M at t = 1 and 387.0M at t = 8.
settingparams (lib)mult-adds (lib)papertop-1
α=1\alpha=1, 2244,210,088568.7M4.2M / 569M70.6%
α=0.75\alpha=0.75, 2242,569,144325.4M2.6M / 325M68.4%
α=0.5\alpha=0.5, 2241,320,648149.5M1.3M / 149M63.7%
α=0.25\alpha=0.25, 224464,60041.0M0.5M / 41M50.6%
α=1\alpha=1, 1924,210,088418.1M4.2M / 418M69.1%
α=1\alpha=1, 1604,210,088290.7M4.2M / 290M67.2%
α=1\alpha=1, 1284,210,088186.4M4.2M / 186M64.4%
MobileNetV1 trade-off points. Params and multiply-adds from the figure's library; top-1 and paper columns from Howard et al. 2017, Tables 6 and 7

Two things to read from the table. Accuracy falls smoothly down to α=0.5\alpha=0.5 and then collapses at 0.250.25. And parameters do not shrink as α2\alpha^2 at small widths: 1,025,000 of the 4.2M parameters are the final FC layer (1024×10001024\times1000 plus biases), which only shrinks linearly with α\alpha.

V2: inverted residuals and linear bottlenecks

Sandler et al. ("MobileNetV2: Inverted Residuals and Linear Bottlenecks", CVPR 2018) changed the block. A ResNet bottleneck is wide → narrow → wide, with the shortcut between the wide ends. MobileNetV2 inverts it: the block input and output are narrow bottlenecks, and the inside is expanded by a factor tt (6 in the paper) so that the depthwise conv works in a high-dimensional space.

x∈Rc  →  1×1, ReLU6    Rtc  →  dw 3×3, ReLU6    Rtc  →  1×1, linear    Rc′y=x+F(x)  if stride 1 and c=c′x\in\mathbb R^{c}\;\xrightarrow{\;1\times1,\ \text{ReLU6}\;}\;\mathbb R^{tc}\;\xrightarrow{\;\text{dw }3\times3,\ \text{ReLU6}\;}\;\mathbb R^{tc}\;\xrightarrow{\;1\times1,\ \textbf{linear}\;}\;\mathbb R^{c'}\qquad y=x+F(x)\ \text{ if stride 1 and } c=c'
tt
expansion factor; the first block uses t=1t=1 and all others t=6t=6
ReLU6\text{ReLU6}
min⁡(max⁡(0,x),6)\min(\max(0,x),6), kept for robustness in low-precision arithmetic
linear\text{linear}
no activation after the projection: the bottleneck stays linear

Why linear? The paper's argument: if the "manifold of interest" lies in a low-dimensional subspace, a ReLU applied in a narrow layer zeroes out whole directions and loses information that later layers cannot recover. Expanding first and applying the ReLU in the wide space loses much less. Applying no ReLU after projecting back to the narrow space loses none. Their ablation found that adding a ReLU after the projection lowers accuracy.

Why connect the narrow ends? The residual shortcut joins the bottlenecks, which carry the information, and the stored tensor between blocks is small. MobileNetV2 has 17 such blocks with widths 16, 24, 32, 64, 96, 160, 320, then a 1×11\times1 conv to 1280 and the classifier.

Worked example: the 24 → 24 bottleneck at 56×56 (block 3), t = 6

  1. 1Expand 1×11\times1: 24→14424\to144 channels, 24⋅144=3,45624\cdot144=3{,}456 weights, 3,456⋅562=10,838,0163{,}456\cdot56^2=10{,}838{,}016 multiply-adds.
  2. 2Depthwise 3×33\times3 on 144 channels: 9⋅144=1,2969\cdot144=1{,}296 weights, 1,296⋅3,136=4,064,2561{,}296\cdot3{,}136=4{,}064{,}256 multiply-adds.
  3. 3Project 1×11\times1: 144→24144\to24, another 3,456 weights and 10,838,016 multiply-adds. No ReLU.
  4. 4Total 25,740,288 multiply-adds and 8,832 parameters including BatchNorm (8,2088{,}208 weights + 2⋅(144+144+24)+\,2\cdot(144+144+24)). Stride 1 and 24=2424=24, so the input is added to the output. The 1×11\times1 convs are 84.2% of the block's cost.

V3: searched, with h-swish and squeeze-and-excitation

Howard et al. ("Searching for MobileNetV3", ICCV 2019) kept the inverted residual block and let search tune it. Platform-aware neural architecture search (MnasNet-style, with measured phone latency in the reward) chose the block-level structure, and NetAdapt then trimmed individual layer widths. Several hand-made changes came on top:

h-swish(x)=x⋅ReLU6(x+3)6,SE(u)=u⊙h-sigmoid(W2 ReLU(W1 pool(u)))\text{h-swish}(x)=x\cdot\frac{\mathrm{ReLU6}(x+3)}{6},\qquad \text{SE}(u)=u\odot\text{h-sigmoid}\big(W_2\,\mathrm{ReLU}(W_1\,\mathrm{pool}(u))\big)
h-swish\text{h-swish}
a piecewise-polynomial stand-in for swish xσ(x)x\sigma(x), cheap and quantization-friendly; used mainly in the deeper layers
SE\text{SE}
squeeze-and-excitation (Hu et al., 2018): reweights the expanded channels by a gate computed from their global average
W1W_1
reduces to 1/4 of the expanded width, then W2W_2 expands back

They also halved the stem to 16 filters and redesigned the expensive last stage: the final 1×11\times1 expansion moves after global pooling, so it runs on a 1×11\times1 map instead of 7×77\times7. MobileNetV3-Large reports 75.2% top-1 at 219M multiply-adds (5.4M parameters), and V3-Small 67.4% at 56M (2.5M). torchvision's versions have 5,483,032 and 2,542,856 parameters.

python
1import torch, torch.nn as nn
2from torchvision.models import (mobilenet_v2, MobileNet_V2_Weights,
3 mobilenet_v3_large, MobileNet_V3_Large_Weights)
4
5class InvertedResidual(nn.Module):
6 def __init__(self, cin, cout, stride=1, t=6):
7 super().__init__()
8 hid = cin * t
9 self.use_res = stride == 1 and cin == cout
10 layers = []
11 if t != 1: # 1x1 expand
12 layers += [nn.Conv2d(cin, hid, 1, bias=False), nn.BatchNorm2d(hid), nn.ReLU6(inplace=True)]
13 layers += [nn.Conv2d(hid, hid, 3, stride, 1, groups=hid, bias=False), # depthwise 3x3
14 nn.BatchNorm2d(hid), nn.ReLU6(inplace=True),
15 nn.Conv2d(hid, cout, 1, bias=False), nn.BatchNorm2d(cout)] # linear bottleneck
16 self.conv = nn.Sequential(*layers)
17
18 def forward(self, x):
19 return x + self.conv(x) if self.use_res else self.conv(x)
20
21count = lambda m: sum(p.numel() for p in m.parameters())
22print(count(InvertedResidual(24, 24))) # 8832
23
24m = mobilenet_v2(weights=MobileNet_V2_Weights.IMAGENET1K_V1).eval()
25print(count(m)) # 3504872
26print(count(mobilenet_v2(width_mult=0.5))) # 1968680: the width multiplier
27print(count(mobilenet_v3_large())) # 5483032 (uses nn.Hardswish and SE blocks)
28
29with torch.no_grad():
30 logits = m(torch.randn(1, 3, 224, 224)) # any input side works: global pooling at the end
An inverted residual block, and the torchvision models

Use when

  • +Very low multiply-adds and parameters for their accuracy
  • +Width and resolution multipliers give a family of models from one design
  • +Most compute is in 1×1 convs, which are plain matrix multiplies
  • +Mature tooling: torchvision/timm weights, TFLite and mobile runtimes optimize them

Avoid when

  • −Depthwise layers are memory-bound: GPU speed-ups are much smaller than the FLOP savings
  • −Lower accuracy than large CNNs or ViTs; α≤0.25\alpha\le0.25 degrades sharply
  • −Depthwise weights and ReLU6/h-swish make int8 quantization more delicate
  • −V3's searched design is tuned for one phone CPU and may not be optimal elsewhere

Check yourself

MobileNetV1 at α = 1, 224×224 costs 569M multiply-adds. Roughly what does α = 0.5 at 224×224 cost?

Check yourself

Why does MobileNetV2 omit the activation after the 1×1 projection of each block?

Remember

  • ✓V1: straight stack of depthwise separable blocks; 4.2M params, 569M mult-adds, 70.6% top-1; 95% of compute is 1×11\times1 convs.
  • ✓Width multiplier α\alpha scales cost by about α2\alpha^2; resolution ρ\rho scales cost by ρ2\rho^2 and leaves parameters unchanged.
  • ✓V2: inverted residuals (narrow → expand t=6t=6 → depthwise → narrow) with a linear projection and shortcuts between bottlenecks; 3.4M, 300M, 72.0%.
  • ✓V3: NAS + NetAdapt, h-swish, squeeze-and-excitation, a cheaper head; Large 75.2% at 219M mult-adds.
  • ✓Low FLOPs do not guarantee low latency: measure on the target device.
← back to Deep Learning roadmap