Definition
ShuffleNet is a family of mobile CNNs from Megvii. ShuffleNet v1 (Zhang, Zhou, Lin and Sun, CVPR 2018) uses pointwise group convolutions for the layers and a channel shuffle that permutes channels between groups. ShuffleNet v2 (Ma, Zhang, Zheng and Sun, ECCV 2018) drops the group convolutions. It is built from channel split units designed with four guidelines for real-device speed.
In a MobileNet-style network the depthwise convs are cheap and the dense convs take over 90% of the multiply-adds. ShuffleNet v1 attacks that remaining cost by splitting the convs into groups, as ResNeXt does for its convs. That divides their cost by , and the saving can be spent on wider layers. The catch is that stacked group convolutions never mix information across groups. The channel shuffle fixes that for free.
At a glance
- v1 unit
- GConv → BN, ReLU → shuffle → dw → BN → GConv → BN, add, ReLU; stride 2: avg-pool shortcut and concat
- v2 unit
- split channels in half; one half is identity, the other goes → dw → (equal widths, no groups); concat, then shuffle
- v1 result
- 1×, : 137M multiply-adds, 32.6% top-1 error; 7.8 points lower error than MobileNet at about 40M multiply-adds; about 13× faster than AlexNet on an ARM phone
- v2 result
- 1×: 146M multiply-adds; torchvision weights 2,278,604 params, 69.36% top-1
- Note
- both papers say "FLOPs" for multiply-adds
- In PyTorch
nn.ChannelShuffle(g),torchvision.models.shufflenet_v2_x1_0(v1 is not in torchvision)
Channel shuffle
With groups, a group convolution's output group depends only on input group . Stack two of them and the network splits into independent narrow towers. The shuffle sends one channel from every group into every group before the next layer. With channels it reshapes the channel axis to , transposes to and flattens:
- number of groups and channels per group,
- the backward pass applies the inverse permutation (reshape to , transpose); no parameters, no arithmetic
For , the new order is 0, 4, 8, 1, 5, 9, 2, 6, 10, 3, 7, 11. Each group of the next convolution now reads one channel from each of the three previous groups. The operation is differentiable, has no parameters and costs only a memory copy, which is often fused into the next layer's read.
- input and output channels
- groups; v1 tried with stage widths 144 → 384 so all versions cost about 140M multiply-adds
Worked example: one v1 unit (g = 3, stage 3, 480 channels on 14×14, stride 1)
- 1Bottleneck width is a quarter of the output: .
- 2 GConv in 3 groups: weights. Depthwise : . GConv : . That is 39,480 weights, plus BN parameters.
- 3Multiply-adds . With dense convs () the same widths would need weights, 2.95× more.
- 4Stage 3 is one stride-2 unit and seven of these: 307,680 parameters and 63.8M multiply-adds in the figure's library, almost half of the network's 137.5M.
v2: FLOPs are an indirect metric
Ma et al. measured networks with the same FLOPs and found very different speeds on GPU and ARM. FLOPs ignore memory access cost (MAC) and degree of parallelism. Their analysis of a conv with multiply-adds gives the first guideline:
- reading the input map and writing the output map
- reading the weights
- AM–GM:
| setting | c₁, c₂ | MAC | vs best |
|---|---|---|---|
| G1: 1:1 | 128, 128 | 819,200 | 1.00× (= bound) |
| G1: 1:2 | 90.5, 181 | 867,899 | 1.06× |
| G1: 1:6 | 52.3, 313.5 | 1,163,503 | 1.42× |
| G1: 1:12 | 37, 443.4 | 1,522,779 | 1.86× |
| G2: | 181, 181 | 1,151,737 | 1.41× |
| G2: | 256, 256 | 1,622,016 | 1.98× |
| G2: | 362, 362 | 2,287,091 | 2.79× |
The four guidelines (Ma et al. 2018)
- 1G1, equal channel width minimizes MAC. For fixed FLOPs, a conv is cheapest in memory traffic when (table, top rows).
- 2G2, excessive group convolution increases MAC. With groups, . At fixed FLOPs the wider layers that groups allow read and write more activations (table, bottom rows). Choose for the target platform, not by FLOPs alone.
- 3G3, network fragmentation reduces parallelism. Many small parallel branches (as in Inception cells or NAS cells) launch many small kernels and synchronize often. That is slow on GPUs even when the FLOPs are low.
- 4G4, element-wise operations are not negligible. ReLU, additions, bias and depthwise convs have few FLOPs but high MAC. Removing the ReLU and shortcut from a ResNet bottleneck made it about 20% faster in their measurements on both GPU and ARM.
ShuffleNet v1 breaks three of them: G2 with its group convs, G1 with its 4:1 bottleneck, and G4 with its residual add. The v2 unit obeys all four. Channel split sends channels through an identity path and through dw with equal widths and no groups (G1, G2). There is a single branch of real work (G3), and concatenation replaces the add (G4). A channel shuffle then mixes the two halves. In the stride-2 unit both halves process the full input with stride 2, which doubles the channel count.
1import torch, torch.nn as nn2from torchvision.models import shufflenet_v2_x1_0, ShuffleNet_V2_X1_0_Weights34def channel_shuffle(x, g):5 b, c, h, w = x.shape6 return x.view(b, g, c // g, h, w).transpose(1, 2).reshape(b, c, h, w)78x = torch.arange(12.).view(1, 12, 1, 1)9print(channel_shuffle(x, 3).flatten().tolist()) # [0, 4, 8, 1, 5, 9, 2, 6, 10, 3, 7, 11]10print(torch.equal(nn.ChannelShuffle(3)(x), channel_shuffle(x, 3))) # True1112class ShuffleV2Unit(nn.Module): # stride 113 def __init__(self, c):14 super().__init__()15 b = c // 216 self.branch = nn.Sequential(17 nn.Conv2d(b, b, 1, bias=False), nn.BatchNorm2d(b), nn.ReLU(inplace=True),18 nn.Conv2d(b, b, 3, padding=1, groups=b, bias=False), nn.BatchNorm2d(b), # depthwise, no ReLU19 nn.Conv2d(b, b, 1, bias=False), nn.BatchNorm2d(b), nn.ReLU(inplace=True))2021 def forward(self, x):22 x1, x2 = x.chunk(2, dim=1) # channel split23 return channel_shuffle(torch.cat([x1, self.branch(x2)], dim=1), 2)2425m = shufflenet_v2_x1_0(weights=ShuffleNet_V2_X1_0_Weights.IMAGENET1K_V1).eval()26print(sum(p.numel() for p in m.parameters())) # 2278604Use when
- +Very low multiply-adds: v1 and v2 1× need about 140M
- +Channel shuffle is free in parameters and FLOPs and restores cross-group flow
- +v2 is fast on real hardware: equal widths, no groups, concat instead of add
- +Simple width scaling (0.5× to 2×) and good torchvision weights for v2
Avoid when
- −v1's group convs and channel shuffles are memory-bound on GPUs
- −Shuffle and split are reshapes/copies that some runtimes and accelerators handle poorly
- −Accuracy now below newer mobile nets (MobileNetV3, EfficientNet-Lite) at similar cost
- −Guidelines are about a 1×1-conv model of cost; real kernels and fusion can change the picture
Check yourself
Two group convolutions with g = 4 are stacked, with no shuffle between them. Output group 2 of the second layer depends on which input groups?
Check yourself
A 1×1 conv on a 28×28 map must cost exactly 25M multiply-adds. Which channel choice gives the lowest memory access cost?
Remember
- ✓v1: group convs cut the dominant cost by ; the channel shuffle (reshape → transpose → flatten) lets groups exchange information.
- ✓v1 1×, : about 137M multiply-adds, 32.6% top-1 error; the shuffle is worth 1.9 points at and 5.2 at .
- ✓v2: FLOPs are indirect. MAC of a conv is , minimized at .
- ✓Four guidelines: equal widths, avoid excessive groups, limit fragmentation, count element-wise ops.
- ✓v2 unit: channel split, a single equal-width branch, concat, shuffle; no group convs.