CycleGAN

CycleGAN (Zhu, Park, Isola, Efros, ICCV 2017) learns image translation from two unpaired collections. Two generators G: X→Y and F: Y→X, each with a PatchGAN discriminator, are trained with LSGAN losses plus a cycle-consistency loss ‖F(G(x))−x‖₁ + ‖G(F(y))−y‖₁ weighted λ = 10.

13 min read1101 wordscore0 viewsupdated 2026-10-12

Definition

CycleGAN is the unpaired image-to-image translation method of Zhu, Park, Isola and Efros, "Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks" (ICCV 2017). It needs only a set of images from domain XX (horses) and a set from domain YY (zebras), with no correspondence between them. It trains two generators, G:X→YG:X\to Y and F:Y→XF:Y\to X, and two discriminators, DYD_Y and DXD_X. Each generator fools its discriminator, and the cycle-consistency loss requires that translating there and back returns the input: F(G(x))≈xF(G(x))\approx x and G(F(y))≈yG(F(y))\approx y.

Pix2Pix showed that a conditional GAN with an L1 loss solves many translation problems, provided you have aligned pairs (x,y)(x,y). For most interesting pairs of domains no such data exists. Nobody has the zebra version of a horse photo, the photo Monet painted from, or summer and winter shots taken from exactly the same viewpoint. Jun-Yan Zhu, Taesung Park, Phillip Isola and Alexei Efros asked what can be learned from two unrelated collections. An adversarial loss alone can make G(X)G(X) look like YY as a distribution, but it says nothing about which yy a given xx should become. Their fix borrows an old idea from language translation: a good translation should translate back. DualGAN and DiscoGAN proposed the same two-way constraint independently in 2017.

Objective

Lcyc(G,F)=Ex∼pdata(x)[∥F(G(x))−x∥1]+Ey∼pdata(y)[∥G(F(y))−y∥1]\mathcal L_{cyc}(G,F)=\mathbb E_{x\sim p_{data}(x)}\big[\|F(G(x))-x\|_1\big]+\mathbb E_{y\sim p_{data}(y)}\big[\|G(F(y))-y\|_1\big]
G:X→YG:X\to Y
e.g. horse → zebra
F:Y→XF:Y\to X
e.g. zebra → horse; trained jointly with G
x→G(x)→F(G(x))x\to G(x)\to F(G(x))
forward cycle consistency
y→F(y)→G(F(y))y\to F(y)\to G(F(y))
backward cycle consistency
∥⋅∥1\|\cdot\|_1
pixel-wise L1; the authors tried an adversarial loss here and saw no improvement
L(G,F,DX,DY)=LGAN(G,DY,X,Y)+LGAN(F,DX,Y,X)+λ Lcyc(G,F),G∗,F∗=arg⁡min⁡G,Fmax⁡DX,DYL,λ=10\mathcal L(G,F,D_X,D_Y)=\mathcal L_{GAN}(G,D_Y,X,Y)+\mathcal L_{GAN}(F,D_X,Y,X)+\lambda\,\mathcal L_{cyc}(G,F),\qquad G^\ast,F^\ast=\arg\min_{G,F}\max_{D_X,D_Y}\mathcal L,\qquad \lambda=10
LGAN(G,DY,X,Y)\mathcal L_{GAN}(G,D_Y,X,Y)
adversarial loss making G(x)G(x) indistinguishable from real yy
λ\lambda
relative weight of cycle consistency; 10 in all experiments
DX, DYD_X,\,D_Y
70×7070\times70 PatchGANs, as in Pix2Pix, but unconditional: there is no paired input to show them

The model can be read as two autoencoders, F∘G:X→XF\circ G:X\to X and G∘F:Y→YG\circ F:Y\to Y, whose latent code is an image of the other domain. The adversarial losses force the code to look like a real image of that domain. Cycle consistency forces it to keep enough information to rebuild the input.

Why the adversarial loss is not enough

With enough capacity, GG can map XX onto any arrangement of YY that has the right distribution. Every permutation of target images is an equally good optimum, so a single photo of a horse could become any zebra. In practice the outcome is often mode collapse: many inputs map to the same output and the optimisation stalls. Cycle consistency rules out mappings that lose information. If two horses go to the same zebra, FF cannot send that zebra back to both. It also ties each output to its input, since FF has to recover xx from G(x)G(x). The figure below makes this exact with rings, where all rotations of a uniform ring have the same distribution.

LSGAN losses, history buffer, identity loss

min⁡G Ex[(DY(G(x))−1)2],min⁡DY 12(Ey[(DY(y)−1)2]+Ex[DY(G(x))2])\min_G\ \mathbb E_x\big[(D_Y(G(x))-1)^2\big],\qquad \min_{D_Y}\ \tfrac12\Big(\mathbb E_y\big[(D_Y(y)-1)^2\big]+\mathbb E_x\big[D_Y(G(x))^2\big]\Big)
(⋅−1)2(\cdot-1)^2
least-squares GAN (Mao et al. 2017) in place of the negative log-likelihood; more stable training and better results in their experiments
DY(⋅)D_Y(\cdot)
raw PatchGAN output, no sigmoid; regressed toward 1 for real and 0 for fake
12\tfrac12
the objective of DD is halved to slow it relative to GG

Two more stabilisers come from earlier work. Following Shrivastava et al. (2017), the discriminators are updated on a history of generated images rather than only the latest one: a buffer of the 50 most recently generated images per domain. In the reference code, once the buffer is full each new fake is either passed through or, with probability 0.5, swapped for a random stored one. This keeps DD from chasing the generator's latest mode. For painting↔photo the authors add an identity loss (after Taigman et al. 2017) that asks each generator to leave images of its target domain unchanged:

Lidentity(G,F)=Ey[∥G(y)−y∥1]+Ex[∥F(x)−x∥1],weight 0.5 λ\mathcal L_{identity}(G,F)=\mathbb E_y\big[\|G(y)-y\|_1\big]+\mathbb E_x\big[\|F(x)-x\|_1\big],\qquad \text{weight } 0.5\,\lambda
G(y)G(y)
a zebra (or a photo) fed to the horse→zebra generator should come out unchanged
0.5λ0.5\lambda
weight used in the paper's painting→photo experiments

Without it, GG and FF may change the colour of the input even when they don't need to. For example, Monet's daytime paintings were sometimes turned into sunset photos. Nothing in the cycle or adversarial losses forbids a reversible tint change. The identity term breaks the tie in favour of changing as little as possible.

Setup in the paper

Generator
Johnson et al. (2016): c7s1-64, d128, d256, residual blocks R256, u128, u64, c7s1-3, with instance normalisation and reflection padding. 6 blocks at 1282128^2, 9 blocks at 2562256^2 and above
Discriminator
70×7070\times70 PatchGAN, C64-C128-C256-C512, instance norm, LeakyReLU 0.2
Losses
LSGAN; λ=10\lambda=10 on Lcyc\mathcal L_{cyc}; identity loss 0.5λ0.5\lambda for painting↔photo
Optimiser
Adam, batch size 1, learning rate 2⋅10−42\cdot10^{-4} for 100 epochs, then linear decay to 0 over the next 100; weights from N(0,0.02)\mathcal N(0,0.02)
Stabilisers
history buffer of 50 generated images per discriminator; DD loss halved
Applications
horse↔zebra, apple↔orange, summer↔winter Yosemite, photo↔Monet/Van Gogh/Cézanne/Ukiyo-e, photo enhancement (shallow depth of field), labels↔photo and map↔aerial photo as quantitative benchmarks
G: (ρ, θ) ↦ (1.00ρ, θ + 60°) · F: (ρ, φ) ↦ (ρ, φ) · x → G(x) → F(G(x))domain Xdomain Y● source · ○ reconstruction · — cycle error · grey ○ real samples of the target domainW₁(G(X), Y)0.000W₁(F(Y), X)0.000‖F(G(x)) − x‖₁1.279‖G(F(y)) − y‖₁1.262identity ‖·‖₁off (1.262)total25.414adversarial: both rings are uniform, so G(X) ~ Y for any a, b, k; the gap is |r − 1|forward round trip F(G(x)): θ ↦ θ + 60°W₁ + λ(fwd + bwd) = 0.000 + 10·(1.279 + 1.262) = 25.414set b = a to make F the inverse of Gselected point 3: |error| = 1.000 (Euclidean, unit ring)W₁ stands in for what a perfect discriminator can detect; the paper uses LSGAN losses
cycle
fig.Two toy domains, each 24 points on a unit ring with its own random angles. G rotates by a and scales the radius by r, and F rotates back by b. Both rings are uniform, so G(X) has the distribution of Y for every rotation. The adversarial term, drawn as the 1-Wasserstein distance a perfect critic would see, is |r − 1| and reads 0.000 at any a and b. The default (a = 60°, b = 0°) fools both discriminators, yet every point comes back 60° away: forward cycle L1 1.279, backward 1.262, total 0 + 10·(1.279 + 1.262) = 25.414. Partial undoing (b = 30°) halves the cycle error (0.661 + 0.654, total 13.149). Only b = a closes both cycles (total 0.000). The fold θ ↦ 2θ wraps X twice around Y, which is still a perfect match in distribution, but F cannot undo it: at a = b = 0 the cycle losses are 1.640 and 1.594 (total 32.334), and no b brings their sum below 3.199. A radius gain of 1.3 is the only thing the adversary sees (W₁ = 0.30, cycle 0.382 + 0.381, total 7.927). The identity loss splits the zero-cycle solutions: at a = b = 60° it adds 0.5·10·2.540 = 12.701, and at a = b = 0° it adds nothing.

Worked example: one generator and discriminator loss

Take 4-pixel images and 2-patch discriminator outputs. Inputs x=(0.5,−0.3,0.9,0.1)x=(0.5,-0.3,0.9,0.1) from XX and an unrelated y=(0.0,0.7,−0.5,0.3)y=(0.0,0.7,-0.5,0.3) from YY.

quantityvalue
D_Y(G(x))(0.6, 0.3)
D_X(F(y))(0.8, 0.4)
F(G(x))(0.45, −0.2, 0.8, 0.1)
G(F(y))(0.1, 0.6, −0.5, 0.5)
G(y) (identity)(0.05, 0.6, −0.45, 0.35)
F(x) (identity)(0.5, −0.25, 0.85, 0.0)
D_Y(y) (real)(0.9, 0.7)
Forward passes for one step (all values made up; losses computed in Python)

Assembling the losses

  1. 1LSGAN generator terms: mean((0.6−1)2,(0.3−1)2)=(0.16+0.49)/2=0.325\mathrm{mean}\big((0.6-1)^2,(0.3-1)^2\big)=(0.16+0.49)/2=0.325 for GG, and (0.04+0.36)/2=0.200(0.04+0.36)/2=0.200 for FF.
  2. 2Forward cycle: ∣0.45−0.5∣+∣−0.2+0.3∣+∣0.8−0.9∣+∣0.1−0.1∣=0.25|0.45-0.5|+|-0.2+0.3|+|0.8-0.9|+|0.1-0.1|=0.25, mean 0.06250.0625. Backward: 0.1+0.1+0+0.2=0.40.1+0.1+0+0.2=0.4, mean 0.10000.1000. Weighted: 10⋅0.1625=1.62510\cdot0.1625=1.625.
  3. 3Without identity, LG,F=0.325+0.200+1.625=2.150\mathcal L_{G,F}=0.325+0.200+1.625=2.150. Most of the signal comes from the cycle term even though both reconstructions are close.
  4. 4Identity: ∥G(y)−y∥1=0.25/4=0.0625\|G(y)-y\|_1=0.25/4=0.0625 and ∥F(x)−x∥1=0.2/4=0.05\|F(x)-x\|_1=0.2/4=0.05, sum 0.11250.1125, times 0.5λ=50.5\lambda=5: 0.56250.5625. Total 2.71252.7125.
  5. 5Discriminator DYD_Y on a real yy and a (buffered) fake: 12[mean((0.9−1)2,(0.7−1)2)+mean(0.62,0.32)]=12(0.05+0.225)=0.1375\tfrac12\big[\mathrm{mean}((0.9-1)^2,(0.7-1)^2)+\mathrm{mean}(0.6^2,0.3^2)\big]=\tfrac12(0.05+0.225)=0.1375.

Results and failure cases

On Cityscapes labels→photo with FCN-scores, CycleGAN reached per-pixel accuracy 0.52, per-class accuracy 0.17 and class IoU 0.11 without any pairs. Pix2Pix, trained on the true pairs, scored 0.71, 0.25 and 0.18 in the same evaluation. Paired supervision still wins clearly. In Turk studies on maps↔aerial photos, CycleGAN outputs were taken for real in about a quarter of trials (26.8% for map→photo, 23.2% for photo→map). The baselines (CoGAN, BiGAN/ALI, SimGAN, feature loss + GAN) almost never fooled raters. In the ablation, removing the GAN losses or the cycle loss substantially degraded results. GAN loss with only one cycle direction often trained unstably and collapsed, particularly for the direction that was not constrained.

The authors document clear limits. Colour and texture changes work well. Geometric changes mostly fail: on dog→cat the learned translation degenerates into minimal changes to the input. The authors attribute this to generator architectures tailored to appearance changes. The training distribution matters too. ImageNet's wild-horse and zebra classes contain no riders, so a man riding a horse got zebra stripes on his skin. Translations can also be ambiguous, for example tree↔building labels on Cityscapes. The cycle loss enforces invertibility, not semantic correctness: Chu, Zhmoginov and Sandler (2017) showed that CycleGAN can hide the information needed to rebuild xx as a near-invisible, high-frequency signal in G(x)G(x).

python
1import random, torch, torch.nn as nn, torch.nn.functional as F
2
3class ResBlock(nn.Module):
4 def __init__(self, c):
5 super().__init__()
6 self.body = nn.Sequential(nn.ReflectionPad2d(1), nn.Conv2d(c, c, 3), nn.InstanceNorm2d(c), nn.ReLU(True),
7 nn.ReflectionPad2d(1), nn.Conv2d(c, c, 3), nn.InstanceNorm2d(c))
8 def forward(self, x):
9 return x + self.body(x)
10
11def resnet_generator(n_blocks=9): # c7s1-64, d128, d256, 9×R256, u128, u64, c7s1-3
12 L = [nn.ReflectionPad2d(3), nn.Conv2d(3, 64, 7), nn.InstanceNorm2d(64), nn.ReLU(True)]
13 for c in (64, 128):
14 L += [nn.Conv2d(c, 2 * c, 3, 2, 1), nn.InstanceNorm2d(2 * c), nn.ReLU(True)]
15 L += [ResBlock(256) for _ in range(n_blocks)]
16 for c in (256, 128):
17 L += [nn.ConvTranspose2d(c, c // 2, 3, 2, 1, output_padding=1), nn.InstanceNorm2d(c // 2), nn.ReLU(True)]
18 L += [nn.ReflectionPad2d(3), nn.Conv2d(64, 3, 7), nn.Tanh()]
19 return nn.Sequential(*L)
20
21def patchgan(): # 70×70 PatchGAN C64-C128-C256-C512, one score per patch
22 L, c = [nn.Conv2d(3, 64, 4, 2, 1), nn.LeakyReLU(0.2, True)], 64
23 for co, s in ((128, 2), (256, 2), (512, 1)):
24 L += [nn.Conv2d(c, co, 4, s, 1), nn.InstanceNorm2d(co), nn.LeakyReLU(0.2, True)]
25 c = co
26 return nn.Sequential(*L, nn.Conv2d(512, 1, 4, 1, 1)) # no sigmoid: LSGAN regresses the raw output
27
28class ImagePool: # history of 50 generated images for the discriminator
29 def __init__(self, size=50):
30 self.size, self.images = size, []
31 def query(self, img):
32 if len(self.images) < self.size:
33 self.images.append(img.detach()); return img.detach()
34 if random.random() < 0.5: # half the time: show an old fake, store the new one
35 i = random.randrange(self.size)
36 old, self.images[i] = self.images[i], img.detach()
37 return old
38 return img.detach()
39
40G, F_, D_X, D_Y = resnet_generator(), resnet_generator(), patchgan(), patchgan() # G: X→Y, F: Y→X
41opt_g = torch.optim.Adam(list(G.parameters()) + list(F_.parameters()), lr=2e-4, betas=(0.5, 0.999))
42opt_d = torch.optim.Adam(list(D_X.parameters()) + list(D_Y.parameters()), lr=2e-4, betas=(0.5, 0.999))
43pool_x, pool_y, lam, lam_idt = ImagePool(), ImagePool(), 10.0, 0.5
44mse = nn.MSELoss()
45gan = lambda d, real: mse(d, torch.full_like(d, 1.0 if real else 0.0))
46
47x = torch.rand(1, 3, 256, 256) * 2 - 1 # a horse (batch size 1, as in the paper)
48y = torch.rand(1, 3, 256, 256) * 2 - 1 # an unrelated zebra: no pairing anywhere
49
50# generators: fool both discriminators, close both cycles, and (optionally) leave targets unchanged
51fake_y, fake_x = G(x), F_(y)
52loss_gan = gan(D_Y(fake_y), True) + gan(D_X(fake_x), True)
53loss_cyc = F.l1_loss(F_(fake_y), x) + F.l1_loss(G(fake_x), y) # ‖F(G(x))−x‖₁ + ‖G(F(y))−y‖₁
54loss_idt = F.l1_loss(G(y), y) + F.l1_loss(F_(x), x) # ‖G(y)−y‖₁ + ‖F(x)−x‖₁
55loss_g = loss_gan + lam * loss_cyc + lam_idt * lam * loss_idt
56opt_g.zero_grad(); loss_g.backward(); opt_g.step()
57
58# discriminators: LSGAN, real → 1, (pooled) fake → 0, halved
59loss_d = 0.5 * (gan(D_Y(y), True) + gan(D_Y(pool_y.query(fake_y)), False)) \
60 + 0.5 * (gan(D_X(x), True) + gan(D_X(pool_x.query(fake_x)), False))
61opt_d.zero_grad(); loss_d.backward(); opt_d.step()
62
63print(fake_y.shape, D_Y(fake_y).shape) # torch.Size([1, 3, 256, 256]) torch.Size([1, 1, 30, 30])
64print(f"{sum(p.numel() for p in G.parameters()) / 1e6:.2f}M per generator, "
65 f"{sum(p.numel() for p in D_Y.parameters()) / 1e6:.2f}M per discriminator") # 11.38M, 2.76M
CycleGAN: ResNet generators, 70×70 PatchGANs, image pool, and one step of LSGAN + 10·cycle + 5·identity (run with PyTorch 2 on CPU)

Use when

  • +Needs only two unpaired image collections
  • +Strong for colour, texture and style changes (season transfer, painting↔photo, horse↔zebra)
  • +Simple, reusable recipe: two GANs plus an L1 cycle term
  • +Cycle consistency rules out many-to-one mappings that a pure adversarial loss allows

Avoid when

  • −Twice the networks of Pix2Pix and six generator passes per step with identity loss
  • −Fails on geometric or shape changes (dog↔cat), where it degenerates into minimal edits
  • −Clearly below paired training when pairs exist (FCN per-pixel accuracy 0.52 vs 0.71)
  • −Deterministic one-to-one mapping; can hide information steganographically instead of translating

Check yourself

Why can't CycleGAN rely on the two adversarial losses alone?

Check yourself

On painting→photo, G turns a daytime Monet into a convincing sunset photo, and F turns it back perfectly. Which term discourages this?

Remember

  • ✓CycleGAN objective: LGAN(G,DY)+LGAN(F,DX)+λ(∥F(G(x))−x∥1+∥G(F(y))−y∥1)\mathcal L_{GAN}(G,D_Y)+\mathcal L_{GAN}(F,D_X)+\lambda\big(\|F(G(x))-x\|_1+\|G(F(y))-y\|_1\big) with λ=10\lambda=10, on unpaired sets XX and YY.
  • ✓Training details: LSGAN losses (D(G(x))−1)2(D(G(x))-1)^2, a history buffer of 50 generated images, an identity loss ∥G(y)−y∥1+∥F(x)−x∥1\|G(y)-y\|_1+\|F(x)-x\|_1 weighted 0.5λ0.5\lambda, ResNet generators and 70×7070\times70 PatchGANs.
  • ✓Adversarial losses match distributions only; cycle consistency makes the mapping invertible, which rules out many-to-one collapse.
  • ✓Works for colour and texture, fails for geometric changes, and stays below paired Pix2Pix when pairs exist.
  • ✓Zero cycle loss does not imply a correct translation: reversible tints and hidden steganographic codes also satisfy it.
← back to Deep Learning roadmap