# CycleGAN

> CycleGAN (Zhu, Park, Isola, Efros, ICCV 2017) learns image translation from two unpaired collections. Two generators G: X→Y and F: Y→X, each with a PatchGAN discriminator, are trained with LSGAN losses plus a cycle-consistency loss ‖F(G(x))−x‖₁ + ‖G(F(y))−y‖₁ weighted λ = 10.

Source: https://www.mlfoundry.in/learn/cyclegan  
Section: Deep Learning › Generative Deep Learning Algorithms  
Level: core · 13 min read · updated 2026-10-12

**CycleGAN:** is the unpaired image-to-image translation method of Zhu, Park, Isola and Efros, "Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks" (ICCV 2017). It needs only a set of images from domain $X$ (horses) and a set from domain $Y$ (zebras), with no correspondence between them. It trains two generators, $G:X\to Y$ and $F:Y\to X$, and two discriminators, $D_Y$ and $D_X$. Each generator fools its discriminator, and the **cycle-consistency loss** requires that translating there and back returns the input: $F(G(x))\approx x$ and $G(F(y))\approx y$.

**Pix2Pix** showed that a conditional GAN with an L1 loss solves many translation problems, provided you have *aligned pairs* $(x,y)$. For most interesting pairs of domains no such data exists. Nobody has the zebra version of a horse photo, the photo Monet painted from, or summer and winter shots taken from exactly the same viewpoint. Jun-Yan Zhu, Taesung Park, Phillip Isola and Alexei Efros asked what can be learned from two unrelated collections. An adversarial loss alone can make $G(X)$ look like $Y$ *as a distribution*, but it says nothing about which $y$ a given $x$ should become. Their fix borrows an old idea from language translation: a good translation should translate back. DualGAN and DiscoGAN proposed the same two-way constraint independently in 2017.

## Objective

$$
\mathcal L_{cyc}(G,F)=\mathbb E_{x\sim p_{data}(x)}\big[\|F(G(x))-x\|_1\big]+\mathbb E_{y\sim p_{data}(y)}\big[\|G(F(y))-y\|_1\big]
$$

- $G:X\to Y$: e.g. horse → zebra
- $F:Y\to X$: e.g. zebra → horse; trained jointly with G
- $x\to G(x)\to F(G(x))$: **forward** cycle consistency
- $y\to F(y)\to G(F(y))$: **backward** cycle consistency
- $\|\cdot\|_1$: pixel-wise L1; the authors tried an adversarial loss here and saw no improvement

$$
\mathcal L(G,F,D_X,D_Y)=\mathcal L_{GAN}(G,D_Y,X,Y)+\mathcal L_{GAN}(F,D_X,Y,X)+\lambda\,\mathcal L_{cyc}(G,F),\qquad G^\ast,F^\ast=\arg\min_{G,F}\max_{D_X,D_Y}\mathcal L,\qquad \lambda=10
$$

- $\mathcal L_{GAN}(G,D_Y,X,Y)$: adversarial loss making $G(x)$ indistinguishable from real $y$
- $\lambda$: relative weight of cycle consistency; 10 in all experiments
- $D_X,\,D_Y$: $70\times70$ PatchGANs, as in **Pix2Pix**, but *unconditional*: there is no paired input to show them

The model can be read as two autoencoders, $F\circ G:X\to X$ and $G\circ F:Y\to Y$, whose **latent code is an image of the other domain**. The adversarial losses force the code to look like a real image of that domain. Cycle consistency forces it to keep enough information to rebuild the input.

## Why the adversarial loss is not enough

With enough capacity, $G$ can map $X$ onto *any* arrangement of $Y$ that has the right distribution. Every permutation of target images is an equally good optimum, so a single photo of a horse could become any zebra. In practice the outcome is often **mode collapse**: many inputs map to the same output and the optimisation stalls. Cycle consistency rules out mappings that lose information. If two horses go to the same zebra, $F$ cannot send that zebra back to both. It also ties each output to its input, since $F$ has to recover $x$ from $G(x)$. The figure below makes this exact with rings, where all rotations of a uniform ring have the same distribution.

## LSGAN losses, history buffer, identity loss

$$
\min_G\ \mathbb E_x\big[(D_Y(G(x))-1)^2\big],\qquad \min_{D_Y}\ \tfrac12\Big(\mathbb E_y\big[(D_Y(y)-1)^2\big]+\mathbb E_x\big[D_Y(G(x))^2\big]\Big)
$$

- $(\cdot-1)^2$: least-squares GAN (Mao et al. 2017) in place of the negative log-likelihood; more stable training and better results in their experiments
- $D_Y(\cdot)$: raw PatchGAN output, no sigmoid; regressed toward 1 for real and 0 for fake
- $\tfrac12$: the objective of $D$ is halved to slow it relative to $G$

Two more stabilisers come from earlier work. Following Shrivastava et al. (2017), the discriminators are updated on a **history of generated images** rather than only the latest one: a buffer of the **50** most recently generated images per domain. In the reference code, once the buffer is full each new fake is either passed through or, with probability 0.5, swapped for a random stored one. This keeps $D$ from chasing the generator's latest mode. For painting↔photo the authors add an **identity loss** (after Taigman et al. 2017) that asks each generator to leave images of its *target* domain unchanged:

$$
\mathcal L_{identity}(G,F)=\mathbb E_y\big[\|G(y)-y\|_1\big]+\mathbb E_x\big[\|F(x)-x\|_1\big],\qquad \text{weight } 0.5\,\lambda
$$

- $G(y)$: a zebra (or a photo) fed to the horse→zebra generator should come out unchanged
- $0.5\lambda$: weight used in the paper's painting→photo experiments

Without it, $G$ and $F$ may change the colour of the input even when they don't need to. For example, Monet's daytime paintings were sometimes turned into sunset photos. Nothing in the cycle or adversarial losses forbids a reversible tint change. The identity term breaks the tie in favour of changing as little as possible.

**Setup in the paper**

- **Generator:** Johnson et al. (2016): c7s1-64, d128, d256, residual blocks R256, u128, u64, c7s1-3, with instance normalisation and reflection padding. 6 blocks at $128^2$, 9 blocks at $256^2$ and above
- **Discriminator:** $70\times70$ PatchGAN, C64-C128-C256-C512, instance norm, LeakyReLU 0.2
- **Losses:** LSGAN; $\lambda=10$ on $\mathcal L_{cyc}$; identity loss $0.5\lambda$ for painting↔photo
- **Optimiser:** Adam, batch size 1, learning rate $2\cdot10^{-4}$ for 100 epochs, then linear decay to 0 over the next 100; weights from $\mathcal N(0,0.02)$
- **Stabilisers:** history buffer of 50 generated images per discriminator; $D$ loss halved
- **Applications:** horse↔zebra, apple↔orange, summer↔winter Yosemite, photo↔Monet/Van Gogh/Cézanne/Ukiyo-e, photo enhancement (shallow depth of field), labels↔photo and map↔aerial photo as quantitative benchmarks

> Figure: Two toy domains, each 24 points on a unit ring with its own random angles. G rotates by a and scales the radius by r, and F rotates back by b. Both rings are uniform, so G(X) has the distribution of Y for every rotation. The adversarial term, drawn as the 1-Wasserstein distance a perfect critic would see, is |r − 1| and reads 0.000 at any a and b. The default (a = 60°, b = 0°) fools both discriminators, yet every point comes back 60° away: forward cycle L1 1.279, backward 1.262, total 0 + 10·(1.279 + 1.262) = 25.414. Partial undoing (b = 30°) halves the cycle error (0.661 + 0.654, total 13.149). Only b = a closes both cycles (total 0.000). The fold θ ↦ 2θ wraps X twice around Y, which is still a perfect match in distribution, but F cannot undo it: at a = b = 0 the cycle losses are 1.640 and 1.594 (total 32.334), and no b brings their sum below 3.199. A radius gain of 1.3 is the only thing the adversary sees (W₁ = 0.30, cycle 0.382 + 0.381, total 7.927). The identity loss splits the zero-cycle solutions: at a = b = 60° it adds 0.5·10·2.540 = 12.701, and at a = b = 0° it adds nothing.

## Worked example: one generator and discriminator loss

Take 4-pixel images and 2-patch discriminator outputs. Inputs $x=(0.5,-0.3,0.9,0.1)$ from $X$ and an unrelated $y=(0.0,0.7,-0.5,0.3)$ from $Y$.

Forward passes for one step (all values made up; losses computed in Python)

| quantity | value |
| --- | --- |
| D_Y(G(x)) | (0.6, 0.3) |
| D_X(F(y)) | (0.8, 0.4) |
| F(G(x)) | (0.45, −0.2, 0.8, 0.1) |
| G(F(y)) | (0.1, 0.6, −0.5, 0.5) |
| G(y)  (identity) | (0.05, 0.6, −0.45, 0.35) |
| F(x)  (identity) | (0.5, −0.25, 0.85, 0.0) |
| D_Y(y)  (real) | (0.9, 0.7) |

**Assembling the losses**

1. LSGAN generator terms: $\mathrm{mean}\big((0.6-1)^2,(0.3-1)^2\big)=(0.16+0.49)/2=0.325$ for $G$, and $(0.04+0.36)/2=0.200$ for $F$.
2. Forward cycle: $|0.45-0.5|+|-0.2+0.3|+|0.8-0.9|+|0.1-0.1|=0.25$, mean $0.0625$. Backward: $0.1+0.1+0+0.2=0.4$, mean $0.1000$. Weighted: $10\cdot0.1625=1.625$.
3. Without identity, $\mathcal L_{G,F}=0.325+0.200+1.625=2.150$. Most of the signal comes from the cycle term even though both reconstructions are close.
4. Identity: $\|G(y)-y\|_1=0.25/4=0.0625$ and $\|F(x)-x\|_1=0.2/4=0.05$, sum $0.1125$, times $0.5\lambda=5$: $0.5625$. Total $2.7125$.
5. Discriminator $D_Y$ on a real $y$ and a (buffered) fake: $\tfrac12\big[\mathrm{mean}((0.9-1)^2,(0.7-1)^2)+\mathrm{mean}(0.6^2,0.3^2)\big]=\tfrac12(0.05+0.225)=0.1375$.

## Results and failure cases

On Cityscapes labels→photo with FCN-scores, CycleGAN reached per-pixel accuracy 0.52, per-class accuracy 0.17 and class IoU 0.11 without any pairs. Pix2Pix, trained on the true pairs, scored 0.71, 0.25 and 0.18 in the same evaluation. Paired supervision still wins clearly. In Turk studies on maps↔aerial photos, CycleGAN outputs were taken for real in about a quarter of trials (26.8% for map→photo, 23.2% for photo→map). The baselines (CoGAN, BiGAN/ALI, SimGAN, feature loss + GAN) almost never fooled raters. In the ablation, removing the GAN losses or the cycle loss substantially degraded results. GAN loss with only one cycle direction often trained unstably and collapsed, particularly for the direction that was not constrained.

The authors document clear limits. **Colour and texture** changes work well. **Geometric** changes mostly fail: on dog→cat the learned translation degenerates into minimal changes to the input. The authors attribute this to generator architectures tailored to appearance changes. The training distribution matters too. ImageNet's wild-horse and zebra classes contain no riders, so a man riding a horse got zebra stripes on his skin. Translations can also be ambiguous, for example tree↔building labels on Cityscapes. The cycle loss enforces invertibility, not semantic correctness: Chu, Zhmoginov and Sandler (2017) showed that CycleGAN can hide the information needed to rebuild $x$ as a near-invisible, high-frequency signal in $G(x)$.

```python
import random, torch, torch.nn as nn, torch.nn.functional as F

class ResBlock(nn.Module):
    def __init__(self, c):
        super().__init__()
        self.body = nn.Sequential(nn.ReflectionPad2d(1), nn.Conv2d(c, c, 3), nn.InstanceNorm2d(c), nn.ReLU(True),
                                  nn.ReflectionPad2d(1), nn.Conv2d(c, c, 3), nn.InstanceNorm2d(c))
    def forward(self, x):
        return x + self.body(x)

def resnet_generator(n_blocks=9):        # c7s1-64, d128, d256, 9×R256, u128, u64, c7s1-3
    L = [nn.ReflectionPad2d(3), nn.Conv2d(3, 64, 7), nn.InstanceNorm2d(64), nn.ReLU(True)]
    for c in (64, 128):
        L += [nn.Conv2d(c, 2 * c, 3, 2, 1), nn.InstanceNorm2d(2 * c), nn.ReLU(True)]
    L += [ResBlock(256) for _ in range(n_blocks)]
    for c in (256, 128):
        L += [nn.ConvTranspose2d(c, c // 2, 3, 2, 1, output_padding=1), nn.InstanceNorm2d(c // 2), nn.ReLU(True)]
    L += [nn.ReflectionPad2d(3), nn.Conv2d(64, 3, 7), nn.Tanh()]
    return nn.Sequential(*L)

def patchgan():                           # 70×70 PatchGAN C64-C128-C256-C512, one score per patch
    L, c = [nn.Conv2d(3, 64, 4, 2, 1), nn.LeakyReLU(0.2, True)], 64
    for co, s in ((128, 2), (256, 2), (512, 1)):
        L += [nn.Conv2d(c, co, 4, s, 1), nn.InstanceNorm2d(co), nn.LeakyReLU(0.2, True)]
        c = co
    return nn.Sequential(*L, nn.Conv2d(512, 1, 4, 1, 1))   # no sigmoid: LSGAN regresses the raw output

class ImagePool:                          # history of 50 generated images for the discriminator
    def __init__(self, size=50):
        self.size, self.images = size, []
    def query(self, img):
        if len(self.images) < self.size:
            self.images.append(img.detach()); return img.detach()
        if random.random() < 0.5:         # half the time: show an old fake, store the new one
            i = random.randrange(self.size)
            old, self.images[i] = self.images[i], img.detach()
            return old
        return img.detach()

G, F_, D_X, D_Y = resnet_generator(), resnet_generator(), patchgan(), patchgan()   # G: X→Y, F: Y→X
opt_g = torch.optim.Adam(list(G.parameters()) + list(F_.parameters()), lr=2e-4, betas=(0.5, 0.999))
opt_d = torch.optim.Adam(list(D_X.parameters()) + list(D_Y.parameters()), lr=2e-4, betas=(0.5, 0.999))
pool_x, pool_y, lam, lam_idt = ImagePool(), ImagePool(), 10.0, 0.5
mse = nn.MSELoss()
gan = lambda d, real: mse(d, torch.full_like(d, 1.0 if real else 0.0))

x = torch.rand(1, 3, 256, 256) * 2 - 1    # a horse (batch size 1, as in the paper)
y = torch.rand(1, 3, 256, 256) * 2 - 1    # an unrelated zebra: no pairing anywhere

# generators: fool both discriminators, close both cycles, and (optionally) leave targets unchanged
fake_y, fake_x = G(x), F_(y)
loss_gan = gan(D_Y(fake_y), True) + gan(D_X(fake_x), True)
loss_cyc = F.l1_loss(F_(fake_y), x) + F.l1_loss(G(fake_x), y)          # ‖F(G(x))−x‖₁ + ‖G(F(y))−y‖₁
loss_idt = F.l1_loss(G(y), y) + F.l1_loss(F_(x), x)                     # ‖G(y)−y‖₁ + ‖F(x)−x‖₁
loss_g = loss_gan + lam * loss_cyc + lam_idt * lam * loss_idt
opt_g.zero_grad(); loss_g.backward(); opt_g.step()

# discriminators: LSGAN, real → 1, (pooled) fake → 0, halved
loss_d = 0.5 * (gan(D_Y(y), True) + gan(D_Y(pool_y.query(fake_y)), False)) \
       + 0.5 * (gan(D_X(x), True) + gan(D_X(pool_x.query(fake_x)), False))
opt_d.zero_grad(); loss_d.backward(); opt_d.step()

print(fake_y.shape, D_Y(fake_y).shape)    # torch.Size([1, 3, 256, 256]) torch.Size([1, 1, 30, 30])
print(f"{sum(p.numel() for p in G.parameters()) / 1e6:.2f}M per generator, "
      f"{sum(p.numel() for p in D_Y.parameters()) / 1e6:.2f}M per discriminator")  # 11.38M, 2.76M
```

CycleGAN: ResNet generators, 70×70 PatchGANs, image pool, and one step of LSGAN + 10·cycle + 5·identity (run with PyTorch 2 on CPU)

> **Cycle consistency is not correctness:** $\mathcal L_{cyc}=0$ says $F$ undoes $G$, not that $G$ is the translation you wanted. Any invertible map that the discriminator accepts satisfies it. Examples are a reversible colour shift (the reason for the identity loss) or a hidden high-frequency code that stores $x$ inside $G(x)$. In the figure, every rotation $a=b$ has zero cycle loss and zero adversarial loss. Only the identity term singles out $a=b=0$.

> **Invertibility rules out collapse:** An adversarial loss constrains only the *distribution* of $G(X)$. Mapping many inputs to one output is cheap for it and fatal for the cycle, because $F$ would have to send one image back to many. The fold $\theta\mapsto2\theta$ in the figure is that case on a ring: it matches the target distribution perfectly, but it is two-to-one, and no $F$ can bring the cycle loss below $3.199$.

> **Backward is not optional:** Both cycle directions matter. With only $F(G(x))\approx x$, $F$ can learn to decode anything $G$ produces, while $G(F(y))$ for real $y$ is unconstrained. The paper's ablation found single-direction variants often unstable and prone to collapse. Implement both, and note that one step needs four generator passes: $G(x)$, $F(G(x))$, $F(y)$ and $G(F(y))$, plus two more with the identity loss.

**Strengths**

- Needs only two unpaired image collections
- Strong for colour, texture and style changes (season transfer, painting↔photo, horse↔zebra)
- Simple, reusable recipe: two GANs plus an L1 cycle term
- Cycle consistency rules out many-to-one mappings that a pure adversarial loss allows

**Limitations**

- Twice the networks of Pix2Pix and six generator passes per step with identity loss
- Fails on geometric or shape changes (dog↔cat), where it degenerates into minimal edits
- Clearly below paired training when pairs exist (FCN per-pixel accuracy 0.52 vs 0.71)
- Deterministic one-to-one mapping; can hide information steganographically instead of translating

**Question:** Why can't CycleGAN rely on the two adversarial losses alone?

- A. Adversarial losses cannot be computed without paired data
- B. They only make G(X) match Y as a distribution, so any rearrangement of outputs, including many-to-one collapse, is equally good
- C. PatchGANs cannot judge 256×256 images
- D. LSGAN losses have no gradient for G

**Answer:** B. They only make G(X) match Y as a distribution, so any rearrangement of outputs, including many-to-one collapse, is equally good — A distribution-matching loss is blind to which output each input gets. Cycle consistency requires $F(G(x))\approx x$, so the mapping must keep each input's information, and it ties every output to its own input.

**Question:** On painting→photo, G turns a daytime Monet into a convincing sunset photo, and F turns it back perfectly. Which term discourages this?

- A. The forward cycle loss
- B. The backward cycle loss
- C. The identity loss ‖G(y) − y‖₁ + ‖F(x) − x‖₁
- D. The history buffer

**Answer:** C. The identity loss ‖G(y) − y‖₁ + ‖F(x) − x‖₁ — A reversible colour change costs nothing under either cycle loss, and a sunset photo is a real-looking photo. The identity loss asks $G$ to leave real photos unchanged (and $F$ real paintings), which penalises gratuitous tint changes. The paper weights it $0.5\lambda$.

**Key points**

- CycleGAN objective: $\mathcal L_{GAN}(G,D_Y)+\mathcal L_{GAN}(F,D_X)+\lambda\big(\|F(G(x))-x\|_1+\|G(F(y))-y\|_1\big)$ with $\lambda=10$, on unpaired sets $X$ and $Y$.
- Training details: LSGAN losses $(D(G(x))-1)^2$, a history buffer of 50 generated images, an identity loss $\|G(y)-y\|_1+\|F(x)-x\|_1$ weighted $0.5\lambda$, ResNet generators and $70\times70$ PatchGANs.
- Adversarial losses match distributions only; cycle consistency makes the mapping invertible, which rules out many-to-one collapse.
- Works for colour and texture, fails for geometric changes, and stays below paired Pix2Pix when pairs exist.
- Zero cycle loss does not imply a correct translation: reversible tints and hidden steganographic codes also satisfy it.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
