Adversarial loss

A learned loss: a discriminator is trained to tell real from generated samples, and the generator is trained to fool it. With an optimal discriminator the game minimizes Jensen–Shannon divergence; variants change the divergence and the gradients.

14 min read1079 wordsadvanced0 viewsupdated 2026-10-11

Definition

Adversarial loss is the training signal of a generative adversarial network (GAN). A discriminator DD is trained to separate real samples x∼pdatax\sim p_{\text{data}} from generated ones G(z)G(z), z∼p(z)z\sim p(z). The generator GG is trained to make DD wrong. Neither network has a fixed target: DD is the generator's loss function, and it is learned at the same time.

Pixel and feature losses compare an output with one specific target. Many tasks have no single right answer: there are countless plausible faces, textures or completions of an image. An adversarial loss asks a different question: does this output look like it came from the data distribution? That lets a model commit to one sharp sample instead of averaging the possibilities into blur, which is why adversarial terms are added to reconstruction and perceptual losses in super-resolution, image translation and the autoencoders behind latent diffusion.

Goodfellow et al. (2014) introduced the minimax game and its analysis. Radford, Metz & Chintala's DCGAN (2016) made it trainable with convolutional networks. Training was notoriously unstable, and much of the following work changed the loss. Mao et al. (2017) proposed least squares (LSGAN). Arjovsky, Chintala & Bottou (2017) used the Wasserstein distance (WGAN) and Gulrajani et al. (2017) its gradient penalty (WGAN-GP). Lim & Ye (2017) introduced the hinge loss that spectral normalization (Miyato et al., 2018) and BigGAN (Brock et al., 2019) adopted. Nowozin, Cseke & Tomioka (2016) showed that a family of such games minimizes any chosen ff-divergence.

The minimax game

min⁡Gmax⁡D V(D,G)=Ex∼pdata[log⁡D(x)]+Ez∼p(z)[log⁡(1−D(G(z)))]\min_G\max_D\ V(D,G)=\mathbb E_{x\sim p_{\text{data}}}\big[\log D(x)\big]+\mathbb E_{z\sim p(z)}\big[\log\big(1-D(G(z))\big)\big]
D(x)∈(0,1)D(x)\in(0,1)
the discriminator's probability that xx is real; in code D(x)=σ(a(x))D(x)=\sigma(a(x)) with a logit aa
G(z)G(z)
a generated sample; its distribution is pgp_g
max⁡D\max_D
DD minimizes binary cross-entropy with label 1 for real and 0 for fake, which is the same as maximizing VV

The optimal discriminator. For a fixed GG, write V=∫pdata(x)log⁡D(x)+pg(x)log⁡(1−D(x)) dxV=\int p_{\text{data}}(x)\log D(x)+p_g(x)\log(1-D(x))\,dx and maximize pointwise. The function alog⁡y+blog⁡(1−y)a\log y+b\log(1-y) peaks at y=a/(a+b)y=a/(a+b), so

D∗(x)=pdata(x)pdata(x)+pg(x),V(D∗,G)=−log⁡4+2 JS(pdata ∥ pg)D^*(x)=\frac{p_{\text{data}}(x)}{p_{\text{data}}(x)+p_g(x)},\qquad V(D^*,G)=-\log4+2\,\mathrm{JS}\big(p_{\text{data}}\,\|\,p_g\big)
JS(p∥q)\mathrm{JS}(p\|q)
Jensen–Shannon divergence 12KL(p∥m)+12KL(q∥m)\tfrac12\mathrm{KL}(p\|m)+\tfrac12\mathrm{KL}(q\|m) with m=12(p+q)m=\tfrac12(p+q); it lies in [0,log⁡2][0,\log2]
−log⁡4-\log4
the value at the global optimum pg=pdatap_g=p_{\text{data}}, where D∗≡12D^*\equiv\tfrac12 everywhere

So with a perfect discriminator, training the generator minimizes the JS divergence to the data, and the unique global minimum is pg=pdatap_g=p_{\text{data}}. The logit of D∗D^* is the log density ratio log⁡pdata(x)−log⁡pg(x)\log p_{\text{data}}(x)-\log p_g(x). That is the deeper reading of every GAN: the discriminator estimates a density ratio, and the generator moves its samples toward regions where the ratio says the data is denser.

Saturation and the non-saturating loss

Write the generator's loss on one fake as a function of the discriminator's logit a=a(G(z))a=a(G(z)). The minimax loss is log⁡(1−σ(a))=−softplus⁡(a)\log(1-\sigma(a))=-\operatorname{softplus}(a), with gradient −σ(a)-\sigma(a). Early in training the fakes are bad, DD rejects them confidently, a≪0a\ll0, and the gradient vanishes. Goodfellow et al. therefore suggested a different generator objective with the same fixed point: maximize log⁡D(G(z))\log D(G(z)).

LGminimax=Ez[log⁡(1−σ(a))],  ∂∂a=−σ(a)LGNS=Ez[−log⁡σ(a)],  ∂∂a=σ(a)−1\mathcal L_G^{\text{minimax}}=\mathbb E_z\big[\log(1-\sigma(a))\big],\ \ \frac{\partial}{\partial a}=-\sigma(a)\qquad\quad \mathcal L_G^{\text{NS}}=\mathbb E_z\big[-\log\sigma(a)\big],\ \ \frac{\partial}{\partial a}=\sigma(a)-1
a→−∞a\to-\infty
minimax gradient →0\to0 (saturates); non-saturating gradient →−1\to-1 (full strength)
LGNS\mathcal L_G^{\text{NS}}
binary cross-entropy of the fakes with the real label 1; in code BCEWithLogits(D(G(z)), ones)

The fix has a cost. With D=D∗D=D^*, Arjovsky & Bottou (2017) showed that the non-saturating gradient is the gradient of KL(pg∥pdata)−2 JS(pdata∥pg)\mathrm{KL}(p_g\|p_{\text{data}})-2\,\mathrm{JS}(p_{\text{data}}\|p_g). The reverse KL heavily penalizes fakes where there is no data but barely penalizes missing modes, one explanation for mode collapse. The figure checks the identity: for pg=N(μ,1)p_g=\mathcal N(\mu,1), KL(pg∥pdata)=μ2/2\mathrm{KL}(p_g\|p_{\text{data}})=\mu^2/2, so the non-saturating gradient in μ\mu is μ−2 ∂JS/∂μ\mu-2\,\partial\mathrm{JS}/\partial\mu.

p_d = N(0, 1) vs p_g = N(4.0, 1.0²)−8−40481½■ data ■ generator ━ D*(x)JS = 0.6327 nats (max ln 2 = 0.6931)V(D*, G) = 2·JS − ln 4 = −0.1209W₁(p_d, p_g) = 4.000∂JS/∂μ = 0.069 · ∂W₁/∂μ = 1.000∂L_G/∂μ via D*: minimax 0.137 · NS 3.863L_G vs D's output a on a fake−40510−10−6−2024━ L_G(a) ┄ ∂L_G/∂a · a = logitnon-saturating −log D at a = −8.00D = σ(a) = 3.4e−4L_G = 8.000 · ∂L_G/∂a = −1.000a = D*'s logit at x = μ: −8.00full signal while D is sure
fig.Data N(0, 1) against a generator N(μ, σ²), with the optimal discriminator D* (orange). At μ = 4 the supports barely overlap: JS = 0.6327 nats, close to its ceiling ln 2 = 0.6931, and V(D*, G) = −0.1209. D*'s logit at the centre of the fakes is −8, so D(G(z)) = 3.4e-4. There the minimax generator's logit gradient is −3.4e-4 while the non-saturating one is −0.9997. Through D*, the gradient in μ is 0.137 for minimax (exactly 2·∂JS/∂μ) and 3.863 for non-saturating. At μ = 6 the minimax gradient falls to 0.012 while W₁ = 6 still has slope 1. Choose LSGAN, WGAN or hinge to see their generator losses: LSGAN's gradient a − 1 grows linearly, WGAN's and hinge's are a constant −1.

Other adversarial losses

When pgp_g and pdatap_{\text{data}} barely overlap, as for distributions on thin manifolds in pixel space, JS is stuck near log⁡2\log2 and the optimal discriminator is nearly perfect, so its gradient carries little information about which way to move. The common variants change the divergence, the shape of the loss, or both. Here ar=D(x)a_r=D(x) and af=D(G(z))a_f=D(G(z)) are the raw discriminator outputs (logits or critic scores):

lossdiscriminator / criticgeneratorwhat it minimizes
minimax (Goodfellow 2014)sp⁡(−ar)+sp⁡(af)\operatorname{sp}(-a_r)+\operatorname{sp}(a_f)−sp⁡(af)-\operatorname{sp}(a_f)2 JS−log⁡42\,\mathrm{JS}-\log4
non-saturating (NS)sp⁡(−ar)+sp⁡(af)\operatorname{sp}(-a_r)+\operatorname{sp}(a_f)sp⁡(−af)\operatorname{sp}(-a_f)same fixed point; gradient of KL(pg∥pd)−2JS\mathrm{KL}(p_g\|p_d)-2\mathrm{JS} at D∗D^*
LSGAN (Mao 2017)12(ar−b)2+12(af−a)2\tfrac12(a_r-b)^2+\tfrac12(a_f-a)^212(af−c)2\tfrac12(a_f-c)^2Pearson χ2\chi^2 when b−c=1, b−a=2b-c=1,\ b-a=2; a,b,c=0,1,1a,b,c=0,1,1 is common
WGAN (Arjovsky 2017)af−ara_f-a_r, with DD 1-Lipschitz−af-a_fWasserstein-1 distance W1W_1
hinge (Lim & Ye 2017)max⁡(0,1−ar)+max⁡(0,1+af)\max(0,1-a_r)+\max(0,1+a_f)−af-a_fa margin version of the game; used by SNGAN, SAGAN, BigGAN
Discriminator and generator losses (all minimized). Expectations over real and fake batches are implicit.

sp⁡(t)=log⁡(1+et)\operatorname{sp}(t)=\log(1+e^t) is softplus, so sp⁡(−a)=−log⁡σ(a)\operatorname{sp}(-a)=-\log\sigma(a): the first two rows are just BCE on logits. LSGAN uses a linear output and squared error, so a fake that is on the correct side of the decision boundary but far from it still gets a gradient proportional to its distance. WGAN replaces the classifier with a critic and estimates W1(pdata,pg)=sup⁡∥f∥L≤1Epdataf−EpgfW_1(p_{\text{data}},p_g)=\sup_{\|f\|_L\le1}\mathbb E_{p_{\text{data}}}f-\mathbb E_{p_g}f (Kantorovich–Rubinstein duality). W1W_1 keeps growing with the distance between the distributions even when their supports are disjoint, so its gradient does not vanish. The Lipschitz constraint was first imposed by weight clipping, then by a gradient penalty λ Ex^(∥∇x^D(x^)∥2−1)2\lambda\,\mathbb E_{\hat x}(\|\nabla_{\hat x}D(\hat x)\|_2-1)^2 on random interpolates (WGAN-GP, λ=10\lambda=10), or by spectral normalization of each layer. Hinge stops pushing on samples once they are classified with margin 1, which in practice is stable and works well with spectral normalization.

Worked example: one real and one fake

The discriminator outputs ar=2a_r=2 on a real image and af=−1a_f=-1 on a fake, so D(x)=σ(2)=0.8808D(x)=\sigma(2)=0.8808 and D(G(z))=σ(−1)=0.2689D(G(z))=\sigma(-1)=0.2689 (all values checked in Node):

lossD lossG loss∂(G loss)/∂a_f
minimax0.1269+0.3133=0.44020.1269+0.3133=0.4402log⁡(1−0.2689)=−0.3133\log(1-0.2689)=-0.3133−0.2689-0.2689
non-saturating0.44020.4402−log⁡0.2689=1.3133-\log0.2689=1.3133−0.7311-0.7311
LSGAN (0, 1, 1)12(1)2+12(−1)2=1.0\tfrac12(1)^2+\tfrac12(-1)^2=1.012(−2)2=2.0\tfrac12(-2)^2=2.0−2-2
WGAN−1−2=−3-1-2=-311−1-1
hingemax⁡(0,−1)+max⁡(0,0)=0\max(0,-1)+\max(0,0)=011−1-1
  1. 1The discriminator's BCE gradients are σ(ar)−1=−0.1192\sigma(a_r)-1=-0.1192 (push the real logit up) and σ(af)=0.2689\sigma(a_f)=0.2689 (push the fake logit down).
  2. 2Every generator loss has a negative gradient in afa_f: all of them push the fake toward "real". They differ in strength. Non-saturating is 2.7×2.7\times stronger than minimax here, and the gap grows without bound as af→−∞a_f\to-\infty.
  3. 3The hinge discriminator loss is already 0: the real is past margin +1+1 and the fake sits exactly at −1-1. The discriminator stops changing on this pair, while the generator still receives gradient −1-1.
  4. 4WGAN's critic values are unbounded, so its loss can be negative. Only differences between real and fake scores are meaningful, and −(D loss)-(\text{D loss}) estimates W1W_1 when the critic is Lipschitz.
python
1import torch, torch.nn.functional as F
2
3bce = F.binary_cross_entropy_with_logits
4
5def d_loss_ns(real_logits, fake_logits): # standard (minimax / NS) discriminator
6 return (bce(real_logits, torch.ones_like(real_logits))
7 + bce(fake_logits, torch.zeros_like(fake_logits)))
8
9def g_loss_ns(fake_logits): # non-saturating: fakes labelled real
10 return bce(fake_logits, torch.ones_like(fake_logits)) # = softplus(-a).mean()
11
12def d_loss_hinge(real, fake):
13 return F.relu(1 - real).mean() + F.relu(1 + fake).mean()
14
15def g_loss_hinge(fake): # also the WGAN generator loss
16 return -fake.mean()
17
18def gradient_penalty(D, real, fake, lam=10.0): # WGAN-GP, images (B, C, H, W)
19 eps = torch.rand(real.size(0), 1, 1, 1, device=real.device)
20 x_hat = (eps * real + (1 - eps) * fake).requires_grad_(True)
21 (g,) = torch.autograd.grad(D(x_hat).sum(), x_hat, create_graph=True)
22 return lam * ((g.flatten(1).norm(dim=1) - 1) ** 2).mean()
23
24d_loss_ns(torch.tensor([2.0]), torch.tensor([-1.0])) # 0.4402
25g_loss_ns(torch.tensor([-1.0])) # 1.3133
26
27def train_step(G, D, opt_g, opt_d, real, z_dim=128):
28 z = torch.randn(real.size(0), z_dim, device=real.device)
29 fake = G(z)
30 opt_d.zero_grad()
31 d_loss_ns(D(real), D(fake.detach())).backward() # detach: no generator update here
32 opt_d.step()
33 opt_g.zero_grad()
34 g_loss_ns(D(fake)).backward() # gradient flows through D into G
35 opt_g.step()
GAN losses on logits, and one training step

Use when

  • +Learns a loss for "looks real" where no hand-written distance exists
  • +Allows sharp, committed outputs instead of the blurry mean of pixel losses
  • +Fast sampling: one generator forward pass per sample
  • +Flexible: conditional, patch-based and auxiliary versions plug into almost any image model

Avoid when

  • −A two-player game, not a minimization: oscillation, divergence and sensitivity to hyperparameters
  • −Mode collapse: the generator can cover only part of the data distribution
  • −Vanishing or uninformative gradients when real and fake supports do not overlap (minimax/JS)
  • −Loss curves say little about sample quality; no likelihood to compare models

Check yourself

Early in training, the discriminator rejects every fake with D(G(z)) ≈ 0.001. Which generator loss still gives a useful gradient?

Check yourself

For a fixed generator, what does the optimal discriminator's logit equal?

Remember

  • ✓GAN: min⁡Gmax⁡D Elog⁡D(x)+Elog⁡(1−D(G(z)))\min_G\max_D\ \mathbb E\log D(x)+\mathbb E\log(1-D(G(z))); DD is trained with BCE (real 1, fake 0).
  • ✓D∗=pdata/(pdata+pg)D^*=p_{\text{data}}/(p_{\text{data}}+p_g) and V(D∗,G)=2 JS−log⁡4V(D^*,G)=2\,\mathrm{JS}-\log4; D∗D^*'s logit is the log density ratio.
  • ✓The minimax generator loss saturates when DD is confident. Use the non-saturating −log⁡D(G(z))-\log D(G(z)) (BCE with label 1 on the fakes).
  • ✓LSGAN (squared error), WGAN (W1W_1 with a Lipschitz critic) and hinge losses change the divergence and keep gradients alive.
  • ✓Implement on logits with binary_cross_entropy_with_logits, detach fakes in the D step, and judge progress by samples, not loss values.
← back to Deep Learning roadmap