# Adversarial loss

> A learned loss: a discriminator is trained to tell real from generated samples, and the generator is trained to fool it. With an optimal discriminator the game minimizes Jensen–Shannon divergence; variants change the divergence and the gradients.

Source: https://www.mlfoundry.in/learn/adversarial-loss  
Section: Deep Learning › Loss Functions  
Level: advanced · 14 min read · updated 2026-10-11

**Adversarial loss:** is the training signal of a generative adversarial network (GAN). A discriminator $D$ is trained to separate real samples $x\sim p_{\text{data}}$ from generated ones $G(z)$, $z\sim p(z)$. The generator $G$ is trained to make $D$ wrong. Neither network has a fixed target: $D$ **is** the generator's loss function, and it is learned at the same time.

Pixel and feature losses compare an output with one specific target. Many tasks have no single right answer: there are countless plausible faces, textures or completions of an image. An adversarial loss asks a different question: does this output look like it came from the data distribution? That lets a model commit to one sharp sample instead of averaging the possibilities into blur, which is why adversarial terms are added to reconstruction and perceptual losses in super-resolution, image translation and the autoencoders behind latent diffusion.

Goodfellow et al. (2014) introduced the minimax game and its analysis. Radford, Metz & Chintala's DCGAN (2016) made it trainable with convolutional networks. Training was notoriously unstable, and much of the following work changed the loss. Mao et al. (2017) proposed least squares (LSGAN). Arjovsky, Chintala & Bottou (2017) used the Wasserstein distance (WGAN) and Gulrajani et al. (2017) its gradient penalty (WGAN-GP). Lim & Ye (2017) introduced the hinge loss that spectral normalization (Miyato et al., 2018) and BigGAN (Brock et al., 2019) adopted. Nowozin, Cseke & Tomioka (2016) showed that a family of such games minimizes any chosen $f$-divergence.

## The minimax game

$$
\min_G\max_D\ V(D,G)=\mathbb E_{x\sim p_{\text{data}}}\big[\log D(x)\big]+\mathbb E_{z\sim p(z)}\big[\log\big(1-D(G(z))\big)\big]
$$

- $D(x)\in(0,1)$: the discriminator's probability that $x$ is real; in code $D(x)=\sigma(a(x))$ with a logit $a$
- $G(z)$: a generated sample; its distribution is $p_g$
- $\max_D$: $D$ minimizes binary cross-entropy with label 1 for real and 0 for fake, which is the same as maximizing $V$

**The optimal discriminator.** For a fixed $G$, write $V=\int p_{\text{data}}(x)\log D(x)+p_g(x)\log(1-D(x))\,dx$ and maximize pointwise. The function $a\log y+b\log(1-y)$ peaks at $y=a/(a+b)$, so

$$
D^*(x)=\frac{p_{\text{data}}(x)}{p_{\text{data}}(x)+p_g(x)},\qquad V(D^*,G)=-\log4+2\,\mathrm{JS}\big(p_{\text{data}}\,\|\,p_g\big)
$$

- $\mathrm{JS}(p\|q)$: Jensen–Shannon divergence $\tfrac12\mathrm{KL}(p\|m)+\tfrac12\mathrm{KL}(q\|m)$ with $m=\tfrac12(p+q)$; it lies in $[0,\log2]$
- $-\log4$: the value at the global optimum $p_g=p_{\text{data}}$, where $D^*\equiv\tfrac12$ everywhere

So with a perfect discriminator, training the generator minimizes the JS divergence to the data, and the unique global minimum is $p_g=p_{\text{data}}$. The logit of $D^*$ is the log density ratio $\log p_{\text{data}}(x)-\log p_g(x)$. That is the deeper reading of every GAN: the discriminator estimates a density ratio, and the generator moves its samples toward regions where the ratio says the data is denser.

## Saturation and the non-saturating loss

Write the generator's loss on one fake as a function of the discriminator's logit $a=a(G(z))$. The minimax loss is $\log(1-\sigma(a))=-\operatorname{softplus}(a)$, with gradient $-\sigma(a)$. Early in training the fakes are bad, $D$ rejects them confidently, $a\ll0$, and the gradient vanishes. Goodfellow et al. therefore suggested a different generator objective with the same fixed point: maximize $\log D(G(z))$.

$$
\mathcal L_G^{\text{minimax}}=\mathbb E_z\big[\log(1-\sigma(a))\big],\ \ \frac{\partial}{\partial a}=-\sigma(a)\qquad\quad \mathcal L_G^{\text{NS}}=\mathbb E_z\big[-\log\sigma(a)\big],\ \ \frac{\partial}{\partial a}=\sigma(a)-1
$$

- $a\to-\infty$: minimax gradient $\to0$ (saturates); non-saturating gradient $\to-1$ (full strength)
- $\mathcal L_G^{\text{NS}}$: binary cross-entropy of the fakes **with the real label 1**; in code `BCEWithLogits(D(G(z)), ones)`

The fix has a cost. With $D=D^*$, Arjovsky & Bottou (2017) showed that the non-saturating gradient is the gradient of $\mathrm{KL}(p_g\|p_{\text{data}})-2\,\mathrm{JS}(p_{\text{data}}\|p_g)$. The reverse KL heavily penalizes fakes where there is no data but barely penalizes missing modes, one explanation for **mode collapse**. The figure checks the identity: for $p_g=\mathcal N(\mu,1)$, $\mathrm{KL}(p_g\|p_{\text{data}})=\mu^2/2$, so the non-saturating gradient in $\mu$ is $\mu-2\,\partial\mathrm{JS}/\partial\mu$.

> Figure: Data N(0, 1) against a generator N(μ, σ²), with the optimal discriminator D* (orange). At μ = 4 the supports barely overlap: JS = 0.6327 nats, close to its ceiling ln 2 = 0.6931, and V(D*, G) = −0.1209. D*'s logit at the centre of the fakes is −8, so D(G(z)) = 3.4e-4. There the minimax generator's logit gradient is −3.4e-4 while the non-saturating one is −0.9997. Through D*, the gradient in μ is 0.137 for minimax (exactly 2·∂JS/∂μ) and 3.863 for non-saturating. At μ = 6 the minimax gradient falls to 0.012 while W₁ = 6 still has slope 1. Choose LSGAN, WGAN or hinge to see their generator losses: LSGAN's gradient a − 1 grows linearly, WGAN's and hinge's are a constant −1.

## Other adversarial losses

When $p_g$ and $p_{\text{data}}$ barely overlap, as for distributions on thin manifolds in pixel space, JS is stuck near $\log2$ and the optimal discriminator is nearly perfect, so its gradient carries little information about **which way** to move. The common variants change the divergence, the shape of the loss, or both. Here $a_r=D(x)$ and $a_f=D(G(z))$ are the raw discriminator outputs (logits or critic scores):

Discriminator and generator losses (all minimized). Expectations over real and fake batches are implicit.

| loss | discriminator / critic | generator | what it minimizes |
| --- | --- | --- | --- |
| minimax (Goodfellow 2014) | $\operatorname{sp}(-a_r)+\operatorname{sp}(a_f)$ | $-\operatorname{sp}(a_f)$ | $2\,\mathrm{JS}-\log4$ |
| non-saturating (NS) | $\operatorname{sp}(-a_r)+\operatorname{sp}(a_f)$ | $\operatorname{sp}(-a_f)$ | same fixed point; gradient of $\mathrm{KL}(p_g\\|p_d)-2\mathrm{JS}$ at $D^*$ |
| LSGAN (Mao 2017) | $\tfrac12(a_r-b)^2+\tfrac12(a_f-a)^2$ | $\tfrac12(a_f-c)^2$ | Pearson $\chi^2$ when $b-c=1,\ b-a=2$; $a,b,c=0,1,1$ is common |
| WGAN (Arjovsky 2017) | $a_f-a_r$, with $D$ 1-Lipschitz | $-a_f$ | Wasserstein-1 distance $W_1$ |
| hinge (Lim & Ye 2017) | $\max(0,1-a_r)+\max(0,1+a_f)$ | $-a_f$ | a margin version of the game; used by SNGAN, SAGAN, BigGAN |

$\operatorname{sp}(t)=\log(1+e^t)$ is softplus, so $\operatorname{sp}(-a)=-\log\sigma(a)$: the first two rows are just BCE on logits. **LSGAN** uses a linear output and squared error, so a fake that is on the correct side of the decision boundary but far from it still gets a gradient proportional to its distance. **WGAN** replaces the classifier with a critic and estimates $W_1(p_{\text{data}},p_g)=\sup_{\|f\|_L\le1}\mathbb E_{p_{\text{data}}}f-\mathbb E_{p_g}f$ (Kantorovich–Rubinstein duality). $W_1$ keeps growing with the distance between the distributions even when their supports are disjoint, so its gradient does not vanish. The Lipschitz constraint was first imposed by weight clipping, then by a gradient penalty $\lambda\,\mathbb E_{\hat x}(\|\nabla_{\hat x}D(\hat x)\|_2-1)^2$ on random interpolates (WGAN-GP, $\lambda=10$), or by spectral normalization of each layer. **Hinge** stops pushing on samples once they are classified with margin 1, which in practice is stable and works well with spectral normalization.

## Worked example: one real and one fake

The discriminator outputs $a_r=2$ on a real image and $a_f=-1$ on a fake, so $D(x)=\sigma(2)=0.8808$ and $D(G(z))=\sigma(-1)=0.2689$ (all values checked in Node):

| loss | D loss | G loss | ∂(G loss)/∂a_f |
| --- | --- | --- | --- |
| minimax | $0.1269+0.3133=0.4402$ | $\log(1-0.2689)=-0.3133$ | $-0.2689$ |
| non-saturating | $0.4402$ | $-\log0.2689=1.3133$ | $-0.7311$ |
| LSGAN (0, 1, 1) | $\tfrac12(1)^2+\tfrac12(-1)^2=1.0$ | $\tfrac12(-2)^2=2.0$ | $-2$ |
| WGAN | $-1-2=-3$ | $1$ | $-1$ |
| hinge | $\max(0,-1)+\max(0,0)=0$ | $1$ | $-1$ |

1. The discriminator's BCE gradients are $\sigma(a_r)-1=-0.1192$ (push the real logit up) and $\sigma(a_f)=0.2689$ (push the fake logit down).
2. Every generator loss has a negative gradient in $a_f$: all of them push the fake toward "real". They differ in strength. Non-saturating is $2.7\times$ stronger than minimax here, and the gap grows without bound as $a_f\to-\infty$.
3. The hinge discriminator loss is already 0: the real is past margin $+1$ and the fake sits exactly at $-1$. The discriminator stops changing on this pair, while the generator still receives gradient $-1$.
4. WGAN's critic values are unbounded, so its loss can be negative. Only differences between real and fake scores are meaningful, and $-(\text{D loss})$ estimates $W_1$ when the critic is Lipschitz.

```python
import torch, torch.nn.functional as F

bce = F.binary_cross_entropy_with_logits

def d_loss_ns(real_logits, fake_logits):              # standard (minimax / NS) discriminator
    return (bce(real_logits, torch.ones_like(real_logits))
            + bce(fake_logits, torch.zeros_like(fake_logits)))

def g_loss_ns(fake_logits):                            # non-saturating: fakes labelled real
    return bce(fake_logits, torch.ones_like(fake_logits))      # = softplus(-a).mean()

def d_loss_hinge(real, fake):
    return F.relu(1 - real).mean() + F.relu(1 + fake).mean()

def g_loss_hinge(fake):                                # also the WGAN generator loss
    return -fake.mean()

def gradient_penalty(D, real, fake, lam=10.0):         # WGAN-GP, images (B, C, H, W)
    eps = torch.rand(real.size(0), 1, 1, 1, device=real.device)
    x_hat = (eps * real + (1 - eps) * fake).requires_grad_(True)
    (g,) = torch.autograd.grad(D(x_hat).sum(), x_hat, create_graph=True)
    return lam * ((g.flatten(1).norm(dim=1) - 1) ** 2).mean()

d_loss_ns(torch.tensor([2.0]), torch.tensor([-1.0]))   # 0.4402
g_loss_ns(torch.tensor([-1.0]))                        # 1.3133

def train_step(G, D, opt_g, opt_d, real, z_dim=128):
    z = torch.randn(real.size(0), z_dim, device=real.device)
    fake = G(z)
    opt_d.zero_grad()
    d_loss_ns(D(real), D(fake.detach())).backward()    # detach: no generator update here
    opt_d.step()
    opt_g.zero_grad()
    g_loss_ns(D(fake)).backward()                      # gradient flows through D into G
    opt_g.step()
```

GAN losses on logits, and one training step

> **Detach, labels and sigmoids:** Three bugs account for most broken GAN losses. (1) Forgetting `fake.detach()` in the discriminator step, which also updates $G$ to help $D$. (2) Training the generator with label 0 and a minus sign (the saturating loss) instead of label 1. (3) Putting `nn.Sigmoid` at the end of $D$ and then calling `BCEWithLogitsLoss`, or using `BCELoss` on raw scores. Keep $D$'s output a raw logit. For WGAN and hinge losses there is no sigmoid at all.

> **The loss value is not a progress bar:** In a classifier, the loss going down means learning. In a GAN, $D$'s and $G$'s losses are opponents. A discriminator loss near 0 usually means $D$ is winning and $G$'s signal is failing, not that training is done. Judge progress by samples and by sample-quality metrics (FID, precision/recall), not by the curves. The exception is the WGAN critic loss, which estimates $W_1$ and does correlate with sample quality.

> **Adversarial loss as one term among several:** Outside pure GANs the adversarial loss is usually an auxiliary term. Pix2pix (Isola et al., 2017) adds an L1 reconstruction term to a patch-based conditional GAN loss. SRGAN combines it with a VGG feature loss, and the VQGAN and latent-diffusion autoencoders combine L1, LPIPS and a patch discriminator. The reconstruction terms pin down the content, and the adversarial term removes the blur that the reconstruction terms alone would produce.

**Strengths**

- Learns a loss for "looks real" where no hand-written distance exists
- Allows sharp, committed outputs instead of the blurry mean of pixel losses
- Fast sampling: one generator forward pass per sample
- Flexible: conditional, patch-based and auxiliary versions plug into almost any image model

**Limitations**

- A two-player game, not a minimization: oscillation, divergence and sensitivity to hyperparameters
- Mode collapse: the generator can cover only part of the data distribution
- Vanishing or uninformative gradients when real and fake supports do not overlap (minimax/JS)
- Loss curves say little about sample quality; no likelihood to compare models

**Question:** Early in training, the discriminator rejects every fake with D(G(z)) ≈ 0.001. Which generator loss still gives a useful gradient?

- A. log(1 − D(G(z))), the original minimax loss
- B. −log D(G(z)), the non-saturating loss
- C. Both give the same gradient
- D. Neither: the generator must wait for D to weaken

**Answer:** B. −log D(G(z)), the non-saturating loss — With logit $a=\log(0.001/0.999)\approx-6.9$, the minimax gradient is $-\sigma(a)\approx-0.001$, essentially zero, while the non-saturating gradient is $\sigma(a)-1\approx-0.999$. Both losses have the same fixed point, but only the second one gives a usable signal when the generator is losing badly.

**Question:** For a fixed generator, what does the optimal discriminator's logit equal?

- A. $\log p_{\text{data}}(x)$
- B. $\log p_{\text{data}}(x)-\log p_g(x)$
- C. $\mathrm{JS}(p_{\text{data}}\|p_g)$
- D. $0$ everywhere

**Answer:** B. $\log p_{\text{data}}(x)-\log p_g(x)$ — $D^*=p_{\text{data}}/(p_{\text{data}}+p_g)$, so $\operatorname{logit}D^*=\log\frac{D^*}{1-D^*}=\log\frac{p_{\text{data}}}{p_g}$. It is 0 everywhere only at the optimum $p_g=p_{\text{data}}$. The discriminator is a density-ratio estimator.

**Key points**

- GAN: $\min_G\max_D\ \mathbb E\log D(x)+\mathbb E\log(1-D(G(z)))$; $D$ is trained with BCE (real 1, fake 0).
- $D^*=p_{\text{data}}/(p_{\text{data}}+p_g)$ and $V(D^*,G)=2\,\mathrm{JS}-\log4$; $D^*$'s logit is the log density ratio.
- The minimax generator loss saturates when $D$ is confident. Use the non-saturating $-\log D(G(z))$ (BCE with label 1 on the fakes).
- LSGAN (squared error), WGAN ($W_1$ with a Lipschitz critic) and hinge losses change the divergence and keep gradients alive.
- Implement on logits with `binary_cross_entropy_with_logits`, detach fakes in the D step, and judge progress by samples, not loss values.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
