Gabriel Poesia

Generative Adversarial Nets (@ NeurIPS 2014)

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio


GANs are a class of generative models that are trained without directly optimizing likelihoods $p(x)$ of samples $x$ from the training set. Instead, they are based on a min-max objective involving two models trained simultaneously: a generator $G$, which tries to generate samples that look like those in $x$, and a discriminator $D$, which tries to tell apart samples generated from $G$ from those drawn from the training set.

The original GAN formulation optimizes:

\(\min_\theta \max_\phi V(D_\phi, G_\theta) = \mathbb{E}_{x \sim p(x)} \left[ \log D_\phi(x) \right] + \mathbb{E}_{x \sim q(z)} \left[ \log (1 - D_\phi(G_\theta(z))) \right] \)

where $q(z)$ is any simple prior distribution for the latent variables (e.g., uniform between -1 and 1).

The paper shows that, even without directly referring to $p(x)$, this objective corresponds to a Jensen-Shannon Divergence to the true distribution $p(x)$. Thus, the global optimum happens when the distribution defined by the generator exactly matches the true distribution $p(x)$.