# Understanding Terminology in Goodfellow's paper on GANS

I am trying to understand Ian Goodfellow et al's paper Generative Adversarial Nets here.

In section 3 the author's write:

The adversarial modeling framework is most straightforward to apply when the models are both multilayer perceptrons. To learn the generator’s distribution $$p_g$$ over data $$x$$, we define a prior on input noise variables $$p_z(z)$$, then represent a mapping to data space as $$G(z;$$ $$\theta_g)$$, where $$G$$ is a differentiable function represented by a multilayer perceptron with parameters $$\theta_g$$.

I have a few questions.

1) From what I understand, the generator is a multi-layer perceptron input with a random vector sampled from a predefined latent space, (where wikipedia gives the example of a multivariate normal distribution). What does it mean to learn the generator's distribution $$p_g$$ over data $$x$$?

2) What does "we define a prior on input noise variables $$p_z(z)$$" mean? I understand that in Bayesian statistical inference, that a prior probability distribution, called a prior for short, gives a probability distribution on outcomes before some evidence is collected.

Any insights appreciated.

Question 1

1) From what I understand, the generator is a multi-layer perceptron input with a random vector sampled from a predefined latent space, (where wikipedia gives the example of a multivariate normal distribution). What does it mean to learn the generator's distribution $$p_g$$ over data x?

Generally, supervised learning means to learn a distribution $$p_{model}(y\mid x)$$ where $$x$$ are your examples and $$y$$ your labels. However, in unsupervised learning it is not about labels but instead you are interested in unconditional probabilities and train a model to learn $$p_{model}(x)$$ (e.g. see p.485/486 in "The Elements of Statistical Learning" by Hastie et al). When applying GANs this $$model$$ is the generator $$g$$, i.e. you are learning $$p_{g}(x)$$. Accordingly, in the original paper the authors state:

The generator $$G$$ implicitly defines a probability distribution $$p_g$$ as the distribution of the samples $$G(z)$$ obtained when $$z ∼ p_z$$.

Question 2

2) What does "we define a prior on input noise variables $$p_z(z)$$" mean? I understand that in Bayesian statistical inference, that a prior probability distribution, called a prior for short, gives a probability distribution on outcomes before some evidence is collected.

Prior means we are talking unconditional probabilities, i.e. $$p(z)$$ is not conditioned on anything related to the task and examples on hand. While for example, $$p_d(y\mid x)$$ (the distribution learned by the Discriminator) is conditioned on $$x$$. Often $$z$$ is sampled from the standard normal distribution $$\mathcal{N}(0,1)$$.

In "NIPS 2016 Tutorial: Generative Adversarial Networks" the authors provide a good summary of the overall concept of $$G$$:

The generator is simply a differentiable function $$G$$. When $$z$$ is sampled from some simple prior distribution, $$G(z)$$ yields a sample of $$x$$ drawn from $$p_{model}$$. Typically, a deep neural network is used to represent $$G$$.

• When the authors write : " To learn the generator’s distribution $p_g(x)$", do they mean for any set of parameters or for a fixed set of parameters? Because this seems if it is the latter, that we are learning a family of probability distributions for each $\theta_g$? – IntegrateThis Mar 17 '20 at 22:36
• @IntegrateThis $p_g(x)$ does not depend on $\theta$. Only $G$ does. So I'd say you are learning $p_g(x)$ by training $G(z_g;\theta_g)$ so $G(z_g;\theta_g)$ produces samples drawn from $p_g(x)$. – Sammy Mar 18 '20 at 8:45