2
$\begingroup$

Since the aim of a Discriminator is to output 1 for real data and 0 for fake data, hence, the aim is to increase the likelihood of true data vs. fake one. In addition, since maximizing the likelihood is equivalent to minimizing the log-likelihood, why are we updating the discriminator by ascending its stochastic gradient as mentioned in Algorithm 1 in https://arxiv.org/pdf/1406.2661.pdf. Shouldn't we update the discriminator by descending its stochastic gradient?

Any help is much appreciated!

$\endgroup$
4
$\begingroup$

In algorithm 1 of the original GAN article (https://arxiv.org/pdf/1406.2661.pdf), the discriminator is said to be updated by "ascending its stochastic gradient". This is referring to equation 1: $$ \min_G \max_D V(D, G)= \mathbb{E}_{x\sim p_{data}(x)}[\log D(x)] + \mathbb{E}_{z\sim p_z(z)}[\log(1 - D(G(z)))] $$

When we want to minimize something, we do grandient descent. When we want to maximize something, we do gradient ascent. In this context, we want to maximize $V(D, G)$ with respect to the discriminator $D$, that is, the $\max_D V(D, G)$ part from equation 1.

I recommend you have a look at NIPS 2016 GAN tutorial video and text. They are very enlightening.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Yes you are right, but from a probabilistic point of view, should the process of maximizing the likelihood be equivalent to minimizing the loss, which is log-likelihood? Therefore, I thought that we should minimize the loss, that is the log(D(x)), And hence the probability/likelihood will increase. Correct me if I'm wrong $\endgroup$ – I. A Aug 8 '17 at 16:27
  • $\begingroup$ Yes, you minimize the following loss: $\mathscr{L}_d = -\dfrac{1}{2}\mathbb{E}_{x \sim p_{data}} \log D(x) -\dfrac{1}{2}\mathbb{E}_z \log (1 - D(G(z)))$ $\endgroup$ – ncasas Aug 8 '17 at 16:31
  • $\begingroup$ There is a typo in the equation. The second item in the sum should be $\mathbb{E}_{z\sim p_z(z)}[\log(1 - D(G(z)))]$. $z$ is passed to the generator and not $x$ $\endgroup$ – Joseph Mar 29 '18 at 17:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.