what is meant by minimizing and maximizing in GANs?

Question

It is a subtle change that involves the generator maximizing the log of the discriminator probabilities for generated images instead of minimizing the log of the inverted discriminator probabilities for generated images.

This is a very tricky and confusing line to understand GANs loss functions, it is taken from machine learning mastery blog. My question is about the terms minimizing the log of the inverted discriminator probabilities, isn't the subtraction of probabilities from 1 give you same thing (referring to the formula of this part of loss function). please clarify it with some kind of analogy or example of some sort.

Answer 1

This basically boils down to the Vanishing Gradient Problem. It is a well known issue that vanishing gradients limit the ability of deep neural nets to learn. And playing the min-max-game for GANs can lead to this issue for G as well:

When you start to train a GAN G will not produce very good results, i.e. D can easily classify these as fake or real with a probability of them being real approaching $0$. Accordingly, the expression $\log(1-D(G(z))$ converges to $0$, too, and results in a vanishing gradient for G, i.e. the generator cannot really learn when it tries to minimize this.

In contrast, $\log D(G(z))$ does not converge to $0$ when $D(G(z))$ is very small (since $\lim \limits_{x \to 0} \log x=-\infty$). Therefore, G does have a gradient to learn from.

Also see the original paper where the authors write:

Early in learning, when G is poor, D can reject samples with high confidence because they are clearly different from the training data. In this case, $\log(1 - D(G(z)))$ saturates. Rather than training G to minimize $\log(1 - D(G(z)))$ we can train G to maximize $\log D(G(z))$. This objective function results in the same fixed point of the dynamics of G and D but provides much stronger gradients early in learning.

As you can see, this is a purely practical consideration in order ease learning for G and not about theoretical equivalence of expressions.