I know that in GANs model, there is min-max game between generator and discriminator which discriminator tries to maximize the loss function and the goal of generator is to minimize it. But why we write the loss function as min-max problem and not max-min? As I know, the loss function is not convex, so there is difference between these approaches.
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
You can use either a min-max or a max-min game, it doesn't matter. Look, this is the "usual" loss function (i. e. if you use the original loss proposed by Goodfellow et al. in their paper on GANs, though I use a slightly different notation):
$$\min_{G}\max_{D}\mathcal L_{\text{GAN}}\left(D, G \right) = \mathbb E_{x \sim p_{\text{data}\left( x\right) }}\left[ \log \left\{ D\left( x \right) \right\} \right] + \mathbb E_{z\sim p_{z}\left( z\right)}\left[ \log\left\{ 1 - D\left( G\left( z\right) \right) \right\}\right].$$
Now, you could also write this as:
$$\max_{G}\min_{D}\mathcal L_{\text{GAN}}\left(D, G \right) = \mathbb E_{x \sim p_{\text{data}\left( x\right) }}\left[ \log \left\{ 1 - D\left( x \right) \right\} \right] + \mathbb E_{z\sim p_{z}\left( z\right)}\left[ \log\left\{ D\left( G\left( z\right) \right) \right\}\right].$$
I hope that clarifies your doubt.
$\begingroup$
$\endgroup$
I also had a similar problem, but I was rather interested in why we maximize with D in inner loop rather than outer loop. Other than that discriminator needs to be good for generator to be any good, there is a nice perspective on mode collapse by Ian Goodfellow in the 15th minute here: https://youtu.be/9JpdAg6uMXs
