In Wasserstein GAN, it's explained that maximizing a certain formula over a set of K-Lipschitz functions approximates the 1-Wasserstein distance and they model the functions as NNs. That much I understand. They also argue that to unsure those functions (which are the critics in the context of GAN) are "somewhat" Lipschitz, they perform weight clipping. Why is that the case?

They mention they also tried projecting the weights on a sphere. What would that also make the output of the NN a Lipschitz function of the inputs?

Here is the approximation of the 1-Wasserstein distance I mentioned although I don't think it's relevant to the question:

$$ K.W\left(\mathbb{P}_r, \mathbb{P}_\theta\right)=\max _{w \in \mathcal{W}} \mathbb{E}_{x \sim \mathbb{P}_r}\left[f_w(x)\right]-\mathbb{E}_{z \sim p(z)}\left[f_w\left(g_\theta(z)\right]\right. $$ $\left\{f_w\right\}_{w \in \mathcal{W}}$ are all $K$-Lipschitz for some $ K$



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.