2
$\begingroup$

A Generative Adversarial Network (GAN) consists of two sub-networks: (1) generator and (2) discriminator.

What does a discriminator should be able to do? Or more specifically, should it be able to distinguish (classify) a real object (for example a vector) from a generated one or should it be able to distinguish a set of generated vectors from a set of real vectors?

I tend to think that the second option is correct. However, if it is the case, how do we build a neural network that classifies a set of vectors instead of a vector?

$\endgroup$
4
$\begingroup$

The discriminator must classify individual elements as being fake (i.e. created by the generator) or real (i.e. taken from the training dataset). The discriminator generates labels (real/fake) for each element in the batch. The loss functions are computed based on those labels.

Elements are fed to the discriminator in batches of the same type (i.e. all elements in the batch are real or all elements in the batch are fake). This is because the fake data batch is directly generated by the generator as part of the same computational graph (i.e. the output of the generator is directly connected to the input of the discriminator). This is so to be able to propagate gradients of the generator parameters through the discriminator.

|improve this answer|||||
$\endgroup$
  • $\begingroup$ but I can imagine that in this case the generator will learn to ignore the input noise and always produce the same output which is a copy (a very close to) one of the real examples. I though that by classifying a batch we can also take into account the diversity (distribution) of the generated entities. For example, if we try to generate numbers coming from a Gaussian distribution, it would be impossible to decide whether a single number is "real" or "generated". In case of a set of number we can, in theory, determine whether it is coming from a real distribution of generated. $\endgroup$ – Roman Oct 12 '17 at 7:31
  • $\begingroup$ The problem of the generator collapsing onto a finite subset of samples happens actually. It is called mode collapse. About your example, the way to decide if a number is a sample taken from a gaussian distribution is a matter of basic probabilities, and certainly a discriminator can learn it, as it is a comparison between the distance to the mean and the variance. $\endgroup$ – ncasas Oct 12 '17 at 17:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.