Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I have understood how GAN works while two networks (generative and discriminative) compete with each other. I have built a DCGAN (GAN with convolutional discriminator and de-convolutional generator) which now successfully generates handwritten digits similar to those in MNIST dataset.
I have read a lot about GAN's applications for extracting features from images. How can use my trained GAN model (on MNIST dataset) to extract feature from MNIST handwritten digist images?
Typically to extract features, you can use the top layer of the network before the output. The intuition is that these features are linearly separable because the top layer is just a logistic regression.
For GANs, you can use the features from the discriminator. These features are supposed to give a probability if the input came from the training dataset, "real images". In Radford's DCGAN paper, they use all the convolutional layers of the discriminator and run a max pooling layer extract features for CIFAR-10.
To evaluate the quality of the representations learned by DCGANs for supervised tasks, we train on Imagenet-1k and then use the discriminator’s convolutional features from all layers, maxpooling each layers representation to produce a 4 × 4 spatial grid. These features are then flattened and concatenated to form a 28672 dimensional vector and a regularized linear L2-SVM classifier is trained on top of them.
Kenny's answer is correct – if you're using convolutional D, output of layers before dense may serve as features. My intuition is that it will work better for AC-GANs (or similar architectures, which make D classify input in addition to determining if it's fake or real).
There is an approach called BiGAN which adds an Encoder component able to map generated and training samples to latent distribution z used to "initialize" generator. Authors show that it can effectively be used as a feature set for transfer learning and other tasks.
as the GAN consists of two parts -- the generator and the discriminator, there are two ways to use GAN as feature extractor:
The second way is more controversial. Some studies [1] thought that, intuitively, as the target of the discriminator is to distinguish the generated samples from the real samples, it will just focus on the difference between these two kinds of samples. But what makes sense is the difference between real samples, which is the samples used by the downstream tasks.
I have tried to study this, and I found that the feature extracted can be factorized into two orthogonal subspaces. The first space contributes to the discriminator task, while the second is free from it. As in most cases, the features used to distinguish the real samples from the generated ones are noise, the second feature space will be noise-free. From this perspective, although the task of discriminator will not focus on the difference between real samples, which are useful for the downstream tasks, the noise-free features contained in the second subspace will work.
[1] Jost Tobias Springenberg. Unsupervised and Semi-Supervised Learning with Categorical Generative Adversarial Networks. arXiv:1511.06390 [cs, stat], April 2016. arXiv preprint. arXiv:1511.06390 [stat.ML]. Ithaca, NY: Cornell University Library.
