as the GAN consists of two parts -- the generator and the discriminator, there are two ways to use GAN as feature extractor:
- Generator based way as presented by Mikhail Yurasov.
- Discriminator based way as presented by Kenny.
The second way is more controversial. Some studies [1] thought that, intuitively, as the target of the discriminator is to distinguish the generated samples from the real samples, it will just focus on the difference between these two kinds of samples. But what makes sense is the difference between real samples, which is the samples used by the downstream tasks.
I have tried to study this, and I found that the feature extracted can be factorized into two orthogonal subspaces. The first space contributes to the discriminator task, while the second is free from it. As in most cases, the features used to distinguish the real samples from the generated ones are noise, the second feature space will be noise-free. From this perspective, although the task of discriminator will not focus on the difference between real samples, which are useful for the downstream tasks, the noise-free features contained in the second subspace will work.
[1] Jost Tobias Springenberg. Unsupervised and Semi-Supervised Learning with Categorical Generative Adversarial Networks. arXiv:1511.06390 [cs, stat], April 2016. arXiv preprint. arXiv:1511.06390 [stat.ML]. Ithaca, NY: Cornell University Library.