4
$\begingroup$

I have a very interesting question. It's one month ago Geoffrey Hinton release his CapsNet paper. It's a complete new way for visual computing in the neural network stack. But my question is. Is it possible to use the CapsNet as a GAN. I only understand the concept of the CapsNet but not technical limitations. So could this be possible?

This sounds for me for a nice field to research:)

$\endgroup$
  • 2
    $\begingroup$ I was wondering this too - in principle the capsules should help force consistency between sub-components of an image and avoid some of the weird monstrosities that CNN-based GANs can produce. However, I don't know whether it is even possible to build the generator component using capsules architecture. A CNN-based generator and CapsNet-based discriminator might be more feasible in the near future $\endgroup$ – Neil Slater Nov 23 '17 at 12:05
  • $\begingroup$ Here you can find the papers, several articles and tutorials about CapsNet as well as current implementations in different programming languages and frameworks. $\endgroup$ – loretoparisi Dec 4 '17 at 23:09
  • $\begingroup$ I am expecting this too. That should worth a sci paper. $\endgroup$ – Origond Jan 4 '18 at 9:25
2
$\begingroup$

It is definitely possible to use them in the Encoder (Discriminator) part of a GAN, since according to Hinton's original paper it's used for classification which means it fits in Discriminators as well even though the proper loss function is a subject to research.

I chose a topic of Capsule Nets in Discriminators of GANs for my Master thesis and I'm working on the model right now. However, I simplified the routing algorithm by not vectorizing the DigitCaps values (otherwise Keras code gets complicated, although I'm not sure if what I've done is 100% correct).

$\endgroup$
0
$\begingroup$

Yes. You can use a CapsuleNet as the discriminator in a GAN. See the following recent paper, which claims this improves over using CNNs as the discriminator:

CapsuleGAN: Generative Adversarial Capsule Network. Ayush Jaiswal, Wael AbdAlmageed, Yue Wu, Premkumar Natarajan. arXiv:1802.06167

The improvement appears to be modest on the tasks they tested. It's not clear how to use a CapsuleNet as the generator, so the paper uses a CNN-based generator network.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.