1
$\begingroup$

When going through an introductory GAN tutorial to generate mnist like handwritten digits I wondered whether the systematic variance in the training data due to the different digits makes the model harder to train.

Would't it be easier to train a model if all real samples would be 1s instead of 0-9s?

My question

  1. Would an approach where I train a separate model to generate each digit (0-9) separately using only training data of one type of digit be feasible at all?

  2. Would training those models for just one digit be faster (i.e. need less epochs to reach a certain accuracy/quality) than a model using all of mnist.

Edit

I found an acceptable answer for me (see below) but obviously I am happy for anyone else to chime in.

$\endgroup$
2
  • $\begingroup$ I am just learning how to train GAN, I have tried using MNIST dataset and I want to adapt to generate specific digit. As per your comment above it says you trained GAN using single digit 3, how did you do that? $\endgroup$ – Nansy Mar 18 at 17:10
  • $\begingroup$ @Nansy if your training sample used to train the GAN is only the 3 pictures provided by mnist, the resulting GAN is only producing 3s. So the answer is to simply prefilter mnist to only the number you need. $\endgroup$ – Fnguyen Mar 18 at 18:47
1
$\begingroup$

After some further research and experimentation, I think I am able to answer my own question.

1. Yes it is feasible!

The quickest way to answer my first question was to try it out, which I did. Training a GAN on only a single digit (3 in my example) worked and produced the expected results:GAN generated handwritten 3s after 100 epochs

Besides the obvious practical example, I was able to find some further articles discussing GAN which mentioned that reducing the classes in the training sample to reduce variance is advised (e.g. in CIFAR-10 concentrate on frogs OR trucks not both). So while I suspected as much it is still nice to confirm that this approach is valid and feasible.

2. I'm not sure it is faster

The second part of my question was whether such an approach is better (i.e. more accurate) and faster.

I was unable to confirm this because mnist is a very simple example and accuracy is very good, very fast in either case. I learned that speed of model fitting depends on the model complexity and number of epochs, which do not change by focusing on just one digit! So naturally covering all digits individual takes a lot longer than just training one whole model.

It might be the case that for more complex classes/image types more complex models and epochs are needed and that cutting down on variance (i.e. just frogs, no trucks) can help cutting down those two main drivers of computation duration but this was not the case for mnist like images.

The only direct benefit I was able to find is the greater utility of individual models. Due to the clear separation I am able to generate a specific number on demand (by calling the relevant model) instead of generating only a random number from 0-9.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.