Suppose a Variational Autoencoder (VAE) is trained with mnist data. To sample, one draws from normal distribution. My question is: suppose I am interested in generating only 1s and no other digits. How can do I do that? do I sample until I generate a 1 and then I keep sampling from the neighborhood of that point? or is there a more controlled way to tell the VAE what digit to generate? Thanks


The ideal encoder would take any input and generate a sample coming from $\mathcal{N}(0,1)$. If the input is 1 or 2 or 3, the samplings will all come from $\mathcal{N}(0,1)$, so, how does the decoder know if the input was 1, 2, or 3?

  • 2
    $\begingroup$ there are 2 approaches, one is sample unitll you get a 1, the other is to have some input point in latent space whose representation is close to 1. In both cases you will have to check you get indeed a 1 back $\endgroup$
    – Nikos M.
    Apr 20, 2021 at 18:01
  • $\begingroup$ about the second approach, if the encoder truly does his job, shouldn't the output of the encoder just be $\mathcal{N}(0,1)$ but it will be the same for digit 2, 3, etc. $\endgroup$
    – Schach21
    Apr 20, 2021 at 20:11
  • $\begingroup$ to my understanding "close" input points should have "close" output representations, if this is what you mean. A 3rd option is to train a VAE only with 1s $\endgroup$
    – Nikos M.
    Apr 20, 2021 at 20:16
  • $\begingroup$ I edited my question to clarify what I mean, please let me know if it's clear or I should clarify further. $\endgroup$
    – Schach21
    Apr 20, 2021 at 20:22
  • $\begingroup$ Are conditional VAEs an option? $\endgroup$
    – Jonathan
    Apr 20, 2021 at 22:08

1 Answer 1


You condition the latent variable on another variable drawn from a categorical distribution (https://wiseodd.github.io/techblog/2016/12/17/conditional-vae/).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.