Why VAE Encoder outputs log variance and not standard deviation?

Question

When talking about VAE (and viewing VAE implementations), the Encoder outputs:

μ, log(variance)

when we train the model (the part of the decoder model), we convert the log(variance) to Standard deviation:

std = exp(0.5 * logvar)

(I took the example from here: https://github.com/AntixK/PyTorch-VAE/blob/master/models/vanilla_vae.py)

If we need to convert the log(variance) to Standard deviation, why won't we output the Standard deviation from the encoder instead of making calculation to convert it to Standard deviation ?

Answer 1

The log variance is necessary to compute the KL divergence term in the loss function.

During training, the VAE minimizes a loss function that includes a reconstruction loss and a KL divergence term. The reconstruction loss measures the error between the input and its reconstruction, while the KL divergence term measures the distance between the encoder's distribution and a prior distribution over the latent space.

See also: Mathematical details from this article.

However, in the github you've mentioned, the standard deviation is a trick to generate images from the model.

It is used in the function generate, which is applied to retrieve images from the model in experiment.py :

    def sample_images(self):
        # Get sample reconstruction image            
        test_input, test_label = next(iter(self.trainer.datamodule.test_dataloader()))
        test_input = test_input.to(self.curr_device)
        test_label = test_label.to(self.curr_device)

#         test_input, test_label = batch
        recons = self.model.generate(test_input, labels = test_label)
        vutils.save_image(recons.data,
                          os.path.join(self.logger.log_dir , 
                                       "Reconstructions", 
                                       f"recons_{self.logger.name}_Epoch_{self.current_epoch}.png"),
                          normalize=True,
                          nrow=12)