How does variational autoencoders actually work in comparison to GAN?

Question

I want to know about how variational autoencoders work. I am currently working in a company and we want to incorporate variational autoencoders for creating synthetic data. I have questions regarding this method though, is this the only way to generate synthetic or artificial data? Is there a difference between VAE and GANs, is one preferred over the other?

I am also not a person with a lot of mathematical background and a bit wary on the implementation of it. Finally, I have gone through many links and videos on the implementation through PyTorch and Tensorflow. Are both similar in implementation?

I went through this link: https://www.youtube.com/watch?v=9zKuYvjFFS8&ab_channel=ArxivInsights

However, still not fully grasped a simpler way to implement this technique. Any help with understanding and its implementation would be greatly appreciated.

Answer 1

VAEs were a hot topic some years ago. They were known to generate somewhat blurry images and sometimes suffered from posterior collapse (the decoder part ignores the bottleneck). These problems improved with refinements. Basically, they are normal autoencoders (minimize the difference between the input image and output image) with an extra loss term to force the bottleneck into a normal distribution.

GANs became popular also a few years ago. They are known for being difficult to train due to their non-stationary training regime. Also, the quality of the output varies, including suffering the problem of mode collapse (always generating the same image). They consist of two networks: generator and discriminator, where the generator generates images and the discriminator tells if some image is fake (i.e. generated by the generator) or real. The generator learns to generate by training to deceive the discriminator.

Nowadays the hot topic is diffusion models. They are the type of models behind the renowned image-generation products Midjourney and DALL-E. They work by adding random noise to an image up to the point they are become only noise, and then learning how to remove that noise back into the image; then, you can generate images directly from noise.

Answer 2

It depends on the architecture chosen, but generally speaking, they do have some differences that can be measured as follows:

• How they learn and the training speed Variational Autoencoder learn by modelling explicit densities. On the other hand GANs are a min-max game and for that reason they learn based on competition. Because of the non-cooperative nature of GANs their convergence is harder to be ensured and for that reason while training you can observe more oscillations and variability. Nevertheless, this is not exactly bad, it will depend on the variability that you want to introduce to your synthetic data. GANs are harder to optimize and they tend to suffer from mode collapse. This can usually be mitigated with the right loss function (aka you want to ensure a more cooperative training). Both are not exclusive, and you can see some architectures that combine the pros and cons of both worlds, such as TimeGAN.
• Sampling space VAE generate new records by reconstructing the data from a low-dimensional representation of the original records. This process introduces some noise, but allows the generation of data with quality. GANs generate data from any random input, which allows the generation of more diverse samples when compare to VAE. This poses a huge benefit in cases of augmentation for instance.

In a nutshell, GANs are harder to train, but when well fine tune can generate outputs with bigger variability and also more realistic when compared to VAE. The choice will mainly depend on the objective of the generated data - if you want to stress test a model or even augment fraud cases, GANs are better candidates. If you just want to replicate more of the same data for compression purposes for instance, VAE are a great way to go.

Attention models can be also very interesting indeed, but will depend on the data types that you want to focus on (structured data, images, text, etc.)