Variational Autoencoders VS Transformers

Question

I'm relatively new to the field, but I'd like to know how do variational autoencoders fare compared to transformers?

Answer 1

Variational AutoEncoder

---

VAE is an autoencoder whose encodings distribution is regularised during the training in order to ensure that its latent space has good properties allowing us to generate some new data. A variational autoencoder (VAE) provides a probabilistic manner for describing an observation in latent space.

• The Intuition Behind Variational Autoencoders
• Variational autoencoders

Transformers

---

Transformers are an architecture introduced in 2017, used primarily in the field of NLP, that aims to solve sequence-to-sequence tasks while handling long-range dependencies with ease. It relies entirely on self-attention to compute representations of its input and output WITHOUT using sequence-aligned RNNs or convolution. The main tasks these Transformers are used for are classification, information extraction, question answering, summarization, translation, text generation, etc

The most popular Transformers are BERT and GPT-2. Hugging Face has other Transformers available for you to experiment with.

• The Illustrated Transformer
• How do Transformers Work in NLP?

Answer 2

TLDR; transformers, in this context, are a method for optimizing the creation of an autoencoder. They aren't different things, so it doesn't make sense to compare them as if they are.