Questions tagged [autoencoder]

Autoencoders are a type of neural network that learns a useful encoding for data in an unsupervised manner.

Filter by
Sorted by
Tagged with
21 votes
1 answer
24k views

What is "posterior collapse" phenomenon?

I was going through this paper on Towards Text Generation with Adversarially Learned Neural Outlines and it states why the VAEs are hard to train for text generation due to this problem. The paper ...
thanatoz's user avatar
  • 2,395
20 votes
3 answers
10k views

Why are autoencoders for dimension reduction symmetrical?

I'm not an expert in autoencoders or neural networks by any means, so forgive me if this is a silly question. For the purpose of dimension reduction or visualizing clusters in high dimensional data, ...
dcl's user avatar
  • 251
15 votes
2 answers
11k views

What is the difference between an autoencoder and an encoder-decoder?

I want to know if there is a difference between an autoencoder and an encoder-decoder.
Kahina's user avatar
  • 614
13 votes
3 answers
22k views

How can autoencoders be used for clustering?

Suppose I have a set of time-domain signals with absolutely no labels. I want to cluster them in 2 or 3 classes. Autoencoders are unsupervised networks that learn to compress the inputs. So given an ...
Tendero's user avatar
  • 243
12 votes
2 answers
16k views

Does it make sense to train a CNN as an autoencoder?

I work with analyzing EEG data, which will eventually need to be classified. However, obtaining labels for the recordings is somewhat expensive, which has led me to consider unsupervised approaches, ...
Kaare's user avatar
  • 267
10 votes
1 answer
8k views

Difference: Replicator Neural Network vs. Autoencoder

I'm currently studying papers about outlier detection using RNN's (Replicator Neural Networks) and wonder what is the particular difference to Autoencoders? RNN's seem to be treaded for many as the ...
Nex's user avatar
  • 285
10 votes
1 answer
3k views

Why is Reconstruction in Autoencoders Using the Same Activation Function as Forward Activation, and not the Inverse?

Suppose you have an input layer with n neurons and the first hidden layer has $m$ neurons, with typically $m < n$. Then you compute the actication $a_j$ of the $j$-th neuron in the hidden layer by ...
Manfred Eppe's user avatar
10 votes
1 answer
824 views

Transforming AutoEncoders

I've just read Geoff Hinton's paper on transforming autoencoders Hinton, Krizhevsky and Wang: Transforming Auto-encoders. In Artificial Neural Networks and Machine Learning, 2011. and would quite ...
Daniel Slater's user avatar
10 votes
1 answer
184 views

Robustness of ML Model in question

While trying to emulate a ML model similar to the one described in this paper, I seemed to eventually get good clustering results on some sample data after a bit of tweaking. By "good" results, I mean ...
Alerra's user avatar
  • 201
9 votes
2 answers
2k views

Transform an Autoencoder to a Variational Autoencoder?

I would like to compare the training by an Autoencoder and a variational autoencoder. I have already run the traing using AE. I would like to know if it's possible to transform this AE into a VAE and ...
Kahina's user avatar
  • 614
9 votes
2 answers
43k views

Why my training and validation loss is not changing?

I used MSE loss function, SGD optimization: ...
sp_713's user avatar
  • 115
9 votes
1 answer
12k views

Validation loss is lower than the training loss

I am using autoencoder for anomaly detection in warranty data. Architecture 1: The plot shows the training vs validation loss based on Architecture 1. As we see in the plot, validation loss is ...
Ashwini's user avatar
  • 235
9 votes
1 answer
7k views

Which type auto encoder gives best results for text

I did I couple of examples for auto encoders for images and they worked fine. Now I want to do an auto encoder for text that takes as input a sentence and returns the same sentence. But when I try to ...
sspp's user avatar
  • 109
8 votes
1 answer
9k views

Using an autoencoder for anomaly detection on categorical data

Say a dataset has 0.5% of its features continuous and 99.5% categorical (binary) with ~2400 features in total. In this dataset, each observation is 1 of 2 classes - Fraud (1) or Not Fraud (0). ...
PyRsquared's user avatar
  • 1,604
7 votes
3 answers
2k views

What is an autoencoder?

I am a student and I am studying machine learning. I am focusing on deep generative models, and in particular to autoencoders and variational autoencoders (VAE). I am trying to understand the concept, ...
J.D.'s user avatar
  • 851
7 votes
1 answer
833 views

How does an encoder-decoder network work?

Let's say I trained an encoder-decoder network on a cat dataset using reconstruction error as loss function. The network is fully trained and the decoder is able to reconstruct good cat images. Now ...
ashukid's user avatar
  • 877
7 votes
2 answers
9k views

Variational Autoencoders VS Transformers

I'm relatively new to the field, but I'd like to know how do variational autoencoders fare compared to transformers?
TheQuantumMan's user avatar
7 votes
1 answer
727 views

How can I prove bottleneck layer of my CNN auto encoder contain useful information?

I am using CNN autoencoder to create a state representation layer which I will later be feed into my Reinforcement Agent. So I trained my CNN autoencoder and it is giving nice state representations. ...
Shamane Siriwardhana's user avatar
7 votes
2 answers
4k views

Tips and tricks for designing time-series variational autoencoders

I am new to VAEs but find them quite fascinating. I was wondering if anyone might have any tips or tricks regarding, how one should build the encoder and decoder layers w.r.t. to time-series data. ...
Astrid's user avatar
  • 205
7 votes
1 answer
917 views

Why autoencoders use binary_crossentropy loss and not mean squared error?

Keras autoencoders examples: (https://blog.keras.io/building-autoencoders-in-keras.html) use binary_crossentropy (BCE) as loss function. Why they use ...
user3668129's user avatar
7 votes
2 answers
5k views

Transformer-based architectures for regression tasks

As far as I've seen, transformer-based architectures are always trained with classification tasks (one-hot text tokens for example). Are you aware of any architectures using attention and solving ...
Damjan Dakic's user avatar
7 votes
1 answer
4k views

Convolutional autoencoders not learning

I'm trying to implement convolutional autoencoders in tensorflow, on the mnist dataset. The problem is that the autoencoder does not seem to learn properly: it will always learn to reproduce the 0 ...
AkiRoss's user avatar
  • 191
6 votes
2 answers
4k views

Latent loss in variational autoencoder drowns generative loss

I'm trying to run a variational auto-encoder on the CIFAR-10 dataset, for which I've put together a simple network in TensorFlow with 4 layers in the encoder and decoder each, an encoded vector size ...
Ali250's user avatar
  • 213
6 votes
2 answers
1k views

How to scale outputs from AutoEncoder from multiple models?

I have a problem for which I have not been able to find any answers in my search so far. BACKGROUND I am working on an anomaly detection problem on machines utilising an auto-encoder. I am building ...
DaytaSigntist's user avatar
6 votes
1 answer
9k views

How to extract features from the encoded layer of an autoencoder?

I have done some research on autoencoders, and I have come to understand that they can also be used for feature extraction (see this question on this site as an example). Most of the examples out ...
user1301428's user avatar
6 votes
1 answer
6k views

Why maximize ELBO in the variational autoencoder?

For a variational autoencoder, we have that: $$\mathcal{L}(x,\theta,\phi) := \mathbb{E}_{z \sim q_\phi(z|x)}[\log p_{\theta}(x|z)] -KL[q_{\phi}(z|x) ||p(z)] $$ This is called the variational lower ...
A_P's user avatar
  • 163
6 votes
1 answer
1k views

How does u-net work?

I have read this paper about U-Net. This kind of network is quite similar to an autoencoder, in addition it has concatenations between the encoder and the decoder parts. I would like to know the ...
Simone's user avatar
  • 705
6 votes
3 answers
4k views

Stuck on deconvolution in Theano and TensorFlow

I'm captivated by autoencoders and really like the idea of convolution. It seems though that both Theano and TensorFlow only support conv2d to go from an array of 2D-RGB (n 3D arrays) to an array of ...
Joseph Catrambone's user avatar
6 votes
0 answers
142 views

Unable to transform (greatly performing) Autoencoder into Variational Autoencoder

Following the procedure described in this SO question, I am trying to transform my (greatly performing) convolutional Autoencoder into a Variational version of the same Autoencoder. As explained in ...
user87590's user avatar
5 votes
1 answer
682 views

Unsupervised feature reduction for anomaly detection with autoencoders

I am collecting a big number of generated numeric features for the task of unsupervised anomaly detection. I can assume that all training data is considered normal. I expect some of the generated ...
Yuval's user avatar
  • 153
5 votes
1 answer
2k views

do autoencoders work well for non images?

I have a classification problem for which a feedforward, fully connected neural net works reasonably well (two classes, true positive and true negative rate close to 80%). I want to get these rates ...
Alejandro Simkievich's user avatar
5 votes
1 answer
3k views

How design a autoencoder architecture

I would like to build an autoencoder (CNN) to learn a representation of my data. I never built such a network and I have some experience in supervised learning (classification). I would like to know ...
alexandre_d's user avatar
5 votes
1 answer
2k views

Keras autoencoder not converging

Could someone please explain to me why the autoencoder is not converging? To me the results of the two networks below should be the same. However, the autoencoder below is not converging, whereas, the ...
Fritz O'Connor's user avatar
5 votes
1 answer
6k views

Understanding autoencoder loss function

I've never understood how to calculate an autoencoder loss function because the prediction has many dimensions, and I always thought that a loss function had to output a single number / scalar ...
Ryan Zotti's user avatar
  • 4,139
5 votes
1 answer
7k views

What is the best architecture for Auto-Encoder for image reconstruction?

I am trying to use Convultional Auto-Encoder for its latent space (embedding layer), specifically, I want to use the embedding for K-nearest neighbor search in the latent space (similar idea to ...
Idan Azuri's user avatar
4 votes
2 answers
5k views

Preprocessing and dropout in Autoencoders?

I am working with autoencoders and have few confusions, I am trying different autoencoders like : ...
Aaditya ura's user avatar
4 votes
1 answer
4k views

1D CNN Variational Autoencoder Conv1D Size

I am trying to create a 1D variational autoencoder to take in a 931x1 vector as input, but I have been having trouble with two things: Getting the output size of 931, since maxpooling and upsampling ...
Celi Manu's user avatar
  • 141
4 votes
1 answer
617 views

Why KL Divergence instead of Cross-entropy in VAE

I understand how KL divergence provides us with a measure of how one probability distribution is different from a second, reference probability distribution. But why are they particularly used (...
Bahauddin Omar's user avatar
4 votes
2 answers
4k views

Autoencoders for the compression of time series

I am trying to use autoencoder (simple, convolutional, LSTM) to compress time series. Here are the models I tried. Simple autoencoder: ...
netang's user avatar
  • 41
3 votes
2 answers
7k views

Autoencoder: using cosine distance as loss function

I'm trying to train an autoencoder (in PyTorch) to reconstruct gene profiles. At the moment I'm using the Mean Squared Error (MSE) loss for training: the model is not overfitting and both the training ...
wrong_path's user avatar
3 votes
2 answers
243 views

How does variational autoencoders actually work in comparison to GAN?

I want to know about how variational autoencoders work. I am currently working in a company and we want to incorporate variational autoencoders for creating synthetic data. I have questions regarding ...
NevMthw's user avatar
  • 47
3 votes
1 answer
2k views

Variational auto-encoders (VAE): why the random sample?

Why do people train variational auto-encoders (VAE) to encode means and variances (regularised towards 0 and 1), and then sample a random Gaussian, rather that simply encode latent vectors and ...
Antoine Savine's user avatar
3 votes
2 answers
6k views

Autoencoder for anomaly detection from feature vectors

I am trying to use an autoencoder (as described here https://blog.keras.io/building-autoencoders-in-keras.html#) for anomaly detection. I am using a ~1700 feature vector (rather than images, which ...
Lafayette's user avatar
  • 604
3 votes
2 answers
4k views

Basic encoder-decoder architecture

I read a couple of posts (like this one) about the encoder-decoder architecture and their implementation. But I still don't understand a couple of things. What is the difference between a basic CNN ...
XOXO's user avatar
  • 31
3 votes
2 answers
5k views

how to find classification accuracy in autoencoders?

how can we find the accuracy autoencoders for classification of images? because we will get the reconstruction of the image and when we will plug in the test data it will spit out the image but how ...
Boris's user avatar
  • 463
3 votes
3 answers
1k views

How to interpreter Binary Cross Entropy loss function?

I saw some examples of Autoencoders (on images) which use sigmoid as output layer and BinaryCrossentropy as loss function. The ...
Boom's user avatar
  • 297
3 votes
1 answer
2k views

KL divergence loss goes to zero while training VAE

I'm trying to train a variational autoencoder to perform unsupervised classification of astronomical images (they are of size 63x63 pixels). I'm using an encoder with 2 convolutional layers and a ...
Rahul Priyadarshan's user avatar
3 votes
1 answer
7k views

What do we visualize in showing a VAE latent space?

I am trying to wrap my head around VAE's and have trouble understanding what is being visualized when people make scatter plots of the latent space. I think I understand the bottleneck concept; we go ...
ITA's user avatar
  • 133
3 votes
1 answer
719 views

What exactly is the input of decoder in autoencoder setup

I am reviewing various autoencoder setups for MNIST reconstruction, Seq2Seq translation and others. My naive understanding of data flow is as follows: Input -> [Encoder] -> Hidden Representation -> [...
aboev's user avatar
  • 133
3 votes
1 answer
439 views

How to implement a convolutional autoencoder?

I would like to implement a convolutional autoencoder in Tensorflow, but it is not clear how the decoder part should work. Each layer of the encoding, is a convolutional layer with activation ...
Iter Ator's user avatar
  • 195

1
2 3 4 5
7