Questions tagged [autoencoder]

Autoencoders are a type of neural network that learns a useful encoding for data in an unsupervised manner.

Filter by
Sorted by
Tagged with
16
votes
3answers
8k views

Why are autoencoders for dimension reduction symmetrical?

I'm not an expert in autoencoders or neural networks by any means, so forgive me if this is a silly question. For the purpose of dimension reduction or visualizing clusters in high dimensional data, ...
15
votes
1answer
11k views

What is "posterior collapse" phenomenon?

I was going through this paper on Towards Text Generation with Adversarially Learned Neural Outlines and it states why the VAEs are hard to train for text generation due to this problem. The paper ...
13
votes
3answers
14k views

How can autoencoders be used for clustering?

Suppose I have a set of time-domain signals with absolutely no labels. I want to cluster them in 2 or 3 classes. Autoencoders are unsupervised networks that learn to compress the inputs. So given an ...
11
votes
2answers
5k views

What is the difference between an autoencoder and an encoder-decoder?

I want to know if there is a difference between an autoencoder and an encoder-decoder.
11
votes
2answers
13k views

Does it make sense to train a CNN as an autoencoder?

I work with analyzing EEG data, which will eventually need to be classified. However, obtaining labels for the recordings is somewhat expensive, which has led me to consider unsupervised approaches, ...
10
votes
1answer
2k views

Why is Reconstruction in Autoencoders Using the Same Activation Function as Forward Activation, and not the Inverse?

Suppose you have an input layer with n neurons and the first hidden layer has $m$ neurons, with typically $m < n$. Then you compute the actication $a_j$ of the $j$-th neuron in the hidden layer by ...
10
votes
1answer
755 views

Transforming AutoEncoders

I've just read Geoff Hinton's paper on transforming autoencoders Hinton, Krizhevsky and Wang: Transforming Auto-encoders. In Artificial Neural Networks and Machine Learning, 2011. and would quite ...
10
votes
1answer
153 views

Robustness of ML Model in question

While trying to emulate a ML model similar to the one described in this paper, I seemed to eventually get good clustering results on some sample data after a bit of tweaking. By "good" results, I mean ...
9
votes
2answers
1k views

Transform an Autoencoder to a Variational Autoencoder?

I would like to compare the training by an Autoencoder and a variational autoencoder. I have already run the traing using AE. I would like to know if it's possible to transform this AE into a VAE and ...
9
votes
1answer
11k views

Validation loss is lower than the training loss

I am using autoencoder for anomaly detection in warranty data. Architecture 1: The plot shows the training vs validation loss based on Architecture 1. As we see in the plot, validation loss is ...
9
votes
1answer
6k views

Which type auto encoder gives best results for text

I did I couple of examples for auto encoders for images and they worked fine. Now I want to do an auto encoder for text that takes as input a sentence and returns the same sentence. But when I try to ...
8
votes
2answers
28k views

Why my training and validation loss is not changing?

I used MSE loss function, SGD optimization: ...
8
votes
1answer
6k views

Using an autoencoder for anomaly detection on categorical data

Say a dataset has 0.5% of its features continuous and 99.5% categorical (binary) with ~2400 features in total. In this dataset, each observation is 1 of 2 classes - Fraud (1) or Not Fraud (0). ...
8
votes
1answer
7k views

Difference: Replicator Neural Network vs. Autoencoder

I'm currently studying papers about outlier detection using RNN's (Replicator Neural Networks) and wonder what is the particular difference to Autoencoders? RNN's seem to be treaded for many as the ...
7
votes
3answers
2k views

What is an autoencoder?

I am a student and I am studying machine learning. I am focusing on deep generative models, and in particular to autoencoders and variational autoencoders (VAE). I am trying to understand the concept, ...
7
votes
1answer
783 views

How does an encoder-decoder network work?

Let's say I trained an encoder-decoder network on a cat dataset using reconstruction error as loss function. The network is fully trained and the decoder is able to reconstruct good cat images. Now ...
7
votes
1answer
583 views

How can I prove bottleneck layer of my CNN auto encoder contain useful information?

I am using CNN autoencoder to create a state representation layer which I will later be feed into my Reinforcement Agent. So I trained my CNN autoencoder and it is giving nice state representations. ...
7
votes
1answer
2k views

Transformer-based architectures for regression tasks

As far as I've seen, transformer-based architectures are always trained with classification tasks (one-hot text tokens for example). Are you aware of any architectures using attention and solving ...
7
votes
1answer
3k views

Convolutional autoencoders not learning

I'm trying to implement convolutional autoencoders in tensorflow, on the mnist dataset. The problem is that the autoencoder does not seem to learn properly: it will always learn to reproduce the 0 ...
6
votes
2answers
504 views

How to scale outputs from AutoEncoder from multiple models?

I have a problem for which I have not been able to find any answers in my search so far. BACKGROUND I am working on an anomaly detection problem on machines utilising an auto-encoder. I am building ...
6
votes
2answers
4k views

Tips and tricks for designing time-series variational autoencoders

I am new to VAEs but find them quite fascinating. I was wondering if anyone might have any tips or tricks regarding, how one should build the encoder and decoder layers w.r.t. to time-series data. ...
6
votes
1answer
1k views

How does u-net work?

I have read this paper about U-Net. This kind of network is quite similar to an autoencoder, in addition it has concatenations between the encoder and the decoder parts. I would like to know the ...
6
votes
3answers
4k views

Stuck on deconvolution in Theano and TensorFlow

I'm captivated by autoencoders and really like the idea of convolution. It seems though that both Theano and TensorFlow only support conv2d to go from an array of 2D-RGB (n 3D arrays) to an array of ...
5
votes
2answers
3k views

Latent loss in variational autoencoder drowns generative loss

I'm trying to run a variational auto-encoder on the CIFAR-10 dataset, for which I've put together a simple network in TensorFlow with 4 layers in the encoder and decoder each, an encoded vector size ...
5
votes
1answer
615 views

Unsupervised feature reduction for anomaly detection with autoencoders

I am collecting a big number of generated numeric features for the task of unsupervised anomaly detection. I can assume that all training data is considered normal. I expect some of the generated ...
5
votes
1answer
1k views

do autoencoders work well for non images?

I have a classification problem for which a feedforward, fully connected neural net works reasonably well (two classes, true positive and true negative rate close to 80%). I want to get these rates ...
5
votes
1answer
6k views

How to extract features from the encoded layer of an autoencoder?

I have done some research on autoencoders, and I have come to understand that they can also be used for feature extraction (see this question on this site as an example). Most of the examples out ...
5
votes
1answer
3k views

Why maximize ELBO in the variational autoencoder?

For a variational autoencoder, we have that: $$\mathcal{L}(x,\theta,\phi) := \mathbb{E}_{z \sim q_\phi(z|x)}[\log p_{\theta}(x|z)] -KL[q_{\phi}(z|x) ||p(z)] $$ This is called the variational lower ...
5
votes
1answer
2k views

Keras autoencoder not converging

Could someone please explain to me why the autoencoder is not converging? To me the results of the two networks below should be the same. However, the autoencoder below is not converging, whereas, the ...
5
votes
1answer
79 views

Why autoencoders use binary_crossentropy loss and not mean squared error?

Keras autoencoders examples: (https://blog.keras.io/building-autoencoders-in-keras.html) use binary_crossentropy (BCE) as loss function. Why they use ...
5
votes
1answer
6k views

Understanding autoencoder loss function

I've never understood how to calculate an autoencoder loss function because the prediction has many dimensions, and I always thought that a loss function had to output a single number / scalar ...
4
votes
1answer
2k views

How design a autoencoder architecture

I would like to build an autoencoder (CNN) to learn a representation of my data. I never built such a network and I have some experience in supervised learning (classification). I would like to know ...
4
votes
1answer
745 views

1D CNN Variational Autoencoder Conv1D Size

I am trying to create a 1D variational autoencoder to take in a 931x1 vector as input, but I have been having trouble with two things: Getting the output size of 931, since maxpooling and upsampling ...
4
votes
1answer
3k views

What is the best architecture for Auto-Encoder for image reconstruction?

I am trying to use Convultional Auto-Encoder for its latent space (embedding layer), specifically, I want to use the embedding for K-nearest neighbor search in the latent space (similar idea to ...
4
votes
1answer
3k views

Autoencoders for the compression of time series

I am trying to use autoencoder (simple, convolutional, LSTM) to compress time series. Here are the models I tried. Simple autoencoder: ...
3
votes
2answers
4k views

Autoencoder: using cosine distance as loss function

I'm trying to train an autoencoder (in PyTorch) to reconstruct gene profiles. At the moment I'm using the Mean Squared Error (MSE) loss for training: the model is not overfitting and both the training ...
3
votes
2answers
4k views

Preprocessing and dropout in Autoencoders?

I am working with autoencoders and have few confusions, I am trying different autoencoders like : ...
3
votes
1answer
1k views

Variational auto-encoders (VAE): why the random sample?

Why do people train variational auto-encoders (VAE) to encode means and variances (regularised towards 0 and 1), and then sample a random Gaussian, rather that simply encode latent vectors and ...
3
votes
2answers
6k views

Autoencoder for anomaly detection from feature vectors

I am trying to use an autoencoder (as described here https://blog.keras.io/building-autoencoders-in-keras.html#) for anomaly detection. I am using a ~1700 feature vector (rather than images, which ...
3
votes
1answer
168 views

ValueError: Input 0 of layer conv2d is incompatible with the layer: expected axis -1 of input shape to have value 1 but received input with shape

I'm trying to create an auto-encoder based model for segmentation, which looks something like this: https://i.stack.imgur.com/4F3Z0.png I haven't added a single step, nor missed one as far as I ...
3
votes
2answers
3k views

how to find classification accuracy in autoencoders?

how can we find the accuracy autoencoders for classification of images? because we will get the reconstruction of the image and when we will plug in the test data it will spit out the image but how ...
3
votes
2answers
4k views

Basic encoder-decoder architecture

I read a couple of posts (like this one) about the encoder-decoder architecture and their implementation. But I still don't understand a couple of things. What is the difference between a basic CNN ...
3
votes
1answer
652 views

What exactly is the input of decoder in autoencoder setup

I am reviewing various autoencoder setups for MNIST reconstruction, Seq2Seq translation and others. My naive understanding of data flow is as follows: Input -> [Encoder] -> Hidden Representation -> [...
3
votes
1answer
416 views

How to implement a convolutional autoencoder?

I would like to implement a convolutional autoencoder in Tensorflow, but it is not clear how the decoder part should work. Each layer of the encoding, is a convolutional layer with activation ...
3
votes
1answer
798 views

What is the purpose of the discriminator in an adversarial autoencoder?

This is specific to the generative adversarial network (GAN) proposed in A. Makhzani et al. "Adversarial Autoencoders". In a traditional GAN, the discriminator is trained to distinguish real samples ...
3
votes
1answer
902 views

Ordered elements of feature vectors for autoencoders?

Here is a newbie question; when one trains an autoencoder or a variational autoencoder, does the order of the objects in the training vector $x$ matter? Suppose I take an MNIST image image $(28\...
3
votes
2answers
7k views

How does strided deconvolution works?

I am trying to understand how the shape of the image changes after deconvolution ? I am trying to understand the example code of convolutional autoencoder from neon. ...
3
votes
1answer
135 views

Why KL Divergence instead of Cross-entropy in VAE

I understand how KL divergence provides us with a measure of how one probability distribution is different from a second, reference probability distribution. But why are they particularly used (...
3
votes
1answer
392 views

VAE generates bad images. due to unbalanced loss functions?

I'm training a variational autoencoder on CelebA dataset using TensorFlow.keras The problem I'm facing is that the generated images are not diverse enough and look kinda bad. (new) Example: What I ...
3
votes
1answer
688 views

Intractability in Variational Autoencoders

I'm having difficulty understanding when integrals are intractable in variational inference problems. In a variational autoencoder with observation $x$ and latent variable $z$ we want to maximize data ...

1
2 3 4 5
7