Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [autoencoder]

Autoencoders are a type of neural network that learns a useful encoding for data.

0
votes
0answers
4 views

Can I make it 'return _sequences = False' in autoencoder, using Keras?

Question Should we use ‘return_sequences=True’ for all the LSTM layers in the encoder of the LSTM autoencoder? My questions are based on the article LSTM Autoencoder for Extreme Rare Event ...
1
vote
0answers
11 views

How smaller does the input data get reduced in a LSTM autoencoder

Question In a LSTM autoencder, how smaller does my input data(59 features) get reduced in a latent vector, which is usually located in the middle between an encoder and a decoder? Why did the ...
1
vote
0answers
27 views

Why maximize ELBO in the variational autoencoder?

For a variational autoencoder, we have that: $$\mathcal{L}(x,\theta,\phi) := \mathbb{E}_{q_\phi(z|x)}[\log p_{\theta}(x|z)] -KL[q_{\phi}(z|x) ||p(z)] $$ This is called the variational lower bound or ...
0
votes
1answer
14 views

Large scale Autoencoder for friendship recommendation

I have a user friendship graph for about 30 Million users. I am trying to use an auto encoder [30 Million, 512, 512, 1024, Dropout(0.3), 512, 512, 30 Million]. But I am not learning anything. Has ...
0
votes
0answers
17 views

How to check quality of latent space like in β-VAE article?

There is a nice plot in the β-VAE article that shows quality of latent space code: Is there a general way to visualize or analyze latent space code dimensions so that is would be clear if they are ...
0
votes
0answers
14 views

Why does Logistic Regression perform better than Autoencoders when classifying imbalanced data?

The 'shuttle' data can be downloaded from the link here. It is imbalanced data and there are two classes in the target variable. The proportion of the two classes are seven percent. I used Logistic ...
0
votes
0answers
19 views

IndexError: list index out of range

I'm implementing a sequence-2-sequence model with RNN-VAE architecture, and I use an attention mechanism. I have problem in the decoder part. I'm struggling with this error: IndexError: list index ...
0
votes
0answers
17 views

Autoencoder and Dimensionality

I'm pretty confused with the input/output/dense portion of an autoencoder. So my data consists of a numpy array of a 9 categorical features which have all been one hot encoded. So the input would ...
0
votes
2answers
20 views

Shouldn't an autoencoder with #(neurons in hidden layer) = #(neurons in input layer) be “perfect”?

I'm exploring autoencoders for the first time. I'm using the Matlab neural networks toolbox. I have created a synthetic dataset consisting of points in 2D space plus some noise. My idea was to ...
1
vote
0answers
44 views

What is the best architecture for Auto-Encoder for image reconstruction?

I am trying to use Convultional Auto-Encoder for its latent space (embedding layer), specifically, I want to use the embedding for K-nearest neighbor search in the latent space (similar idea to ...
0
votes
0answers
9 views

Variational auto-encoders for text generation

How are the variational auto-encoders used for text generation? Can variational auto-encoders be used for character based text generation?
1
vote
1answer
76 views

How to explain get_weight with autoencoder in keras?

I built an autoencoder model of three layers with 9 5 9. Input dim =9, encoder dim =5, output dim=9 When I get the model weights, ...
2
votes
1answer
52 views

What do we visualize in showing a VAE latent space?

I am trying to wrap my head around VAE's and have trouble understanding what is being visualized when people make scatter plots of the latent space. I think I understand the bottleneck concept; we go ...
0
votes
0answers
43 views

InvalidArgumentError: incompatible shapes: [32,153] vs [32,5] , when using VAE

I'm working on a sequence to sequence model using LSTM, the model worked perfectly with an autoencoder, but when I try to use a Variational autoencoder by adding the mean and deviation layer and ...
1
vote
1answer
108 views

What is “posterior collapse” phenomenon?

I was going through this paper on Towards Text Generation with Adversarially Learned Neural Outlines and it states why the VAEs are hard to train for text generation due to this problem. The paper ...
0
votes
0answers
18 views

How is the standard deviation of VAE's obtained?

I am trying to build a Variational Autoencoder. I was looking at various codes online and found most of them in some way or another copy Francois Chollet (Google researchers) code. Now my main ...
1
vote
0answers
54 views

Variational AutoEncoder giving negative loss

I'm learning about variational autoencoders and I've implemented a simple example in keras, model summary below. I've copied the loss function from one of Francois Chollet's blog posts and I'm getting ...
0
votes
1answer
414 views

Why use Variational Autoencoders VAE insted of Autoencoders AE in Anomaly Detection?

I have read many papers that recommends using Variational Autoencoders over Autoencoders since they have a more probabilistic approach and the ability to use KL divergence on the latent dimension. But ...
0
votes
0answers
28 views

Using VAE with Sequence to Sequence Approach

In the code below, I'm using a VAE with a seq-to-seq approach for translation. At the beginning I sarted only by using a simple seq-to-seq approach which implements a RNN-AE, until this step I had not ...
1
vote
0answers
24 views

Understanding ELBO Learning Dynamics for VAE?

As I understand it, I'm basically minimizing the KL Divergence between the Prior and Encoder latent distribution, and the log probability of the decoder distribution. I have a model that does generate ...
1
vote
0answers
15 views

Tensor Shape 'Reshape' during the dataflow

I would like to combine an Autoencoder with a LSTM. However, the 'timestep' is a block for the implements and I would like to train them together. Is there a solution to the tensor 'reshape'? I mean, ...
1
vote
0answers
41 views

Various Distances between probability distributions

I was reading about different distribution distance - and came across Kullback-Leibler divergence, Jensen-Shannon divergence, MMD, and Wasserstein distance - the book was too abstract for me to absorb....
2
votes
1answer
70 views

Variational auto-encoders (VAE): why the random sample?

Why do people train variational auto-encoders (VAE) to encode means and variances (regularised towards 0 and 1), and then sample a random Gaussian, rather that simply encode latent vectors and ...
0
votes
1answer
17 views

Help required to implement the below model using Bi-GRU

As you can see in above images I need to model Bi-GRUs stacked as shown in table which takes input (N,1,64) and outputs (N,204). The input data is binary number stream and so is output data. Can ...
0
votes
1answer
62 views

what is correct way to perform normalization on data in Auto encoder?

working on anomaly detection problem. i'm using auto-encoder to denoise given input. I trained network with normal data(anomaly free). so model predict normal state of given input. Normalization of ...
4
votes
2answers
125 views

Transform an Autoencoder to a Variational Autoencoder?

I want to compare between the training by an Autoencoder and a variational autoencoder. I have already run the traing using AE. I want to know if it's possible to transform this AE into a VAE and ...
0
votes
0answers
19 views

Encoder Decoder Network Image Compression

Could you train an encoder decoder network to take an image in and attempt to recreate that image as an output. I am basically interested at looking at the intermediate feature vector representation ...
1
vote
0answers
161 views

How to set the Reconstruction error threshold for anomaly detection using autoencoders?

Hi I am doing anomaly detection using auto encoders.I have trained the model using 'Non Anomalous' values.Now when I give anomalous points as test data. What should be the Reconstruction error ...
1
vote
1answer
23 views

Autoencoder Dimensionality Error

Thanks for taking a look! I have an auto-encoder that I am trying to use for anomaly detection. I have 2 log files, logfile.log and testfile.log. They're essentially the same logfile, I just split ...
1
vote
0answers
20 views

Train autoencoder on 1D data with high and low amplitudes

I'm using an autoencoder-RNN combination to predict colliding waves in 1D. The training data is obtained by using the 1D Euler equations along with boundary conditions to preduce to waves (left and ...
1
vote
1answer
21 views

Autoencoder doesn't learn to reduce dimesions

I coded a neural network from scratch in Python. I tried it with the XOR problem and it learned correctly. So I tried to encode an Autoencoder with 3 inputs (and therefore with also 3 outputs) to ...
1
vote
0answers
17 views

Any heuristic for minimal DCGAN latent space dimension?

I am highly interested in approaching minimal latent space dimension (as many other may be) for DCGANs or autoencoders. In this example of DCGAN on the MNIST dataset, the person uses a 100-...
0
votes
0answers
154 views

CNN for Keras Autoencoder

I'm working on my first Autoencoder with Keras. To do so, I followed these two tutorials: Version 1) https://blog.keras.io/building-autoencoders-in-keras.html Version 2) https://ramhiser.com/post/...
0
votes
0answers
14 views

How do you feed a VAE input layer a complex dataset form - an array with multiple sub-arrays?

I am currently trying to to pipe data into a simple VAE using Tensorflow but have encountered an issue. All the literature on VAEs are for images where tensors are typically squashed into 1D. The ...
1
vote
0answers
17 views

Perplexity calculation in variational neural topic models

I'm looking at this 2016 paper from Miao et al. https://arxiv.org/abs/1511.06038 where they use a variational autoencoder for topic modelling. To evaluate the effectiveness of their model, they use ...
0
votes
0answers
18 views

LSTM Autoencoder on Patterns of Labels

Currently, I am trying to do anomaly detection on univariate data consisting of labels. For example: [A, A, B, C] is good but [A, A, A, A] is anomalous. I'm dealing with more than just ABC. Is an LSTM ...
2
votes
0answers
50 views

Build an Autocomplete model for document titles

I want to build an autocomplete model using RNN where input is article names (documents title). X: ['Billing', 'Loan status', 'Filling loan application', 'Contact Info', ...] The article name can ...
0
votes
1answer
215 views

KL divergence in VAE

If I understand correctly KL-divergence is relative entropy of two distributions. To calculate KL divergence of two distributions, you would need two vectors of random variables. What I do not ...
1
vote
1answer
175 views

Training multiple keras models and combining outputs to determine losses

I'm trying to predict the future states of a 1D travelling wave (square, triangle and sawtooth) using a deep learning setup in Keras. The waves are discretised in a 1024 data points. As this gives a ...
1
vote
1answer
79 views

Training 128x128 autoencoders on 512x512 images, produces strange gridline after recombining

So I'm training an autoencoder that can recreate 128x128 images, so it can recreate any images by splitting them into 128x128 patches first, running it through the autoencoder, and having them ...
1
vote
0answers
103 views

What is the mathematical definition of the latent space [closed]

I have been seeking a mathematical explanation of the latent space of a neural network. The best I have gotten is "The latent space is the space in which the data lies in the bottleneck layer." ...
1
vote
1answer
44 views

Do anomalous input features to autoencoder result in high errors on the corresponding output features?

An autoencoder is trained by replicating each training instance to both input and output. However, when predicting for anomaly detection, will the output error be local to the same output feature(s) ...
0
votes
0answers
28 views

Encoder Decoder networks with varying image sizes

Encoder Decoder Network - Computerphile : At the very beginning of this video, Michael Pound goes on to say: So it (encoder decoder network) makes no assumptions about the size of the input the ...
3
votes
1answer
2k views

Validation loss is lower than the training loss

I am using autoencoder for anomaly detection in warranty data. Architecture 1: The plot shows the training vs validation loss based on Architecture 1. As we see in the plot, validation loss is ...
0
votes
2answers
511 views

How to use Autoencoders for outlier detection on images

I have a bunch of images taken from a camera showing a pipe and would like to detect if the pipe is leaking or not. There are very few examples of leaking pipe in the data set. So considering this ...
1
vote
1answer
37 views

Is train/test-Split in unsupervised learning of neural network necessary?

I am using autoencoder for anomaly detection in warranty data. It is unsupervised. I calculate the reconstruction error by the model and the records with high reconstruction error value is considered ...
1
vote
0answers
78 views

What is an intuitive explanation for the Importance Weighted Autoencoder?

I have been reading a paper by Burda et al. on Importance Weighted Autoencoders(IWAE) but I can't quite grasp what they mean by sampling the terms h1...hk. Do they mean you have separate models from ...
0
votes
1answer
42 views

Denoising Autoencoder Parameter Search

I have ran a hyperparameter search for a denoising autoencoder and the results suggest I should make the sizes of my hidden layers as large as possible (within the range of values I allowed it to ...
3
votes
1answer
361 views

What mu and sigma vector really mean in VAE?

In standard autoencoder, we encode data to bottleneck, then decode with using initial input as output to compute loss. We do activate matrix multiplication all over the network and if we are good, ...
3
votes
1answer
227 views

How can I prove bottleneck layer of my CNN auto encoder contain useful information?

I am using CNN autoencoder to create a state representation layer which I will later be feed into my Reinforcement Agent. So I trained my CNN autoencoder and it is giving nice state representations. ...