Questions tagged [autoencoder]
Autoencoders are a type of neural network that learns a useful encoding for data.
0
votes
0answers
4 views
Can I make it 'return _sequences = False' in autoencoder, using Keras?
Question
Should we use ‘return_sequences=True’ for all the LSTM layers in the encoder of the LSTM autoencoder?
My questions are based on the article LSTM Autoencoder for Extreme Rare Event ...
1
vote
0answers
11 views
How smaller does the input data get reduced in a LSTM autoencoder
Question
In a LSTM autoencder, how smaller does my input data(59 features) get reduced in a latent vector, which is usually located in the middle between an encoder and a decoder?
Why did the ...
1
vote
0answers
27 views
Why maximize ELBO in the variational autoencoder?
For a variational autoencoder, we have that:
$$\mathcal{L}(x,\theta,\phi) := \mathbb{E}_{q_\phi(z|x)}[\log p_{\theta}(x|z)] -KL[q_{\phi}(z|x) ||p(z)] $$
This is called the variational lower bound or ...
0
votes
1answer
14 views
Large scale Autoencoder for friendship recommendation
I have a user friendship graph for about 30 Million users. I am trying to use an auto encoder [30 Million, 512, 512, 1024, Dropout(0.3), 512, 512, 30 Million]. But I am not learning anything.
Has ...
0
votes
0answers
17 views
How to check quality of latent space like in β-VAE article?
There is a nice plot in the β-VAE article that shows quality of latent space code:
Is there a general way to visualize or analyze latent space code dimensions so that is would be clear if they are ...
0
votes
0answers
14 views
Why does Logistic Regression perform better than Autoencoders when classifying imbalanced data?
The 'shuttle' data can be downloaded from the link here.
It is imbalanced data and there are two classes in the target variable. The proportion of the two classes are seven percent. I used Logistic ...
0
votes
0answers
19 views
IndexError: list index out of range
I'm implementing a sequence-2-sequence model with RNN-VAE architecture, and I use an attention mechanism. I have problem in the decoder part.
I'm struggling with this error: IndexError: list index ...
0
votes
0answers
17 views
Autoencoder and Dimensionality
I'm pretty confused with the input/output/dense portion of an autoencoder.
So my data consists of a numpy array of a 9 categorical features which have all been one hot encoded.
So the input would ...
0
votes
2answers
20 views
Shouldn't an autoencoder with #(neurons in hidden layer) = #(neurons in input layer) be “perfect”?
I'm exploring autoencoders for the first time. I'm using the Matlab neural networks toolbox. I have created a synthetic dataset consisting of points in 2D space plus some noise. My idea was to ...
1
vote
0answers
44 views
What is the best architecture for Auto-Encoder for image reconstruction?
I am trying to use Convultional Auto-Encoder for its latent space (embedding layer), specifically, I want to use the embedding for K-nearest neighbor search in the latent space (similar idea to ...
0
votes
0answers
9 views
Variational auto-encoders for text generation
How are the variational auto-encoders used for text generation? Can variational auto-encoders be used for character based text generation?
1
vote
1answer
76 views
How to explain get_weight with autoencoder in keras?
I built an autoencoder model of three layers with 9 5 9.
Input dim =9, encoder dim =5, output dim=9
When I get the model weights,
...
2
votes
1answer
52 views
What do we visualize in showing a VAE latent space?
I am trying to wrap my head around VAE's and have trouble understanding what is being visualized when people make scatter plots of the latent space. I think I understand the bottleneck concept; we go ...
0
votes
0answers
43 views
InvalidArgumentError: incompatible shapes: [32,153] vs [32,5] , when using VAE
I'm working on a sequence to sequence model using LSTM, the model worked perfectly with an autoencoder, but when I try to use a Variational autoencoder by adding the mean and deviation layer and ...
1
vote
1answer
108 views
What is “posterior collapse” phenomenon?
I was going through this paper on Towards Text Generation with Adversarially Learned
Neural Outlines and it states why the VAEs are hard to train for text generation due to this problem. The paper ...
0
votes
0answers
18 views
How is the standard deviation of VAE's obtained?
I am trying to build a Variational Autoencoder. I was looking at various codes online and found most of them in some way or another copy Francois Chollet (Google researchers) code.
Now my main ...
1
vote
0answers
54 views
Variational AutoEncoder giving negative loss
I'm learning about variational autoencoders and I've implemented a simple example in keras, model summary below. I've copied the loss function from one of Francois Chollet's blog posts and I'm getting ...
0
votes
1answer
414 views
Why use Variational Autoencoders VAE insted of Autoencoders AE in Anomaly Detection?
I have read many papers that recommends using Variational Autoencoders over Autoencoders since they have a more probabilistic approach and the ability to use KL divergence on the latent dimension. But ...
0
votes
0answers
28 views
Using VAE with Sequence to Sequence Approach
In the code below, I'm using a VAE with a seq-to-seq approach for translation. At the beginning I sarted only by using a simple seq-to-seq approach which implements a RNN-AE, until this step I had not ...
1
vote
0answers
24 views
Understanding ELBO Learning Dynamics for VAE?
As I understand it, I'm basically minimizing the KL Divergence between the Prior and Encoder latent distribution, and the log probability of the decoder distribution. I have a model that does generate ...
1
vote
0answers
15 views
Tensor Shape 'Reshape' during the dataflow
I would like to combine an Autoencoder with a LSTM. However, the 'timestep' is a block for the implements and I would like to train them together. Is there a solution to the tensor 'reshape'? I mean, ...
1
vote
0answers
41 views
Various Distances between probability distributions
I was reading about different distribution distance - and came across Kullback-Leibler divergence, Jensen-Shannon divergence, MMD, and Wasserstein distance - the book was too abstract for me to absorb....
2
votes
1answer
70 views
Variational auto-encoders (VAE): why the random sample?
Why do people train variational auto-encoders (VAE) to encode means and variances (regularised towards 0 and 1), and then sample a random Gaussian, rather that simply encode latent vectors and ...
0
votes
1answer
17 views
Help required to implement the below model using Bi-GRU
As you can see in above images I need to model Bi-GRUs stacked as shown in table which takes input (N,1,64) and outputs (N,204). The input data is binary number stream and so is output data. Can ...
0
votes
1answer
62 views
what is correct way to perform normalization on data in Auto encoder?
working on anomaly detection problem. i'm using auto-encoder to denoise given input. I trained network with normal data(anomaly free). so model predict normal state of given input. Normalization of ...
4
votes
2answers
125 views
Transform an Autoencoder to a Variational Autoencoder?
I want to compare between the training by an Autoencoder and a variational autoencoder. I have already run the traing using AE. I want to know if it's possible to transform this AE into a VAE and ...
0
votes
0answers
19 views
Encoder Decoder Network Image Compression
Could you train an encoder decoder network to take an image in and attempt to recreate that image as an output. I am basically interested at looking at the intermediate feature vector representation ...
1
vote
0answers
161 views
How to set the Reconstruction error threshold for anomaly detection using autoencoders?
Hi I am doing anomaly detection using auto encoders.I have trained the model using 'Non Anomalous' values.Now when I give anomalous points as test data.
What should be the Reconstruction error ...
1
vote
1answer
23 views
Autoencoder Dimensionality Error
Thanks for taking a look!
I have an auto-encoder that I am trying to use for anomaly detection. I have 2 log files, logfile.log and testfile.log. They're essentially the same logfile, I just split ...
1
vote
0answers
20 views
Train autoencoder on 1D data with high and low amplitudes
I'm using an autoencoder-RNN combination to predict colliding waves in 1D. The training data is obtained by using the 1D Euler equations along with boundary conditions to preduce to waves (left and ...
1
vote
1answer
21 views
Autoencoder doesn't learn to reduce dimesions
I coded a neural network from scratch in Python. I tried it with the XOR problem and it learned correctly. So I tried to encode an Autoencoder with 3 inputs (and therefore with also 3 outputs) to ...
1
vote
0answers
17 views
Any heuristic for minimal DCGAN latent space dimension?
I am highly interested in approaching minimal latent space dimension (as many other may be) for DCGANs or autoencoders.
In this example of DCGAN on the MNIST dataset, the person uses a 100-...
0
votes
0answers
154 views
CNN for Keras Autoencoder
I'm working on my first Autoencoder with Keras. To do so, I followed these two tutorials:
Version 1) https://blog.keras.io/building-autoencoders-in-keras.html
Version 2) https://ramhiser.com/post/...
0
votes
0answers
14 views
How do you feed a VAE input layer a complex dataset form - an array with multiple sub-arrays?
I am currently trying to to pipe data into a simple VAE using Tensorflow but have encountered an issue. All the literature on VAEs are for images where tensors are typically squashed into 1D.
The ...
1
vote
0answers
17 views
Perplexity calculation in variational neural topic models
I'm looking at this 2016 paper from Miao et al. https://arxiv.org/abs/1511.06038 where they use a variational autoencoder for topic modelling.
To evaluate the effectiveness of their model, they use ...
0
votes
0answers
18 views
LSTM Autoencoder on Patterns of Labels
Currently, I am trying to do anomaly detection on univariate data consisting of labels. For example: [A, A, B, C] is good but [A, A, A, A] is anomalous. I'm dealing with more than just ABC. Is an LSTM ...
2
votes
0answers
50 views
Build an Autocomplete model for document titles
I want to build an autocomplete model using RNN where input is article names (documents title).
X: ['Billing', 'Loan status', 'Filling loan application', 'Contact Info', ...]
The article name can ...
0
votes
1answer
215 views
KL divergence in VAE
If I understand correctly KL-divergence is relative entropy of two distributions. To calculate KL divergence of two distributions, you would need two vectors of random variables.
What I do not ...
1
vote
1answer
175 views
Training multiple keras models and combining outputs to determine losses
I'm trying to predict the future states of a 1D travelling wave (square, triangle and sawtooth) using a deep learning setup in Keras. The waves are discretised in a 1024 data points. As this gives a ...
1
vote
1answer
79 views
Training 128x128 autoencoders on 512x512 images, produces strange gridline after recombining
So I'm training an autoencoder that can recreate 128x128 images, so it can recreate any images by splitting them into 128x128 patches first, running it through the autoencoder, and having them ...
1
vote
0answers
103 views
What is the mathematical definition of the latent space [closed]
I have been seeking a mathematical explanation of the latent space of a neural network. The best I have gotten is "The latent space is the space in which the data lies in the bottleneck layer." ...
1
vote
1answer
44 views
Do anomalous input features to autoencoder result in high errors on the corresponding output features?
An autoencoder is trained by replicating each training instance to both input and output. However, when predicting for anomaly detection, will the output error be local to the same output feature(s) ...
0
votes
0answers
28 views
Encoder Decoder networks with varying image sizes
Encoder Decoder Network - Computerphile : At the very beginning of this video, Michael Pound goes on to say:
So it (encoder decoder network) makes no assumptions about the size of the input the ...
3
votes
1answer
2k views
Validation loss is lower than the training loss
I am using autoencoder for anomaly detection in warranty data.
Architecture 1:
The plot shows the training vs validation loss based on Architecture 1.
As we see in the plot, validation loss is ...
0
votes
2answers
511 views
How to use Autoencoders for outlier detection on images
I have a bunch of images taken from a camera showing a pipe and would like to detect if the pipe is leaking or not. There are very few examples of leaking pipe in the data set. So considering this ...
1
vote
1answer
37 views
Is train/test-Split in unsupervised learning of neural network necessary?
I am using autoencoder for anomaly detection in warranty data. It is unsupervised. I calculate the reconstruction error by the model and the records with high reconstruction error value is considered ...
1
vote
0answers
78 views
What is an intuitive explanation for the Importance Weighted Autoencoder?
I have been reading a paper by Burda et al. on Importance Weighted Autoencoders(IWAE) but I can't quite grasp what they mean by sampling the terms h1...hk. Do they mean you have separate models from ...
0
votes
1answer
42 views
Denoising Autoencoder Parameter Search
I have ran a hyperparameter search for a denoising autoencoder and the results suggest I should make the sizes of my hidden layers as large as possible (within the range of values I allowed it to ...
3
votes
1answer
361 views
What mu and sigma vector really mean in VAE?
In standard autoencoder, we encode data to bottleneck, then decode with using initial input as output to compute loss. We do activate matrix multiplication all over the network and if we are good, ...
3
votes
1answer
227 views
How can I prove bottleneck layer of my CNN auto encoder contain useful information?
I am using CNN autoencoder to create a state representation layer which I will later be feed into my Reinforcement Agent. So I trained my CNN autoencoder and it is giving nice state representations. ...
