Questions tagged [autoencoder]
Autoencoders are a type of neural network that learns a useful encoding for data in an unsupervised manner.
335
questions
1
vote
1
answer
46
views
Unsupervised Machine Translation System Using Variational Autoencoder Models
I want to work on an unsupervised machine translation system using a variational autoencoder. I did a literature review but didn't find any related work, and most of the work is based on denoising ...
1
vote
1
answer
20
views
numpy in call method: how to run without eager execution?
I wrote an implementation of a feedback recurrent autoencoder in Keras. The key difference to a regular autoencoder is, that the decoded output is fed back to the input layers of both, encoder and ...
- 21
0
votes
0
answers
33
views
How to work with multiple feature types on autoencoder?
This is my first post here.
I am working on an adversarial autoencoder that receives different features, encodes them, and decodes them.
For instance, suppose you have a dataset from a large survey ...
0
votes
0
answers
42
views
How to detect rare events in Time series?
I am working on a time series dataset in which each time step can be classified under 4 classes:
~EOI : P(~EOI) = .85
...
- 125
0
votes
1
answer
11
views
Does the input data representation when training an autoencoder matter?
Suppose I want to train an autoencoder on English words. Does it matter what kind of input data representation I use? E.g., if I use word2vec as my input or bag of words, will the quality of my ...
- 101
0
votes
0
answers
37
views
Understanding the desired behavior of the loss function of Variational Autoencoders
So I understood that when training VAE, we need to weight the KL part of the loss with a weight less than 1 so that the reconstruction loss can get a chance to learn (avoiding the posterior collapse).
...
0
votes
0
answers
43
views
Modelling LSTM Autoencoder for anomaly detection with multiple Time Series
I'm currently working on LSTM autoencoder for anomaly detection. My main problem is I have multiple time series - each individual time series corresponds to a different customer, detailing their sales ...
- 1
0
votes
0
answers
9
views
Architectures for continuous vector labels prediction
I have a small dataset of feature vectors of size 200 to each corresponds a larger vector of size 1000. I would like to "predict" a large vector for every small one. The task sounds like ...
1
vote
0
answers
27
views
Confusion over taking gradients in Variational Autoencoder (VAE)
I am confused as to when to hold certain parameters constant in a VAE. I will explain with a concrete example.
We can write $\operatorname{ELBO}(\phi, \theta) = \mathbb{E}_{q_{\phi}(z)}\left[\log
\...
- 11
0
votes
1
answer
73
views
How does a VQ-VAE produce new images?
I'm implementing a VQ-VAE for a LDM for biological time series data. I trained the VQ-VAE, and reconstructions works somewhat reasonable, but I have an understanding problem with how a VQ-VAE works.
...
- 11
0
votes
1
answer
73
views
Constructing an LSTM autoencoder for variable-lentgth sequences
I would like to construct an LSTM autoencoder model for sequence anomaly detection where the sequences can be varying in length. I understand based on this answer that padding and masking can be used ...
- 123
0
votes
0
answers
69
views
Seq2seq Transformer Autoencoder returns same results when unfolded using only memory and initial value
I have numeric signals from two sensors, and I would like to create a mapping using sequence-to-sequence autoencoder. I used the transformer architecture, and it seems to be learning - the loss is ...
- 151
0
votes
0
answers
16
views
Binary latent representation
I've been working on a problem where I got stuck at encoding my data into a binary latent representation, most of the methods out there aren't really working for my case.
I have input for the encoder ...
0
votes
0
answers
22
views
Using Autoencoder in Python for Data Selection and Risk Level Calculation
I have a dataset that includes 100 data points from each sensor, representing various measurements. These measurements can be used to calculate the level of risk associated with each sensor. However, ...
- 1
3
votes
2
answers
193
views
How does variational autoencoders actually work in comparison to GAN?
I want to know about how variational autoencoders work. I am currently working in a company and we want to incorporate variational autoencoders for creating synthetic data. I have questions regarding ...
- 47
0
votes
0
answers
16
views
Multiagent observations encoding - LSTM or Transformer?
In multi agent learning problems where agents can join or leave, to address the issue of varying observation space I am trying to build a latent representation either using LSTM or transformer which ...
0
votes
0
answers
13
views
How can I ask a conditional auto encoder to put more emphasis on the condition without setting epochs too high?
Here's an example of what I'm testing on. I have the mnist digits data and I randomly add a random number from 1 to 10 to each digit and divide it by 11. I do this because I wanted to test out the ...
- 113
0
votes
0
answers
15
views
Counting Strikes from Dynamic Probing: Reliable Impulse Detection
Im looking for general guidance and advice to solve a specific application problem.
Let me explain the contect briefly: In geotechnics, there is a method called "dynamic probing" to examine ...
0
votes
0
answers
21
views
LSTM Autoencoder Failing to Capture Amplitudes of Input Signals
I'm trying to create an LSTM autoencoder to encode some high frequency time series data. The data has 206 time steps and I'm trying to both encode this down to a lower dimension and also recreate the ...
- 1
0
votes
0
answers
36
views
Minimizing the difference between two distributions with TensorFlow
Suppose that the encoding neural network of a variational autoencoder (VAE) outputs a distribution from which latent samples will be drawn. To do this, the layer ...
0
votes
0
answers
183
views
Transformer-based autoencoder generates same output and bad embeddings
I am trying to implement the transformer-based autoencoder presented in this paper: https://arxiv.org/abs/2210.08288
The paper seems rather vague to me and I do not fully understand how the model is ...
0
votes
0
answers
188
views
Incompatible Shape Error [32, 150,150,3] vs [32] for an Autoencoder
Please Help, I am trying to make an Autoencoder from a dataset of 533,248 images but I can't seem to get the shape right when I go to fit my model. The code I have is below.
...
0
votes
0
answers
26
views
Pixelated output of an autoencoder network
I have an autoencder that encodes an input size of (76, 400, 1) in a 2D convolutional layer, and decodes it an output size of (125, 400, 1). Both downsampling and upsampling are performed using a '...
0
votes
1
answer
86
views
Deepstack autoencoder
I'm trying to go through the first edition tabular challenge on Kaggle. Obviously my first few trials results did not satisfy me, so I went to see how other people did, and in the post of the first ...
- 63
1
vote
0
answers
18
views
Generating artificial training data with encoder and classical algorithm
I would like to know if this idea has been tried before, and if so, where I can find more information about it.
This is an approach to generating artificial training data for segmentation tasks using ...
- 11
0
votes
0
answers
51
views
autoencoder feature extraction
I'm very new to deep learning and come across an idea of feature extraction by using autoencoder. I went through many example online and came across these following.
https://www.thepythoncode.com/...
0
votes
0
answers
13
views
Larger Latent Space Dim for Point Cloud Autoencoder
So I'm trying to follow a paper that uses a AE to learn point clouds. The thing is, the dimension of the point cloud data is 3 (x, y, z), but the dimension of the latent space from what I can tell is ...
- 101
1
vote
0
answers
64
views
Threshold for auto encoder anomaly detection
I have fitted an auto encoder on 25-dimensional time series data hoping to be able to detect anomalies. training set is 100k observations, testset for threshold setting is 10k observations. all ...
- 21
0
votes
0
answers
28
views
Cluster-Based Anomaly Detection (and PCA)
I have a dataset of user operations (250 types of operations) in a trading platform and my task is to extract features, flagging rules, other insight for fraud prevention/anomaly detection, or some ...
- 1
0
votes
1
answer
17
views
How can I show data clustering after building a Convolutional Variational Autoencoder?
I am brand new to Machine Learning, and just followed Tensorflow Convolutional Variational Autoencoder Tutorial using my own dataset instead of the MNIST set. I was able to successfully display the 2D ...
2
votes
1
answer
182
views
Early anomaly detection / Failure prediction on time series
My problem here is that I want to predict failures in advance with respect to their occurrence. I have sensors mounted on my machine and with a certain frequency, they send data to my database. ...
- 45
0
votes
1
answer
212
views
What is the dataset during testing a Variational auto-encoder?
I am getting confused in the testing dataset of a VAE. After training the VAE, what should be the testing data-set of the VAE?
I understand that during testing the VAE only has the decoder part. Hence,...
- 103
0
votes
0
answers
48
views
Measuring the "smoothness" of the latent space of a (variational) autoencoder
Suppose I have an autoencoder (or variational autoencoder) that encodes language.
I want to compare different training regimes and evaluate the AE / VAE on how "smooth" its latent space is. ...
1
vote
1
answer
117
views
Autoencoder: How should hidden layer be used?
I'm building a variational autoencoder to generate faces. I'm using gray-scale images with the size 30x30. I started with a very simple model:
Input Layer, 900 nodes, values 0-1
Latent Space, 10 nodes ...
- 11
0
votes
0
answers
32
views
Feature extraction using autoencoder produces latent vector with large elements
Good day.
I am experimenting with a convolution autoencoder, represented below, for feature extraction. The latent vectors have large numbers, > 10 and < -10, that cause problems for later ...
1
vote
0
answers
29
views
Python train convolutional on numerical values shape issue
I want to train a convolutional neural network autoencoder on a csv file which contains values pixel neighborhood position of an original image of 1024x1024. When I try to train it, I have the ...
- 137
1
vote
1
answer
54
views
Why VAE Encoder outputs log variance and not standard deviation?
When talking about VAE (and viewing VAE implementations), the Encoder outputs:
μ, log(variance)
when we train the model (the ...
- 363
0
votes
1
answer
170
views
Tips for making an autoencoder converge?
I am currently trying to make an autoencoder that compresses a 3D volume where each value represents the density of said volume. The architecture is a UNet without skip connections. The optimizer is ...
- 101
0
votes
1
answer
45
views
How to train encoder in BiGAN?
I have some difficulties training a BiGAN. In particular, the encoder seems not learning the map between the images x and the latent space z.
I have the following encoder:
...
- 1
0
votes
0
answers
68
views
Is it possible to use text Auto Encoders without text generation?
I have a use case where I have large texts, and a lot of it. Pretty often the sequence length exceeds 1000 tokens. I need a lower dimensional compression of the texts as an input for a classifier. The ...
- 101
0
votes
2
answers
367
views
Autoencoder vs Pre-trained network for feature extraction
I wanted to know if anyone has any sort of guidance on what is better for image classification on a lot of classes (about 400) with a small amount of samples per class (around 20), for relatively big ...
0
votes
0
answers
54
views
Practical application of denoising autoencoders
I have been reading into autoencoders for the purpose of denoising data. In the examples i found (eg. [1, 2, 3], which are the first few google results) they have the following input/output:
Input ...
0
votes
1
answer
69
views
Questions on GLM: General Language Model Pretraining with Autoregressive Blank Infilling
For GLM: General Language Model Pretraining with Autoregressive Blank Infilling ,
May I ask how is the sampling for input division in step (b) being done ?
why in step (c), the green ...
- 283
0
votes
1
answer
20
views
What says the output of autoencoder?
What is the meaning of output of autuencoders? Can we say it is the noise removed version of actual dataset and should it be symmetrical?
- 11
1
vote
0
answers
43
views
How can I use autoencoders for noise detection and removal
How can I use autoencoders for noise detection and removal in a dataset with only 2 features and no labels? How should my architecture be like, such as 2 1 1 1 2 or any other? And does the output of ...
- 11
0
votes
1
answer
64
views
Trimming "unused" neurons from the bottleneck of an autoencoder
I'm working with autoencoding data in segments, and working with the latent space afterwards (I am also working on VAEs, but this segment of the project concerns deterministic AEs).
I've noticed that ...
- 101
1
vote
0
answers
92
views
How to implement simple VAE with sparse tensor in Tensorflow
thanks for reading.
I have been attempting to train a simple VAE on very sparse 2D and 3D data. So far I have been training using dense tensors which - I think - is resulting in horrible training due ...
- 11
1
vote
0
answers
40
views
Training a straight "copy the input to the output" autoencoder for audio is strangely slow
As a learning exercise, I'm training a "perfect" audio autoencoder. It has a hidden layer just as wide as the input layer, with linear activation. The expectation is that the network should ...
1
vote
0
answers
28
views
Deep learning model to predict the actual input values
I have some observed parameters to be used as input to the deep learning model. This problem comes from the wireless field where we transmit $x$. The channel $h$ is random in nature. The received ...
- 11
0
votes
1
answer
53
views
Autoencoder general questions and poor loss
I'm trying to get a simple autoencoder working on the iris dataset to explore autoencoders at a basic level. I'm running into an issue where the loss of the model is extremely high (>20).
Can ...
