Questions tagged [autoencoder]
Autoencoders are a type of neural network that learns a useful encoding for data in an unsupervised manner.
207
questions
0
votes
0answers
15 views
LSTM autoencoder reconstructs input in ascending order
I implemented an autoencoder LSTM using Keras just as indicated in this article: article. The problem is that the reconstructed input of the time-series is given in ascending order with respect to the ...
0
votes
0answers
9 views
Differences between autoencoders and dynamic time warping?
While they work quite differently in terms of implementation, the end result for unsupervised learning is quite similar:
Dynamic time warping measures the distance between timeseries-like data (which ...
1
vote
1answer
21 views
What is the feedforward network in a transformer trained on?
After reading the 'Attention is all you need' article, I understand the general architecture of a transformer. However, it is unclear to me how the feed forward neural network learns.
What I learned ...
-1
votes
0answers
14 views
0
votes
0answers
21 views
Analogy between Autoencoder and PCA
I know that Autoencoders can be regarded as non-linear generalisations of PCA, but I struggle to understand in depth the analogy between the two.
Once PCA has been performed on a function $F(\vec{\...
0
votes
2answers
23 views
Auto-Encoder customized layer training
My question is related with model-weights optimization during back propagation.
In this image I'm trying to represent an auto-encoder having 7 layers where 4th one is center layer.
If my ...
1
vote
0answers
23 views
Keras Encoder - Decoder Model
Im training a model of 400 samples .
The dataset contains 400 images of faces as input (X) and also 400 faces with glasses as output (Y) .
im training the model by code below :
...
2
votes
0answers
16 views
Variational Autoencoder Latent Space size
Variational Autoencoder: Imagine we use a batch size of e.g. 32. Furthermore we got 2 Linear Layers (mu, sigma) which are 300 long. The output dimension of the encoder (conv2d layer) is (32, 64 , 64, ...
0
votes
0answers
40 views
Tensorflow2 graph tensor leaking when using layer.output
I am trying to build a Contractive Auto Encoder using Tensorflow 2.0. The model's loss function uses the encoder output in its calculations. The problem is that every time i retrieve the output, it ...
0
votes
0answers
31 views
Autoencoder for anomaly detection, output layer activation function
I am building an Autoencoder to detect anomalies. I have mixed data, i.e continuous and categorical. I have one-hot encoded the categorical data. Scaled the data with a MinMax scaler.
To determine if ...
0
votes
0answers
23 views
InvalidArgumentError: Incompatible shapes while training
InvalidArgumentError: Incompatible shapes: [15,31744] vs. [15,31680][[{{node loss_4/output_loss/logistic_loss/mul}}]]
Has anyone of you ever got this kind of ...
1
vote
2answers
41 views
How to train the machine so that it can give 'out of bound/classes' as an output for neural network
I know I was not able to word the title of the question properly. So I am trying to explain the problem here: Suppose, I built and trained a CNN to identify numbers from 0 to 9. However, when I ...
3
votes
0answers
62 views
Unable to transform (greatly performing) Autoencoder into Variational Autoencoder
Following the procedure described in this SO question, I am trying to transform my (greatly performing) convolutional Autoencoder into a Variational version of the same Autoencoder. As explained in ...
2
votes
0answers
24 views
Loss function for Autoencoder of sparse 3D Image
I have 3D structure data of molecules. I represented the atoms as points in a 100*100*100 grid and applied a gaussian blur to counter the sparseness. (nearly all of the grid cells contain zeros) I am ...
1
vote
0answers
36 views
Issue with implementation of Sequence to Sequence Autoencoder
Hi,
I want to implement Sequence to Sequence Autoencoder for text message classification purpose whether it is spam or ham.
The complete code is:
...
0
votes
0answers
15 views
How to correctly mirror / inverse an encoder architecture for the decoder?
For the purpose of autoencoders, It seems to be common practice to build the decoder architecture in a way so that it mirrors the encoder architecture. However, there seem to be 2 approaches of doing ...
1
vote
1answer
33 views
Dense output from neural network
I would like to create a loss function that encourages the output of the embedding of an autoencoder to be dense. I don't have an explicit condition for how density is defined, but one option would be ...
0
votes
2answers
33 views
What Non-linearities are best in Denoising RNN Autoencoders and where should the go?
I’m employing a denoising RNN autoencoder for a project relating to motion capture data. This is my first time using auto encoder architectures and I was just wondering what non-linearities should be ...
0
votes
0answers
9 views
Finding differences in smooth embedding spaces?
I'm trying to find statistically significant differences between embeddings (by an autoencoder) of different datasets.
At first I tried to embed them separately, and cluster them. However, the ...
1
vote
1answer
218 views
How to extract features from the encoded layer of an autoencoder?
I have done some research on autoencoders, and I have come to understand that they can also be used for feature extraction (see this question on this site as an example). Most of the examples out ...
0
votes
0answers
15 views
How do I use Conv2DTranspose to create my decoder?
I am constructing VAE of input 80x60.
I have my encoder below but I am troubles making the decoder as it does not conform to 80 x 60.
Here is my decoder.
Below is the code for constructing the ...
2
votes
0answers
18 views
Are Denoising Variational Autoencoders deterministic?
I have a pretty good understanding of regular autoencoders and, to a certain extent, of variational autoencoders, where the latent representation is forced to follow specific probabilistic ...
1
vote
1answer
37 views
How to use Variational Autoencoder's μ and σ with user-generated z?
My understanding of VAE is that unlike Autoencoders, it does not directly give you a discrete encoding (latent code vectors n-dim) instead, it gives you both mu and sigma (n- dim mean vectors and n-...
1
vote
1answer
68 views
Steps on how to use autoencoders to reduce dimensions
I have a dataset that contains text columns. I have used tf-idf to convert those text columns to numerical columns. I want to reduce the dimension of the dataset since tf-idf creates a multitude of ...
1
vote
0answers
18 views
How to estimate total correlation KL[q(z)||Πjq(zj)] of VAE after training (useful for latents disentanglement evaluation)
FactorVAE and β-TCVAE both use total correlation (TC) batch estimation for their objectives. Where TC is:
$$
KL\bigl( q(z)||\prod\nolimits_{j} q(z_{j})\bigr)
$$
both estimates are applied to $q(z|x)$...
0
votes
0answers
15 views
Do I need to train a separate DeepFake model for every input person?
I would like to create a deep fake model of a specific person (we will call him Steve). I would then like to be able to upload a video of any random person and swap their face with Steve's.
So far I ...
0
votes
0answers
14 views
What's the difference between encoder/decoder and autoencoder? [duplicate]
What's the difference between the encoder-decoder model and autoencoder model? Or are they the same thing? Thanks!
1
vote
0answers
30 views
chatbot encoder/decoder: why do we need to use chatbot answer as the decoder inputs?
I am looking into the chatbot tutorial at:
https://medium.com/predict/creating-a-chatbot-from-scratch-using-keras-and-tensorflow-59e8fc76be79
It uses sequence to sequence model with encoder/decoder ...
0
votes
0answers
17 views
Can I use/modify an Autoencoder to handle missing data?
I am about to implement an Autoencoder to detect anomalies. Therefore, e.g., in my test set, there is a situation where the data stream broke for some days. This results in a lack of data and should ...
2
votes
0answers
58 views
Results are too good.. what is wrong? How to predict correctly?
I am about to evaluate a neural network and want to check whether the predictions make sense.
The variables:
...
0
votes
0answers
26 views
normalization/standardization of input/output of autoencoder and Gaussian Process
I have two machine learning algorithms that deal with time series data.
My data consist of 1500 time series, each of 500 time components.
The first machine learning algorithm is an autoencoder, ...
0
votes
0answers
23 views
Looking for a good text autoencoder model (for text reconstruction)
I am looking for a simple and good model that can learn to encode and reconstruct text sentences to use it in some downstream task. I tried a tensorflow seq2seq model here, but it doesn't do ...
0
votes
0answers
11 views
Removing layers from a convolutional encoding-decoding network
I've been reading this paper on Style-transfer (Universal Style Transfer via Feature Transforms):
A crucial part of the algorithm (Section 3.3) uses a pre-trained VGG-19 network as an encoder to ...
0
votes
1answer
54 views
Validation generator in Autoencoder returning NaN
I am trying to build a fairly simple autoencoder using Keras on the OpenImages dataset. Here is the architecture of the ae:
...
0
votes
1answer
53 views
Keras - Autoencoder different from Encoder + Decoder
I build a CNN 1d Autoencoder in Keras, following the advice in this SO question, where Encoder and Decoder are separated. My goal is to re-use the decoder, once the Autoencoder has been trained. The ...
0
votes
0answers
14 views
VAE latent space dimension with Weibull PDFs as input data
I am currently using a Variational Autoencoder to reconstruct PDFs of Weibull distributions with varying parameters (sampled uniformly from a given parameter span). The PDFs are generated by sampling ...
0
votes
1answer
35 views
why do we operate with graphical models in VAE, if there are no probabilites involved?
In the variational autoencoder, I often see graphical models e.g. $P(X|Z)$ for the decoder, but when I looked at code, I don't see any random variables, I see just deterministic network, with ...
0
votes
1answer
14 views
how to design sigma(std) vector to have values above > 0 in VAE autoencoder
I am trying to understand how to design mu and sigma vector in VAE autoencoder, as we know std value can not be larger than 0. If I use linear function, the value can be lower than zero, so I think ...
2
votes
1answer
430 views
Autoencoders for the compression of time series
I am trying to use autoencoder (simple, convolutional, LSTM) to compress time series.
Here are the models I tried.
Simple autoencoder:
...
0
votes
0answers
193 views
How can we use resnet in autoencoders?
I am creating an unsupervised classifier model, for which i want to use resnet 50 on a custom database and used the top layers of resnet as start point of my autoencoder. How to proceed with this. ...
0
votes
0answers
27 views
How to measure performance of a denoising autoencoder?
Is there a way to theoretically estimate (avoiding brute force) what is the minimum signal-to-noise ratio required for a denoising autoencoder (like a regular filter)? What I want to know specifically ...
0
votes
0answers
26 views
Combining multiple loss functions in the right way
I am trying to build an (variational) Autoencoder to generate fake but representative data from a generic data set with a couple of numeric and categorical columns.
So far I have built functions that ...
0
votes
1answer
44 views
Autoencoder or layer-based dimensionality reduction?
I have a few TB of wide data. I want to reduce the number of features in my dataset before feeding my dataset into a classification model... or should I not?
Obviously, I will want to try both ...
0
votes
1answer
15 views
How to design n-dimensional feature descriptor similar as the input image?
I am re-writing the H-Net code in Keras for cross-domain image similarity. The network architecture is described in the attached paper. I wrote the encoder and decoder parts but unable to get similar ...
1
vote
1answer
551 views
Autoencoder: using cosine distance as loss function
I'm trying to train an autoencoder (in PyTorch) to reconstruct gene profiles. At the moment I'm using the Mean Squared Error (MSE) loss for training: the model is not overfitting and both the training ...
1
vote
0answers
19 views
Autoencoders linear latent space
According to "Linear interpolation in latent space" in https://hackernoon.com/latent-space-visualization-deep-learning-bits-2-bd09a46920df and others, the latent space representation of an autoencoder ...
1
vote
0answers
53 views
Disadvantages of Mean Squared Error? [closed]
I'm using mean squared error as reconstruction error for my autoencoder. The dataset is ECG (time series) and model is conv1d.
I assumed MSE as the best option for reconstruction error, but it's ...
7
votes
1answer
712 views
How does an encoder-decoder network work?
Let's say I trained an encoder-decoder network on a cat dataset using reconstruction error as loss function. The network is fully trained and the decoder is able to reconstruct good cat images.
Now ...
0
votes
1answer
269 views
Fully Convolutional Variational Autoencoder
I would like to make a neural network which uses black and white images as input and outputs a colored version of it. The important thing in that process is that the size of the images must stay the ...
0
votes
0answers
25 views
What is the meaning of constraint two neural networks against each other?
I have two identical autoencoders with the same number of layers and parameters. I would like to find the similarity between two images from different domains such as an image captured from the camera ...
