Questions tagged [autoencoder]

Autoencoders are a type of neural network that learns a useful encoding for data in an unsupervised manner.

157 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
1answer
3k views

Autoencoders for the compression of time series

I am trying to use autoencoder (simple, convolutional, LSTM) to compress time series. Here are the models I tried. Simple autoencoder: ...
3
votes
0answers
234 views

Chess deep learning siamese network overfitting when shouldn't in theory

TLDR: My network is training with pairs so instead of 10^6 samples it has 10^12 samples (The number of samples squared) . With that large of a data set is shouldn't overfit but it does after very few ...
3
votes
0answers
101 views

Unable to transform (greatly performing) Autoencoder into Variational Autoencoder

Following the procedure described in this SO question, I am trying to transform my (greatly performing) convolutional Autoencoder into a Variational version of the same Autoencoder. As explained in ...
3
votes
0answers
113 views

Encoder-Decoder Sequence-to-Sequence Model for Translations in Both Directions

Is it possible to use a pre-trained sequence to sequence encoder-decoder model which translates an input text in source language to an output in target language to do an inverse translation? That is, ...
3
votes
0answers
493 views

Autoencoder behavior with All White/Black MNIST

I am using a stock auto-encoder anomaly detector from Deeplearning4j. I was getting unexpected results from my own variant of the auto-encoder, which looks for anomalies in my own (non-image) data, ...
3
votes
0answers
571 views

Using an autoencoder to mimic independent component analysis?

I'm trying to use autoencoders in keras to create a linear transformation similar to independent component analysis (ICA) (using this to denoise electroencephalographic data, time series of 64x100000 ...
2
votes
2answers
135 views

How to interpreter Binary Cross Entropy loss function?

I saw some examples of Autoencoders (on images) which use sigmoid as output layer and BinaryCrossentropy as loss function. The ...
2
votes
0answers
39 views

Convolutional autoencoder - why keras example is asymmetry model?

I'm looking on keras convolutional autoencoder example, and confused with the model structure: ...
2
votes
0answers
21 views

Deep Continious Clustering algorithm - just one output cluster

I use the DCC algorithm to cluster some data. The whole algorithm is available here, but shortly it is: construct mkNN graph of the data points (the connected components of it are the clusters). ...
2
votes
1answer
677 views

How to Save Model that has a TensorFlow Probability Regularizer?

Consider the following minimal VAE: ...
2
votes
1answer
339 views

How to choose the good number dimension of autoencoder?

I'm using Autoencoder for feature extracting. I stuck with how to choose good number of dimension of encoder layer (latent layer). After training dataset, the model gave the latent layer (embedding ...
2
votes
0answers
87 views

Keras: DepthwiseConv3DTranspose or doing transposed Conv. with a Conv. layer

I am building an autoencoder for 3D images and would like to use Depthwise convolutions. For the encoder, I found an implementation of a depthwise 3D convolutional layer (DepthwiseConv3D). For the ...
2
votes
0answers
91 views

Variational Autoencoder Latent Space size

Variational Autoencoder: Imagine we use a batch size of e.g. 32. Furthermore we got 2 Linear Layers (mu, sigma) which are 300 long. The output dimension of the encoder (conv2d layer) is (32, 64 , 64, ...
2
votes
0answers
150 views

Loss function for Autoencoder of sparse 3D Image

I have 3D structure data of molecules. I represented the atoms as points in a 100*100*100 grid and applied a gaussian blur to counter the sparseness. (nearly all of the grid cells contain zeros) I am ...
2
votes
0answers
21 views

Are Denoising Variational Autoencoders deterministic?

I have a pretty good understanding of regular autoencoders and, to a certain extent, of variational autoencoders, where the latent representation is forced to follow specific probabilistic ...
2
votes
0answers
99 views

Results are too good.. what is wrong? How to predict correctly?

I am about to evaluate a neural network and want to check whether the predictions make sense. The variables: ...
2
votes
1answer
571 views

Keras - Autoencoder different from Encoder + Decoder

I build a CNN 1d Autoencoder in Keras, following the advice in this SO question, where Encoder and Decoder are separated. My goal is to re-use the decoder, once the Autoencoder has been trained. The ...
2
votes
0answers
116 views

Can autoencoders take time series into account?

Here, I read the following: The first key to understanding is that HTM relies on data that streams over time (...) By contrast, conventional deep learning uses static data and is therefore time ...
2
votes
0answers
370 views

How smaller does the input data get reduced in a LSTM autoencoder

Question In a LSTM autoencder, how smaller does my input data(59 features) get reduced in a latent vector, which is usually located in the middle between an encoder and a decoder? Why did the ...
2
votes
1answer
4k views

IndexError: list index out of range

I'm implementing a sequence-2-sequence model with RNN-VAE architecture, and I use an attention mechanism. I have problem in the decoder part. I'm struggling with this error: IndexError: list index ...
2
votes
1answer
2k views

Variational AutoEncoder giving negative loss

I'm learning about variational autoencoders and I've implemented a simple example in keras, model summary below. I've copied the loss function from one of Francois Chollet's blog posts and I'm getting ...
2
votes
0answers
204 views

What is the difference between KL-divergence, JS-divergence, Wasserstein distance and MMD?

I was reading about different distribution distances, and came across Kullback-Leibler divergence Jensen-Shannon divergence Wasserstein distance Maximum mean discrepancy (MMD) The book was too ...
2
votes
0answers
75 views

Build an Autocomplete model for document titles

I want to build an autocomplete model using RNN where input is article names (documents title). X: ['Billing', 'Loan status', 'Filling loan application', 'Contact Info', ...] The article name can ...
2
votes
0answers
118 views

What is an intuitive explanation for the Importance Weighted Autoencoder?

I have been reading a paper by Burda et al. on Importance Weighted Autoencoders(IWAE) but I can't quite grasp what they mean by sampling the terms h1...hk. Do they mean you have separate models from ...
2
votes
1answer
3k views

Variational Autoencoder TIme Series

Can anyone suggest a blog where Variational Autoencoder has been used for time series forecasting?
2
votes
0answers
745 views

How to scale data for LSTM autoencoder?

I am working on an LSTM autoencoder in keras. The aim here is to obtain a latent space representation for the time sequences which I intend to use for clustering. My input sequences (each feature) ...
2
votes
0answers
2k views

Feature extraction using autoencoder and assigning sub-features to the classes

I have a dataset with N records and D numerical attributes belonign to C different classes. ...
2
votes
0answers
548 views

Is there any implementation of Recursive Auto Encoders in Tensorflow?

I am looking for the implementation of Recursive Auto Encoders (RAE) in tensor flow python. I want to model English sentence representations from a sequence to sequence neural network model. RAE is ...
1
vote
0answers
5 views

quereies related to autoencoder

i want to design an deep auto encoder after following keras tutorial. Input is a simple 2-dimensional image consists of 512 rows and 50 columns matrix My trial code is ...
1
vote
0answers
13 views

input of Auto-Encoder as a feature extraction for training is similar to data that we use later for a classification model?

I have a data set of images, for example, 200 images, I want to use Autoencoder as a feature compressor. I use for example 150 for train the autoencoder and 50 for evaluation. after train and evaluate ...
1
vote
1answer
34 views

Multioutput prediction using LSTM encoder decoder with Attention

(I am working on Jupter notebook with python version 3.6.12, running Tensorflow 2.4.0 version.) I have a dataset that consists of 5 input features and 3 output features (that requires to be predicted)....
1
vote
0answers
12 views

Deep autoencoder: validation loss doesn't change

I'm trying to understand autoencoders and reproduced some code from Keras documentation: ...
1
vote
0answers
34 views

Word embedding autoencoder

I'm trying to train a word embedding autoencoder, but it either doesn't train, or trains but doesn't make predictions. I know I'm doing something wrong, so any help is greatly appreciated. Here is my ...
1
vote
0answers
12 views

Trouble with anomaly/novelty detection (on microscale) - need easy practical guide with Keras

I am relatively new to the field of machine learning. However, I already have solved simple image classification tasks with Keras (for example building CNNs and classifying MNIST...). The rough deep ...
1
vote
1answer
41 views

Autoencoder not learning walk forward image transformation

I have a series of 15 frames with (60 rows x 50 columns). Over the course of those 15 frames, the moon moves from the top left to the bottom right. Data = https://github.com/aiqc/AIQC/tree/main/...
1
vote
0answers
14 views

If the input to the autoencoder is normalized, do we need to use sigmoid on the last layer?

According to: https://stackoverflow.com/questions/65307833/why-is-the-decoder-in-an-autoencoder-uses-a-sigmoid-on-the-last-layer The last layer activation function contains sigmoid in order to the ...
1
vote
0answers
160 views

Autoencoder: Size of out_backprop doesn't match computed

This question was asked before and non of the answered worked for, I have the code ...
1
vote
1answer
48 views

Anomaly detection using LSTM AutoEncoder

Having a sequence of 10 days of sensors events, and a true / false label, specifying if the sensor triggered an alert within the 10 days duration: sensor_id timestamp feature_1 feature_2 ...
1
vote
0answers
41 views

How to derive Evidence Lower Bound in the paper "Zero-Shot Text-to-Image Generation"?

Can someone share the derivation of Evidence Lower Bound in this paper ? Zero-Shot Text-to-Image Generation The overall procedure can be viewed as maximizing the evidence lower bound (ELB) (Kingma &...
1
vote
0answers
34 views

How does an autoencoder 'fill in the blanks' in the context of a recommender system?

My understanding is that an autoencoder takes an input, produces a lower dimensional representation of the input, which should explain the original features in the dataset, and then reconstructs the ...
1
vote
0answers
56 views

Is there any problem with the following Python+TF+Keras code for a custom loss function and network?

I am trying to code a custom loss function for variational autoencoder. I am not using mse for reconstruction loss since I am not learning p(x|z) ~ N(mu,I). Instead ...
1
vote
0answers
16 views

How does bottleneck layer reduce computations without compromising with performance?

I was reading an article explaining the google's inception model . There it was mentioned , that to reduce the number of computations , we use a bottleneck layer. But I was surprised , if the model ...
1
vote
0answers
13 views

Autoencoder for Extremely Sparse Data

I am attempting to train an autoencoder on data that is extremely sparse. Each datapoint is only zeros and ones and contains ~3% 1s. Being that the data is mostly zero the autoencoder learns to ...
1
vote
0answers
31 views

Sampling for the encoder part of the VAE

my question regards the code utilized to implement the sampling function in the encoder part of VAE. Supposing that we chose a latent dimension of 2. Before the latent representation, we have 4 ...
1
vote
1answer
115 views

Training a Variational Autoencoder (VAE) for Random Number Generation

I have a complicated 20-dimensional multi-modal distribution and consider training a VAE to learn an approximation of it using 2000 samples. But particularly, with the aim to subsequently generate ...
1
vote
0answers
86 views

Autoencoder fails to reconstruct

I'm trying to use an autoencoder to reduce dimensionality of my features. My features are of dimension 2048. I tried to train an autoencoder to reduce the dimensionality to 50. I'm using a single ...
1
vote
0answers
221 views

Data augmentation for recommendation systems

I have a user-item matrix that I use to train a denoising autoencoder to predict the top-k items to recommend to the different users. The idea is to corrupt the matrix by erasing a percentage ...
1
vote
0answers
25 views

Using Iterative Hard/Soft Thresholding in autoencoder with non linear activation

Can someone please give an intuitive explanation of the difference between the Iterative Hard Thresholding VS Iterative Soft thresholding algorithm? And if we can use these algorithms in an ...
1
vote
2answers
204 views

MNIST data shape

In going through the different tutorials on CNN, autoencoders, and so on I trained myself on the MNIST problem. The different images are stored in a 3D array which shape is (60000,28,28). In some ...
1
vote
0answers
56 views

How to pass noisy images as input and original images as labels in Keras - Autoencoders

I want to make denoising autoencoder, I've added some noise to images, and i want to use them for training, original images will be used as label. Reading the keras documentation I've found that I ...