Questions tagged [autoencoder]

Autoencoders are a type of neural network that learns a useful encoding for data in an unsupervised manner.

164 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6 votes
0 answers
142 views

Unable to transform (greatly performing) Autoencoder into Variational Autoencoder

Following the procedure described in this SO question, I am trying to transform my (greatly performing) convolutional Autoencoder into a Variational version of the same Autoencoder. As explained in ...
user87590's user avatar
4 votes
2 answers
4k views

Autoencoders for the compression of time series

I am trying to use autoencoder (simple, convolutional, LSTM) to compress time series. Here are the models I tried. Simple autoencoder: ...
netang's user avatar
  • 41
3 votes
0 answers
541 views

Autoencoder: Size of out_backprop doesn't match computed

This question was asked before and non of the answered worked for, I have the code ...
fromSAS2Spark's user avatar
3 votes
0 answers
343 views

Chess deep learning siamese network overfitting when shouldn't in theory

TLDR: My network is training with pairs so instead of 10^6 samples it has 10^12 samples (The number of samples squared) . With that large of a data set is shouldn't overfit but it does after very few ...
EXTORY's user avatar
  • 31
3 votes
0 answers
404 views

What is the difference between KL-divergence, JS-divergence, Wasserstein distance and MMD?

I was reading about different distribution distances, and came across Kullback-Leibler divergence Jensen-Shannon divergence Wasserstein distance Maximum mean discrepancy (MMD) The book was too ...
asahi kibou's user avatar
3 votes
0 answers
143 views

What is an intuitive explanation for the Importance Weighted Autoencoder?

I have been reading a paper by Burda et al. on Importance Weighted Autoencoders(IWAE) but I can't quite grasp what they mean by sampling the terms h1...hk. Do they mean you have separate models from ...
deZakelijke's user avatar
3 votes
0 answers
114 views

Encoder-Decoder Sequence-to-Sequence Model for Translations in Both Directions

Is it possible to use a pre-trained sequence to sequence encoder-decoder model which translates an input text in source language to an output in target language to do an inverse translation? That is, ...
Amir's user avatar
  • 193
3 votes
0 answers
675 views

Autoencoder behavior with All White/Black MNIST

I am using a stock auto-encoder anomaly detector from Deeplearning4j. I was getting unexpected results from my own variant of the auto-encoder, which looks for anomalies in my own (non-image) data, ...
Stevod's user avatar
  • 43
3 votes
0 answers
686 views

Using an autoencoder to mimic independent component analysis?

I'm trying to use autoencoders in keras to create a linear transformation similar to independent component analysis (ICA) (using this to denoise electroencephalographic data, time series of 64x100000 ...
Russell Butler's user avatar
2 votes
0 answers
120 views

Convolutional autoencoder - why keras example is asymmetry model?

I'm looking on keras convolutional autoencoder example, and confused with the model structure: ...
user3668129's user avatar
2 votes
0 answers
24 views

Deep Continious Clustering algorithm - just one output cluster

I use the DCC algorithm to cluster some data. The whole algorithm is available here, but shortly it is: construct mkNN graph of the data points (the connected components of it are the clusters). ...
Ilya.K.'s user avatar
  • 157
2 votes
1 answer
1k views

How to Save Model that has a TensorFlow Probability Regularizer?

Consider the following minimal VAE: ...
Parker Wieck's user avatar
2 votes
0 answers
223 views

Keras: DepthwiseConv3DTranspose or doing transposed Conv. with a Conv. layer

I am building an autoencoder for 3D images and would like to use Depthwise convolutions. For the encoder, I found an implementation of a depthwise 3D convolutional layer (DepthwiseConv3D). For the ...
PascalIv's user avatar
  • 433
2 votes
0 answers
177 views

Variational Autoencoder Latent Space size

Variational Autoencoder: Imagine we use a batch size of e.g. 32. Furthermore we got 2 Linear Layers (mu, sigma) which are 300 long. The output dimension of the encoder (conv2d layer) is (32, 64 , 64, ...
Shoorty's user avatar
  • 31
2 votes
0 answers
216 views

Loss function for Autoencoder of sparse 3D Image

I have 3D structure data of molecules. I represented the atoms as points in a 100*100*100 grid and applied a gaussian blur to counter the sparseness. (nearly all of the grid cells contain zeros) I am ...
PascalIv's user avatar
  • 433
2 votes
0 answers
33 views

Are Denoising Variational Autoencoders deterministic?

I have a pretty good understanding of regular autoencoders and, to a certain extent, of variational autoencoders, where the latent representation is forced to follow specific probabilistic ...
Victor Deleau's user avatar
2 votes
0 answers
139 views

Results are too good.. what is wrong? How to predict correctly?

I am about to evaluate a neural network and want to check whether the predictions make sense. The variables: ...
Ben's user avatar
  • 540
2 votes
1 answer
942 views

Keras - Autoencoder different from Encoder + Decoder

I build a CNN 1d Autoencoder in Keras, following the advice in this SO question, where Encoder and Decoder are separated. My goal is to re-use the decoder, once the Autoencoder has been trained. The ...
guest's user avatar
  • 21
2 votes
0 answers
129 views

Can autoencoders take time series into account?

Here, I read the following: The first key to understanding is that HTM relies on data that streams over time (...) By contrast, conventional deep learning uses static data and is therefore time ...
Ben's user avatar
  • 540
2 votes
0 answers
520 views

How smaller does the input data get reduced in a LSTM autoencoder

Question In a LSTM autoencder, how smaller does my input data(59 features) get reduced in a latent vector, which is usually located in the middle between an encoder and a decoder? Why did the ...
Eiffelbear's user avatar
2 votes
1 answer
4k views

IndexError: list index out of range

I'm implementing a sequence-2-sequence model with RNN-VAE architecture, and I use an attention mechanism. I have problem in the decoder part. I'm struggling with this error: IndexError: list index ...
Kahina's user avatar
  • 614
2 votes
1 answer
4k views

Variational AutoEncoder giving negative loss

I'm learning about variational autoencoders and I've implemented a simple example in keras, model summary below. I've copied the loss function from one of Francois Chollet's blog posts and I'm getting ...
Jed's user avatar
  • 141
2 votes
0 answers
78 views

Build an Autocomplete model for document titles

I want to build an autocomplete model using RNN where input is article names (documents title). X: ['Billing', 'Loan status', 'Filling loan application', 'Contact Info', ...] The article name can ...
quintin's user avatar
  • 121
2 votes
2 answers
3k views

Variational Autoencoder TIme Series

Can anyone suggest a blog where Variational Autoencoder has been used for time series forecasting?
Abrar Zahin's user avatar
2 votes
0 answers
863 views

How to scale data for LSTM autoencoder?

I am working on an LSTM autoencoder in keras. The aim here is to obtain a latent space representation for the time sequences which I intend to use for clustering. My input sequences (each feature) ...
aditya ramesh's user avatar
2 votes
0 answers
2k views

Feature extraction using autoencoder and assigning sub-features to the classes

I have a dataset with N records and D numerical attributes belonign to C different classes. ...
Mo-'s user avatar
  • 1,235
2 votes
0 answers
553 views

Is there any implementation of Recursive Auto Encoders in Tensorflow?

I am looking for the implementation of Recursive Auto Encoders (RAE) in tensor flow python. I want to model English sentence representations from a sequence to sequence neural network model. RAE is ...
Nomiluks's user avatar
  • 461
1 vote
0 answers
32 views

Can unsupervised pretraining (autoencoders) be used for u-nets?

TLDR: Will a u-net pretrained as an autoencoder be able to learn a latent representation of the data if the encoder weights are frozen (can't game the system and pass forward the unmodified image)? ...
Gordon Chen's user avatar
1 vote
0 answers
48 views

Confusion over taking gradients in Variational Autoencoder (VAE)

I am confused as to when to hold certain parameters constant in a VAE. I will explain with a concrete example. We can write $\operatorname{ELBO}(\phi, \theta) = \mathbb{E}_{q_{\phi}(z)}\left[\log \...
Joel's user avatar
  • 11
1 vote
0 answers
18 views

Generating artificial training data with encoder and classical algorithm

I would like to know if this idea has been tried before, and if so, where I can find more information about it. This is an approach to generating artificial training data for segmentation tasks using ...
dave4422's user avatar
1 vote
0 answers
74 views

Threshold for auto encoder anomaly detection

I have fitted an auto encoder on 25-dimensional time series data hoping to be able to detect anomalies. training set is 100k observations, testset for threshold setting is 10k observations. all ...
Student's user avatar
  • 21
1 vote
1 answer
157 views

Autoencoder: How should hidden layer be used?

I'm building a variational autoencoder to generate faces. I'm using gray-scale images with the size 30x30. I started with a very simple model: Input Layer, 900 nodes, values 0-1 Latent Space, 10 nodes ...
lev1248's user avatar
  • 11
1 vote
0 answers
29 views

Python train convolutional on numerical values shape issue

I want to train a convolutional neural network autoencoder on a csv file which contains values pixel neighborhood position of an original image of 1024x1024. When I try to train it, I have the ...
user979974's user avatar
1 vote
1 answer
66 views

Why VAE Encoder outputs log variance and not standard deviation?

When talking about VAE (and viewing VAE implementations), the Encoder outputs: μ, log(variance) when we train the model (the ...
user3668129's user avatar
1 vote
0 answers
46 views

How can I use autoencoders for noise detection and removal

How can I use autoencoders for noise detection and removal in a dataset with only 2 features and no labels? How should my architecture be like, such as 2 1 1 1 2 or any other? And does the output of ...
MobiusT's user avatar
  • 11
1 vote
0 answers
112 views

How to implement simple VAE with sparse tensor in Tensorflow

thanks for reading. I have been attempting to train a simple VAE on very sparse 2D and 3D data. So far I have been training using dense tensors which - I think - is resulting in horrible training due ...
Zephrom's user avatar
  • 11
1 vote
0 answers
44 views

Training a straight "copy the input to the output" autoencoder for audio is strangely slow

As a learning exercise, I'm training a "perfect" audio autoencoder. It has a hidden layer just as wide as the input layer, with linear activation. The expectation is that the network should ...
Alexander Ljungberg's user avatar
1 vote
0 answers
28 views

Deep learning model to predict the actual input values

I have some observed parameters to be used as input to the deep learning model. This problem comes from the wireless field where we transmit $x$. The channel $h$ is random in nature. The received ...
Riva11's user avatar
  • 11
1 vote
0 answers
53 views

Does Sliced Wasserstein Distance work in higher than 2 dimensions?

I had thought that it only worked for 2D distributions. I am trying to implement a sliced Wasserstein autoencoder and I was wondering if my latent space can be larger than 2D.
nighthawk's user avatar
1 vote
0 answers
182 views

CNN auto-encoder performs much worse than Fully connected auto-encoder

I am trying to develop a one-class classifier, that will learn regular examples, and, hopefully, will have hard times reconstructing anomaly observations. I have 1D signals which I tried to ...
David Harar's user avatar
1 vote
0 answers
85 views

How should I think when I want to compare mu and sigma for different images in VAE?

I'm searching for a way to compare mu and sigma values of the encoder network's output of variational autoencoders. In detail, imagine I trained my VAE on the MNIST digits dataset using the official ...
BlackCode's user avatar
1 vote
0 answers
116 views

What does it means (concretly) that a VAE encode inputs as distribution?

From this post we can read that VAEs encode inputs as distributions instead of simple points ? What does it mean concretely ? If the encoder consists of the weights between the input image and the ...
RandomFellow's user avatar
1 vote
0 answers
5 views

quereies related to autoencoder

i want to design an deep auto encoder after following keras tutorial. Input is a simple 2-dimensional image consists of 512 rows and 50 columns matrix My trial code is ...
simond's user avatar
  • 73
1 vote
0 answers
57 views

input of Auto-Encoder as a feature extraction for training is similar to data that we use later for a classification model?

I have a data set of images, for example, 200 images, I want to use Autoencoder as a feature compressor. I use for example 150 for train the autoencoder and 50 for evaluation. after train and evaluate ...
Shiva Parsa Rad's user avatar
1 vote
0 answers
53 views

Deep autoencoder: validation loss doesn't change

I'm trying to understand autoencoders and reproduced some code from Keras documentation: ...
deethereal's user avatar
1 vote
0 answers
141 views

Word embedding autoencoder

I'm trying to train a word embedding autoencoder, but it either doesn't train, or trains but doesn't make predictions. I know I'm doing something wrong, so any help is greatly appreciated. Here is my ...
user1513335's user avatar
1 vote
0 answers
116 views

Trouble with anomaly/novelty detection (on microscale) - need easy practical guide with Keras

I am relatively new to the field of machine learning. However, I already have solved simple image classification tasks with Keras (for example building CNNs and classifying MNIST...). The rough deep ...
Patrick's user avatar
  • 11
1 vote
0 answers
260 views

Custom keras callbacks and changing weight (beta) of regularization term in variational autoencoder loss function

The variational autoencoder loss function is this: Loss = Loss_reconstruction + Beta * Loss_kld. I am trying to efficiently implement Kullback-Liebler Divergence Cyclic Annealing--that is changing the ...
Jared Frazier's user avatar
1 vote
1 answer
69 views

Autoencoder not learning walk forward image transformation

I have a series of 15 frames with (60 rows x 50 columns). Over the course of those 15 frames, the moon moves from the top left to the bottom right. Data = https://github.com/aiqc/AIQC/tree/main/...
Kermit's user avatar
  • 519
1 vote
0 answers
200 views

If the input to the autoencoder is normalized, do we need to use sigmoid on the last layer?

According to: https://stackoverflow.com/questions/65307833/why-is-the-decoder-in-an-autoencoder-uses-a-sigmoid-on-the-last-layer The last layer activation function contains sigmoid in order to the ...
user3668129's user avatar