I want to reduce the dimensionality of a dataset using a stacked Autoencoder. The size of the dataset and the computing power at my disposal make it very difficult to train the Network using simple, full-batch Gradient Descent.

Does it make sense to use Mini-Batch Gradient Descent for this kind fo task? I guess that would generate a noisy reduction of dimensionality?


In general, using an autoencoder for dimensionality reduction using mini-batch gradient descent is perfectly fine.

You should pay attention to the neural architecture you use in order to have the appropriate inductive bias, e.g. image autoencoders tend to use convolutional layers to profit from pattern locality.

You should also assess how well the input data is reconstructed, to ensure that the information is properly retained in the compressed representation.

However, as you did not specify the final destination of the data (i.e. what the reduced-dimensionality data is going to be used for), we can't tell whether the dimensionality reduction would be appropriate in for your specific purposes.

  • $\begingroup$ I need Autoencoders for two separate projects: 1) To reduce training time and multicollinearity before a classification task; 2) To train a word2vec model on a corpus $\endgroup$ – Leevo Jul 1 '19 at 14:47
  • $\begingroup$ For the classification use case, an autoencoder should be fine. For reducing the dimensionality of word2vec, you may want to take a look at specific approaches like this one. $\endgroup$ – noe Jul 2 '19 at 12:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.