I want to reduce the dimensionality of a dataset using a stacked Autoencoder. The size of the dataset and the computing power at my disposal make it very difficult to train the Network using simple, full-batch Gradient Descent.
Does it make sense to use Mini-Batch Gradient Descent for this kind fo task? I guess that would generate a noisy reduction of dimensionality?
In general, using an autoencoder for dimensionality reduction using mini-batch gradient descent is perfectly fine.
You should pay attention to the neural architecture you use in order to have the appropriate inductive bias, e.g. image autoencoders tend to use convolutional layers to profit from pattern locality.
You should also assess how well the input data is reconstructed, to ensure that the information is properly retained in the compressed representation.
However, as you did not specify the final destination of the data (i.e. what the reduced-dimensionality data is going to be used for), we can't tell whether the dimensionality reduction would be appropriate in for your specific purposes.
