I am working with a large dataset (approximately 55K observations x 11K features) and trying to perform dimensionality reduction to about 150 features. So far, I tried PCA, LDA, and autoencoder. The autoencoder that I tried was 12000-8000-5000-100-500-250-150-, all layers were Dense with sigmoid activation, except the final layer, which had a linear activation in order to reproduce the continuous data from the input. The autoencoder loss effectively plateaus after 10-15 epochs, regardless of the learning rate (here, I used the ReduceLROnPlateau feature in Keras). For the record, I am normalizing each feature by z-score prior to the training. I'm not sure how to get this loss to stop reaching a plateau.
Should my next attempt be to use a convolutional neural network on this dataset to see if I can reduce the dimensionality more successfully? Are there any pre-trained convolutional autoencoders that I could use? Training a convolutional autoencorder from scratch seems to require quite a bit of memory and time, but if I could work off of a pre-trained CNN autoencoder this might save me memory and time.