I am trying to use a single layer autoencoder with linear activation function to perform dimensionality reduction on a dataset before clustering.
The data consists of 5000 samples with 2000 features each. I'm using keras (tensorflow backend). The model that I am using is returned by the function below
model = Sequential() model.add(Dense(n_hidden, input_dim=n_features, kernel_initializer='he_uniform',activation='linear',name='reduce_layer',use_bias=False)) model.add(Dense(n_features, activation='linear', kernel_initializer='he_uniform',use_bias=False)) model.compile(loss='mse', optimizer=Adam(learning_rate=,.01,beta_1=0.9,beta_2=0.95)) return model
and I am using the following to get the reduced representation of my samples
red_dim_model = Model(inputs=model.input, outputs=model.get_layer('reduce_layer').output)
The trained model severely underperforms when compared to standard pca. The mean squared error is about 2 orders of magnitude higher than pca and this doesn't seem to change despite messing with hyperparameters (omptimizer, learning rate, error function, number of epochs, batch size, etc.). The reduced representation results in unexpected and unenlightening clusters.
Why would my autoencoder significantly underperform when compared to pca? Is it an issue with my model, is the number of samples too slow, or is it something else? Any ideas on what I can try to improve performance?