I am trying to use a single layer autoencoder with linear activation function to perform dimensionality reduction on a dataset before clustering.

The data consists of 5000 samples with 2000 features each. I'm using keras (tensorflow backend). The model that I am using is returned by the function below

    model = Sequential()
    model.add(Dense(n_hidden, input_dim=n_features, kernel_initializer='he_uniform',activation='linear',name='reduce_layer',use_bias=False))
    model.add(Dense(n_features, activation='linear', kernel_initializer='he_uniform',use_bias=False))
    model.compile(loss='mse', optimizer=Adam(learning_rate=,.01,beta_1=0.9,beta_2=0.95))
    return model

and I am using the following to get the reduced representation of my samples

red_dim_model = Model(inputs=model.input,  outputs=model.get_layer('reduce_layer').output)

The trained model severely underperforms when compared to standard pca. The mean squared error is about 2 orders of magnitude higher than pca and this doesn't seem to change despite messing with hyperparameters (omptimizer, learning rate, error function, number of epochs, batch size, etc.). The reduced representation results in unexpected and unenlightening clusters.

Why would my autoencoder significantly underperform when compared to pca? Is it an issue with my model, is the number of samples too slow, or is it something else? Any ideas on what I can try to improve performance?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.