I have done some research on autoencoders, and I have come to understand that they can also be used for feature extraction (see this question on this site as an example). Most of the examples out there seem to focus on autoencoders applied to image data, but I would like to apply them to a more general data set.
Therefore, I have implemented an autoencoder using the
keras framework in Python. For simplicity, and to test my program, I have tested it against the Iris Data Set, telling it to compress my original data from 4 features down to 2, to see how it would behave.
The encoder seems to be doing its job in compressing the data (the output of the encoder layer does indeed show only two columns). However, the values of these two columns do not appear in the original dataset, which makes me think that the autoencoder is doing something in the background, selecting/combining the features in order to get to the compressed representation.
Here is the complete working example:
from pandas import read_csv from numpy.random import seed from sklearn.model_selection import train_test_split from keras.layers import Input, Dense from keras.models import Model # Get input data and separate features from labels df = read_csv("iris.data") Y = df.iloc[:,4] X = df.iloc[:, : 4] # Split data set in train and test data X_train, X_test, Y_train, Y_test = train_test_split(X, Y, train_size=0.5, random_state=seed(1234)) # Input information col_num = X.shape input_dim = Input(shape=(col_num,)) # Encoding information encoding_dim = 2 encoded = Dense(encoding_dim, activation='relu')(input_dim) # Decoding information decoded = Dense(col_num, activation='sigmoid')(encoded) # Autoencoder information (encoder + decoder) autoencoder = Model(input=input_dim, output=decoded) # Train the autoencoder autoencoder.compile(optimizer='adadelta', loss='mean_squared_error') autoencoder.fit(X_train, X_train, nb_epoch=50, batch_size=100, shuffle=True, validation_data=(X_test, X_test)) # Encoder information for feature extraction encoder = Model(input=input_dim, output=encoded) encoded_input = Input(shape=(encoding_dim,)) encoded_output = encoder.predict(X_test) # Show the encoded values print(encoded_output[:5])
This is the output from this example:
[[ 0.28065908 6.151131 ] [ 0.8104178 5.042427 ] [-0. 6.4602194 ] [ 3.0278277 2.7351477 ] [ 0.06134868 5.064625 ]]
Basically, my idea was to use the autoencoder to extract the most relevant features from the original data set. However, so far I have only managed to get the autoencoder to compress the data, without really understanding what the most important features are though.
My question is therefore this: is there any way to understand which features are being considered by the autoencoder to compress the data, and how exactly they are used to get to the 2-column compressed representation?