# How to extract features from the encoded layer of an autoencoder?

I have done some research on autoencoders, and I have come to understand that they can also be used for feature extraction (see this question on this site as an example). Most of the examples out there seem to focus on autoencoders applied to image data, but I would like to apply them to a more general data set.

Therefore, I have implemented an autoencoder using the keras framework in Python. For simplicity, and to test my program, I have tested it against the Iris Data Set, telling it to compress my original data from 4 features down to 2, to see how it would behave.

The encoder seems to be doing its job in compressing the data (the output of the encoder layer does indeed show only two columns). However, the values of these two columns do not appear in the original dataset, which makes me think that the autoencoder is doing something in the background, selecting/combining the features in order to get to the compressed representation.

Here is the complete working example:

from pandas import read_csv
from numpy.random import seed
from sklearn.model_selection import train_test_split
from keras.layers import Input, Dense
from keras.models import Model

# Get input data and separate features from labels
Y = df.iloc[:,4]
X = df.iloc[:, : 4]

# Split data set in train and test data
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, train_size=0.5, random_state=seed(1234))

# Input information
col_num = X.shape[1]
input_dim = Input(shape=(col_num,))

# Encoding information
encoding_dim = 2
encoded = Dense(encoding_dim, activation='relu')(input_dim)
# Decoding information
decoded = Dense(col_num, activation='sigmoid')(encoded)
# Autoencoder information (encoder + decoder)
autoencoder = Model(input=input_dim, output=decoded)

# Train the autoencoder
autoencoder.fit(X_train, X_train, nb_epoch=50, batch_size=100, shuffle=True, validation_data=(X_test, X_test))

# Encoder information for feature extraction
encoder = Model(input=input_dim, output=encoded)
encoded_input = Input(shape=(encoding_dim,))
encoded_output = encoder.predict(X_test)

# Show the encoded values
print(encoded_output[:5])


This is the output from this example:

[[ 0.28065908  6.151131  ]
[ 0.8104178   5.042427  ]
[-0.          6.4602194 ]
[ 3.0278277   2.7351477 ]
[ 0.06134868  5.064625  ]]


Basically, my idea was to use the autoencoder to extract the most relevant features from the original data set. However, so far I have only managed to get the autoencoder to compress the data, without really understanding what the most important features are though.

My question is therefore this: is there any way to understand which features are being considered by the autoencoder to compress the data, and how exactly they are used to get to the 2-column compressed representation?

• Thank you for this answer, it confirmed my suspicions that weights were involved. You wrote "Answer is you can check the weights assigned by the neural network for the input to Dense layer transformation to give you some idea." Do you happen to have a code example on how to do this in the code above? If I just do print(encoder.weights) I get [<tf.Variable 'dense_1/kernel:0' shape=(4, 2) dtype=float32_ref>, <tf.Variable 'dense_1/bias:0' shape=(2,) dtype=float32_ref>], but it would be nice to see the actual weights assigned to the output columns. – user1301428 Dec 9 '19 at 9:27
• Unfortunately the first option returns an empty array, and the second one gives me this error: Attempting to use uninitialized value dense_1/bias :/ But I think this could be a separate question for stackoverflow. Thanks for the help! – user1301428 Dec 9 '19 at 18:20