0
$\begingroup$

I don't understand how convolutional autoencoders achieve dimensionality reduction. For FFNN based autoencoder, the reduction is easy to understand: the input layer has N neurons, and the hidden ones have M neurons, where N is greater than M. Instead, in a convolutional autoencoder, the input image is wide and thin, and it becomes small and thick. It results in an amount of information that is greater than the initial ones.

I report a practical example to explain better what I mean:

# INPUT: 28 x 28 x 1 (wide and thin)

conv1 = Conv2D(32, (3, 3), activation='relu', padding='same')(input_img) #28 x 28 x 32
pool1 = MaxPooling2D(pool_size=(2, 2))(conv1) #14 x 14 x 32
conv2 = Conv2D(64, (3, 3), activation='relu', padding='same')(pool1) #14 x 14 x 64
pool2 = MaxPooling2D(pool_size=(2, 2))(conv2) #7 x 7 x 64
conv3 = Conv2D(128, (3, 3), activation='relu', padding='same')(pool2) 

# OUTPUT: 7 x 7 x 128 (small and thick)

In this example, we start from a 28x28 single-channel image (784), and the encoder output will be 7x7x128 (6272). Is this a dimension reduction?

$\endgroup$
0
$\begingroup$

Why don't you use a lower number of filters in the last convolution? Instead of 128 you can just choose whatever number you want, e.g. 10.

Also, normally after the convolutional (and pooling layers), you flatten the output (therefore losing the spatial information) and then project with a dense layer onto the final representation space. You can control the dimensionality of the representation space with the shape of the matrices of the last dense layer.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.