0
$\begingroup$

I have a question regarding the number of filters in a convolutional Autoencoder. As far as I have understood, as the network gets deeper, the amount of filters in the convolutional layer increases. A typical pattern would be to $16, 32, 64, 128, 256, 512 ...$

Right now I am looking into Autoencoders and on the Keras Blog I noticed that they do it the other way around. They start with $16$ filters in the first layer, then the number of filters is decreased:

https://blog.keras.io/building-autoencoders-in-keras.html

Now I am wondering if this is usual when working with Autoencoders or does it depend on what kind of features the network should learn?

Thanks in advance,

Cheers,

Michael

Improve this question
$\endgroup$

2 Answers 2

Reset to default
0
$\begingroup$

Autoencoders are meant to reduce the dimensionality of your data. Increasing the number of filters would do the opposite.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ Thanks for your comment, though Iam not sure if I agree. Assuming that I use Maxpooling layers I would reduce the spatial dimension of the image. Even though I would increase the amount of filters throughout my network, the final dimension in the bottleneck layer would be smaller than the input dimension. Please correct if Iam wrong! $\endgroup$
    – Michael Lempart
    Commented Jul 4, 2019 at 10:09
  • $\begingroup$ Our assertions are compatible. Max-Pooling is another way of reducing the size of image data. It's quite drastic though: the simplest 2x2 maxpool would make you loose 75% of the pixel data at your disposal! That's why I'm always a bit reluctant on Max-Pooling (but that's just my personal preference). You can try both a see which works best. $\endgroup$
    – Leevo
    Commented Jul 4, 2019 at 10:56
  • $\begingroup$ Yes, I get your point. It seems like there are no real pre-defined rules. One have to test what works best depending on the underlying data... $\endgroup$
    – Michael Lempart
    Commented Jul 4, 2019 at 10:58
  • $\begingroup$ Yeah, that is the case in most of ML unfortunately. That is what I think: use Max-Pool if you need to make your model lighter, and run it faster. Use conv layers if you want to keep as much as information as possible through the Autoencoder. The solution is a tradeoff between the two, I guess $\endgroup$
    – Leevo
    Commented Jul 4, 2019 at 11:02
  • 1
    $\begingroup$ Thank you so much for your input... I will try to run my autoencoder with only convolutational layers as well. Maybe Maxpooling is the reason why Iam getting som artefacts in the different channels... $\endgroup$
    – Michael Lempart
    Commented Jul 4, 2019 at 11:28
0
$\begingroup$

You should take into account that the size of the images in the Keras example is as little as 28x28 and that they are grayscale images (i.e.a single channel), so if you want to actually compress the information you don't have much margin to increase the number of channels in the convolutional layers.

The architecture of the encoder in normal autoencoders (with input images of larger sizes and in color) resembles the typical convolutional network meant for classification, where the width and height and progressively reduced and the number of channels increases, and there is a final flattening operation to express the information into a single dense vector.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks, that makes sense. I tested both increasing and decreasing the amount of filters with input images of size 256x256 with 3 channels, whereas the first channel is a grayscale image and the other two channels consists of binary masks. I noticed that I get strange artefacts when increasing the filters (i.ex. channel 1 can be seen in channel 3). If I decrease the amount of filters the images are reconstructed nicely. I guess that has something to do with my dataset though. Maybe I need to collect more low level features? $\endgroup$
    – Michael Lempart
    Commented Jul 4, 2019 at 10:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.