Autoencoders are fitting anomalies too good

Question

I have a set of ~ 5000 greyscale images with resolution of 64x128. I want to do an unsupervised anomaly detection. As a first try, I chose convolutional autoencoders (AE) and trained an AE model. I chose AE because I found it easier to understand the algorithm and visualize the results. Note that I have no prior experience in the field of machine leaning and wanted to try out using ML for making better sense of my data. Since this is an unsupervised algorithm, I used the same set of data to train as well as test the model. I used the standard mean-squared-error as the loss metric. The model was trained and the histogram of the MSE showed some outliers. The outliers detected by the model were indeed real outliers. However, the model missed to detect other kinds of outliers. Particularly the ones where the size of the white region in the middle is much smaller than the others. Ideally this is an outlier, however the AE model was able to reconstruct this with minimal error.

Histogram: https://imgur.com/a/7yTX7Se

A good and an anomalous image: https://imgur.com/a/cyH0ZO4

Here the anomaly is having a smaller white region.

Any suggestions on how to tune the AE model to detect such anomalies? Or is AE not suitable for such use cases. A quick search indicates such an issue can be solved using AE with "memory", but I thought of checking before exploring other algorithms.

Answer 1

In principle it's a feasible way to use CAE's (Convolutional AutoEncoders) for this. It's a bit hard to really tell you what to do better without knowing the code. But I can give you some points I found useful for myself:

• try a different metric for the reconstruction error/loss, e.g. the SSIM (Structural Similarity Index) - maybe it's better for the model to distinguish "good" from "bad"
• try to optimize the CAE structure, e.g.
  • different layer combinations (Conv2D, BatchNormalization, some ReLu function, MaxPooling, Dropout, ...)
  • more or less layers
  • tune the number of filters, kernel size, dropout rate, latent dimension
• try different optimizers, e.g. Adam, RMSprop, etc. - also with tunable hyperparameters

Maybe that helps. Just an ideation...

Answer 2

I also have a similar problem. i asked here. The autoencoder I trained became too powerful and started to reconstruct even anomalies.

So there are 2 strategies:

1. train the autoencoder, and use the encoder output (as reduced dimension) to train an OC SVM OR

2. use the whole trained autoencoder, calculate the reconstruction error, and use that with a threshold to detect an anomaly.

In my understanding. These two approaches have opposite ways of finding the same thing. I think it would be best to use both approaches and check both scores.