CNN for unsupervised anomaly detection

Question

I'm wondering if the following strategy has been already used and could work

Let's says you have a CNN which work well to classify image data, dog and cat. You only have cat and dog image as training data. Is there any way to use it to detect image of horse as anomaly?

For example, with a ruled based system we could says

if P(cat) and P(dog) ~0.5 then it's an anomaly

another way could be to take feature vector at last fully connected layer and compare vector very different from other could be considerated as anomaly

Do you have any related paper? is it a totaly dumb idea?

Answer 1

I'm not really sure if if P(cat) and P(dog) ~0.5 then it's an anomaly would be sound.

But what you are referring to is not a strange idea: A CNN forms a conceptual description of inputs based on patterns in previously seen data. New data that is less like the train data doens't compress and reconstruct well. The approach usually used for this is called autoencoding.

Roughly: If you would build an autoencoder by reconstructing images of cats and dogs, the in and out images would eventually (after enough data) approach each other (the distance you'd define would decrease). A new image from another domain would reconstruct less well (The distance between reconstruction and input would be larger than average).

Answer 2

Using a CNN in an autoencoder (mentioned by S van Balen), is the most established way of doing anomaly detection. You can possibly use a pre-trained network as a base for this. And it should be possible to train only the decoder, keeping the encoder frozen. There are some works that show using regularization constraints to make the decoder layers the inverse of the encoder, which could maybe make this more efficient.

Another approach would be to use the last feature layer of the CNN, and pass that to an anomaly detection model. Like classic methods clustering approaches, or even an autoencoder. Effectively using the CNN as a feature extractor only.

Both approaches require access to (some of) the training data to work, but can leverage the pretraining of the CNN.