How to build a network that detects common object in images?

Question

I have a group of images. All contain some common object (let's assume all at the same size and no other alterations). I want to train a CNN that will learn the filter for the common object and gives me back a boolean flag of whether it exists in a given image or not.

(for simplicity, I assume the "object" is smaller than 5x5 pixels)

I thought of something like:

• A Conv layer with 1 filter of 5x5
• A MaxPooling layer with pool_size = input size (meaning - returns a single value: the maximal activation of the filter)
• Output max activation (or a variation of it)

Some questions:

1. Would you output the raw max activation and use a threshold to decide? sigmoid of it? or force the network to output "object" and "no_object" classes and do softmax?
2. Quite related - what loss function?
3. Is it a problem to have a large MaxPooling layer (one that reduces all activation to a single one)? What happens to the gradient on backprop?
   It seems like it would just go randomly to one of the activations and I'm not sure that it can "learn" the object that way..

I'm wondering how to look at this problem as it's not really a 2-class classification problem. Even if I can come up with images that doesn't contain the object, there is nothing to "learn" from this class except of the fact that it doesn't contain something (which is a valuable information here, I must agree).

I've tried playing with it on simulated small black-white images and a cross "object" that I randomly added. Couldn't get the filter to learn the cross pattern although on some tries it got the horizontal line..

Any idea would help! Thanks!

Answer 1

You probably just need a deeper network to learn the cross test example. Convolutional layers need to have multiple feature channels and be stacked in order to learn anything more complex that edge/corner counting. Look at example networks for e.g. MNIST and you will usually see typically 2 or 3 CNN layers, with multiple channels (e.g. 32 or 64 channels, each of which is associated with previous layer channels amount of NxN kernels) plus 1 or 2 fully connected layers before classifier.

For a binary choice, yes train on object present/not present classes. There is no major difference between a single sigmoid output or softmax with two outputs. Use cross-entropy loss function, multiclass version if you use softmax.

I suggest just adapt an MNIST example network to your problem by changing input shape and number of outputs. So that would use softmax and multiclass cross-entropy loss.

Is it a problem to have a large MaxPooling layer

It could work in some situations, but by having a single layer, single channel CNN, you have made the network too simple to classify even moderately complex shapes. Reducing the activations from this layer to just the max is going to limit the network to essentially classifying by strongest matching 5x5 template to the filter in the image. With enough training examples it should be able to create such a filter, but it would not cope very well with noise, rotation or any partial matches.

You might get something more effective with a large pooling layer over say a 2-deep multi-channel CNN, and then a small fully-connected layer over those max channel outputs. It would still be an unusual choice of architecture, but I suspect it could learn problems like your cross test.

Answer 2

Also, you could take some ideas of an architecture from

https://github.com/soumith/convnet-benchmarks

https://github.com/jcjohnson/cnn-benchmarks

There, are multiple state-of-the-art cnn architectures