Why are support vector machines good at classifying images?

Question

I'm trying to figure out if I can identify styles of art using support vector machines and I don't quite understand what makes them good. Perhaps I can make the question more accurate. Support vector machines are basically a way of doing classification with a linear model. You find $\vec{\beta}$ and $\alpha$ such that $sgn(\beta \cdot x + \alpha)$ determines what the class is. So I believe that in order to do image classification, you need logic. It does not suffice to find the average color of an image and decide whether it is greater or smaller than a number to decide which number an image represent. You need logic. I believe you need intermediate logic, like comparing one part of the image to another part of an image. Kind of like a multilayered support vector machine, which now looks like a neural net. Is this intuition correct?

Sincerely, Still figuring it out.

Answer 1

Support vector machines have one built-in "layer" that helps with having an interpretation of the data - the kernel. You could even use output from some other image classifier, including a neural network, as the kernel. E.g. you could measure how far apart two images are in "classifier space" from a trained neural network (maybe one trained against different targets from the ones you want to classify with the SVM)

There is also an implied "layer" in image feature construction. Common image features are histograms of pixel values, histograms of pixel differences (edge detection), bags of "visual words" and there are many more possible transformations and statistics.

Typically if you are using a classifier other than a deep neural network (which is supposed to discover the features automatically), then you will pre-process the image into a set of features. That pre-processing covers the majority of an approach being "good at images".

Answer 2

To add some stuffs to what Neil Slater said:

• First I'm not sure you got (as I read your question) the fact that SVMs are quite good, as long as you use them with a proper kernel. Which means that you must find a way of transforming your images such that the resulting input data are linearly separable in the sens of your formula. If the SVM algorithm is very simple, using kernel is nontrivial.
• Then the best approach nowadays for image classification is deep neural network. Not because they are magic but mostly because of the use of convolutional layers. Let say that for 10 000 neurons in a network, 100 will do what SVM do: classification. The rest of it just act as a kernel. In the case of images, since features to recognize are intricately local and global, the use of successive convolutional layers can decorticate your image to retrieve the specifications you need (in your example, you could recognize brushstroke as well as global shapes).

To sum up:

• Shallow learning (SVM for example) work great but you have to design your own kernel, or treat your data before using them.
• Deep learning (convolutional networks for example) do the work for you, but are harder to set up and to interpret.