I have a classification problem with large number of classes: feature set is 512 Dimension, number of classes are around 3000. This is a face identification problem. (identify among 3000 celebrities, whose face it is. The feature is extracted using FaceNet.)

The issue for training such a SVM model is too slow:

I used sklearn SVC, it came with below result:

RAM usage: > 100GB // I eventually ended up using virtual memory of 100GB
Training time: > 30 hours
Classification time: > 1 hour per face
Other issue: Single CPU usage, no parallelization

To summarize, it is not practical to use above naive way of training for such a SVM model.

My Question: What is a practical way of optimizing SVM training / usage of this scale input data? (~ 3000 classes, feature_size = 512)


I think that using SVMs for your model is the main problem.

SVMs (linear or otherwise) are originally intended for binary classification. There are various procedures for extending them to a multi-class problem. The most common methods involve transforming the problem into a set of binary classification problems. The most common strategy is using "One-Versus-All" classifiers (OVA classification), and choosing the class which classifies the test data with the greatest margin. Another strategy is to build a set of One-Versus-One classifiers, and to choose the class that is selected by the most classifiers. Both these strategies are computationally expensive both in training and in testing. I know there are other, more complicated methods, but I don't think that any of them is intended for such a high number of classes.

The most trivial way would be to use a neural-network model for such a problem. If you want to use one of the more classic algorithms, perhaps using decision trees will work (or even k-nearest neighbors).

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.