When trying to train a SVM on some Kaggle data, I have encountered a situation where the linear kernel fails to give any results.
This doesn't make sense to me because the RBF kernel works just fine, and my understanding is that a linear kernel is a strictly simpler version that doesn't map the data into higher dimensions. In fact, all my research suggests that not only is a linear kernel possible, but that in most cases, it should be faster to converge (with the trade-off that the results might not be as accurate).
However, this has not proved to be the case. While the RBF kernel was able to produce a result during cross-validation after ~6 minutes, the linear kernel just sat there with no output after ~6 hours. After this point, I just force quit the training and moved onto testing other SVM parameters.
A breakdown of some facts:
- Roughly 60,000 examples, around 120 features (Sometimes cutting down to 30 features helps, and linear kernel will produce a result after many hours)
- I'm using SKLearn GridSearchCV to perform my training
- I have tested this used
SVC(kernel='linear')as well as
LinearSVC(), both with the same outcome
- This issue has come up with multiple data sets across different competitions
My current hypothesis is that the training stops possibly because the data is not linearly separable. I might be missing something else obvious though. Any guidance is appreciated, thanks!