I am performing phishing URL classification, and I am comparing several ML classifiers on a balanced 2-class data-set (legitimate URL, phishy URL).
The ensemble and boosting classifiers such as Random Forest, Ada Boost, Extra Trees etc AND K-NN achieve an accuracy about 90%, and a False Positive Rate about 11-12%. (fig.)
On the other hand, classifiers such as SVM, Logistic Regression, Multinomial NB and Bernoulli NB seem to perform poorly with accuracies fluctuating between 70% - 80% and higher false positives.
Here is the thing. I also tried Gaussian NB and although it yields by far the worst accuracy 58.84% it has an incredibly low False Positive Rate 2.14% (and thus a high FNR)
- I have no idea why this is happening. Any ideas?
- Why some classifiers perform so poorly and others not?
I parametrized them all with Grid Search, they are used on the same dataset (about 30k records of each class) and I perform a 3-fold cross validation. It doesn't make any sense to me, especially for SVM. At last I use about 20 features.
P.S: I use python's sk-learn library