What do this Classification evaluation results mean to you? Do they are suspicious or not?

Question

I have collected dataset with two class labels and used the SVM Method to classify the dataset, and this is the results. Does this appear suspicious or not?

scikit-learn classifiers with SVM SVC train on 114859 instances, test on 49227 instances Excution (Training) Time: 9.82799983025

Excution (Testing) Time: 3.75

accuracy: 0.999837487558

Precision-Recall AUC: 1.00

ROC AUC: 1.00

pos precision: 0.999822253822

pos recall: 1.0

pos F-measure: 0.999911119012

neg precision: 1.0

neg recall: 0.998107404779

neg F-measure: 0.999052806062

Answer 1

The statistics here are obviously very good, in fact too good for any practical data set. Your model is almost perfect... Unfortunately, it's practically useless and I'll explain.

In machine learning, if you see something like this you know you are in trouble. That can happen if there are problems with your data workflow. For example, you might have removed all outliers that you shouldn't, or you actually used a subset of your training data for the test set.

It's fine if you're just toying SVM, but you'll never encounter something like this in real life.

Answer 2

In the dark I would say that it is suspicious. However this really depends on the problem you are solving. Certain datasets will have a lot of clear examples, a very easy task to solve etcetera.

What could have happened is that during the preprocessing somehow your targets leaked into your features. Some more advanced preprocessing techniques like aggregating numerical features and the target per category and appending those to your features allow for such mistakes and will lead to extremely high accuracy. Any information that is not available during new predictions should not be used in your training set.