Interpreting Machine Learning Classification Metrics

Question

I'm trying to understand the results of a classifier I used to predict two possible classes. Here is what I get:

     Precision     Recall     F1-score     Support

 0      0.97        1.00        0.98        341091
 1      0.60        0.18        0.27        12629

 Avg    0.96        0.97        0.96        353720

If I'm trying to predict Class 1, does this model have any value? Can I improve the results of Class 1 prediction?

Answer 1

I assume the results you show have been evaluated according to a train-validation-test split approach. With the information you have provided, it is possible to figure your confusion matrix. It has to be something like :

label=0 and pred=0 : 330.000
label=0 and pred=1 : 4800
label=1 and pred=0 : 11.000
label=1 and pred=1 : 7200

According to this, your model has some value but and is better that what random choices could give you. Thinking of a random choice between the two classes given the distribution of the classes, you would get something close to :

label=0 and pred=0 : 328.000
label=0 and pred=1 : 12.600
label=1 and pred=0 : 12.600
label=1 and pred=1 : 484

So, I think you model has some value. Although, I think you have to confront this performance with the use-case you are working on, the features and model you are using and obviously the level of performance you think acceptable.

To increase your model performance, it exists many approaches you can try. Here are some of them :

• Try other classification algorithms (Naive Bayes, Logistic regression, SVM, tree-based technics...).
• Clean your input variables if there are dirty and extract insights from it. This is called features engineering.
• Diagnose your results and search if you are in an underfitting or an overftitting situation. Adapt your algorithm if needed.
• Apply technics relative to unbalanced classes. See more here.