Precision, recall and importance of them in the imbalance problem

Question

I have a test dataset. The dataset is an imbalanced dataset. The total training instances for the dataset is 543 among them minority class(yes) is 75 and the majority class(No) is 468. The class of interest is minority class(yes). I used the Naive Bayes classifier for prediction. The confusion matrix I got

TP   TN   FP  FN
33   391  77  42

The total instances for No class are 468, The classifier truly predicted 391 instances as negative. However the total negative class that the classifier predict is 391+42 = 433, Those, 42 false negatives are actually positive class but the classifier predict them as negative. Am I right with this explanation?

Secondly, the classifier predicted 33 instances as true positive. However, total prediction of positive class TP+FP = 33+77 = 110. Now these false positive are actually negative class.

So, if I calculate TP+FN I will get 33+42 = 75 which is the total number of positive instances in the test set.

If I calculate TN+FP I will get 391+77 = 468, which is the total number of negative instances in the test set.

Now, the precision is True positive/(True positive + False positive), As I have mentioned earlier False positive is noting but some negative instances, So, my question is what does precision actually mean?

For recall is True positive/(True positive + False negative), As I have mentioned earlier False negative means positive instances. (True positive+Flase negative ) total number of positive instances. Now, what does it mean by True positive/ Total number of positive instances?

Lastly, in the class imbalance problem if the majority class is our class of interest which metric (precision and recall) should we consider?

Thank you.

Answer 1

The total instances for No class are 468, The classifier truly predicted 391 instances as negative. However the total negative class that the classifier predict is 391+42 = 433, Those, 42 false negatives are actually positive class but the classifier predict them as negative. Am I right with this explanation?

Yes, this is correct.

Now, the precision is True positive/(True positive + False positive), As I have mentioned earlier False positive is noting but some negative instances, So, my question is what does precision actually mean?

Precision represents the proportion of correct instance among the instances predicted as positive. In other words, this is the probability that a case predicted positive is truly positive.

For recall is True positive/(True positive + False negative), As I have mentioned earlier False negative means positive instances. (True positive+Flase negative ) total number of positive instances. Now, what does it mean by True positive/ Total number of positive instances?

Recall is the proportion of instance predicted positive by the system among all the truly positive instances. In other words, it represents the probability that the system correctly "finds" that an instance is positive.

Lastly, in the class imbalance problem if the majority class is our class of interest which metric (precision and recall) should we consider?

It's pretty rare that the majority class is of interest, usually the minority class is chosen as the positive class. But anyway this wouldn't change the answer: one should use precision and recall (or F1-score if a single value is needed), but in this case one should use a higher precision (number of digits after the comma).

Answer 2

I suggest you to read this article and this one. In the second article, it is nicely explained most situations you can probably face: High Precision versus High recall, and so on..

Quick example: Suppose a multi-label classification problem where the model may have to predict correctly more than one label. Here, predictions can be partially correct or incorrect. If metrics are such that you have low precision and high recall this that your model predict more than necessary label but most of them are incorrect. High precision and high recall is the best result: most of predicted labels are relevant.