I was wondering mathematically how Precision Recall curve is plotted. How is this curve useful?
In short, the precision-recall curve shows the trade-off between the two values as you change the strictness of the classifier.
There is a great explanation here, using the classification of images of airplanes and geese as an example.
A good way to characterize the performance of a classifier is to look at how precision and recall change as you change the threshold. A good classifier will be good at ranking actual airplane images near the top of the list, and be able to retrieve a lot of airplane images before retrieving any geese: its precision will stay high as recall increases. A poor classifier will have to take a large hit in precision to get higher recall. Usually, a publication will present a precision-recall curve to show how this tradeoff looks for their classifier.
What is the x and y axis of this scatter plot? If precision and recall are on these axes, then the range of both axes would be 0 - 1. I'm assuming one point would then represent one model, in this case you'd want to select the model on the top right, I.e. where precision and recall are both high.
As for the concepts behind precision and recall, wiki has a good write up https://en.m.wikipedia.org/wiki/Precision_and_recall
To Plot Recall- Precision graph one can simply compute the confusion matrix for say 10 different threshold.
Calculate the other metrics like precision and recall for each threshold from confusion matrix and plot the graph by plotting the value of recall and precision for each threshold.
Many a times AUC and accuracy won't able to determine the performance of the model perfectly and one needs to analyse the performance of their model using other performance metrics like precision, recall, F score , etc.
Generally, precision and recall are used when the data set is highly imbalanced. For example: Identify terrorist in Airport from all travelers. Then in that case out of 10000 people there are only 19 terrorist. Here in this case if I check for the non-terrorist group then out of 9981 I am able to predict 9979 correctly that ultimately gives me accuracy of 99.99% and AUC 99.xx%. But I wrongly classified two terrorist as civilians that is not permissible. So using Precision and recall we can actually check the performance of model on each class. Here in this case precision for non-terrorist group is high and suggest model is good but when you compute the precision and recall for terrorist group , you will come to know that model fails to predict terrorist and precision and recall for this class is quite low compare to non terrorist group.