Plotting a no-skill model in a precision-recall curve

Question

I am following this tutorial to apply threshold tuning using precision-recall curve for an imbalanced dataset

Within the tutorial, a no-skill model is defined as:

A no-skill model is represented by a horizontal line with a precision that is the ratio of positive examples in the dataset (e.g. TP / (TP + TN)), or 0.01 on our synthetic dataset. perfect skill classifier has full precision and recall with a dot in the top-right corner.

The no skill model is the plotted using this code:

no_skill = len(testy[testy==1]) / len(testy)
pyplot.plot([0,1], [no_skill,no_skill], linestyle='--', label='No Skill')

Within my dataset, the no-skill model would be plotted at precision 0.1, which is the ratio of positive examples in train_y.

But my question is: wouldn't a no-skill model in an imbalanced dataset automatically label all observations as being from the majority class? Seeing as precision = TP/(TP+FP), in this case TPs=0 making precision=0? So shouldn't the no-skill model be plotted at precision=0?

Answer 1

For an explanation using conditional probabilities, see https://en.wikipedia.org/wiki/Precision_and_recall#No-Skill_Classifiers

For different realizations of Noskill classifiers, see dummy classifiers in sklearn

Answer 2

Assuming the dummy model yielding constant probability for all samples, there are two possible thresholds: either everything is classified as positive or negative. So starting point will be (1, positive_class_ratio) and the ending point (0, 1) (by convention, as precision is undefined with no positives).

Tutorial version is still reasonable as its AUC conforms with sklearn average precision score for this case.