When I am training and evaluating classifiers or hyperparameter tuning I don't like to look at precision and recall metrics alone because those numbers depend on a threshold that I will set afterwards to cut the output probabilities and make the final prediction. This threshold usually defaults to .5. But that depends entirely on the problem and I always end up changing it. I like to decide its value at the end, once I have selected the ~3 best models from the previous evaluation phase. Sometimes I need a conservative classifier and sometimes a more relaxed one. So before going into that, I need something to use to select the best models and the metric I look at the most is the area under the precision-recall curve. If I had to choose only one metric to look at it would be that. However it is not that popular and I see people using precision and recall numbers alone (threshold=0.5) to evaluate their models.
Why isn't it used to grid-search/model selection more often? Maybe I am missing an important drawback?