1
$\begingroup$

I have just finished reading an article about the F1 score, recall and precision. Everything was clear except the fact that the author, in his example (see https://towardsdatascience.com/beyond-accuracy-precision-and-recall-3da06bea9f6c#a9f0), is using a threshold for his model output to play on the F1 score, recall and precision values.

So my question is: is it conscientious to use a threshold as the author did in his article?

In my opinion, this is like a kind of manual over-fitting, but maybe there is something I misunderstood...

$\endgroup$
1
$\begingroup$

Good question. Using a threshold is perfectly fine and is not "manual overfitting".

It is not manual because this is a step that can (and should) be done automatically. It is not overfitting as it doesn't modify the model itself. It modifies how you interpret the model's output.

What the user did is actual called cost-sensitive learning. It is a technique where you define which error is more costly and these costs are reflected in the performance metric you use.

I can see why it may feel like overfitting but it is important to understand that all you are doing here is handcrafting a performance metric with which you will evaluate your model. Your model will simply try to optimize this metric.

$\endgroup$
2
  • $\begingroup$ Thank for this helpful answer! :) Just to be sure: even if you have an extreme threshold such as 0.2 or 0.8, it will not be a problem, because this threshold just changes the model output interpretation, right? $\endgroup$ – Kaharon May 29 '20 at 3:24
  • $\begingroup$ Indeed. The threshold can be "anything" between 0 and 1 and will only modify how you interpret the output, but won't change that output itself $\endgroup$ – Valentin Calomme May 29 '20 at 6:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.