2
$\begingroup$

Is it possible to come up with a model that minimises both false positive and false negative?

Minimising can be done to a point, such as the Bayes error threshold.

Improve this question
$\endgroup$
6
  • 2
    $\begingroup$ In effort of trying to understand the confusion matrix for a machine learning algorithm, I have written this document for my understanding. drive.google.com/open?id=1E2pPWBVXrxybHbt1OnOxRNMmmCIRQsDj As I understand, if we increase precision, we will have a reduced recall and increasing recall will reduce accuracy. Increased accuracy mean decreased False Positive, and increased recall mean decreased False Negative. According to my understanding, we can not have a model that will minimize both False Negative and False Positives. Looking for comments from experts. $\endgroup$
    – vkj
    Jul 15, 2018 at 21:26
  • $\begingroup$ if a model tries to predict better decrease false positive and false negative together. $\endgroup$
    – parvij
    Jul 16, 2018 at 6:16
  • 1
    $\begingroup$ Those friends that believe this question is so broad and may have so many answers, please share your opinions and don't vote for closing. This is a very good question. $\endgroup$
    – Green Falcon
    Jul 16, 2018 at 14:27
  • 1
    $\begingroup$ @vkj One of the approaches for finding a moderated value for learning is to employ F1 score but I've not seen something that tries to minimize them both in machine learning era. There is a famous quote that says there is not always a good model that describes entirely your data. In ML we have bias/variance trade-off. As apposed to that, in deep learning we have other tools that may let you handle high variance and high bias problems without any conflict. But I guess your question may have good answers that I've not faced :) $\endgroup$
    – Green Falcon
    Jul 16, 2018 at 14:34
  • 1
    $\begingroup$ @SeanOwen Actually you are right if you are attempting to speak about tasks like regression and statistical approximations. On the contrary, in deep learning it is not really true due to the fact that people never speak about that popular tradeoff. People always speak about increasing the size of training data to achieve better learning. About minimising, it can be interpreted as the Bayes error which is the best thing that can be achieved. $\endgroup$
    – Green Falcon
    Jul 17, 2018 at 13:56

1 Answer 1

Reset to default
5
$\begingroup$

Yes and No! depending on what do you mean by minimization. When you say minimizing $f$ and $g$ according to something, you are actually looking for a point which minimizes both. It does not mean that this point necessarily finds the minimum of $f$ or $g$. So yes in this sense.

But in case you mean a point in which both of them are in their minimum, this case is not realistic in practice as usually increasing one of them means decreasing the other one. Have a look at the plot bellow. FRR means False Rejection Rate, FAR means False Acceptance Rate and EER means Equal Error Rate.This is the terminology used in Biometric, a ML-based field studying the person identification/recognition.enter image description here

It illustrates what I explained above.

Note

You should consider the importance of each error. In some use-cases (including some biometric use-cases) you may sacrifice one error for the other one. In those cases you minimize the error you need to be minimum without really caring about the other one (of course trying to keep the other one as low as possible as well but not at top priority).

For example I have a Face Recognition system which opens the door of a highly confidential military site to permitted users. Should I try to keep the EER low? Sure not. If sometimes the door is not open for a permitted person it's ok (at most they complain a bit) but door MUST NOT be opened for someone who is not allowed so here you care about keeping FAR low.

Hope it helped

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ Thanks for your answer. I just want to say whenever you design a network which is too big which can memorise (hefz kone) your data, you have high variance problem which is somehow doing the desired behaviour in the question. My opinion. $\endgroup$
    – Green Falcon
    Jul 16, 2018 at 14:43
  • 2
    $\begingroup$ Thanks for the comment @Media! you are right ... and one solution to avoid overfitting is statistical model selection which I did not explain. Good and necessary point! damet garm $\endgroup$
    – Kasra Manshaei
    Jul 16, 2018 at 17:35
  • $\begingroup$ You are welcome :)) $\endgroup$
    – Green Falcon
    Jul 16, 2018 at 17:50
  • 1
    $\begingroup$ Thanks. I asked this question because I have same understanding. I recently came across a business requirement that says "False Positive should be 0%, and False Negative should be 0%". And I was puzzled by this. People happily provided solution too, and I know that is not possible. So, I thought I should ask this question to make sure I am not missing some understanding. $\endgroup$
    – vkj
    Jul 17, 2018 at 2:07
  • $\begingroup$ your understanding is exactly right my friend and the task is certainly not rational. $\endgroup$
    – Kasra Manshaei
    Jul 17, 2018 at 9:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.