0
$\begingroup$

enter image description here

While applying sigmoid activation function (in finding y label), I have calculated it as below:

y = 0.35 + (0.8 * 0.1) + (0.3 * 0.6) + (-0.2 * 0.4) = 0.53

sigmoid_y = 0.625

how do we take threshold value here?

now, how can we say that the given label (y) belongs to 0 or 1?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

This is a design decision.

You can take the naive approach of using 0.5 as the threshold value, and assign 1 to every value greater than or equal to 0.5, and 0 to the rest.

But you can also, choose your threshold to obtain a classifier that meets certain properties, like achieving a certain false positive rate, o a certain false positive rate. You can see these answers for different objectives when choosing the threshold value: this, this, this.

Note that your question is not specific to the sigmoid activation or to neural networks, but applied also to any probabilistic classifier.

Improve this answer
$\endgroup$
5
  • $\begingroup$ yes, my question not specific to 'sigmoid'. if we have balanced data then we can take 0.5 as threshold. but, still am not cleared how we choose this value incase of imbalanced ? how can we select threshold in case of soft max activation function (in balanced , imbalanced)? @noe $\endgroup$
    – tovijayak
    Commented Jun 20, 2023 at 10:51
  • $\begingroup$ The links I included detail how to choose a threshold value depending on what exactly you what to optimize for, e.g. FPR, FNR. The specific criterion is problem-specific, so one should decide what is more important in their classification problem. This is applicable to both balanced and imbalanced problems. $\endgroup$
    – noe
    Commented Jun 20, 2023 at 10:59
  • $\begingroup$ Thank you for the notes @noe, how can we choose in case of softmax activation function? $\endgroup$
    – tovijayak
    Commented Jun 20, 2023 at 12:02
  • $\begingroup$ As I already commented, the fact that you are using softmax is not relevant to the way you choose the threshold. $\endgroup$
    – noe
    Commented Jun 20, 2023 at 12:06
  • $\begingroup$ ok, Thank you!! will go through the link again.. $\endgroup$
    – tovijayak
    Commented Jun 20, 2023 at 12:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.