Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I'm working in binary classifier problem, where I have used Tensorflow low level API's. Last layer wrapped with Sigmoidal Function and just returning a single value.
For my prediction I just set a standard threshold value of 0.5 and hence if it's >0.5, I'm predicting as 1, otherwise 0.
I'm also looking for a confidence level of each prediction. Is it possible to get the same with this architecture, or does it need some modification?
Since a neural net that ends with a sigmoid activation outputs probabilities, you can take the output of the network as is.
If you're referring to scikit-learn's predict_proba, it is equivalent to taking the sigmoid-activated output of the model in tensorflow. In fact that's exactly what scikit-learn does.
E.g. if tour model outputs $0.8$ for class $1$, you would classify this as $1$ (since $0.8 > 0.5$), with a probability of $0.8$. S
Usually binary classifiers are implemented with one output node and Sigmoid activation function. In that case the output you get is the predicted probability of an observation being of class 1 (compared to 0). If you want a probability distribution you can simply pair that y predicted, with 1-y, meaning "the probability of the other class".
Alternatively, you could implement the model with two output nodes, and Softmax activation function. The output would then be a probability distribution on the two classes.
I'm also looking for a confidence level of each prediction. Is it possible to get the same with this architecture, or does it need some modification?
Neural Networks do not calculate confidence levels. Many studies have been made on how to estimate them. The one I know better is based on dropout as a perturbation method. It is based on using varying levels of droput after the model was trained on multiple prediction, and building standard confidence intervals on those. You can read an introduction to it here. I have to say though, the implementation is not straightforward, and it's a computationally intensive procedure.
