How to interpret a Bayesian neural network prediction for binary classification in comparison to deterministic neural network?

Question

Allow me to clarify my current understanding:

Interpreting a binary classification prediction made by a deterministic neural network

On one hand, point estimates fall on a sigmoid curve (between 0-1, where 0.5 is the classification threshold). So they already have a pseudo probability/ strength associated with them given how far away the estimate is from 0.5 (e.g. a prediction of 0.85 is much more certain than a prediction of 0.54)

Interpreting a binary classification prediction made by a bayesian (probabilistic) neural network

On the other hand, a BNN not only provides a point estimate (mean, also on a sigmoid curve) but also provides confidence intervals. So could I say something like, "here we see a sigmoid value of 0.6, AND we are X% certain that it is above 0.5?"

---

Is my understanding correct/ missing anything important?

cross post: https://stackoverflow.com/questions/75848574/how-to-interpret-a-bayesian-neural-network-prediction-for-binary-classification?noredirect=1#comment133789503_75848574

Answer 1

Bayesian neural networks do not provide confidence intervals, confidence intervals are a frequentist statistic.

Bayesian neural networks often estimate a distribution of parameter values for each parameter, whereas traditional neural networks estimate a point estimate for each parameter.