I'm a little bit new to machine learning.

I am using a neural network to classify images. There are two possible classes. I am using a `Sigmoid` activation at the last layer so the scores of images are between 0 to 1. 

I expected the scores to be sometimes close to 0.5 when the neural net is not sure about the class of the image, but all scores are either 1.0000000e+00 (due to rounding I guess) or very close to zero (for exemple 2.68440009e-15). In general, is that a good or bad thing ? How can this behaviour be avoided?

In my use case I wanted to optimize for recall by setting a lower threshold but this has no impact because of what I described above.

More generally, how can I minimize the number of false negatives when in training the neural net only cares about my not ad-hoc loss ? I am ok with decreasing accuracy a little bit to increase recall.