I'm a little bit new to machine learning.

I am using a neural network to classify images. There are two possible classes. I am using Sigmoid activation at the last layer so the scores of images are between 0 to 1.

I expected the scores to be sometimes close to 0.5 when the neural net is not sure about the class of the image, but all score are either 1.0000000e+00 (due to rounding I guess) or very close to zero (for exemple 2.68440009e-15). In general, is that a good or bad thing ? I have the feeling it's not. If it is not, why? and how can it be avoided?

In my use case I wanted to optimize for recall by manually setting the necessary score to classify an image as beonging to class 1 to be 0.6 or 0.7 but this has no impact.

More generally, how can I minimize the number of false negatives when in training the neural net only cares about my not ad-hoc loss ?