I saw some examples of Autoencoders (on images) which use
sigmoid as output layer and
BinaryCrossentropy as loss function.
The input to the Autoencoders is normalized [0..1]
sigmoid outputs values (value of each pixel of the image) [0..1]
I tried to evaluate the output of
BinaryCrossentropy and I'm confused.
Assume for simplicity we have image [2x2] and we run Autoencoder and get 2 results. One result is close to the True value and the second is same as the true value:
import numpy as np import tensorflow as tf bce = tf.keras.losses.BinaryCrossentropy() y_true = [0.5, 0.3, 0.5, 0.9] y_pred = [0.1, 0.3, 0.5, 0.8] print(bce(y_true, y_pred).numpy()) y_pred = [0.5, 0.3, 0.5, 0.9] print(bce(y_true, y_pred).numpy())
As you can see, the second example (which is the same as the true value) gets low score (low loss value, but still it's not 0 or close to 0).
It seems that
It seems that using
BinaryCrossentropy as loss function won't give us the best results. (We never get values close to zero) ?
Does the best value will be close to 0.5 ?
What am I missing ?