This it a problem that has come on my path a few times now and I don't have a satisfying solution yet. The goal is to predict probabilities or fractions based on some $x$ where our training $y$ has these probabilties or fractions and thus is in the domain $[0,1]$ as opposed to $\{0,1\}$. My question is with regards to my loss function. In the case of fractions, if the error between 0.4 and 0.5 and the error between 0.89 and 0.99 is the same I can just use MSE if I want to predict the expected value.
In case of probabilities where we want to approach it similarly as classification problems, where the difference between 0.89 and 0.99 is much bigger than that of 0.4 and 0.5, we want to put this in our loss function. Does cross entropy still work properly if I feed it fractions in $y$?
$\mathcal{L}(y,\hat{y})=-y\log(\hat{y}) - (1-y)\log(1-\hat{y})$
Let's say our $y=0.5$ and our current prediction is $\hat{y}=0.6$ we would get:
$\mathcal{L}(0.5,0.6)=-0.5\log(0.6) - 0.5\log(0.4)$
I don't really see why this would go wrong? The function is still convex. Everywhere it says that the target should be in $\{0, 1\}$ however. Maybe my math is lacking or I'm missing something obvious, why is this a bad idea?