I'm a bit confused by the coexistence of Loss and Accuracy metrics in Neural Networks. Both are supposed to render the "exactness" of the comparison of $y$ and $\hat{y}$, aren't they? So isn't the application of the two redundant in the training epochs? Moreover, why aren't they correlate?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
Log loss has the nice property that it is a differentiable function. Accuracy might be more important and is definitely more interpretable but is not directly usable in the training of the network due to the backpropagation algorithm that requires the loss function to be differentiable. When your preferred loss is not directly optimizable (like the accuracy) you use a loss function that behaves similarly to proxy the true metric. In case of binary classification you would use a sigmoid at the end and a log loss to approximate accuracy. They are highly correlated.
answered Aug 25, 2016 at 13:50
$\begingroup$
$\endgroup$
4
Yes, they both measure the exactness of y and y_hat and yes they're usually correlated. Sometimes the loss function might not be accuracy but you're still interested in measuring the accuracy even though you're not optimizing it directly. Google's TensorFlow MNIST example minimizes/optimizes cross entropy loss but displays accuracy to the user when reporting results, and this is completely fine.
Sometimes you don't want to optimize accuracy directly. For example, if you have serious class imbalance, your model will maximize accuracy by simply always picking the most common class, but this would not be a useful model. In this case entropy / log-loss would be a better loss function to optimize.
answered Aug 25, 2016 at 13:33
-
7$\begingroup$ More importantly, accuracy is a not a differentiable function so you cannot backpropagate through it. $\endgroup$ Commented Aug 25, 2016 at 13:45
-
$\begingroup$ @JanvanderVegt Yes, that's a great point $\endgroup$ Commented Aug 25, 2016 at 13:45
-
$\begingroup$ I learnt that in Keras I can put a "custom" evaluation metrics (by custom in this case I mean that no built-in implementation in Keras, like AUC or F1-Score) in the compilation function. I presume that in this case this "custom" metrics will be used/displayed instead of Accuracy everywhere where
show_accuracyparameter is set to True (like at fitting or in evaluation). Is that correct? $\endgroup$– HendrikCommented Aug 25, 2016 at 14:10 -
1$\begingroup$ @Hendrik yes you can, just create a
def your_own_metric(y_true, y_pred)function and pass it tomodel.compile(..., metrics=[your_own_metric])$\endgroup$ Commented Aug 25, 2016 at 15:31