How does Keras calculate accuracy from the classwise probabilities? Say, for example we have 100 samples in the test set which can belong to one of two classes. We also have a list of the classwise probabilites. What threshold does Keras use to assign a sample to either of the two classes?
$\begingroup$
$\endgroup$
4
-
1$\begingroup$ are you using model.evaluate in keras? $\endgroup$– Hima VarshaCommented Oct 7, 2016 at 8:15
-
1$\begingroup$ Yes, I am using model.evaluate. More specifically, model.evaluate_generator. $\endgroup$– pseudomonasCommented Oct 7, 2016 at 10:10
-
$\begingroup$ datascience.stackexchange.com/questions/13920/… $\endgroup$– Hima VarshaCommented Oct 13, 2016 at 11:47
-
1$\begingroup$ Possibly related @SO: How does Keras evaluate the accuracy?) $\endgroup$– desertnautCommented Jul 3, 2018 at 14:52
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
7
For binary classification, the code for accuracy metric is:
K.mean(K.equal(y_true, K.round(y_pred)))
which suggests that 0.5 is the threshold to distinguish between classes. y_true should of course be 1-hots in this case.
It's a bit different for categorical classification:
K.mean(K.equal(K.argmax(y_true, axis=-1), K.argmax(y_pred, axis=-1)))
which means "how often predictions have maximum in the same spot as true values"
There is also an option for top-k categorical accuracy, which is similar to one above, but calculates how often target class is within the top-k predictions.
-
$\begingroup$ Thank you for the answer. Does that mean even for binary classification, the labels need to be one hot encoded? $\endgroup$ Commented Mar 20, 2017 at 5:02
-
$\begingroup$ @Raghuram No, for binary classification you just need 0 or 1 as class, no need to one hot encode them. Since K.mean(K.equal(y_true, K.round(y_pred))) will match 2 float values for each case, so it has to be 0 or 1 and not [0,1],[1,0]. $\endgroup$ Commented Jul 4, 2017 at 20:13
-
$\begingroup$ For categorical accuracy, use
categorical_accuracy. $\endgroup$ Commented Dec 23, 2017 at 11:12 -
2$\begingroup$ for a multi-class problem (with more than two classes), is there a difference between using "accuracy" vs "categorical_accuracy" $\endgroup$ Commented Nov 6, 2018 at 20:03
-
2