2
$\begingroup$

I have a dataset with 600 examples of target class yes and 300 of no. After classification I get high accuracy(75%) but low kappa (0.4). What can it mean? It seems that kappa is really low, am I right? Other metrics like precission or AUR are about 0.7-0.8 what seems to be good.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I'm not sure that anything is wrong here. With a binary target, chance is 50%, and 75% is half way to perfect accuracy of 100%. Kappa, which ranges from 0 at chance to 1 at perfect separation, is 0.4, which is again a little less than half way to perfect prediction. So it seems to be corroborating the performance of your model.

Kappa is defined as: (observed accuracy - expected accuracy)/(1 - expected accuracy)

Since your classes appear balanced, you have (0.75 - 0.5) / (1 - 0.5) = 0.5

Improve this answer
$\endgroup$
2
  • $\begingroup$ So is it correct score for the dataset? How big should be the accuracy then to get 0.6-0.8 kappa? $\endgroup$
    – Mateusz
    Nov 5, 2017 at 19:45
  • 1
    $\begingroup$ And class is not so balanced becuase 2/3 is Yes, 1/3 is No $\endgroup$
    – Mateusz
    Nov 5, 2017 at 20:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.