I am trying to validate the accuracy of my KNN algorithm for the movie rating prediction.

I have $2$ vectors: $Y$ - with the real ratings, $Y'$ - with predicted ones.

When I calculate Standard Error of the Estimate (is it the one I need to calculate?) using following formula:

$$\sigma_{est} = \sqrt{\frac{\sum (Y-Y')^2}{N}}$$

I'm getting result of $\sim 1.03$. But I thought that it can't be $> 1$. If it is not, then what does this number say to me?

results = load('first_try.mat');
Y = results(:,1);
Y_predicted = results(:,2);

o = sqrt(sum((Y-Y_predicted).^2)/rows(Y))
  • 2
    $\begingroup$ Why did you think it can't be bigger than 1? $\endgroup$ Commented Oct 2, 2018 at 6:54
  • $\begingroup$ I thought that 1 means that 100% match $\endgroup$ Commented Oct 2, 2018 at 6:54
  • 2
    $\begingroup$ That's not what a standard error (SE) is, the SE is the standard deviation (SD) of your estimate, which goes from 0 to infinity. $\endgroup$ Commented Oct 2, 2018 at 6:55
  • 1
    $\begingroup$ Depends what you mean by accuracy, the mean is usually used as your accuracy measure, the SE is used to show how variable this measure is (for ex. building a 95 % CI). Whether 1.03 is big in your case depends entirely on your data. $\endgroup$ Commented Oct 2, 2018 at 7:10
  • 2
    $\begingroup$ "I thought that 1 means that 100% match" if we had a perfect match (i.e. $Y=Y'$), then wouldn't the numerator of your expression equal $0$ and hence give you $\sigma_{est}=0$? $\endgroup$ Commented Oct 2, 2018 at 8:30

1 Answer 1


K-NN is a measure of distance, thus the result of your equation will depend on the scale of your data. If the ratings are in a scale from 0 to 100. Then if you always predict very poorly you are evidently going to have values much larger than 1.

For example, for a very bad predictor

import numpy as np

Y = [100, 90, 100, 90]
Y_p = [10, 10, 10, 10]

np.sqrt(np.sum(np.subtract(Y, Y_p)**2)/len(Y))



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.