0
$\begingroup$

I'm looking for a statistical approach to compare ranks produced by 2 versions of an ranking algorithm (A & B) against the actual ranks in the system. This is about ranking hospitals and the algorithm uses k-Means clustering. Version A of the algorithm is what is live now. We plan to improve the algorithm's (version B) ranking performance (so that the ranks more closer to the actual rank) by including additional features. I'm looking for a reliable approach to prove if version B is really moving the ranks closer to the actual rank.

Actual Rank
Hospital_ID Rank
H1          1   
H2          2
H3          3
H4          4
H5          5

Algorithm Version A (Existing version)
Hospital_ID Rank
H1          3
H2          4
H3          5
H4          4
H5          2

Algorithm Version B (Improved version)
Hospital_ID Rank
H1          2
H2          3
H3          1
H4          2
H5          5

The example above is just a sample, there could actually be 300 - 500 hospital ranks to be compared & evaluated. I know one way is to use "Root Mean Squared Error (RMSE)" which is a standard error metric in Regression, however this could hide matching trends in my opinion.

I think there might also be a way to prove this via visualization, but I'm not sure what type of chart will help with this. Any comments related to this is also appreciated.

Improve this question
$\endgroup$
1
  • $\begingroup$ Although it is equivalent, it is surprising that you are not using the word correlation in this context. $\endgroup$
    – Valentas
    Jul 4 at 20:38

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.