Can anyone recommend an algorithm/toolkit to rank items that have been rated in a hot-or-not style that gives statistical significance?

For example, out of a set of N images, two images are shown to users and the user picks the one that appeals most to him/her. The final output is a ranking: worst to best: image1 image5 image3 image8 etc.

I've come across the Bradley-Terry model with maximum-likelihood inference but I haven't found a toolkit that shows statistical significance of the rating.

Any toolkits in python especially would be much appreciated (though R is fine too).

  • $\begingroup$ I found this out: The BradleyTerry-Luce model computes a p-value that express how the visualizations compare to one specific visualization only, which serves as reference and is a parameter of the formula. Thus you need to perform additional to tests for for each comparison. To counteract the problem of making multiple comparisons tests you use a Bonferroni correction $\endgroup$ – user1011332 Aug 3 '18 at 18:22
  • $\begingroup$ can you please elaborate on your answer? sorry I can't comment and don't know how to contact you. I am facing a similar situation $\endgroup$ – ha554an Mar 27 at 19:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.