I am trying to find the top 10 useful item recommendation. Items are divided into categories and then top10 in each category is calculated. There are six features based on which a score is assigned to each item and then they are sorted in decreasing order to get the top 10.

I am not sure are these the best top items? How to validate the output? I need to see which feature is dominating the most in calculating the score?

I have already calculated correlation, any other statistical measure?

  • $\begingroup$ Validation on a recommender can be tough, what context is it in? sales? $\endgroup$ – TBSRounder Feb 5 '16 at 17:49
  • $\begingroup$ @MarkHeiler Yes. $\endgroup$ – maggs Feb 5 '16 at 18:20

There are a lot of ways to see how well a recommender works and I think it really depends on your final goal. A list of evaluation metrics are here that might be useful to looks through. Although a measure like Accuracy/Precision/RMSE might suit your needs.

For simplicity, lets say you have a year worth of historical sales. You can build your system on the first 8 months of sales, then for each following month you can see what products would be recommended in that month and what products were actually purchased in that month. So within that you can get an idea of how well it is performing (of the 5 products someone bought, we had all 5 in the in the spots 1,2,6,8,9 of the top-10 list). You would want the product purchases to be closer to the top, which is why ranking metrics could be useful for you.

Of course there is a lot more to consider, but I think this is a good place to start. Are you trying to increase the pure number of sales or the revenue? Do certain products sell more depending on seasons/holidays? Are you using ratings or just associated sales (how do you recommend new products?). Factors like these can change your recommender and how you would want to evaluate it.

  • $\begingroup$ We don't have historical data about user. We have set of features of hotels on which we want to show the best recommended hotels so that our sales can increase at a good speed and user doesn't have to dig deep into the filters to select the best hotel. I am using the following approach - datascience.stackexchange.com/questions/10090/… $\endgroup$ – maggs Feb 8 '16 at 12:04
  • $\begingroup$ Hmmm, so the idea is to have a dynamic top-10 that changes for each filter they select? $\endgroup$ – TBSRounder Feb 8 '16 at 23:43
  • $\begingroup$ Its one of the steps. But I am not stuck at that. The problem is to show the best 25 hotels on initial page load. We don't know which will be the best, we have to make a algo. that gives us the best possible guess. Filters and similarity comes when a user starts interacting. Its the second phase of the requirement. $\endgroup$ – maggs Feb 9 '16 at 6:46
  • $\begingroup$ Can you see what hotel a customer would end up booking with, then go back and see where that hotel was on their top-25 list in order to validate it? $\endgroup$ – TBSRounder Feb 9 '16 at 14:05
  • $\begingroup$ Its again a customer data. Think of it as a cold start problem, with a difference that all users are new. We cannot show some random hotels to them, so we need to show in such a way that it fulfils all my features and their priorities. Thanks for your effort. If you need any further explanation please let me know. $\endgroup$ – maggs Feb 9 '16 at 16:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.