2
$\begingroup$

I am trying to find the top 10 useful item recommendation. Items are divided into categories and then top10 in each category is calculated. There are six features based on which a score is assigned to each item and then they are sorted in decreasing order to get the top 10.

I am not sure are these the best top items? How to validate the output? I need to see which feature is dominating the most in calculating the score?

I have already calculated correlation, any other statistical measure?

$\endgroup$
2
  • $\begingroup$ Validation on a recommender can be tough, what context is it in? sales? $\endgroup$
    – TBSRounder
    Feb 5, 2016 at 17:49
  • $\begingroup$ @MarkHeiler Yes. $\endgroup$
    – maggs
    Feb 5, 2016 at 18:20

1 Answer 1

3
$\begingroup$

There are a lot of ways to see how well a recommender works and I think it really depends on your final goal. A list of evaluation metrics are here that might be useful to looks through. Although a measure like Accuracy/Precision/RMSE might suit your needs.

For simplicity, lets say you have a year worth of historical sales. You can build your system on the first 8 months of sales, then for each following month you can see what products would be recommended in that month and what products were actually purchased in that month. So within that you can get an idea of how well it is performing (of the 5 products someone bought, we had all 5 in the in the spots 1,2,6,8,9 of the top-10 list). You would want the product purchases to be closer to the top, which is why ranking metrics could be useful for you.

Of course there is a lot more to consider, but I think this is a good place to start. Are you trying to increase the pure number of sales or the revenue? Do certain products sell more depending on seasons/holidays? Are you using ratings or just associated sales (how do you recommend new products?). Factors like these can change your recommender and how you would want to evaluate it.

$\endgroup$
5
  • $\begingroup$ We don't have historical data about user. We have set of features of hotels on which we want to show the best recommended hotels so that our sales can increase at a good speed and user doesn't have to dig deep into the filters to select the best hotel. I am using the following approach - datascience.stackexchange.com/questions/10090/… $\endgroup$
    – maggs
    Feb 8, 2016 at 12:04
  • $\begingroup$ Hmmm, so the idea is to have a dynamic top-10 that changes for each filter they select? $\endgroup$
    – TBSRounder
    Feb 8, 2016 at 23:43
  • $\begingroup$ Its one of the steps. But I am not stuck at that. The problem is to show the best 25 hotels on initial page load. We don't know which will be the best, we have to make a algo. that gives us the best possible guess. Filters and similarity comes when a user starts interacting. Its the second phase of the requirement. $\endgroup$
    – maggs
    Feb 9, 2016 at 6:46
  • $\begingroup$ Can you see what hotel a customer would end up booking with, then go back and see where that hotel was on their top-25 list in order to validate it? $\endgroup$
    – TBSRounder
    Feb 9, 2016 at 14:05
  • $\begingroup$ Its again a customer data. Think of it as a cold start problem, with a difference that all users are new. We cannot show some random hotels to them, so we need to show in such a way that it fulfils all my features and their priorities. Thanks for your effort. If you need any further explanation please let me know. $\endgroup$
    – maggs
    Feb 9, 2016 at 16:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.