What is the standard way for evaluating and comparing different algorithms while developing recommendation system? Whether we need to have a predetermined annotated ranked dataset and then compare with precision/recall/F measure of different algorithms ? Is this the best way for evaluation ? Or is there any other way to compare results of various recommendation algorithms ?


2 Answers 2


The standard way to evaluate a recommendation engine is by using the RMSE (root mean square error) of the predicted values and the ground truth.

It is almost a SOP that, after finishing developing a recommendation engine, we will evaluate this engine by comparing its RMSE with other famous, common recommendation algorithms like SVD, tranditional CF, even RBM, etc.

Some terms mentioned above do not seem to be related with recommendation, but you can easily find on the internet how these techniques can be used in this topic.


Metric for recommendation system:

  • Coverage: the proportion of unique items that might recommend out of the total number of unique items.
  • Mean Reciprocal Rank: evaluate a model's ability to generate a relevant recommendation at the top-ranked position.
  • Normalized Discounted Cumulative Gain: tell you how well about the model ranks recommendation.
  • ... more

Here have some metrics specific for the recommendation system - Amazon Personalize- Evaluating a solution version with metrics


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.