I have a set of user transactions with a service and I suspect that there is a strong correlation between how users rate the service and how likely they are to use the service again.

I have chosen to represent user ratings - for each user in the dataset - as a number [1-5] which is the average of all ratings the given user has given in the past. There is more to it in regards to how to represent trends etc, but this is not in context for the problem at hand.

My issue is that there is a high number of users that have never left any ratings at all and I am not sure how to deal with these users, in terms of finding the right value for the rating-related features for them.

What I have tried so far is to represent ratings in a different way, shown below:

UID    rating_high    rating_medium    rating_low
  A              2                0             1
  B              0                0             0

where each of the predictors represent the count of ratings - in each rating category - by each of the users in the dataset.

In the above case, user A has left a good rating in 2 occasions, a low rating in one occasion and has never left a 'medium' rating. User B has never left any ratings for the service at all, therefore, he is assigned a count of 0 for all 'rating categories'.

on the 1-5 rating scale, I class all ratings below 3 as rating_low, all those equal to 3 as rating_medium and the ones above 3 as rating_high.

I have not managed to find any other way to represent this data, but I would very much rather not drop this part of the dataset as I believe it caries valuable semantics for my problem.

Any ideas on how to best deal with this issue are most appreciated!

  • Missing data is a common problem in recommender systems; you will find many leads if you search for this term. See if the matrix completion approach applies to your problem. Welcome to DataScience.SE and good luck! – Emre Aug 26 '16 at 17:24
  • Thanks @Emre. The issue is not exactly missing data really. The fact that the data is missing is in fact informative for the problem at hand. In this case it means that some of the users have not given any ratings, as opposed to 'not knowing' what their ratings are. In other words, the fact that I have no ratings for some of the users carries some value in itself, it's just that I'm not sure how to represent this in my dataset. Any ideas? Thanks for the help – Thanos Aug 26 '16 at 17:38
  • You can introduce an additional, boolean variable to indicate that an item has been used, and set the rating to None if the item has not been rated, therefore your table would look like (uid, rating, consumed). You see this scheme with implicit recommender systems. – Emre Aug 26 '16 at 17:55
  • Exactly, the only issue then being that I cannot really have numerical continuous values for some data points and 'None' for some other data points. I like the extra Boolean feature, but the problem of the actual 'rating' feature value remains. Thanks for the help! – Thanos Aug 26 '16 at 18:06
  • Why not? That's your missing data. – Emre Aug 26 '16 at 18:12

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.