5
$\begingroup$

I do at the moment some data experiments with the Graphlab toolkit. I have at the first next SFrame, with the three columns:

Users Items Rating

The pair in the same row from every Users and Items values build the unique key and the Rating is the corresponded float value. These values are not normalised. First of all, I do someself next normalisation:

  1. Division of every rating value of specific user by the rating maximum from this user (scale between 0 and 1)
  2. Take the logarithm by every rating value

Afterward I create a recommender model and evaluate the basic metrics for it.

In this topic I invite everybody to discuss another interesting normalisation methods. If anybody could tell some good method for data preparation, it would be great. The results could be evaluated because of the metrics and I can publish it here.

PS

My dataset is comming from some music site, the users rated some tracks. I have approximately 100 000 users and 300 000 tracks. Total number of ratings is over 3 millions (actually the matrix is sparse). This is the most simple data set, which I analyze now. In the future I can (and will) use some additional information about the users and tracks (f.e. duration, year, genre, band etc). At the moment I just interest to collect some methods for rating normalisation without to use additional information (users & items features). My problem is, the data set doesn't have any Rating at the first. I create someself the column Rating, based on the number of events for unique User-Item pair (I have this information). You can of course understand that some users can hear some tracks many times, and another users only one time. Consequently the dispersion is very high and I want to reduce it (normalise the ratings value).

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Can you tell us where you got the data set from? I would suspect that the data-prep/normalization methods will depend on the source of the data. $\endgroup$
    – nfmcclure
    Commented Nov 19, 2014 at 18:57
  • $\begingroup$ Can you also tell us some other basic characteristics of the dataset like the # of users, # of items, total # of ratings in your dataset? Why are you taking the log of the ratings instead of the raw ratings? $\endgroup$
    – Nitesh
    Commented Nov 20, 2014 at 0:32
  • $\begingroup$ Thank you, I add some additional information to the main post $\endgroup$
    – Guforu
    Commented Nov 20, 2014 at 7:43

1 Answer 1

Reset to default
1
$\begingroup$

Another good normalization is zScore normalization. Was already implemented in python in the module scipy.stats

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.