I have a dataset of book reviews:

| user_id | ISBN | vote | votes_for_user | average_user_vote | ISBN_categ |
   213      3242X   4.5        12                  3.4             1 
   563      1245X   3.2        74                  2.3             2


  vote = rating given by user to a certain book
  votes_for_user = number of votes the user has in the dataset (nr of rows)
  average_user_vote = average of a user's votes
  ISBN_categ = integer categorical of the ISBN (since that is a string).

I want to apply a clustering algorithm such as DBSCAN to see how many clusters I can form with this dataset.

My question is: Should I apply the clustering on the dataframe as it is (minus the ISBN column) or should I construct more features for every user and construct a dataframe where every user appears only once, together with their features, and cluster that?

Remember, the intent here is to cluster users (by user_id), not data points (votes).

  • 1
    $\begingroup$ I think you answered your own question already. Running DBSCAN as is will cluster rows (votes) instead of users. Engineer your data so that each user is a row and the columns are features for that user, then do your clustering. $\endgroup$ – Pallie Apr 24 '19 at 11:11

If your objective is to find clusters of users, then you are interested in finding groups of "similar" reviewers.

Therefore you should:

  1. Retain information which relates to the users in a meaningful way - e.g. votes_for_user.
  2. Discard information which has no meaningful relationship to a user - e.g. user_id (unless perhaps it contains some information such as time / order).
  3. Be mindful of fields which may contain implicit relationships involving a user - e.g. vote may be a result of the interaction between user and ISBN.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.