Suppose, I am building a hotel recommendation system, that learns user profile based on his/her interaction with the system.

I have two classes, "Like" and "Dislike". For example, a user likes 5 "5-star" hotels, likes 3 "1-star" hotels and dislikes 10 "1-star" hotels. Now, a Naive Bayes Classifier will put a new 5 star hotel in the "Like" class and a new 1 star hotel in "Dislike" class. Now, what I am trying to achieve is a bit different, I don't want to classify, instead I want to extract features/attributes more closely related with the "Like" class. For example, I need "5-star" attribute from the "Like" class.

Now, obviously, I have many more attributes, than the hotel star rating. What I want to know is that, can Naive Bayes be modified to extract features from a class. Or are there any specialized algorithms that are more suited for this task?

I would have data in this form

user_id | hotel_id | hotel_rating | hotel_attr1 | .... | hotel_attr(n)| preference

1       1        5 star        some value   some value    like
1       2        1 star        some value   some value    dislike

Another thing, If I want to extract features such as "5 star" and "1 star" from a class, Is the data formatted above correct? or instead of hotel_rating as the feature, I need 5_star and 1_star as the features, instead of them being values of the hotel_rating feature?

  • $\begingroup$ You can look up for how to select features based on information gain, calculating the entropies. That's exactly what you need. Decision tree classifiers are also based on these calculations. $\endgroup$ – Syed Ali Hamza May 8 '16 at 22:04

To me, what you described, sounds very much like collaborative filtering. In collaborative filtering, the features of the user(usually individual preferences) and features of the item(e.g. type and attribute of hotels) are trained collaboratively, gradually improving each other through iterations and converge to an optimal. Take a look at Andrew Ng's lecture, you can get a very quick idea of how it works.

In my opinion, collaborative filtering is a relatively basic technique for providing recommendations. Decision tree based models are more generally used in real life recommendations by big companies.


As you have two classes, like and dislike, it's a supervised and not a clustering problem.

Why don't you just try naive bayes and decision trees?

  • $\begingroup$ As I said, I don't want to actually classify, I just want those features that define a "likes", "dislikes" for user more strongly than other features. Haven't looked into Decision Trees though...I'll have a look. $\endgroup$ – panther1 May 8 '16 at 19:55
  • $\begingroup$ So you want feature selection? Decision trees do also select features; and you can also analyze the features by their naive bayes scores... $\endgroup$ – Has QUIT--Anony-Mousse May 8 '16 at 19:57
  • $\begingroup$ Yes, basically..I will look more into naive bayes scoring... $\endgroup$ – panther1 May 8 '16 at 20:08
  • $\begingroup$ Try this link cran.r-project.org/web/packages/Boruta/Boruta.pdf $\endgroup$ – Milan Amrut Joshi May 9 '16 at 12:27

Try this link https://cran.r-project.org/web/packages/Boruta/Boruta.pdf...

it is called Boruta algorithm will give u relevant feature selection , It is done

in r language, It kind of gives you best features required for your analysis..its

like wrapping around random Forest


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.