1
$\begingroup$

I'm working on a recommender system that will recommend movies to users.

Movie ratings

Movie User Rating
100 201 5
105 256 8
... ... ...

Movie tags

Movie Tag
100 1
100 2
100 8
105 2
105 5
... ...

On top of collaborative filtering, I would like the model to take tags into account. How do I prepare a correct dataframe for this task, considering that one movie can have multiple tags?

Obvious solution is one-hot encoding, where I would make each tag a column with 1 or 0 depending on whether a movie has this tag, like so:

Movie User Rating has_tag_1 has_tag_2 has_tag_3 ...
100 201 5 1 1 0 ...
105 256 8 0 1 0 ...

The problem is - I have more than 5000 unique tags to work with, dataframe will be huge. Should I just leave only top 10 tags and drop the rest? What would be the correct approach here?

$\endgroup$

1 Answer 1

0
$\begingroup$

After one hot encoding of the tags, your features will increase. Then the curse of dimensionality will occur. In this case, you should apply PCA(Principal Component Analysis) to reduce the dimensionality or you can plot the correlation matrix and remove the low-correlated & high-correlated features with labels or Choose a machine learning algorithm that is suitable for your recommender system task, such as decision trees, random forests, gradient boosting models, or linear models to find out the important features.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.