How to prepare data if each item has multiple categories (like tags)

Question

I'm working on a recommender system that will recommend movies to users.

Movie ratings

Movie	User	Rating
100	201	5
105	256	8
...	...	...

Movie tags

Movie	Tag
100	1
100	2
100	8
105	2
105	5
...	...

On top of collaborative filtering, I would like the model to take tags into account. How do I prepare a correct dataframe for this task, considering that one movie can have multiple tags?

Obvious solution is one-hot encoding, where I would make each tag a column with 1 or 0 depending on whether a movie has this tag, like so:

Movie	User	Rating	has_tag_1	has_tag_2	has_tag_3	...
100	201	5	1	1	0	...
105	256	8	0	1	0	...

The problem is - I have more than 5000 unique tags to work with, dataframe will be huge. Should I just leave only top 10 tags and drop the rest? What would be the correct approach here?

Answer 1

After one hot encoding of the tags, your features will increase. Then the curse of dimensionality will occur. In this case, you should apply PCA(Principal Component Analysis) to reduce the dimensionality or you can plot the correlation matrix and remove the low-correlated & high-correlated features with labels or Choose a machine learning algorithm that is suitable for your recommender system task, such as decision trees, random forests, gradient boosting models, or linear models to find out the important features.