0
$\begingroup$

Hi I am working on a movie recommendation system and I have like 80.000 movies in my data set. The problem is there are like thousands of directors which is a problem when I represent a movie as a vector. One of the features of a movie is its director and I do not want my movie vector to be like 5000 in length. One hot encoding is advised everywhere but I want to do something else that will make my movie vectors small in length. I heard integer encoding but I have never seen an example of it nor it is advised. One thing that comes to my mind is to create a small interval like from 0 to 5 and let's say I have 6000 directors and I will assing first director to 0 and the next director will be assigned to 0+6/6000 and third one will be assigned to 0+2×6/6000 etc. However, I have a lot of doubts. what can you suggest? I am a rookie and not well-versed on the topic.

$\endgroup$
1
$\begingroup$

There are many other options instead of doing one-hot encoding. You can do target encoding, count encoding and many others. Here is a nice library for encoding categorical variables. Explore all the operations in here. Here's the link.

Target encoding would be my bet.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.