I'm working on a recommendation problem, broadly following the Youtube paper on theirs. Their surrogate problem is to recommend the next video a user will watch. One feature they include in their model is a representation of watch history (I.E. which videos a user has watched), which they achieve by embedding the Video ID's and then averaging that embedding.
So for data, they might have...
[
[1234, 5678, 9012],
[1234],
[9012, 1234, 8245],
]
and each of those IDs is embedded into say 8 features, those 8 features are averaged for every observation in the training data and those 8 average features are input to the deep layers of the network.
Now I can replicate that in keras using a combination of tf.keras.preprocessing.sequence.pad_sequences
plus Embedding()
and GlobalAveragePooling1D()
, and that works reasonably well. My problem is that I'd like to evaluate other algorithms on this same problem (after all, I have WAY less data than Youtube does so I'm not sure I need an NN at all). I can't really conceptualise how to deal with this encoding of historical "watches" though in a way that would work in a more traditional algorithm like XGBoost or RandomForests in sklearn. Ordinal Encoding and taking the average seems plain silly, I obviously can't OneHot encode the 1st to nth video watch and so on. Any ideas of how to tackle this would be greatly appreciated.
EDIT:
I guess possible FeatureHasher fits the bill? Rather than an average, it produces a sum of each of the hashed input categories. That's still (be default) gonna lead to 10s of thousands of features though so I'm not sure that it's going to be a significant improvement over OneHotEncoding!
EDIT 2:
Alternatively, perhaps I could treat this as a text embedding problem by concatenating the array of categories into a single string and use CountVectorizer to create an array of (n_samples, n_categories), then use TruncatedSVD to reduce that to a more manageable number of components.
I guess I've a couple of ideas there, anyone know if they're worth trying?