I'm writing a KNN (collaborative filtering) hybrid similarity recommender and I need some advice. It is based on this paper.

I've currently got 2 datasets. The first one is ratings (approximately 3500 films, 1,028,751 ratings) and the second one is personality (1812 users) where the personality of the user is measured using a FFM vector. They are from here.

The similarity between 2 users is calculated as follows:

$$sim(u,v) = A*sim_{ratings}(u,v) + (1-A)*sim_{personality}(u,v) $$

The rating similarities and personality similarities are both done using a Pearson correlation, as opposed to only ratings similarity (which is supported in some recommender modules).

I read up on the module Surprise but as far as I can tell, I wouldn't be able to write my own algorithm as it won't support compound similarity calculation. I went on to write things from scratch i.e. build a personality-personality similarity matrix and then a ratings-ratings similarity matrix between users with the intention of combining them, but as you can imagine, that is very computationally intensive and I simply don't get the results in time, not to mention the testing that would then come afterwards. I even tried to parallelise some aspects, but I'm simply not skilled enough to do that (but if someone believes this is the way to go, please comment)

I'm not sure how to leverage sklearn for this use, but I'd be really open to hearing suggestions (for sklearn or other modules) as I feel totally at a loss.

Alternatively, if you know of a better architecture in literature, please do share. I'm currently using Python.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.