Calculate similarity on boolean data

Question

I am trying to implement simple recommender system and I am trying to understand different approaches to achieve my goal.

My dataset consists of users and items that they bought. I have information about what items user bought and descriptions of these items in form of titles.

At first I though I could use user based collaborative filtering approach but I am stuck at this. I am not quite sure how to calculate similarity for boolean data.

When I have data like this for example

   1  2  3  4
A  0  1  0  1
B  0  1  0  1
C  1  0  1  1
D  0  1  0  0
E  0  0  1  1

And I want to recommend items for user E, so how should I calculate similarity in this case? I chosen for example cosine similarity from scikit learn module in python. But I am not quite sure what should be considered as input. From what I read it should be only vectors of items that two users for which similarity is calculated have in common.

So for example if I wanted to compute similarity between user E and C what should be my input? Because if I input only values that they have in common it does not make sense right? Beacuse input will be [1, 1] and [1, 1] and for that similarity is 1.

Then I tried to input the whole vector like this:

from sklearn.metrics.pairwise import cosine_similarity
from numpy import array, reshape

c = array([1, 0, 1, 1])
e = array([0, 0, 1, 1])

result = cosine_similarity(c.reshape(1, -1), e.reshape(1, -1))

>>> result is 0.81649658

And this approach I think makes more sense but I am not sure if it is acceptable based on what I learned about this type of recommendation.

Answer 1

You should look at the Jaccard Index, is the de facto similarity between set of items, where the sets are represented using a boolean vector. In this boolean vector each coordinate represents an item, 1 means the item is present, 0 otherwise. For example: for an universe of items banana, orange and apple. the set banana, orange will be represented by (1, 1, 0). The Jaccard Index is the intersection of the sets over union the sets so for a set (1, 1) it's value is 1.

Cosine similarity is for real-valued vectors, but whether cosine similarity is better than the Jaccard index and vice versa depends on the application. You should do a test on your data and verify which is better, for a discussion see this question