1
$\begingroup$

I want to find out which users are similar to each other using their personal/organisational data, such as department, company, site, etc.

I have this data in a boolean format, as shown below:

       Dep1 Dep2 Comp1 Comp2 Site1 Site2
    U1  0    1     0     1     0     1
    U2  1    0     0     1     1     0
    U3  1    0     1     0     1     0
    U4  0    1     0     1     0     1
    U5  0    1     0     1     1     0
    U6  1    0     1     0     0     1

I want to select a user and identify the other users that are similar to them, so that I can recommend them software/hardware that those similar users are using.

I have looked into Cosine and Jaccard similarity but I have not seen much help when it comes to calculating them on Boolean data.

I'm using Python and am not new to the language, but I am fairly new to Data Analytics and ML. Any advice is appreciated!

$\endgroup$
  • $\begingroup$ Here in my view, Cosine similarity must still be useful. Can you please help with the steps that you followed for comparision? $\endgroup$ – SKB Jan 10 at 12:13
0
$\begingroup$

You can use the scipy module to calculate similarities. For your example, this would work like this:

import pandas as pd
users = ["U1", "U2", "U3", "U4", "U5", "U6"]
X = pd.DataFrame({
"Dep1": [0, 1, 1, 0, 0, 1],
"Dep2": [1, 0, 0, 1, 1, 0],
"Comp1": [0, 0, 1, 0, 0, 1],
"Comp2": [1, 1, 0, 1, 1, 0],
"Site1": [0, 1, 1, 0, 1, 0],
"Site2": [1, 0, 0, 1, 0, 1],
"Sitex": [1, 0, 0, 1, 0, 1]

}, index =users)    
X 

Out[2]:
    Comp1  Comp2  Dep1  Dep2  Site1  Site2  Sitex
U1      0      1     0     1      0      1      1
U2      0      1     1     0      1      0      0
U3      1      0     1     0      1      0      0
U4      0      1     0     1      0      1      1
U5      0      1     0     1      1      0      0
U6      1      0     1     0      0      1      1


from scipy.spatial.distance import pdist
from scipy.spatial.distance import squareform

# calculate the jaccard distances of the users, you can change jaccard to 'cosine'
distances = pdist(X.values, metric='jaccard')

# pdist returns a non-redundant vector of distances
# distances matrices are always symmetric, and diagonals are always zero
# so it is unnecessary to save all the information
# however if we want a matrix, we can use squareform()

distance_matrix = squareform(distances)

# the distances are bewteen 0 and 1, we get the similiarities by using 1-distances
similiarities = 1-distance_matrix

pd.DataFrame(similiarities, columns = users, index = users)

Out[3]:
          U1        U2   U3        U4   U5        U6
U1  1.000000  0.166667  0.0  1.000000  0.4  0.333333
U2  0.166667  1.000000  0.5  0.166667  0.5  0.166667
U3  0.000000  0.500000  1.0  0.000000  0.2  0.400000
U4  1.000000  0.166667  0.0  1.000000  0.4  0.333333
U5  0.400000  0.500000  0.2  0.400000  1.0  0.000000
U6  0.333333  0.166667  0.4  0.333333  0.0  1.000000
| improve this answer | |
$\endgroup$
  • $\begingroup$ I'm not at work right now but when I'm back I'll look into this. Thanks! I'll get back to you and mark your answer as correct if it works. $\endgroup$ – Top Lit Jan 10 at 12:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.