0
$\begingroup$

I have the following data

 Prefix Label   Year    Symbols Tags    %_Men   Prop    Uni_Role    Uni_Role_2  #_Cars  #_App
    0   IN  0.0     1996    []        []    0.08    0         11         22           0     1
    1   IT  0.0     2000    []        []    0.00    0         2          46           1     1
    2   ES  1.0     1999    []        []    0.10    0         10         30           0     1
    3   UK  1.0     1999    [!]      [!]    0.07    1         10         25           0     1
    4   USA 0.0     2007    []        []    0.07    0         11         23           0     1
    6   Republic of San Marino 0.0  1996    []  []  0.07  0   10         20           0     1
    7   USA 0.0     2002    []        []    0.05    0         5          22           0     1
    8   IN  0.0     2008    []        []    0.03    0         4          18           0     1
    9   IN  0.0.    1997    []        []    0.04    0         9          19           1     1
    10  IN  0.0     1997    []        []    0.06    0         8          20           3     1

I would like to study correlation among variables (target variable is Label). Since there is a mix of categorical (including lists, under Symbols and Tags columns), numerical and binary variables. For categorical variables probably I should consider Chi-Square, for numerical Pearson. I do not know how to get information from binary, but I know that it should not be possible to apply Pearson. The problem is a classification one, where I should predict the status of a citizen. Have you had similar problems? Any idea on how I could do that?

$\endgroup$
1
$\begingroup$

There are some distance/dissimilarity functions for two binary/boolean vectors such as Jaccard distance and Hamming distance. You can check them from scipy's documentation. You can calculate similarities between two boolean columns by using these distance functions.

from scipy.spatial import distance
distance.hamming([1, 0, 0], [0, 1, 0])
0.66666666666666663

from scipy.spatial import distance
distance.jaccard([1, 0, 0], [0, 1, 0])
1.0
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.