0
$\begingroup$

I am currently working on a project that requires me to cluster the unlabeled input. The records contain personal information such as name, DOB, height, sex, etc. We need to cluster the same person in one group, here is the sample data:

+------------------------------------+
|        Record1     Record2         |
+------------------------------------+
| First Name   'Harry'     'Harry'   |
| Middle Name  'Jay'       'J'       |
| Last Name    'Potter'    'Potter'  |
| DOB Month    1           1         |
| DOB Day      1           1         |
| DOB Year     1993        1993      |
| Dr License   'A1234567'  Null      |
| Sex          1           1         |
| Address      Hogwarts'  'Hagwarts' |
+------------------------------------+

Those two records should be clustered in the same group.
I am still in the beginning phase of the project. I want to try with EM clustering using Gaussian Mixture model. But I do not know how I should preprocess the string, or should I do it and how to deal with the Null value.

$\endgroup$
1
$\begingroup$

Convert your key-Value Table to a table in wide format (1 row per person) , then calculate the distance matrix using the Jaccard Distance (which can convert categorical values into numeric values)

library(dummies)
dummy_dat <- dummy.data.frame(my_data)

jaccard_dist <- dist(dummy_dat, method = "binary")

#  Distance Matrix
jaccard_dist
$\endgroup$
0
$\begingroup$

Gaussian mixture modeling is not appropriate to use on text.

Text just is not Gaussian.

Choose appropriate distribution assumptions instead.

$\endgroup$
1
  • $\begingroup$ Thank you very much! Could you please share some link or hint about where I can start? I am a beginner on ML field. I really appreciate your help! $\endgroup$ – roy Jun 19 '18 at 14:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.