10
$\begingroup$

I have a bunch of customer profiles stored in a cluster. These profiles are now used for creation of target groups for our email subscriptions.

Target groups are now formed manually using elasticsearch faceted search capabilities (like get all male customers of age 23 with one car and 3 children).

How could I search for interesting groups automatically - using data science, machine learning, clustering or something else?

programming language seems to be a good tool for this task, but I can't form a methodology of such group search. One solution is to somehow find the largest clusters of customers and use them as target groups, so the question is:

How can I automatically choose largest clusters of similar customers (similar by parameters that I don't know at this moment)?

For example: my program will connect to elasticsearch, offload customer data to CSV and using R language script will find that large portion of customers are male with no children and another large portion of customers have a car and their eye color is brown.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ What do you mean under "interesting groups"? Do you have some predefined important feature list? $\endgroup$
    – yatul
    Commented May 14, 2014 at 9:08
  • $\begingroup$ Interesting groups are any groups of size greater than some threshold an that are much bigger than other possible clusters. $\endgroup$
    – Konstantin V. Salikhov
    Commented May 14, 2014 at 9:11
  • 1
    $\begingroup$ It isn't clear how you will perform preparement steps of your data. But you should look at algorithms described at en.wikipedia.org/wiki/Anomaly_detection . If I were you, I've checked SVM method first $\endgroup$
    – yatul
    Commented May 14, 2014 at 9:31
  • $\begingroup$ I've read about SVM and I think its more about classification of newly created data after manual training on existing dataset - not about clustering existing data and finding abnormally big clusters. Am I right? If I am then this method isn't what I want. $\endgroup$
    – Konstantin V. Salikhov
    Commented May 14, 2014 at 16:15

1 Answer 1

Reset to default
6
$\begingroup$

One algorithm that can be used for this is the k-means clustering algorithm.

Basically:

  1. Randomly choose k datapoints from your set, $m_1$, ..., $m_k$.

  2. Until convergence:

    1. Assign your data points to k clusters, where cluster i is the set of points for which m_i is the closest of your current means
    2. Replace each $m_i$ by the mean of all points assigned to cluster i.

It is good practice to repeat this algorithm several times, then choose the outcome that minimizes distances between the points of each cluster i and the center $m_i$.

Of course, you have to know k to start here; you can use cross-validation to choose this parameter, though.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.