3
$\begingroup$

I have a big .CSV database of 25k users with various attributes of the last user's activity and events during the past 6 weeks

This is an example of the data:

username        (B)      (C)             (D)        (E)  
nicole          524      329             203        787
asteria         197      186             286        120

I want to create a common behavior pattern based on the values of the attributes of each user and run an algorithm to find a common pattern that defines this group's behavior and to find out if there is any correlation in the dimensions values and which dimension define this list of users. I am fully aware that correlation does not necessarily equal causation.

Now I see several challenges in front of me and would greatly appreciate some input from others, or some good resources to find further information.

What is the model for this problem ? What kind of Algorithm is the best to deal with this situation? What is the tools that you recommend to use of the project ?

Any ideas would be great.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

The most common approach is to create business rules handmade, based on the univariate and multivariate analysis of the variable.

Basically, do some frequency count, see if you could isolate some subset of your data just looking at one or two variables.

Then when you have your labels, create a linear or so model with this new variable as output. For exemple, a linear discriminant analysis. The analysis will supply you with new insights on your group.

If you want to rely on an algorithm, two solutions:

As you don't seems to have a lot of variables, a non-supervised segmentation could do the job. For exemple, a k-Nearest Neighbor or a decision tree are basic and good aproachs.

With a few more variables, I like is to do a principal component analysis then a non-supervised classification to define your group on the result of the PCA. Note that a PCA + handmade rules based on the analysis of your PCA result may be enough.

Each time, in the end, a discriminant analysis and a profile of your groups to asses the quality of your results.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for a great answer , in fact i have more attributes and i am aslo including both behaviour and demographic attributes. So according to your answer i should do principal component analysis then a non-supervised classification. Do you recommend any particular tools ? Thanks again $\endgroup$
    – Wassimply
    Commented Jan 26, 2016 at 17:20
  • $\begingroup$ Sure, If you already know R, then R for the data management, profiling, first analysis and the h2o package/software for your pca (the R implementation of PCA are a little bit slow and may not handle millions of rows). $\endgroup$
    – YCR
    Commented Jan 26, 2016 at 17:26
  • $\begingroup$ Thanks again for your valuable help , I have implemented PCA and reduced my multidimensional data to lower key dimensions. But now how I can i proceed to find out the pattern that i have to follow to define this group of users , The result i want to have is a table of attributes with a number range that defines the group pattern. $\endgroup$
    – Wassimply
    Commented Jan 27, 2016 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.