3
$\begingroup$

I need to analyse a dataset about mobile phone usage (#calls, #sms, #internetConnections) per each cell and hour in the different days.

[date] [CDR/Position] [#calls] [#sms] [#internetConnections]

My purpose is detecting similarities in the data (Monday-Tuesday is similar... or Monday night is different...). After this, I'd like to find the reason they are similar/dissimilar.

What can I apply?

Improve this question
$\endgroup$
4
  • $\begingroup$ Do you have data already aggregated by date? Is it the case when your columns are something like: date_1, #calls, #sms, #internetConnections? How many days do you have such data for (in terms of number of rows)? $\endgroup$
    – Nitesh
    Nov 11, 2014 at 17:56
  • $\begingroup$ @Nitesh yes exactly. I have 12 months, millions of records. $\endgroup$
    – marcodena
    Nov 11, 2014 at 21:36
  • $\begingroup$ This needs a fair bit more information. What do you think drives similarity, and what things are you looking for similarity in. Times? $\endgroup$
    – Sean Owen
    Nov 13, 2014 at 4:39
  • $\begingroup$ @SeanOwen sorry, I think the right term is "patterns". Similar amount of calls or similar behaviours (Friday night have a peak like in Saturday night) $\endgroup$
    – marcodena
    Nov 13, 2014 at 14:41

1 Answer 1

Reset to default
1
$\begingroup$

There are two straight forward (vanilla) ways without going for any fancy featurization:

Clustering:

Run a clustering algorithm. Something like k-means should work well with this kind of a dataset. While doing this, I would not feed the day_of_week information into the clustering algorithm.

I would suggest running k-means (after normalizing each of the columns). Choose a small number of clusters that is easy to investigate (or you could use the number of clusters that maximizes the BIC).

Investigate the clusters to understand membership by day_of_week in each of these clusters.

Multi-class Classification:

Treat the day_of_week as the response that you would like to predict. Build a decision tree of a fixed depth to predict the day_of_week given the columns. By examining this tree, you can easily tell, which decisions led to a set of leaves being labeled Sunday vs the set of decisions that led to a set of leaves being labeled Monday. These decisions will also help you understand the similarities between different days.

Improve this answer
$\endgroup$
2
  • $\begingroup$ first of all thx. In Clustering: do you mean I should cluster all the days (24 features) and see how it works? And what about trends? I mean, during winter there is a different usage than in summer $\endgroup$
    – marcodena
    Nov 12, 2014 at 9:40
  • 1
    $\begingroup$ Yes, for clustering, consider clustering all the 12 months of data using all the features and see how it works. Since its unsupervised, it might as well be the case that clusters are created by trends (for example a cluster with all winter data and another with all summer data and so on). Thinking about this a little more carefully and reading your question again, I think multi-class classification might be a better approach. $\endgroup$
    – Nitesh
    Nov 12, 2014 at 16:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.