3
$\begingroup$

I need to analyse a dataset about mobile phone usage (#calls, #sms, #internetConnections) per each cell and hour in the different days.

[date] [CDR/Position] [#calls] [#sms] [#internetConnections]

My purpose is detecting similarities in the data (Monday-Tuesday is similar... or Monday night is different...). After this, I'd like to find the reason they are similar/dissimilar.

What can I apply?

$\endgroup$
  • $\begingroup$ Do you have data already aggregated by date? Is it the case when your columns are something like: date_1, #calls, #sms, #internetConnections? How many days do you have such data for (in terms of number of rows)? $\endgroup$ – Nitesh Nov 11 '14 at 17:56
  • $\begingroup$ @Nitesh yes exactly. I have 12 months, millions of records. $\endgroup$ – marcodena Nov 11 '14 at 21:36
  • $\begingroup$ This needs a fair bit more information. What do you think drives similarity, and what things are you looking for similarity in. Times? $\endgroup$ – Sean Owen Nov 13 '14 at 4:39
  • $\begingroup$ @SeanOwen sorry, I think the right term is "patterns". Similar amount of calls or similar behaviours (Friday night have a peak like in Saturday night) $\endgroup$ – marcodena Nov 13 '14 at 14:41
1
$\begingroup$

There are two straight forward (vanilla) ways without going for any fancy featurization:

Clustering:

Run a clustering algorithm. Something like k-means should work well with this kind of a dataset. While doing this, I would not feed the day_of_week information into the clustering algorithm.

I would suggest running k-means (after normalizing each of the columns). Choose a small number of clusters that is easy to investigate (or you could use the number of clusters that maximizes the BIC).

Investigate the clusters to understand membership by day_of_week in each of these clusters.

Multi-class Classification:

Treat the day_of_week as the response that you would like to predict. Build a decision tree of a fixed depth to predict the day_of_week given the columns. By examining this tree, you can easily tell, which decisions led to a set of leaves being labeled Sunday vs the set of decisions that led to a set of leaves being labeled Monday. These decisions will also help you understand the similarities between different days.

$\endgroup$
  • $\begingroup$ first of all thx. In Clustering: do you mean I should cluster all the days (24 features) and see how it works? And what about trends? I mean, during winter there is a different usage than in summer $\endgroup$ – marcodena Nov 12 '14 at 9:40
  • 1
    $\begingroup$ Yes, for clustering, consider clustering all the 12 months of data using all the features and see how it works. Since its unsupervised, it might as well be the case that clusters are created by trends (for example a cluster with all winter data and another with all summer data and so on). Thinking about this a little more carefully and reading your question again, I think multi-class classification might be a better approach. $\endgroup$ – Nitesh Nov 12 '14 at 16:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.