1
$\begingroup$

Dataset: I'm given the number of minutes individual customers use a product each day and am trying to cluster this data in order to find common usage patterns.

My question: How can I format the data so that, for example, a power user with high levels of use for a year looks the same as a different power user who has only been able to use the device for a month before I ended data collection?

So far I've turned each customer into an array where each cell is the number of minutes used that day. This array starts when the user first uses the product and ends after the user's first year of use. All entries in the cells must be double values (e.x. 200.0 minutes used) for the clustering model. I've considered either setting all cells/days after the last day of data collection to either -1.0 or NULL. Are either of these a valid approach? If not what would you suggest?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I believe your problem boils down to clustering time-series of different lengths. According to your question, you want the longer time-series of a power user to be considered similar to time-series of similar pattern but much shorter.
Therefore you should look into clustering techniques and distance metrics which allow for these properties. I don't know your language of choice but here are some of the many packages in R that you might find interesting :
- Fréchet distance - one of the packages offering this is kmlShape
- Dynamic Time Warping included in base R
- Permutation Distribution Clustering - package pdc
This would also solve your data formatting problem as to setting values to -1 or NULL would not be needed anymore. hth.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for the response :) I'll probably accept this answer in a day or two it's super helpful! A few comments/questions: In Looking up DTW I came across this paper which said DTW can't be used to compare sequences of different lengths? (see example 3 in the paper). I'm actually using python at the moment. I know I didn't ask but are you familiar with libraries which include the Frechet distance or PDC in python? I'm having trouble finding both (makes me think about jumping into r). Thanks! $\endgroup$
    – Chris F.
    Commented Apr 29, 2016 at 19:19
  • $\begingroup$ Glad it helped. Unfortunately I'm not familiar with python libraries but I found this and this for Frechet. Couldn't find anything on PDC. I guess you could look into translating it to python (here is the author A.Brandmaier's thesis on this topic). Alternatively, you could also look into calling R from python libraries. $\endgroup$
    – davidski
    Commented May 2, 2016 at 7:43
  • $\begingroup$ Update : You might find this link useful. $\endgroup$
    – davidski
    Commented May 3, 2016 at 9:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.