1
$\begingroup$

Say I have a dataset, which consists of n customers. A customer is represented by a name, an order, an orderAmount and a timestamp (time of day) for when the order was placed. It could look like this:

name     |order|orderAmount|timestamp    
customer1|tv   |1          |08:30
customer2|hifi |1          |12:00
customer3|hifi |3          |12:30
customer4|tv   |2          |09:30
customer5|cd   |10         |10:00
customer1|tv   |2          |11:30
...      |...  |...        |...

What I'm interested in is clustering these customers, so it would be possible for me to clearly differentiate between them. By looking at the above set, it's clear that there's a difference in what, how much and when they buy, but I would like to automate this process, and I assume clustering is one way to do it, but please correct me if I'm wrong.

One thing in particular I'm not sure of is how I would represent this dataset in a "customer-matrix". I know, for example, a distance-based clustering method such as KMeans requires some sort of normalized input, but how would I get started on that with a dataset such as this?

When/if I can produce good clusters, I imagine I could assign labels to these clusters and use these labels to train a classifier?

I'm a beginner to data science, so there might be a step or two missing in my process so please bear with me.

$\endgroup$
2
$\begingroup$

Suggestion: use an indicator variable for your "order"+"orderAmount" data. The table would look like this, which seems more suitable for distance metrics:

name      | tv | hifi | cd | timestamp
customer1 | 1  | 0    | 0  | 8:30
customer2 | 0  | 1    | 0  | 12:00
customer3 | 0  | 3    | 0  | 12:30
customer4 | 2  | 0    | 0  | 9:30
...

If you do so, then I'd also suggest to normalize those indicator variables (make sure they are in the interval [0, 1], otherwise you could have a order of 1000 "cd" completely dominating the maximum of 10 "tv" for example).

Another thing to watch out is the possibility of binning. If the time of the day doesn't matter much, you could group all the orders from the same day and customer in a single row.

Also, if you can manually label some examples, you could use a semi-supervised algorithm, and maybe it would have better performance than completely unsupervised clustering. Some possible algorithms are HMRF-KMeans, co-training variants, and Spy EM.

$\endgroup$
  • $\begingroup$ manual labeling (like indicator variable) or labeling defeats the purpose of unsupervision. Once labeling is done the task becomes supervised. $\endgroup$ – mnm Oct 22 '17 at 13:05
  • $\begingroup$ Thanks, I like the idea of making columns for each order type. As for the timestamp, if I were to actually need it as a feature, how would I go about normalizing it? Is it just a matter of crunching the 24 hours of a day in between 0 - 1? $\endgroup$ – Khaine775 Oct 22 '17 at 13:38
  • $\begingroup$ @Ashish it's very different to label your 100,000 instances training set, or to label 5 to 10 instances to guide your learning algorithm; semi-supervised learning is very useful in practice. $\endgroup$ – Mephy Oct 22 '17 at 14:16
  • $\begingroup$ @Mephy I reiterate labeling a dataset defeats the purpose of un-supervision. If training a It can be a two step process; wherein at step 1, clustering can be used to identify the patterns or features and in step 2, use the identified features as labels for a supervision task. This two step process is aptly termed "semi-supervised" learning as you have mentioned. $\endgroup$ – mnm Oct 23 '17 at 7:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.