# Predicting and clustering at the same time?

I want to build a segmentation to substitute the existing RFM segmentation which is a basic segmentation based on the Recency, Frequency and Monetary values.

The new segmentation will be used for two purposes:

1. Rank customers per value
2. Have homogeneous groups of customers, so there should be an interpretation and meaning for each group.

I think this shouldn't be an unsupervised learning, but a mix of both supervised and unsupervised learning.

So What I did is that I defined a target variable that takes 1 if the customer is active in the next period and 0 if not. then I used a decision tree to predict this value, so that the leaves represent the clusters. the percentage of active customers in each group is used to rank these clusters.

Do you think that's a good solution and do you have other ideas ?

• You are using some regularization method, such as limiting maximum depth or minimum samples per leaf, on your decision tree, right? Otherwise, you may have a awful number of clusters and they can vary dramatically, since the tree will grow until each leaf only has customers of one type which is subject to noise. Jun 15, 2016 at 11:20
• Exactly! I added conditions on minimum leaf's size. So now I have 13 leaves which is reasonable I think. Jun 15, 2016 at 14:45

Your solution seems interesting. However, it implies that the customers’ value is assessed through whether they are active or not in next period. It might be interesting to look at the purchase by customer for, say future 8 weeks (or future 6 weeks or 12 weeks for that matter – the time frame depends a lot on the particularity of the business). This measurement reflects whether customers are active or not in the next period and also quantifies their 'activity’.

For a somewhat similar project I implemented the following solution:

• Start by randomly selecting one breakpoint for each of the three dimensions (RFM); by breakpoint I mean a random value among the unique values of Recency, another randomly selected value for Frequency, and another one for Monetary.

• Draw some initial segments based on these three breakpoints by selecting groups of customers that are relatively similar in terms of RFM according to the selected breakpoints. As there are 3 breakpoints (one per variable), you will get 2^3 segments because each variable will be broken down in 2 parts leading to 8 segments in total. In other words, you will have:

Segm 1: Recency < breakpoint_Recency & Frequency < breakpoint_Frequency & Monetary < breakpoint_Monetary

Segm 2: Recency < breakpoint_Recency & Frequency >= breakpoint_Frequency & Monetary < breakpoint_Monetary

Segm 3: Recency < breakpoint_Recency & Frequency < breakpoint_Frequency & Monetary >= breakpoint_Monetary

Segm 4: Recency < breakpoint_Recency & Frequency >= breakpoint_Frequency & Monetary >= breakpoint_Monetary

Segm 5: Recency >= breakpoint_Recency & Frequency < breakpoint_Frequency & Monetary < breakpoint_Monetary

Segm 6: Recency >= breakpoint_Recency & Frequency >= breakpoint_Frequency & Monetary < breakpoint_Monetary

Segm 7: Recency >= breakpoint_Recency & Frequency < breakpoint_Frequency & Monetary >= breakpoint_Monetary

Segm 8: Recency >= breakpoint_Recency & Frequency >= breakpoint_Frequency & Monetary >= breakpoint_Monetary

• Compute the mean of future 8 weeks purchases for each of the resulted segments

• Then, of course, this segmentation was random and is not the optimal one. To get the optimal segmentation you then implement a Genetic Algorithm to re-do the previous procedure and explore the space for better breakpoints. By this you will likely get the optimal segmentation of homogeneous customers in terms of their purchasing behavior (as characterized by RFM) – optimal with respect to their value (as you decide to define their value).

By the way, the fitness function for the GA aims to maximize the mean of future 8 weeks purchases per segment. Or you can also set your own objective on how to measure the value of customers and define the fitness function accordingly.

Hope this helps. Good luck!

Update: This solution allows you to find 'the homogenous groups of customers' and 'rank segments per value'. I am not sure if this is what you meant by 'rank customers per value'. In my view supervised clustering defeats the idea of individual ranking (but maybe I am wrong). Moreover, customers change behaviors from one period to the other and it is perhaps better to cluster them in terms of purchasing behavior, not to rank individuals at a point in time. In this way, when a customer changes behavior (in terms of RFM), we know where s/he went (i.e. in which cluster), and thus we can assess whether s/he 'improved' or not. In the latter case, marketing people can take action to incentivize the concerned customer to move back into the better off segment.

• Thanks a lot for sharing, very interesting! I didn't know about GA and it's useful for optimizing problems having a global minimum (convex functions). Sep 26, 2016 at 10:16
• Ranking segments is not the main aim of the segmentation. But it's useful when performance is needed. The main application is, as you said, detecting when the customer is changing segment. best regards, Sep 26, 2016 at 10:23
• Just a quick thought on the GA fitness, shouldn't it be more like a standard clustering method, i.e. maximising the distances between segments and minimising the inner distances? If you try to maximise the mean for every segment you will receive segments with almost identical purchase volume.
– m-dz
Mar 31, 2017 at 14:15