1
$\begingroup$

We did a POC for customer segmentation and followed the below approach

a) extract data from source system (SAP business objects)

b) Use python jupyter notebook to manipulate, merge and group data (multiple csv files)

c) We cluster based on some preset variables. So, we use the below 4 variables a) Recency (R)

b) Frequency (F)

c) Cutomer duration with our company (indicates loyalty) (Y)

d) No of different market segments entered by the customers (indicates cross-selling) (P)

d) Run 1d kmeans algorithm (Jenks Breaks algo) for each variable. So, 4 algos are run (for 4 variables)

e) For the sake of interpretability and for easy modifications of rules based on business criteria, we also incorporate a rule to finally come up with meaningful customer segments like below

f) based on each business users defined requirement, we send out automated emails on a monthly basis

enter image description here

Now, my questions are as follows

a) How can I make this automated? my data gets updated every 45 days. We are always looking to create clusters of 4/5 for Recency and Frequency variable and 2 and 3 for Prod and years variable. This will not change.

b) But since, we provide results to sales users to follow up with customers, we want to be able to track the results across each run and have a dashboard to know whether a customer who needed attention is now moved to loyalist or champions segment because our sales users continuously followed up with them. We would like to measure that transition between each segments and this is planning to be used as a KPI for sales users. How can we do this?

c) Is 1d-kmeans algorithm considered as an AI algorithm?

d) How can this be made as a pipeline and any suggestions on how to improve this project further is welcome

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

The main advantage of unsupervised learning is to be able to make meaningful clusters and hence valid scenarios.

That's why, it is not always necessary to make a fully automated solution, but rather a robust one that could be later automated.

I don't know how many features you have, but UMAP is great for clusterizing non-linear data, even if you have more than 20 features. It is also a random algorithm and it has reproducibility.

I recommend to use UMAP with reproducibility and then K-Means to classify the clusters automatically.

Once it is done, get the range of the data for each cluster(i.e. min/max of each cluster + stardard deviation or mean values if interesting), so that you can detect the different groups, and make valid classifications rules. Those rules can be applied for any new data without needing to go through the UMAP/K-Means process.

If there is a lot of new data in the future, it could be necessary to repeat UMAP/K-Means because of new potential groups. It depends on the data complexity over time.

Here is an example how to achieve this.

More information:

Understanding UMAP interactively.

Basic clustering with UMAP.

How exactly UMAP works.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Nice. detailed answer. Do you have any comments on the way I do segmentation based on your experience? is 1d-kmeans considered an AI algorithm in the 1st place? $\endgroup$
    – The Great
    Aug 15, 2022 at 10:10
  • $\begingroup$ Segmentation is clustering, right? Clusters are done automatically through UMAP and it would group data that are correlated. 1D KMeans is enough if you only have 1 or 2 input features, but if the ranges are quite obvious, maybe you don't even need 1D KMeans and you can set logical rules manually. How many features do you have? $\endgroup$
    – Nicolas Martin
    Aug 15, 2022 at 12:09
  • $\begingroup$ We have more than 30 features. Since, we use purchase transaction history da, we use recency, frequency, duration and prod count variables. So, for the sake ofi nterpretability and to have control over criteria, we used rule based criteria to name clusters $\endgroup$
    – The Great
    Aug 16, 2022 at 0:48
  • $\begingroup$ I used to have good clusters with umap and it is fully explainable because you can retrieve the ranges of each cluster and see if there is any outlier. In addition to that, umap is not linear, which is better than most linear algorithms. K-means is mostly used to detect clusters. It could be used to generate clusters, but generated clusters could be wrong. If you’re not convinced, you can compare the ranges of each algorithm and see which one has most meaningful and valid clusters. $\endgroup$
    – Nicolas Martin
    Aug 16, 2022 at 20:53
  • $\begingroup$ Does it answer your question? If not, please let me know. $\endgroup$
    – Nicolas Martin
    Aug 23, 2022 at 14:10
0
$\begingroup$

Whatever model you go for but you must evaluate your model prepare valuation data set using expert judgement. For example take any 100 customers and as your employees to read them from 1 to 10.

Prepare a logistic classification model. Provide weights to your features. Find out the cost of model. Minimise the cost

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.