# Clustering algorithm prior to model building?

I would like to understand, how a clustering algorithm can be used (if possible) to identify naturally occurring groups within a data set, prior to building predictive models/model, and to hence improve accuracy of models/model

In clustering the outcome variable or the response is unknown, this is why it's called clustering. Irrespective, of the fact the data being labeled or unlabelled, clustering can be applied as a data preprocessing algorithm. Essentially, you must proceed by employing the initial data preprocessing tasks (like missing value treatment, collinearity, skewness etc). Once, the data is "statistically clean", then you can apply any clustering technique. However, remember clustering requires data to be "grouped" such that data points within a group are related to each other and unrelated to other data points belonging to another group. This can be achieved only if you have a "statistically clean" data. The next important point to consider is, "how to determine the possible number of clusters". Because any clustering algorithm will divide the data points into groups oblivious of the fact whether the groups exist or not. Therefore, you will have to prove mathematically/statistically the occurrence of groups in the dataset. In literature, there exist several methods like the "Principal Component Analysis (PCA)" or the "elbow method". Once, you have determined such groups, you can then label the groups and perform predictive analytics.

• How would one determine the labels of these groups, where data points could be grouped in an implicit way ? Also, If there does not exist any naturally occuring groups within a data set, how would we be able to determine this ? (since the clustering algorithm groups the data, regardless of whether there are legitimate groups or not).
– Gale
Mar 5, 2018 at 13:45
• @Gale, what is the logic of applying clustering, if there does not exist any "naturally occurring groups" in the dataset? PCA, is a dimensionality reduction technique only that its unsupervised in nature. The elbow method, scree plot are some ways of determining if there exists "natural groups" in the data or not.
– mnm
Mar 5, 2018 at 13:56
• I didn't imply that we should use clustering when there are no naturally occurring groups. Wanted to know how we could determine the existence of groups, as you've answered :-P
– Gale
Mar 5, 2018 at 14:07
• Once, you have determined such groups, you can then label the groups and perform predictive analytics. Can you elaborate from here: To create a model on each group ? or to introduce a new feature, where levels = groups, and then to create a single model ? or other suggestions ? Thanks
– Gale
Mar 5, 2018 at 14:32

Clustering is primarily useful for you to understand your data.

It probably does not help much to automate that and try to use it for prediction. You can try, but I don't think it will even improve results much to use cluster information.

There are two things that you need to consider:

1. Clustering is very difficult, expensive, and hard to parameterize. That means it won't be easy to automate in a useful way - it breaks all the time when data changes.
2. Most good clustering (not kmeans) cannot predict the cluster label on new data. So using it will mean you first need to train a classifier to predict the cluster, then use this in another classifier to predict your class.
• Interesting response. Most good clustering (not kmeans) cannot predict the cluster label on new data. which clustering algorithms would you say are "better" ? So using it will mean you first need to train a classifier to predict the cluster, then use this in another classifier to predict your class. A little confused here, can you expand on what you mean by predicting the cluster then class, thanks.
– Gale
Mar 6, 2018 at 11:04
• k-means is usually the worst. It is fast, but very limited. Consider e.g. Hierarchical Clustering with average linkage, DBSCAN, OPTICS, ... these often produce much better results, but cannot predict for new data. So you need to train a classifier instead to predict the cluster label. Catch-22. Mar 6, 2018 at 12:20