How does ML Clustering put to a practical real-world use?

Question

Newbie alert to data science and ML. I'm learning Supervised and Unsupervised learning at the moment and Supervised learning is easy to digest and I can relate to a lot of practical use cases. Unsupervised learning is where I just couldn't correlate to a the real-world use cases (although I found numerous of quotes where people say they use it for Customer Segmentation, Fraud Detection etc).

For argument sake, I'll just quote a sample taken from one of the MS Azure Studio examples.

The dataset contains Countries and their average protein intakes in various forms of food).

When this dataset is run through a KMeans algorithm, it creates 3 clusters and fits the country names in these clusters.

So in this specific example what is the problem I'm trying to solve?

Am I looking to find similar countries based on protein intake habits?

Am I creating groups based on the given dataset and then there is a human intelligence to qualify these groups (or clusters) to say "Vegetarian Rich Countries", "Red Meat Rich Countries" etc. Then when a new country comes we predict whether this country falls in which cluster?

In this case, there is an intermediate human intelligence is needed in the workflow which requires labelling the cluster (as opposed to labelling each datapoint in classification). Is this a correct understanding?

Answer 1

You want to understand the data.

So you run a clustering, then study how the points in a cluster differ from the points not in a cluster. Then based on these observations, you form a hypothesis. For example, you may notice that a cluster contains countries who eat a lot of fast-food and who are overweight. Then you can formulate the hypothesis that fast-food causes overweightedness, and then test that hypothesis.

This is a form of explorative data analysis. There is not a mathematical function to maximize, but it is a tool for humans to understand their data and then be able to formulate new hypotheses that would not have sprung to your mind otherwise.

Answer 2

Clustering is a really subjective problem. In most cases, you have a data set of unlabeled samples. One scenario is that you know there are k groups or clusters in the data, you just need to find those k clusters. Here, k-means or Gaussian mixture models (or any other relevant method) can be used to discover your clusters. Another scenario would be that you even don’t know how many clusters or groups exists in your data, you want to find the number of clusters using a clustering method. For example, a clustering algorithm which doesn't need to know the number of clusters beforehand like x-means. And in some cases you want to preprocess your data, partition it and then use the results in a supervised learning algorithm.

Answer 3

This is a pretty nice demo of how clustering works.

https://www.pythonforfinance.net/2018/02/08/stock-clusters-using-k-means-algorithm-in-python/

This is good too.

https://towardsdatascience.com/dbscan-clustering-for-data-shapes-k-means-cant-handle-well-in-python-6be89af4e6ea

Finally, take a look at this.

https://www.kaggle.com/dhanyajothimani/basic-visualization-and-clustering-in-python