Overfitting in an unsupervised technique

Question

I am trying to understand if over-fitting can happen in an unsupervised technique like kmeans clustering. Could someone help me understand if and how this would happen?

Thanks.

Answer 1

I'm not sure if this is valid but how about two trivial clustering examples:

• Every object belongs to cluster which contain only this object. So for example if you would like to cluster N cars, there will be N clusters - one for each car.
• On the other hand there could be case when algorithm will pick one cluster which will contain all elements inside it - one cluster with all N cars.

Those will be valid clusters but obviously they will not give you any useful information.

Answer 2

Yes, overfitting occurs in unsupervised learning as well

Overfitting means your algorithm is finding patterns in attributes that only exist in this dataset and don't generalize to new, unseen data. In addition to finding real patterns, when overfitting, the algorithm is also finding "patterns" that are only stochastic noise.

Example for clustering

For clustering this means the clusters you are finding only exist in your dataset and can't be seen in new data.

Your algorithm might find two clusters in the dataset that don't exist for new data, because both clusters are actually subset of one bigger cluster. Your algorithm is overfitting, your clustering is too fine (e.g. your k is too small for k-means) because you are finding groupings that are only noise.