2
$\begingroup$

I am trying to understand if over-fitting can happen in an unsupervised technique like kmeans clustering. Could someone help me understand if and how this would happen?

Thanks.

$\endgroup$
  • 1
    $\begingroup$ Mostly if you allow a model to have too many parameters, then it will appear to fit the data well. $\endgroup$ – Has QUIT--Anony-Mousse Jul 10 '17 at 23:56
  • $\begingroup$ How do you test your results for overfitting in a k-means run? Some people have said use a training set. I have about 1500 records and about 20 fields. $\endgroup$ – guest Mar 26 '19 at 19:33
2
$\begingroup$

I'm not sure if this is valid but how about two trivial clustering examples:

  • Every object belongs to cluster which contain only this object. So for example if you would like to cluster N cars, there will be N clusters - one for each car.
  • On the other hand there could be case when algorithm will pick one cluster which will contain all elements inside it - one cluster with all N cars.

Those will be valid clusters but obviously they will not give you any useful information.

| improve this answer | |
$\endgroup$
2
$\begingroup$

Yes, overfitting occurs in unsupervised learning as well

Overfitting means your algorithm is finding patterns in attributes that only exist in this dataset and don't generalize to new, unseen data. In addition to finding real patterns, when overfitting, the algorithm is also finding "patterns" that are only stochastic noise.

Example for clustering

For clustering this means the clusters you are finding only exist in your dataset and can't be seen in new data.

Your algorithm might find two clusters in the dataset that don't exist for new data, because both clusters are actually subset of one bigger cluster. Your algorithm is overfitting, your clustering is too fine (e.g. your k is too small for k-means) because you are finding groupings that are only noise.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.