1
$\begingroup$

I got this definitional difference between distortion and inertia from here:

Two values are of importance here — distortion and inertia. Distortion is the average of the euclidean squared distance from the centroid of the respective clusters. Inertia is the sum of squared distances of samples to their closest cluster centre

However, when I searched for an example from here:


distortions.append(sum(np.min(cdist(X, kmeanModel.cluster_centers_,
                                        'euclidean'), axis=1)) / X.shape[0])

I took one inertia value from:

enter image description here

and 1 distortion value from :

enter image description here

however the values 217.64 and 3.45 aren't simply a division of 217.64 by count of variables in x (17).

Can anyone detail on what exactly is the formula for distortion?

$\endgroup$

1 Answer 1

1
$\begingroup$

The reason is that

distortions.append(sum(np.min(cdist(X, kmeanModel.cluster_centers_,
                                        'euclidean'), axis=1)) / X.shape[0])

does not square the distance as stated in the definition

Distortion is the average of the euclidean squared distance from the centroid of the respective clusters

If you replace

    mapping1[k] = sum(np.min(cdist(X, kmeanModel.cluster_centers_,
                                   'euclidean'), axis=1)) / X.shape[0]

with

    mapping_squared[k] = sum(np.square(np.min(cdist(X, kmeanModel.cluster_centers_,
                                   'euclidean'), axis=1))) / X.shape[0]

you will get the desired result:

IN: for key, val in mapping_squared.items():
    print(f'{key} : {val}')

OUT:  
1 : 12.802768166089965
2 : 4.025210084033612
3 : 0.954621848739496
4 : 0.7467787114845938
5 : 0.5647058823529413
6 : 0.42156862745098034
7 : 0.29901960784313725
8 : 0.24019607843137258
9 : 0.16666666666666669

That is, the values in mapping_squared can be multiplied by the number of samples to match inertia:

IN: np.array(distortions_square) * X.shape[0]

OUT:
array([217.64705882,  68.42857143,  16.22857143,  12.6952381 ,
         9.66666667,   7.51666667,   5.66666667,   4.08333333,
         3.        ])

On a side note: Distortion and SSE are usually used interchangeably. See, for example, the paper Scaling Clustering Algorithms to Large Databases:

Distortion is the sum of the L2 distances squared between the data items and the mean of their assigned cluster

This question touches the subject too.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.