Testing for gender composition between groups

Question

I did kmeans with k=4 and I would like to find out if there are any differences in the composition of gender between the 4 clusters.

Do I use Fisher's Exact Test of Independence? Is there any recommended library in r that can handle this as well as post-hoc tests between groups?

Answer 1

For analysis of your clusters you can use the silhouette coefficient or silhouette width. These are available in cluster and factoextra package in R.

I will explain what basically in silhouette analysis:

The silhouette coefficient is calculated as follows:

1) For each observation i, it calculates the average dissimilarity between i and all the other points within the same cluster which i belongs. Let’s call this average dissimilarity “Di”.

2) Now we do the same dissimilarity calculation between i and all the other clusters and get the lowest value among them. That is, we find the dissimilarity between i and the cluster that is closest to i right after its own cluster. Let’s call that value “Ci”

3) The silhouette (Si) width is the difference between Ci and Di (Ci — Di) divided by the greatest of those two values (max(Di, Ci)).

Si = (Ci — Di) / max(Di, Ci)

So, the interpretation of the silhouette width is the following:

• Si > 0 means that the observation is well clustered. The closest it is to 1, the best it is clustered.
• Si < 0 means that the observation was placed in the wrong cluster.
• Si = 0 means that the observation is between two clusters.

Basic code to calculate the above and visualise:

library(cluster)
library(factoextra)

sil <- silhouette(clustering$cluster, dist(input))
fviz_silhouette(sil)