I have a data set that has an attribute(A) with 300 different nominal values. Attribute A has a lot of noise. I decide to cluster my data based on other attributes that related to A. I hope to reach clusters with high correlation to A.

Now, I would like to know, how successful I was. In my opinion, more correlation between A and the clusters means more success. I found chi-squared as a method to check the correlation between two nominal attributes. However, chi-squared gives a value (X-squared) that doesn't have a specific range.

So by different numbers of clusters, how can I recognize which clustering is better?

Can I use p-value?

Is there any better way to evaluate clusters for this problem?

  • $\begingroup$ Does this help: stats.stackexchange.com/questions/21807/…? $\endgroup$ – pincopallino Aug 17 '16 at 8:42
  • $\begingroup$ You want to use the p-value of what, in particular? If would be useful if you could provide an example of the data set you're using. Moreover it's not really clear what you're asking (cluster, correlations X-squared? are different things, how do you relate them all together?). $\endgroup$ – gented Aug 18 '16 at 20:31
  • $\begingroup$ @gennaro-tedesco I cluster data and expect result(clusters) will be similar to attribute A. how should I measure similarity between attribute A and clusters? $\endgroup$ – parvij Aug 21 '16 at 4:18
  • $\begingroup$ Possible duplicate of How to evaluate clusters base on an attribute of the dataset? $\endgroup$ – Has QUIT--Anony-Mousse Aug 21 '16 at 21:00

Silhouette index is a nice measure to evaluate the quality of clustering.

  • $\begingroup$ The silhouette value is a measure of how similar an object is to its own cluster (cohesion) compared to other clusters (separation). But we have another variable(A) to compare. $\endgroup$ – parvij Aug 21 '16 at 4:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.