I have a project for comparison between clustering techniques using the data set of SSA for birth names from 1910-2013 years for the different states.

I have finished applying my clustering techniques on my data set and the output of the clusters were the clusters of the states for each year.

Now I can know from my results; which states are close to each other with the birth names and which were not. By looking

I would like to have interesting results to make my project report interesting, Which states are similar in birth names are not enough to make the read be excited about my project.

My questions:

  1. any ideas of what can be learned from my project?

  2. How can I show the comparison between the clustering techniques? anyway other than seeing how Homogeneous the clusters are?

  • $\begingroup$ I think the question is incomplete ("By looking" ... ?) and it's likely too open ended to ask "what can I learn". Can you narrow this down specifically, to maybe your second point? what are you hoping to compare and what have you tried? $\endgroup$
    – Sean Owen
    Apr 12 '15 at 10:34

It's common to perform clustering on certain features, and then use characteristics such as mean differences between other features (not used in clustering) to show how clusters differ. It's not always meaningful to say that clusters differ merely by lower distances between cluster centers and objects assigned to them (as in k-means).

As an example, you might acquire other features (variables) for the states and then average each feature among objects assigned to each cluster. Then show bar charts of averages for those variables among all the clusters to assist in explaining how the clusters differ. You might also test for significantly different means of the supplemental features across clusters.

  • $\begingroup$ Thanks for your answer, can you explain more about your example of acquiring other features (variables) for the states and then average each feature among objects assigned to each cluster? I didn't completely understand it $\endgroup$
    – shosha
    Apr 12 '15 at 16:05
  • $\begingroup$ Maybe get population size of each state during the given calendar year, percent ethnicity for a variety of race/ethnicity combinations, average education, average household income. Migration could affect names as well. Next, average the population sizes of the states within a cluster, do this for each cluster, then show a bar chart with the mean (average) population size for each cluster. A high bar means large average population size of states in that cluster, whereas small bars reflect clusters having states with low population size. (each bar represents a cluster). $\endgroup$
    – user9086
    Apr 12 '15 at 16:34

For question 1, I feel there are a great deal of things you can look into with this. The basic conclusion is which states are culturally similar to one another (physical proximity would lead to such similarity). Having a 'cultural similarity score' from year to year, you could determine the connectedness of states over time, see if major events effect how connected the country is..... there is a lot of potential here, and reenforces the notion that you need to know what question you're trying to answer before you get too far into an analysis.

As for two, this is a bit trickier. The visual I see is each state getting a node at it's center, and like states having color coded connectors. The visual I'm seeing is similar to the graph they use for the scikit learn site for affinity propagation:



To compare clustering results from different clustering techniques you can use the Entropy Scorer node in KNIME (www.knime.org) THis node overlaps and compares results from different techniques. The quality measure at the end tells you how similar the clusterization is.

-- Rosaria


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.