You cannot use the labels you obtain through k-means to treat the problem as a supervised classification problem. This is because k-means will assign an arbitrary label to every cluster it forms.
It would be only a matter of luck if you get the arbitrary labeling aligned in a way that the classical accuracy measure makes sense.
What you should be looking for is something called the Average Clustering Accuracy measure. This measure gives you the accuracy of your clustering no matter what the actual labeling of any cluster is, as long as the members of one clusters are together.
Disclaimer: everything I am about to write is thanks to a github script which can be found here. If you want you can skip what is coming, and directly go to the link and apply the function defined there called cluster_acc
The smart way to do it is to try to figure out what is the best setting that would yield me the maximum clustering accuracy. By setting here I mean: what labels in my prediction correspond to what labels in the ground truth.
You can do this in python using sklearn.utils.linear_assignment_.linear_assignment. This function uses the Hungarian algorithm to solve what is called a bipartite graph. Solving said graph is what I described above by 'figuring out the best setting'.
Now, I will try to explain in details how to obtain the bipartite graph, and how to obtain the cluster accuracy from the results of the Hungarian method.
- Let y and y' be the ground truth and the predicted clustering assignments, respectively. For example, y=[1,1,1,2,2,2,3,3,3]; y'=[2,2,1,3,3,3,1,1,2].
Notice how in this example, a classical accuracy measure will give an accuracy of 11%, where the more fair clustering accuracy measure will give a 78% as will be shown
- Construct the matrix W, which is a DxD zeros matrix where we will store points. D is the maximum value (label) among the predicted assignments and the ground truth.
For the same example above, W will be 3x3.
- Go over every pair of predicted assignment/ground truth. Add one point to every entry in W where an intersection between the rows described by y' and columns described by y takes place. For our example, this will produce:
W = [1 0 2; 2 0 1; 0 3 0] (using MATLAB notation).
- Subtract W from the maximum value it has. This will place a zero whenever the maximum value (in our case 3) takes place. This gives us our bipartite graph = [2 3 1; 1 3 2; 3 0 3].
- Then, solving this using the sklearn function mentioned above gives us the following table of matching: [ 1 3; 2 1; 3 2]. This tells us that ones in the prediction correspond to threes in the ground truth, twos to ones, and threes to twos, which can be easily confirmed. This matching table tells us which entries in W we should take into consideration when we are measuring the accuracy
- Finally, all we have to do is go to the entries (1,3),(2,1),and (3,2) in W and add them up, and take the average. This gives us a clustering accuracy of 78%.
Hope this was helpful, and thanks again to the authors of the attached github repository.