How to create a confusion matrix for k-means with two features?

Question

I have the need to do a confusion matrix for data run through k-means with two features. I am aware that this is a clustering algorithm and not a classification algorithm but I have seen some articles and questions where it has been done. I am just to thick to decompose the answers and apply it to my situation.

I have data which looks like this:

Total Packets	Total TCP
2	0
0	0
0	0
4	0
1	1
4	2
0	0
0	0
0	0
1	1
0	0
93	85
1234	1232
699	695
4	4
2	2
0	0
0	0
0	0
0	0
0	0
4	0
0	0
4	0
6	4
3	3
0	0
0	0
0	0

Thats the top of the data file with the anomalies/outliers being anything over 200 in the Total TCP column. Where the confusion starts is understanding what is meant in the answer in this link k-means question where the responder mentions k-means labels and truth labels in his answer about how to do a confusion matrix. I have provided a quote for context:

"Assuming that you have some gold standard for the classification of your headlines into k groups (the truth), you could compare this to the KMeans clustering (the prediction).

The only problem with this is that KMeans clustering is agnostic to your truth, meaning the cluster labels that it produces will not be matched to the labels of the gold standard groups. There is, however, a work-around for this, which is to match the kmeans labels to the truth labels based on the best possible match."

Has anyone an idea of what the labels would be with my example? I have followed a tutorial in another link Outlier Detection with K-means and with a K of one it seemed to pick up the outliers as seen in this plot:

The red circles are around the outliers. In terms of where I am I have the program to a point where I can get the outliers but I would like to do a confusion matrix on top of this. I think that has to to with the K-means labels and truth labels mentioned previously but I am a bit lost in how to proceed. Any help would be greatly appreciated and I hope there is enough information in the post.

Answer 1

The question doesn't mention it clearly but apparently the goal is to detect outliers, in this case defined as instances with "anything over 200 in the Total TCP column". So every instance can be labelled as outlier or not:

• class 0 (negative) if total TCP <200
• class 1 (positive) if total >= 200

If you add a third column is_outlier which represents the true outlier status, you obtain an annotated dataset that could be used for binary classification.

Now let's assume you want to cluster with k-means and obtain a confusion matrix. In this case you're using k-means for doing classification without supervision (no training with labelled instances). Let's say $k=2$ since you're actually doing binary classification, so k-means predicts two clusters with no particular meaning or order. Before evaluating against the true labels you would need a method to match the predicted clusters with the true classes. In this particular case it would make sense to take the largest predicted cluster as corresponding to class 0 (not an outlier) and the smallest as class 1 (outlier). Once this is done, you can count the number of instances for every pair (predicted outlier status, true outlier status).

While this is perfectly doable, this approach is highly questionable: you have a deterministic method to find outliers with a simple test on the total TCP value, so why using ML in the first place? Testing the value directly is much more efficient and achieves 100% performance. Also here it's unclear why one would use clustering this way if the goal is actually classification.