Measuring performance of different classifiers without class type in data

Question

To measure the performance of a classification algorithms on a dataset that has an attribute for class type, I divide my dataset to training and test samples and then create a confusion matrix for False Positive, False Negative, True Positive and True Negative samples. Hence there is a class type attribute(e.gYes or No ), the confusion matrix is pretty easy to calculate.

Now suppose that I have a dataset that lacks a class type attribute and all samples are of the class Yes .

How can I measure the performance of different classification algorithms using these kind of datasets?

Answer 1

Normal classification will not work in this case, classifiers learn functions that are able to seperate the training examples. In your case you only have one class which will not work for all the classical examples. There is a machine learning task called One-class classification, see this Wikipedia page. In your case PU Learning (Positive and Unlabeled) seems most appropriate. Evaluating the performance is extremely difficult without having some negative examples as well however. See this question and answer on the Stats stackexchange.