0
$\begingroup$

To measure the performance of a classification algorithms on a dataset that has an attribute for class type, I divide my dataset to training and test samples and then create a confusion matrix for False Positive, False Negative, True Positive and True Negative samples. Hence there is a class type attribute(e.gYes or No ), the confusion matrix is pretty easy to calculate.

Now suppose that I have a dataset that lacks a class type attribute and all samples are of the class Yes .

How can I measure the performance of different classification algorithms using these kind of datasets?

$\endgroup$
  • $\begingroup$ What are you classifying then? Every classifier will return Yes and all will do very good $\endgroup$ – Jan van der Vegt Aug 19 '16 at 10:06
  • $\begingroup$ @Jan I have a set of attributes "a1" to "an" in my dataset. The attribute "an' is either "yes" or "no" . I train the classifier using train data and the classifier labels the test data as either "yes" or "no" and then I compare them with the actual "yes" and "no" to measure performance. Now my question is how to compare the labels when there are no actual "yes" and "no"(all of my data are actually "yes" and I don't have the attributes a1 to an-1 for "no" samples), $\endgroup$ – danrah Aug 19 '16 at 10:25
1
$\begingroup$

Normal classification will not work in this case, classifiers learn functions that are able to seperate the training examples. In your case you only have one class which will not work for all the classical examples. There is a machine learning task called One-class classification, see this Wikipedia page. In your case PU Learning (Positive and Unlabeled) seems most appropriate. Evaluating the performance is extremely difficult without having some negative examples as well however. See this question and answer on the Stats stackexchange.

|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.