I got a list of fake accounts through an algorithm and I would like to determine the precision/accuracy of this result comparing it with the labelled dataset. The lists contain only fake accounts and not only the accounts were identified by my algorithm, so the length of the lists is different. The predicted list (via algorithm) is the following:

['A','B','C','G','L'] # these values are unique; not have duplicate in the list

whereas the labelled dataset, i.e. the source dataset containing labelled data, is the following:

['A','C','D','H','J', 'L']

The length is different, as you can see. I was thinking of using a confusion matrix, but probably this is not the case. There are 3 finding in common (A,B and L) and 2 not in common. Any idea on how I could consider 'good' my model?


A confusion matrix is indeed the right tool to do the job, if you view both the labelled data and your result not as sequences, but as assignments of binary labels (1 = fake, 0 = not fake) to all the accounts (in your case, probably A through L).

To construct the confusion matrix, you can either calculate the binary labels by a "contains" operation on both lists and use standard confusion matrix routines, or use set operations (if the labelled list is L and your result R, the true positive count would be L & R, the false negative count L - R, etc.)

Once you have the confusion matrix, you can use standard classification metrics to evaluate the accuracy of your model.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.