7
$\begingroup$

I have a dataset that I want to classify as fraud/not fraud and I have many weak learners. My concern is that there is much more fraud than not fraud, so my weak learners perform better than average, but none perform better than 50% accuracy in the complete set.

My question is whether I should set up testing and training sets that are half fraud and half not fraud or if I should just use a representative sample.

$\endgroup$
  • 1
    $\begingroup$ How many samples do you have of each? An alternative could be to try some outlier detection approach and test it against your fraud data. $\endgroup$ – jpmuc Jun 16 '15 at 19:36
  • $\begingroup$ bias–variance tradeoff should be there in training data. $\endgroup$ – Sreejithc321 Jun 19 '15 at 7:19
  • $\begingroup$ Have you seen this question:datascience.stackexchange.com/questions/6200/… $\endgroup$ – image_doctor Jul 14 '15 at 8:44
  • $\begingroup$ You might also consider layering a cost-matrix over your classification algorithm, as there is an imbalance in the penalty for making different errors in classification. $\endgroup$ – image_doctor Jul 14 '15 at 8:47
  • $\begingroup$ Is there a particular reason you want to use Adaboost in this context ? $\endgroup$ – image_doctor Jul 14 '15 at 8:48
1
$\begingroup$

Is it possible that adding generated data to your data set will decrease the fraud/non fraud ration and make your dataset more representative / usable ?

At GenieLog, we are producing test data for designing and testing fraud detection tools. We our generator GEDIS Studio we can define regular profiles and fraudster profiles, instantiating each category to a customizable ratio (for ex. 2 % of customer will have fraudulent usage of generated events.)

We did it successfully for telecom CDR (http://www.gedis-studio.com/online-call-detail-records-cdr-generator.html) and Credit Card usages. There's a freely available access to the online generator on http://www.data-generator.com

I'm pretty sure that even if the tool is not matching your needs at least the approach can be valuable. Otherwise I would be interested to read any objection :)

Regards

$\endgroup$
0
$\begingroup$

Training set must represent the dataset your application/algorithm is actually going to face. I suggest you to take a representative sample instead of dividing the training and test set with exactly half fraud an half non-fraud. But please make sure that the training set contains both positive and negative example for fraud for your classifier to perform better.

$\endgroup$
  • $\begingroup$ This is true for your test set, but not training. Oversampling is necessary for problems like these since the vast majority of instances will not be fraud. $\endgroup$ – David Jul 13 '15 at 15:32
0
$\begingroup$

In situations where a particular class is really a minority, I suggest using rare category detection. In this particular case of fraud/non-fraud, fraud is a rare category. Its an active field of research - Refere to Rare Category Detection

$\endgroup$
0
$\begingroup$

I think that it depends on your data set. There are many ways to handle unbalanced data sets, just search, for example https://www.quora.com/In-classification-how-do-you-handle-an-unbalanced-training-set . I think that the simplest way is to use the same distribution of classes in the train and test sets.

If you have really small amount of minority class you can try one-class classification.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.