7
$\begingroup$

I have a dataset that I want to classify as fraud/not fraud and I have many weak learners. My concern is that there is much more fraud than not fraud, so my weak learners perform better than average, but none perform better than 50% accuracy in the complete set.

My question is whether I should set up testing and training sets that are half fraud and half not fraud or if I should just use a representative sample.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ How many samples do you have of each? An alternative could be to try some outlier detection approach and test it against your fraud data. $\endgroup$
    – jpmuc
    Commented Jun 16, 2015 at 19:36
  • $\begingroup$ bias–variance tradeoff should be there in training data. $\endgroup$
    – Sreejithc321
    Commented Jun 19, 2015 at 7:19
  • $\begingroup$ Have you seen this question:datascience.stackexchange.com/questions/6200/… $\endgroup$
    – image_doctor
    Commented Jul 14, 2015 at 8:44
  • $\begingroup$ You might also consider layering a cost-matrix over your classification algorithm, as there is an imbalance in the penalty for making different errors in classification. $\endgroup$
    – image_doctor
    Commented Jul 14, 2015 at 8:47
  • $\begingroup$ Is there a particular reason you want to use Adaboost in this context ? $\endgroup$
    – image_doctor
    Commented Jul 14, 2015 at 8:48

4 Answers 4

Reset to default
1
$\begingroup$

Is it possible that adding generated data to your data set will decrease the fraud/non fraud ration and make your dataset more representative / usable ?

At GenieLog, we are producing test data for designing and testing fraud detection tools. We our generator GEDIS Studio we can define regular profiles and fraudster profiles, instantiating each category to a customizable ratio (for ex. 2 % of customer will have fraudulent usage of generated events.)

We did it successfully for telecom CDR (http://www.gedis-studio.com/online-call-detail-records-cdr-generator.html) and Credit Card usages. There's a freely available access to the online generator on http://www.data-generator.com

I'm pretty sure that even if the tool is not matching your needs at least the approach can be valuable. Otherwise I would be interested to read any objection :)

Regards

Improve this answer
$\endgroup$
0
$\begingroup$

Training set must represent the dataset your application/algorithm is actually going to face. I suggest you to take a representative sample instead of dividing the training and test set with exactly half fraud an half non-fraud. But please make sure that the training set contains both positive and negative example for fraud for your classifier to perform better.

Improve this answer
$\endgroup$
1
  • $\begingroup$ This is true for your test set, but not training. Oversampling is necessary for problems like these since the vast majority of instances will not be fraud. $\endgroup$
    – David
    Commented Jul 13, 2015 at 15:32
0
$\begingroup$

In situations where a particular class is really a minority, I suggest using rare category detection. In this particular case of fraud/non-fraud, fraud is a rare category. Its an active field of research - Refere to Rare Category Detection

Improve this answer
$\endgroup$
0
$\begingroup$

I think that it depends on your data set. There are many ways to handle unbalanced data sets, just search, for example https://www.quora.com/In-classification-how-do-you-handle-an-unbalanced-training-set . I think that the simplest way is to use the same distribution of classes in the train and test sets.

If you have really small amount of minority class you can try one-class classification.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.