0
$\begingroup$

I am working on a dataset that has a dependent variable that is binary, but it contains 98% of 0's and 2% of 1's. I am trying to use Logistic regression to predict purchase of a product. But because of the huge number of 0's, the model is not predicting well and getting a large number of false positive result.

Kindly suggest how can I approach this.

$\endgroup$
  • $\begingroup$ Can you give a sample of your dataset please? Are you trying to predict 0/1 in the future? $\endgroup$ – JahKnows Mar 12 '18 at 6:11
0
$\begingroup$

This kind of problem is call Data Imbalance issue, this is a very common issue in Financial Industry, Health Care Industry(Cancer Cell Detection) like Banks and Insurance (for Fraud Detection)

To overcome such issues, we use different techniques like Over-sampling or Under-sampling.

Over-sampling tries to increase that minority records by duplicating those records to make balance in the data

Under-sampling tries to decrease the majority records by removing some records which are not significant to make balance in the data.

There are different algorithms for implementing the same.

you can go through these Link-1,Link-2, for Explanation and Implementation of the same.

Let me know if you need anything else.

$\endgroup$
1
$\begingroup$

This is a class imbalance problem. Your data has more number of 0's, thats why the model is also predicting 0's only. There are many solutions to this problem, like over/under sampling, SMOTE etc. Here are the links which show how to tackle the problem-

Binary classification with strongly unbalanced classes

Imbalanced classification problem

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.