Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I am working on a dataset that has a dependent variable that is binary, but it contains 98% of 0's and 2% of 1's. I am trying to use Logistic regression to predict purchase of a product. But because of the huge number of 0's, the model is not predicting well and getting a large number of false positive result.
Kindly suggest how can I approach this.
This kind of problem is call Data Imbalance issue, this is a very common issue in Financial Industry, Health Care Industry(Cancer Cell Detection) like Banks and Insurance (for Fraud Detection)
To overcome such issues, we use different techniques like Over-sampling or Under-sampling.
Over-sampling tries to increase that minority records by duplicating those records to make balance in the data
Under-sampling tries to decrease the majority records by removing some records which are not significant to make balance in the data.
There are different algorithms for implementing the same.
you can go through these Link-1,Link-2, for Explanation and Implementation of the same.
Let me know if you need anything else.
This is a class imbalance problem. Your data has more number of 0's, thats why the model is also predicting 0's only. There are many solutions to this problem, like over/under sampling, SMOTE etc. Here are the links which show how to tackle the problem-
