1
$\begingroup$


I wanted to start off by saying this is not an exact duplicate of the other question. I checked it and it didn't have what I urgently need.
So here is the problem. I have a dataset with 30 features and 100000 rows of train data. I want to make a Binary classification model which determines whether the person is eligible or not for membership at our club. I am a rookie data scientist, and binary classification is a first for me. So please help me and tell me which model would be the most accurate for this purpose. Also, the time taken by the model to train doesn't matter... Thank you so much

update: Ok, I have used logistic regression and instead of giving some members as accepted(1) it is showing me al members do not get the membership...So I thought there might be a mistake in choosing the Model, so I am looking for other models...

Improve this question
$\endgroup$
9
  • $\begingroup$ You can use any classification model for binary classification. What is the problem exactly? $\endgroup$
    – Ankit Seth
    Jul 5, 2018 at 6:16
  • $\begingroup$ Ok, I have used logistic regression and instead of giving some members as accepted(1) it is showing me al members do not get the membership...So I thought there might be a mistake in choosing the Model, so I am looking for other models... $\endgroup$
    – omkaartg
    Jul 5, 2018 at 6:18
  • $\begingroup$ Out of your 1000000 rows, how many have membership and how many don't have ? $\endgroup$
    – Ankit Seth
    Jul 5, 2018 at 6:19
  • $\begingroup$ 14000 have membership, the rest dont $\endgroup$
    – omkaartg
    Jul 5, 2018 at 6:20
  • 1
    $\begingroup$ 14% of your data points are labeled as Class A (Accepted) and 86% is labelled as Class N (Not Accepted) just by looking at that it is clear that you have a class imbalance problem. So just by predicting class N your model is getting 86% accuracy. Hence it might misguide you into believing that it is accurate enough but it isnt'. I think Alexis has covered the point very well in the answer below. $\endgroup$
    – Kaustubh
    Jul 5, 2018 at 6:40

3 Answers 3

Reset to default
1
$\begingroup$

As your data is highly imbalanced, and as per your task, it is a case of anomaly detection.

Anomaly detection is a case where your data has one kind of examples in very low number and other in very high number, like your membership division here. Other examples are like detecting flaw in car engines- out of 10000 engines, you get flaw in 40 only. Similarly, members compared to non-members are very less. So treat those person who are members as anomaly.

https://www.allerin.com/blog/machine-learning-for-anomaly-detection

As you can check in above link, there are both supervised and unsupervised methods available for these kind of tasks. I suggest you try those methods. Also you can check this link for some more explanation-

https://www.datascience.com/blog/python-anomaly-detection

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you, I have solved the problem because of you and @Alexis $\endgroup$
    – omkaartg
    Jul 5, 2018 at 10:49
5
$\begingroup$

Looks like to me this is a classic imbalance binary classification problem (see comments above). What loss are you using ? It looks like your model is predicting the non-membership class because it’s minimising it’s averaging loss. Here are some techniques you might wanna try to solve this issue:

  • use regularization
  • over sampling the membership class
  • under sampling the non-membership class
  • select variables, linked to regularization above with l1 and l2 penalties
  • feature engineering: you specified you have 30 features but are all of them useful ? How do you preprocess them to feed the model ? Are they numerical or categorical ?

I hope this gives you some ideas about how tackling the problem. Changing the model won’t miraculously solve your issue.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ I would add that there is a possibility that a non-linear model, such as XGBoost, could be worth a try as an automated attempt to get a better accuracy. However, the OP still needs to take care. For instance there is no indication that they have any cross-validation or test data, and these become more important to protect against over-fitting when using non-linear models. $\endgroup$
    – Neil Slater
    Jul 5, 2018 at 6:52
  • $\begingroup$ Thank you, I have solved the problem because of you and @Ankit _Seth $\endgroup$
    – omkaartg
    Jul 5, 2018 at 10:48
1
$\begingroup$

Apart from using regularization, I would use a stratified-shuffle-split when you divide the data in train and test sets in order to deal with class imbalance problem. It is also important to avoid using accuracy directly as a performance measure and, instead of this, use f1-score. Accuracy is not a good indicator of performance if you have class imbalance.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Sorry, I am a bit ignorant but what is the difference between train_test_split() and stratified-shuffle-split()? $\endgroup$
    – omkaartg
    Jul 6, 2018 at 8:47
  • $\begingroup$ the stratified keeps the percentage of samples of each class at both train and test sets $\endgroup$
    – Miquel Artigues
    Jul 9, 2018 at 11:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.