1
$\begingroup$

I'm working with an unbalanced (10:1) dataset for classification. I also have a bunch of missing data on certain columns. If I discard them all, I still have a 5:1 ratio, so I guess I can afford to discard them?

What should I do with my majority class rows?

  • Predict/Fill the missing values and use them all for undersampling.
  • Discard the rows with missing values since the dataset is already unbalanced. Use the remaining rows for undersampling.

Sorry for the noob question, junior DS here.

$\endgroup$

2 Answers 2

2
$\begingroup$

I am quite reluctant to discard data because you might never know how they really affect the overall dataset. I would only discard them if they make up very little portion of the dataset (maybe like 5%?).

For your situation, it reduces half of the majority dataset which is ALOT. I would suggest you try and find correlations with other columns and find the mean value.

$\endgroup$
0
$\begingroup$

There are plenty of books and online resources about dealing with missing data and imbalanced class values.

Without knowing and becoming familiar with the data and its context (which is your job by the way) it's impossible to give a standard answer. There are many factors to consider when making such choices. But removing data should be a last resort.

My advice is to get reading!

One good introduction to handling missing values is Chapter 2 of Data Mining and Predictive Analytics, 2nd Edition.

Class imbalance can be dealt with using bootstrap resampling, for example. Again, it depends on your data and the end goal. For classification using ensemble methods, bootstrap resampling can be very effective. But you'll need to consider cost-sensitivity.

Here's an outtake of some course notes on this topic:

Building useful classifiers on these datasets can be challenging because classifiers such as traditional decision trees are likely to simply suggest that all records are non-cancerous patients and achieve 99% classification accuracy. However, these trees will not be useful since they will always mis-classify a cancerous patient, which are often more important to correctly classify. These trees will also not be able to reveal useful knowledge on the cancerous patients and fraudulent clients.

In cost sensitive datasets misclassification costs (i.e. the consequence of a misclassification) vary from class value to class value. For instance, the cost/consequence of the misclassification of a cancerous patient as non-cancerous can be significantly higher than the cost/consequence of the misclassification of a non-cancerous patient as cancerous. In the latter case, a non-cancerous patient may need to go through some medical tests, which will waste some time of the patient and the clinicians. However, in the former case a cancerous patient may remain undetected until it is too late and face serious consequences due to the misclassification or incorrect diagnosis.

A cost sensitive classification algorithm should aim to reduce misclassification of a costly class value (such as cancerous patients) even if that increases the misclassification of a non-costly class value (such as non-cancerous patients) in order to reduce the overall misclassification cost. A traditional classification algorithm such as a decision forest algorithm may not be able to take this cost sensitivity into account and be tolerant to some misclassification of a non-costly class value in order to reduce mis-classifications of a costly class value. Regular classification algorithms need to be modified to handle the class imbalance and cost sensitivity issues of datasets.

If cost-sensitivity is relevant then a CS-capable classifier is necessary.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.