1
$\begingroup$

I have been working on a dataset that has data from a famous drug supply chain company. The first few records of the dataset look like the following;

enter image description here

Another data accompanies this (primary) dataset. Which has information on the 1115 drug stores of the company. In the latter dataset (the stores data), one of the columns pertain to the no. of months since when a competitor (nearest perhaps) of that store was established. It has 15% missing values (around 375 missing out of 1115).

Following is how the values in the variable look like;

enter image description here

I have the following idea in mind;

Make buckets of the data. I wish to bin the records in form of quarters (i.e. 3 months ago equal 1 quarter, 6 months ago refers to 2 quarters, etc.). I assume that the missing values might be referring to the fact that the exact time since when the competitor (drug store) was established was not known. So I want to denote the missing values as a separate category. In short, I want to convert a discrete variable into a categorical one.

My questions are;

  1. Whether it's a good idea?
  2. If not, how could I deal with such a dataset? (A domain oriented answer would be helpful)
  3. How do we treat with the missing values generally?
$\endgroup$

1 Answer 1

0
$\begingroup$

Yes, your binning strategy makes sense.

Alternatively, it’s also reasonable to fill missing values with the rounded average of the training dataset (ignoring the missing ones). You could also assume missing value denotes “no current competitor” and set it to a large negative number. Regardless of what works best for you, at test or predict time you would need to fill in with the exact same value.

Alternatively, if that column is meaningful to learn with at all, you could train a model excluding the rows having no value. This may bias learning if the missing values are a systematic phenomenon plus behaviour is ambiguous if missing values can occur at predict time.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.