Input contains NaN, infinity or a value too large for dtype('float32'). Without ruining Dataset

Question

I have a datasheet that is all in Binary but sometimes there are missing cases. In one of the inputs, there is a blank, I don't want to completely remove the whole column because some of the rows have values that maybe could be useful.

Example: Column 9 has all data except for Row 15 (random row) which is blank. How can I make my decision Tree without having to remove the whole column or make the data bad (By Averaging).

I'm not sure if averaging in the situation is will strengthen the data because it might correlate to a different output.

Answer 1

If NaNs are not too frequent you can try to drop the rows containing it.

Otherwise if your algorithm can't handle NaNs (i.e. lightgbm that is also a tree based algorithm can) you have to fill those values somehow. Usually it's a good idea to use a value that is out of the range of normal values (like -99 for a variable with only positive values) so tree can learn to treat those values differently.

In your case with binary (0/1) data it's a little different. I would put 0.5 so model can learn with a single cut to put missing values to either group (0 or 1) depending on which one makes more sense. Don't worry if all 3 groups are very different model should be also able to learn it given that you have enough data for the tree to make another cut.

Answer 2

Use Multiple Imputation. The idea is that you can consider the column with missing values as target in which you have values for many outputs (training+validation data) and you don't for some (test data to be predicted). Then you train a model based on those rwos that you have their value and use it to prdict the missing value. You may have a look here.

Answer 3

What you could also do is create a new column to identify whether that specific feature is missing or not.

Example:

You have feature I, where all rows have values except for row J. You could then create the feature I_na where for row J it would have the value of 1, but for every other row it would be 0.