Fill missing values AND normalise

Question

I have two columns of training data for a neural net which are missing values. (There are many other columns which aren't missing values.)

For example

Height  | Weight
180     | 70
175     | N/A
N/A     | N/A

I want to fill missing values and normalise the columns.

The data is heights and weights so I thought a good fill value would be 0 or -1. This is based on the book Deep Learning in Python:

In general, with neural networks, it's safe to input missing values as 0, with the condition that 0 isn't already a meaningful value.

EDIT I assumed 0 wasn't meaningful in a dataset with values from 150-200

I was also recommended to normalise the data by subtracting the mean and dividing by the std for each column.

Both of those are fine on their own - I understand how and why to do them. What I don't get is how to combine them. I can either ...

• Fill missing values then normalise, but then a) my zeros will no longer be zeros (will my network still learn they are a special value?), and b) the zeros will affect the mean/std to a degree determine by how many values are missing. I suppose I'm concerned this would give a weird distribution
• Normalise then fill missing values. But after I've normalised my data, 0 is now the mean of my column and so isn't a fill value of the same kind. I'd rather let the network know the values are unfilled than assume they all take the mean value

I'm using Keras, Numpy and Pandas with Dense layers for a multiclass classification problem.

Answer 1

I don't understand why you would like to fill values with zeros ! This would basically mean, "this guy, who is 170 cm tall, weights 0 kg" and would fool your network. In my opinion, you have two options:

• discard missing values (the entire row): you end up with less but more consistent training data
• if you really need these rows, then fill missing values with some heuristics: for example, give them the mean of their column, or apply a simple linear regression. Beware that this will add a bias in the learning process, but it would be definitely better than giving random values.

At very least, if you have a lot of missing values, then maybe you should think of selecting a specialized model for partial training data. You actually don't tell us what network you are using, but you might modify it to handle missing values more intelligently than by filling missing values.

Finally, the need of normalizing once again depend on your model (which you don't describe). But this would definitely come after the processing part.

Answer 2

Trial and error is an important part of Deep Learning. There are situations where missing data has meaning and there situations where missing data is just noise. As an example, when tracking facial features like eyes, nose, or ears, missing data is informing the neural network that the feature is outside of view. Other times it is noise from bad data collection.

I recommend fitting your deep learning model with the following data:

1. Use 0 for missing data.
2. Remove rows with missing data.
3. Use the mean for missing data.
4. Single variable feature imputation or Multivariate interpolation.
5. Use Multivariate feature imputation