1
$\begingroup$

I have two columns of training data for a neural net which are missing values. (There are many other columns which aren't missing values.)

For example

Height  | Weight
180     | 70
175     | N/A
N/A     | N/A

I want to fill missing values and normalise the columns.

The data is heights and weights so I thought a good fill value would be 0 or -1. This is based on the book Deep Learning in Python:

In general, with neural networks, it's safe to input missing values as 0, with the condition that 0 isn't already a meaningful value.

EDIT I assumed 0 wasn't meaningful in a dataset with values from 150-200

I was also recommended to normalise the data by subtracting the mean and dividing by the std for each column.

Both of those are fine on their own - I understand how and why to do them. What I don't get is how to combine them. I can either ...

  • Fill missing values then normalise, but then a) my zeros will no longer be zeros (will my network still learn they are a special value?), and b) the zeros will affect the mean/std to a degree determine by how many values are missing. I suppose I'm concerned this would give a weird distribution
  • Normalise then fill missing values. But after I've normalised my data, 0 is now the mean of my column and so isn't a fill value of the same kind. I'd rather let the network know the values are unfilled than assume they all take the mean value

I'm using Keras, Numpy and Pandas with Dense layers for a multiclass classification problem.

$\endgroup$
2
$\begingroup$

I don't understand why you would like to fill values with zeros ! This would basically mean, "this guy, who is 170 cm tall, weights 0 kg" and would fool your network. In my opinion, you have two options:

  • discard missing values (the entire row): you end up with less but more consistent training data
  • if you really need these rows, then fill missing values with some heuristics: for example, give them the mean of their column, or apply a simple linear regression. Beware that this will add a bias in the learning process, but it would be definitely better than giving random values.

At very least, if you have a lot of missing values, then maybe you should think of selecting a specialized model for partial training data. You actually don't tell us what network you are using, but you might modify it to handle missing values more intelligently than by filling missing values.

Finally, the need of normalizing once again depend on your model (which you don't describe). But this would definitely come after the processing part.

$\endgroup$
  • $\begingroup$ It really depends on the neural network you're using. if this is a multilayer perceptron, this doens't make sense. $\endgroup$ – debzsud Jul 26 '18 at 12:07
  • 1
    $\begingroup$ i've added the quote to my question. do you disagree with it? or have i misunderstood it? $\endgroup$ – Joel Berkeley Jul 26 '18 at 12:16
  • $\begingroup$ A zero is ok since it "inhibit" the input corresponding neuron. If you fill with zero after, it would basically be the same as my second option (taking the mean). Another option is to normalize between 0 and 1, and you won't have collusions anymore between actual values and missing values. $\endgroup$ – debzsud Jul 26 '18 at 12:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.