0
$\begingroup$

I want to train an Artificial Neural Network on some data however some of the fields are optional, dependent upon the values contained in other fields. The data in these optional fields is missing, but correctly so. I am wondering how I should factor this into my neural network.

I feel like what I might need to do is turn certain inputs on/off dependent upon the values contained in other inputs but then I am wondering how this would affect weighting in the hidden layer. Could I have some kind of conditional hidden layer or something for only when these optional values are 'activated'?

What would be the standard approach to this? It seems surprisingly hard to find any material on this online!

Thanks in advance for any help!

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

Several options, depending on your algorithms:

  1. Default to 0
  2. Default to 1
  3. Default to Average
Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ Missing values should not be replaced by zeros or ones if there already exist such values in the data. $\endgroup$
    – tuomastik
    Commented Jul 25, 2017 at 13:54
  • $\begingroup$ Am I right in thinking that by defaulting to the average that this would ultimately have no contribution on the overall output? $\endgroup$
    – Taylrl
    Commented Jul 25, 2017 at 16:12
  • $\begingroup$ @tuomastik As there was no input on how the data is actually used within his algorithms, or what it meant for the input when the optional data is empty, I just considered it possible options. Whether that works for him or not, is not up for me to say here. $\endgroup$
    – Lucas Lorenz
    Commented Jul 26, 2017 at 11:11
  • $\begingroup$ @Lucas Lorenz - Zeros and ones are generally pretty common in a dataset. More robust value for replacement could be -9999, for example. Generally speaking, the value should be chosen with a data-driven approach instead of just randomly picking one. Scikit-learn has Imputer class which might be extended in the future by k-NN imputation as discussed here. $\endgroup$
    – tuomastik
    Commented Jul 26, 2017 at 12:57
  • $\begingroup$ @tuomastik: -9999 would be fine for a decision tree algorithm, but would be a problem for a neural network. It would cause numerical problems and loss of performance. $\endgroup$
    – Neil Slater
    Commented Oct 23, 2017 at 8:39
1
$\begingroup$

Zeros can be okay because they will have no impact on the input sum of the next layer. But you should train so the network that it has to know what to do when those are missing (no value coming from them). So you should dropout the input layer, possibly the way your data can be missing. Or you could just dropout with a probability between 0-1 the way that you make this probability random.

I have to deal with missing data too, so I think I will try it out.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.