3
$\begingroup$

Let me preface this post with I am incredibly new to machine learning/neural networks. I am currently working on a classification neural network using TensorFlow whose input is multiple features of continuous data and whose output is an array of confidence levels for a group number (softmax). In some instances, the data passed into the model could have some undefined values in various rows. I understand from research and testing that input tensors' elements must all be of the same type. I have looked into a couple of options on how to fix the issue of these undefined values:

  1. I could simply set these undefined points equal to some constant like 0 or -1 (I believe this to be my best option, as it does not sacrifice other features)
  2. I could remove any row of data with an undefined value. I'm not a fan of this idea as I am working with high-dimensional data, so if I remove one row, my model would be missing out on quite a few columns worth of data.

Beyond these two, I have been unable to find any additional information.

I have tested both of these ideas, and while they fix the issue, they do have some negative impacts on the accuracy of my model. My question is this: What are some other effective ways of handling undefined values when working with neural networks?

I understand that the question is relatively vague, and I apologize if it has any necessary information missing. Please let me know if there is anything I can clarify.

$\endgroup$
2
  • $\begingroup$ There are many other approaches e.g. Imputation with Mean/Median to handle NaN. Why are you not using any of them. Read $\endgroup$
    – 10xAI
    Commented Jun 22, 2021 at 5:13
  • $\begingroup$ As I said in the original post, I am very new to the field. My two approaches were just simply what I could find while researching. I have not used the method you mentioned because I have not heard about it yet, but I will be sure to look into your suggestion. Thank you! $\endgroup$
    – Tis
    Commented Jun 22, 2021 at 14:06

2 Answers 2

0
$\begingroup$

One option is to remove the specific input node which has a null value for that training instance. This is similar to dropout. Thus, the connections between that input node and the next layer would not be present and would not contribute to prediction.

$\endgroup$
0
$\begingroup$

There is not one absolutely correct method to handle this issue. Depending on dataset and algorithm used (eg NNs, SVMs) it is more appropriate in certain cases to simply add default values, but in other cases these default values can skew the results, so it is better to remove these data points altogether. Another method is to remove only the problematic features, in case there are always the same features with undefined values, and not the whole data point.

In machine learning rarely, if ever, is there a one-size-fits-all method. They all depend on various parameters, including type of problem, dataset and learning method used.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.