Suppose I have a neural network which accepts two sets of features as inputs and generates corresponding outputs, for instance, generate average final grade from: 1. working hours for N students in a class, 2. mid-term grade for N students in the same class. During training, the neural network is trained with many different classes. What I would like to during inference is to turn off one set of the features (i.e. feed only working hours as inputs) and predict outputs using the same neural network. Obviously, setting the mid-term grades to be all 0s would not be a good option. I wonder if anyone know what would be a good way to do this?


  • $\begingroup$ Why not only train of the features available at test time? $\endgroup$
    – kbrose
    May 25 '18 at 16:52

As far as I know, you cannot do that.
First :

Obviously setting the mid-term grades to be all 0s would be be a good option

No, actually that is a really bad option, neural networks do not understand magical parameters, meaning, if you put a 0 there it will think that the mid-term grades is 0, therefore it will probably give you a very low final grade prediction for you.

My advise, if you are very interested in doing that, create boolean factors, training inputs:

  1. working_hours
  2. mid-term_grade
  3. mid-term_grade_present

mid-term_grade_present should be a 1 when you have the mid-term_grade training data, and 0 when you do not have it. BTW, yes, generate training data without the mid-term_grade.

If you are working with non linear machine learning algorithms that should be enough, if you are using linear algorithms you should one hot encode the mid-term_grade_present and multiply it by the mid-term_grade, ending with something like this:

  1. working_hours
  2. mid-term_grade_present_1 * mid-term_grade
  3. mid-term_grade_present_0 * mid-term_grade
  • $\begingroup$ Sorry that was a typo, it should be Obviously setting the mid-term grades to be all 0s would not be a good option.... I just updated it in the text. If I set mid-term_grade_present to be 0, what should I use as inputs, specifically what should be the data for mid-term_grade part? If it is the same with mid-term_grade_present=1 case, then this boolean flag basically has no effect, if it is something else, what should it be? $\endgroup$ Mar 26 '18 at 15:23
  • $\begingroup$ if you have enough data you could try taking 20% of it and setting mid-term_grade_present=0 $\endgroup$
    – Kailegh
    Mar 26 '18 at 15:36
  • $\begingroup$ If I set mid-term_grade_present=0 I do not think mid_term_grade data would be ignored by the neural network since: 1. it is already included as part of inputs, 2. it is correlated with outputs. $\endgroup$ Mar 26 '18 at 15:41
  • $\begingroup$ 1.-it is a neural network, it can learn non-linear relationships, and learn features interactions, trust me, it will learn that when mid-term_grade_present=0 it cant rely on mid-term_grade 2.- it is not correlated, you randomly pick a 20% of the data and remove the mid-term_grade info, no correlation in there, give a try, but I sincerely think it should work $\endgroup$
    – Kailegh
    Mar 26 '18 at 18:57
  • $\begingroup$ did this idea finally work for you? or do we have to come up with something else? $\endgroup$
    – Kailegh
    Mar 27 '18 at 14:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.