I need to train a regression algorithm with multiple features and a single label (predicted value). The problem is that this algorithm has to be able to do on-line learning and the number of features it will receive will vary. Let me give a clear example:
The algorithm is trained on a dataset of shape:
[--------Features-----------------] [Label]
[-- context11 -- | -- context12 --] [label1]
Then, for the next training example, one of the contexts might be missing, so the training example might either be:
[--------Features-----------------] [Label]
[-- context21 -- ] [label2]
or
[--------Features-----------------] [Label]
[-- context22 --] [label2]
How can I deal with this situation? So far, I thought about two possibilities:
- Replace the missing part of the features with zeros. I am not sure how this will affect the algorithm though.
- Use a decision tree or random forest? Do these have a more natural way of dealing with a variable number of features?
Any other ideas?