So from Algorithm 3 of https://arxiv.org/pdf/1603.02754v3.pdf, it says that an optimum default direction is determined and the missing values will go in that direction. However, or perhaps I have misunderstood/missed the explanation from the article, it doesn't say what exactly is the input.
For example, I have (parent) node A (with 50 inputs) splitting into node B and node C. Now, say of the 50 inputs there are 7 missing values.
The other 43 inputs are split into B and C accordingly. What I seem to be understanding is that, it will allocate the remaining 7 into B and C and determine which one gives a higher gain score; that will be the optimal direction.
However, given the 7 values are missing (Which means I don't know what are these 7 values), how does allocating missing values into any of the child nodes change the gain score, or rather minimize the loss function? This seems to suggest that Xgboost is inputting something for the missing values. I can't seem to find out what is Xgboost inputting for these missing values. I hope this question isn't too vague/general and easy.
Edit: I think "Missing values" may be a vague term. What I meant here is (From wiki) "In statistics, missing data, or missing values, occur when no data value is stored for the variable in an observation."
From the author himself (https://github.com/dmlc/xgboost/issues/21), he said " tqchen commented on Aug 13, 2014 xgboost naturally accepts sparse feature format, you can directly feed data in as sparse matrix, and only contains non-missing value.
i.e. features that are not presented in the sparse feature matrix are treated as 'missing'. XGBoost will handle it internally and you do not need to do anything on it."
" tqchen commented on Aug 13, 2014 Internally, XGBoost will automatically learn what is the best direction to go when a value is missing. Equivalently, this can be viewed as automatically "learn" what is the best imputation value for missing values based on reduction on training loss."