What are the best practices to train xgboost (eXtreme gradient boosting) models on data that is to big to hold it in memory at once? Splitting the data and train multiple models? Are there more elegant solutions?
You can train xgboost, calculate the output (margin) and then continue the training, see example in boost from prediction.
I‘ve not tried it myself, but maybe you could train on the first subset of your data (say 10%) and then continue on another subset, etc.
Step by step procedure
- Split the data into N manageable subsets, set n=1
- Train xgboost on n-th subset
- Calculate the prediction (margin) for n+1 subset using the model obtained from previous
- Add the margin into the n+1 subset via setinfo
- Increment n
Steps 2-5 to be repeated N times.
I don't think what you are asking for is possible. See this issue.
I understand that you want to train the model on A PART of the data and then continue the training on another PART and so. So @aivanov's answer will not help in this regard.